I think it might be a duplicate form ER8411 v1.0 Firmware 1.3.6 issues.

ill add what i had posted in that topic  as well:

What We Discovered: The Complete Picture

Note: URLs are redacted as "hxxps://[site]" - replace 'xx' with 'tt' for actual URLs.

The Core Problem

My network has a packet reordering issue that only affects new TCP/TLS connections. Let me break this down step by step.

How a Normal HTTPS Connection Works

When I visit hxxps://[url], here's what happens:

Step 1: TCP Handshake (works fine for me)

Me → Server: SYN (let's connect)
Server → Me: SYN-ACK (ok, I'm ready)
Me → Server: ACK (great, connected!)

This part works perfectly every time

Step 2: TLS Handshake (THIS IS WHERE IT BREAKS)

Me → Server: ClientHello (here's my encryption info)
Server → Me: ServerHello + Certificate + Key Exchange
              ^^^ THIS IS THE PROBLEM ^^^

The server's response is too big to fit in one packet, so it gets split into multiple TCP packets:

Normal scenario (working):

Packet 1: Bytes 1-1440    (ServerHello start)
Packet 2: Bytes 1441-2880 (Certificate data)
Packet 3: Bytes 2881-4097 (Certificate end)

My computer receives them in order (1, 2, 3), reassembles them, completes the TLS handshake - SUCCESS

My broken scenario:

Packet 2: Bytes 1441-2880 arrives FIRST
Packet 3: Bytes 2881-4097 arrives SECOND
Packet 1: Bytes 1-1440    arrives NEVER (or very late)

My computer says: "I got packet 2 and 3, but I'm missing packet 1!" and waits for packet 1 that never arrives (or arrives too late). After 5-10 seconds: timeout - FAILURE

Why Does Connection Reuse Work?

Once a TLS connection is successfully established, I can reuse it forever:

Connection Reuse (HTTP Keep-Alive)

First attempt: New connection
  → TCP handshake - SUCCESS
  → TLS handshake (might fail due to packet reordering) - MAYBE
  → If successful: Connection is now OPEN

Second request on SAME connection:
  → No new TCP handshake needed
  → No new TLS handshake needed
  → Just send: "GET /page2 HTTP/1.1" on existing connection - SUCCESS
  → Works perfectly!

PowerShell Example (why it works 10/10):

Invoke-WebRequest "hxxps://[url]"

PowerShell maintains a connection pool. It does this:

Request 1: Create new connection (might get lucky, no packet reordering)
         → Connection stays OPEN in pool
Request 2: Reuse connection from pool - SUCCESS
Request 3: Reuse connection from pool - SUCCESS
Request 4: Reuse connection from pool - SUCCESS

curl Example (why it fails alternating):

curl "hxxps://[url]"  # New connection each time!

curl creates a brand new connection for each request:

Request 1: New connection → Path A → Works - SUCCESS
Request 2: New connection → Path B → Broken (packets reordered) - FAILURE
Request 3: New connection → Path A → Works - SUCCESS
Request 4: New connection → Path B → Broken - FAILURE

Why Is It Alternating?

This is the mysterious part. My network has two paths that traffic alternates between.

Since I tested on my neighbor's network (same ISP, same area) and they have no issues, this rules out ISP-level problems. The issue is specific to my ER8411 gateway.

Theory: Connection Tracking Hash in ER8411

My gateway uses a hash of the connection to decide internal packet processing:

Connection hash = hash(source_port + dest_ip + dest_port + timestamp)

Hash is EVEN → Path A (works) - SUCCESS
Hash is ODD  → Path B (packet reordering) - FAILURE

Because source ports increment:

• Connection 1: Port 54321 → hash = even → Path A - SUCCESS

• Connection 2: Port 54322 → hash = odd → Path B - FAILURE

• Connection 3: Port 54323 → hash = even → Path A - SUCCESS

This suggests the ER8411's hardware offload or packet processing engine has two internal paths, and one of them has a bug that reorders packets.

The tcpdump Proof

We captured this with tcpdump:

Working connection (attempt 1, 3, 5...):

15:45:46.953806 Me → Server: ClientHello (517 bytes)
15:45:46.960253 Server → Me: seq 1:2880 (TLS ServerHello starts)
15:45:46.960339 Server → Me: seq 2881:4097 (continues)
15:45:46.962407 Server → Me: seq 4097:5537 (continues)
Packets arrive IN ORDER, TLS handshake completes - SUCCESS

Broken connection (attempt 2, 4, 6...):

15:45:47.988312 Me → Server: ClientHello (517 bytes)
15:45:47.995990 Server → Me: seq 2897:4097 ← ARRIVES FIRST (wrong!)
15:45:47.996000 Me → Server: SACK {2897:4097} (I got this but missing earlier data)
15:45:47.999812 Server → Me: seq 6993:7085 ← MORE OUT OF ORDER DATA
15:45:52.984604 Me → Server: FIN (giving up after 5 seconds)
Packet 1 (seq 1:2880) NEVER ARRIVED, connection times out - FAILURE

The SACK (Selective Acknowledgment) proves my computer is saying: "I received bytes 2897-4097, but I'm still waiting for bytes 1-2896!"

Real World Impact

Apps/Tools That WORK:

• Browsers (Chrome, Firefox, Edge) - aggressively reuse connections

• PowerShell Invoke-WebRequest - connection pooling

• curl with keepalive - reuses connection

• Mobile apps after initial load - maintain persistent connections

Apps/Tools That FAIL:

• curl (default) - new connection every request

• wget (default) - new connection every request

• Mobile apps on first launch - establishing new connections

• Any tool that doesn't reuse connections

Why My Phone Apps Failed Intermittently:

Open app:
  Connection 1 (login API): Works - SUCCESS
  Connection 2 (fetch data): Fails - FAILURE → App shows error
  User retries:
  Connection 3 (fetch data): Works - SUCCESS → App loads

I just thought the app was "slow" or "glitchy" and retried until it worked!

Why We Still Don't Know the Root Cause

We've eliminated:

• Multi-WAN load balancing (disabled, still happens)

• Multiple VLANs (only VLAN 6 active now, still happens)

• MTU issues (tested many values, still happens)

• My local config (works on 5G, so it's not my devices)

• ISP issue (tested on neighbor's network with same ISP - they have no issues)
   

What's left:

1. ER8411 firmware bug (firmware 1.3.6) - Hardware offload in the gateway's SoC is reordering packets

2. Hardware defect in my specific ER8411 unit - The packet processing ASIC might be faulty

3. Specific configuration interaction - Some combination of my settings triggers the bug

The fact that my neighbor (same ISP, same area, different router) has zero issues strongly points to the ER8411 being the culprit.

Bottom Line

My ER8411 gateway has two internal packet processing paths. One path works perfectly, one path scrambles the packets. Every new connection randomly picks one of these paths, giving me a 50/50 success rate.

Connection reuse works because once I'm on a path (good or bad), I stay on it - and if I got lucky with a good path, I can keep using it forever.

This is why it appears to "alternate" - I'm not really alternating between good and bad, I'm just seeing the statistical result of randomly picking between two paths for each new connection.

Since my neighbor with the same ISP has no issues, this is almost certainly an ER8411 firmware bug or hardware defect.

I found a temporary solution for windows pc's or devices that can set custom MTU... So this is not a solution for android or tv! Anyone got an idea how to set it vlan wide?
```

PS C:\Users\john> netsh interface ipv4 set subinterface "Ethernet" mtu=1492 store=persistent
Ok.

PS C:\Users\john> function Test-Connection-Pattern {
>>     param(
>>         [string]$Url,
>>         [int]$Attempts = 10,
>>         [int]$Timeout = 10
>>     )
>>
>>     1..$Attempts | ForEach-Object {
>>         $code = curl.exe -s -o nul -w "%{http_code}" --max-time $Timeout $Url 2>$null
>>
>>         if ($code -eq "000") {
>>             Write-Host "Attempt $_ - FAILED" -ForegroundColor Red
>>         } elseif ($code -match "^(200|302)$") {
>>             Write-Host "Attempt $_ - SUCCESS ($code)" -ForegroundColor Green
>>         } else {
>>             Write-Host "Attempt $_ - $code" -ForegroundColor Yellow
>>         }
>>
>>         Start-Sleep -Seconds 1
>>     }
>> }
PS C:\Users\john> Test-Connection-Pattern -Url "hxxps://[URL]" -Attempts 5 -Timeout 3
Attempt 1 - SUCCESS (302)
Attempt 2 - SUCCESS (302)
Attempt 3 - SUCCESS (302)
Attempt 4 - SUCCESS (302)
Attempt 5 - SUCCESS (302)
```

 Hey @Ethan-TP ,

Yes setting DHCP option 26 to a lower MTU solves it for a couple of my devices. Not all, as not all devices adhere to this DHCP setting, being able to set the MTU of the internal VLAN would solve my issue.

We encountering the same issue, after migrating for Omada Software Controller v5 to V6. 

Gateway: ER8411 (1.3.2 firmware) - setup/controlled via Omada

PPPoE connection with ISP, requests are randomly falling (2 out of 5 requests), 

The only workaround we've found for now was manually setting the MTU to 1492 on essential machines. But that's impossible for 200+ devices.

Also tries with DHCP Option 26 - MTU, but that ignored by most of our devices.

The easiest way to reproduce it, 

> docker pull redis:latest

>

> Error response from daemon: Head "https://registry-1.docker.io/v2/library/redis/manifests/latest": net/http: TLS handshake timeout

We rolling back to the previous Omada Software Controller version.

Will be greatly appreciated if we get a resolution here ASAP. Thanks !

  @Ethan-TP Hey somehow i can not open the ticket url. ticket.omadanetworks.com/hc/requests/6XXXX gets me redirected to support.omadanetworks.com/nl/...