Business Community > All Threads > ER8411 Loss of WAN connectivity after power outage
< All Threads

ER8411 Loss of WAN connectivity after power outage

ER8411 Loss of WAN connectivity after power outage

ER8411 Loss of WAN connectivity after power outage
ER8411 Loss of WAN connectivity after power outage
2024-06-10 12:01:35 - last edited 2024-06-12 04:03:20
Tags: #WAN Setup #Network Connectivity #Highlighted Thread
Model: ER8411  
Hardware Version: V1
Firmware Version: 1.2.1

Hello,

I faced another issue with ER8411 last night. I had a power outage at my site that lasted 3 hours. After the power restoration (ER8411 has no backup power unit as of now, both power sockets are connected), the ER8411 was unable to raise again the PPPOE connection.

 

SFP+ WAN1: PPPoE connection was disconnected by peer. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:38    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:33    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:50    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:08    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:22:26    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)

 

I though this was caused by the power outage, maybe affecting the ISP connectivity but as test I moved the SFP+ transceiver to two different routers and PPPoE connectivity went immediately up.

This ER8411 is managed by Omada SDN Controller Linux 5.14.20.9.

I tried to move the WAN connection to the 3 SFP+ 1G port of the ER8411 and also to LAN 1 (RJ45), configuring the proper settings for PPPoE connection but had the same errors reported above. At this point, I tried multiple reboots of the ER8411.

Omada SDN reported no connectivity issues towards ER8411 and the WAN interfaces were just marked as "not connected" (connection was still KO also after manually trying to bring it up via connect button).

 

Then, I tried to re-adopt the device from the controlled = nothing changed.

 

Lastly, I downloaded diagnostic file of ER8411 and let the controller to forgot it.

As soon as I booted to default/standalone mode, I reconfigured the PPPoE connection, with same SFP+ IF and transceiver and it went up immediately. I performed the adoption from Omada Controller and everything was back up as before. So we are definelly facing a software (ER8411 1.2.1) or Omada SDN bug which prevented raising a PPPOE connection, even after multiple software reboots. I did not perform any hard reset so far.

 

As said, I have both the ER8411 diagnostic and the controlled export of running logs (I did not reboot yet the controller) and config export available.

 

I look forward to your support as this device should be TP-Link's diamond tip router and such issues shouldn't really happen. I'm available to provide every log/diagnostic needed.

Attached some PPP logs about the disconnections and the last connection brought up with no further events or config changes on my end.

Thank you.

 

File:
er8411_pppoe.txt.zipDownload
  0      
 
  0      
 
#1
Options
1 Accepted Solution
Re:ER8411 Loss of WAN connectivity after power outage-Solution
2024-06-12 03:48:06 - last edited 2024-06-12 03:48:15

Hi @Bianco8 

Thanks for posting in our business forum.

Bianco8 wrote

  @Clive_A 

Clive_A wrote

Hi @Bianco8 

Thanks for posting in our business forum.

I cannot say this is a problem with the router.

As long as the PPPoE is sent out, it is not up to the router to determine whether to connect or disconnect. It is your ISP to determine that.

Bianco8 wrote

  
SFP+ WAN1: PPPoE connection was disconnected by peer. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:38    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:33    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:50    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:08    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:22:26    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
 

 

This log should not be a problem with us at all as it is disconnected by the peer server. The router should just send the discovery and wait for the peer server to allow its connection.

Understanding PPPoE Interaction and PPPoE PADI Timeout


And, based on your log,

 

(Did you manually censor the MAC address?)

 

The same thing happened to the SFP+ WAN as well. I think it should be a line issue or the ISP. The only way to find this out is to Wireshark during the down time and see if the ISP actually sends the LCP response.

So far I cannot judge it to be a problem with the router.

Many thanks for your prompt reply.

I saw the peer disconnect message, which led me to think about ISP issues. 

I forgot to add some context: ER8411 is connected to a Nokia XGS-PON ONT 10G (provided by ISP).

After several hours of outage, I took a look on the setup and saw that the ONT lights were reporting nominal status. So I connected 2 different routers and PPPoE went up immediately. As soon as I plugged the ER8411, PPPOE was stuck down with such messages. I also physically rebooted the ONT but nothing changed on ER8411 end (ONT state was OK).

The only device I did not physically reboot was the ER8411 only. These details let me think of a software issue and not to ISP/line anomaly and let me confirm this:

 

Checking the debug logs downloaded from ER8411 (just before removing it from controller), syslog.log is reporting an additional message:

 

2024-06-10 12:11:56 pppoe_client[7484]: <5> 08214206 WAN4: PPPoE session is over
2024-06-10 12:11:56 pppoe_client[7484]: <5> 08214212 [OMADA]WAN4: PPPoE connection was disconnected by peer. (AC-MAC=MAC_ADDR, Session-ID=0x0548)
2024-06-10 12:11:56 ppp[7484]: <4> 02313107 [OMADA]<null>: PPPOE PPPOE_USERNAME failed to connect to the server because IPCP negotiation failed and local IPv4 addresses duplicate.
2024-06-10 12:11:55 ppp[7484]: <5> 02313103 [OMADA]WAN4: PPPOE_USERNAME authenticated.
2024-06-10 12:11:55 ppp[7484]: <5> 02313101 [OMADA]WAN4: LCP negotiation succeeded. (MTU=1492, AUTH=<null>)
2024-06-10 12:11:55 pppoe_client[7484]: <5> 08214211 [OMADA]WAN4: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0548)
2024-06-10 12:11:52 pppoe_client[6282]: <5> 08214202 WAN4: PPPoE began connecting automatically.

 

###

 

2024-06-10 12:24:48 pppoe_client[28082]: <5> 08214211 [OMADA]WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:45 pppoe_client[26839]: <5> 08214202 WAN1: PPPoE began connecting automatically.
2024-06-10 12:24:33 pppoe_client[24702]: <5> 08214206 WAN1: PPPoE session is over
2024-06-10 12:24:33 pppoe_client[24702]: <5> 08214212 [OMADA]WAN1: PPPoE connection was disconnected by peer. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:33 ppp[24702]: <4> 02313107 [OMADA]<null>: PPPOE PPPOE_USERNAME failed to connect to the server because IPCP negotiation failed and local IPv4 addresses duplicate.
2024-06-10 12:24:33 ppp[24702]: <5> 02313103 [OMADA]WAN1: PPPOE_USERNAME authenticated.
2024-06-10 12:24:32 ppp[24702]: <5> 02313101 [OMADA]WAN1: LCP negotiation succeeded. (MTU=1492, AUTH=<null>)
2024-06-10 12:24:32 pppoe_client[24702]: <5> 08214211 [OMADA]WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:30 pppoe_client[23724]: <5> 08214202 WAN1: PPPoE began connecting automatically.
2024-06-10 12:24:17 pppoe_client[16986]: <5> 08214206 WAN1: PPPoE session is over
2024-06-10 12:24:17 ppp[16986]: <4> 02313122 [OMADA]WAN1: LCP sending CONFIG-REQUEST timeout.

 

MAC_ADDR / PPPOE_USERNAME are censored by me for posting the comment here in the forum.

 

I have one more question, my ONT is the default gateway once the PPPOE connection is established, the default gateway IP is 192.168.100.1. I setup a MGMT VLAN weeks ago with VLAN ID 100 and the ER8411 has IP 192.168.100.254. This could cause conflicts being the ONT and another VLAN on the same subnet or it is correctly segregated/virtualized at Omada level? Otherwise, the system should prevent using the same IPs or provide an error message:

 

/**************************************************************************************************/
/*                                            vnete ifconfig                                           */
/**************************************************************************************************/
 
kern0.1   Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.1.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1458111 errors:0 dropped:310 overruns:0 frame:0
          TX packets:684744 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:109193077 (104.1 MiB)  TX bytes:56727782 (54.0 MiB)
kern0.100 Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.100.254  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2600 (2.5 KiB)
kern0.200 Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.200.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2562 (2.5 KiB)
kern0.99  Link encap:Ethernet  HWaddr XXX  
          inet addr:192.168.99.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2762 (2.6 KiB)
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:680 (680.0 B)  TX bytes:680 (680.0 B)
veth1     Link encap:Ethernet  HWaddr MAC_ADDR_USED_BY_ER8411_FOR_PPPOE  
          inet addr:169.254.11.22  Bcast:0.0.0.0  Mask:255.255.255.252 <--- Fallback?
          inet6 addr: fe80::IPV6_ADDR2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:682325 errors:0 dropped:0 overruns:0 frame:0
          TX packets:775696 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:56391692 (53.7 MiB)  TX bytes:61942817 (59.0 MiB)

 

I'll dig a bit on my available logs and let you know in case of further finding.

Oh. I thought you were using a public IP address on the WAN but well, it is not.

See the highlighted part, the IP was duplicated and that was the reason why the router rejected the connection.

 

And the system log indeed shows you that the IP was duplicated in your highlighted yellow parts.

Seems to be resolved now. You should avoid the WAN and LAN subnet overlapping.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update! ☛Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
Recommended Solution
  1  
  1  
#4
Options
7 Reply
Re:ER8411 Loss of WAN connectivity after power outage
2024-06-11 02:28:12

Hi @Bianco8 

Thanks for posting in our business forum.

I cannot say this is a problem with the router.

As long as the PPPoE is sent out, it is not up to the router to determine whether to connect or disconnect. It is your ISP to determine that.

Bianco8 wrote

  
SFP+ WAN1: PPPoE connection was disconnected by peer. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:38    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:33    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:50    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:08    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:22:26    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
 

 

This log should not be a problem with us at all as it is disconnected by the peer server. The router should just send the discovery and wait for the peer server to allow its connection.

Understanding PPPoE Interaction and PPPoE PADI Timeout


And, based on your log,

(Did you manually censor the MAC address?)

 

The same thing happened to the SFP+ WAN as well. I think it should be a line issue or the ISP. The only way to find this out is to Wireshark during the down time and see if the ISP actually sends the LCP response.

So far I cannot judge it to be a problem with the router.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update! ☛Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
  1  
  1  
#2
Options
Re:ER8411 Loss of WAN connectivity after power outage
2024-06-11 07:32:22

  @Clive_A 

Clive_A wrote

Hi @Bianco8 

Thanks for posting in our business forum.

I cannot say this is a problem with the router.

As long as the PPPoE is sent out, it is not up to the router to determine whether to connect or disconnect. It is your ISP to determine that.

Bianco8 wrote

  
SFP+ WAN1: PPPoE connection was disconnected by peer. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:38    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:33    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:50    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:08    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:22:26    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
 

 

This log should not be a problem with us at all as it is disconnected by the peer server. The router should just send the discovery and wait for the peer server to allow its connection.

Understanding PPPoE Interaction and PPPoE PADI Timeout


And, based on your log,

(Did you manually censor the MAC address?)

 

The same thing happened to the SFP+ WAN as well. I think it should be a line issue or the ISP. The only way to find this out is to Wireshark during the down time and see if the ISP actually sends the LCP response.

So far I cannot judge it to be a problem with the router.

Many thanks for your prompt reply.

I saw the peer disconnect message, which led me to think about ISP issues. 

I forgot to add some context: ER8411 is connected to a Nokia XGS-PON ONT 10G (provided by ISP).

After several hours of outage, I took a look on the setup and saw that the ONT lights were reporting nominal status. So I connected 2 different routers and PPPoE went up immediately. As soon as I plugged the ER8411, PPPOE was stuck down with such messages. I also physically rebooted the ONT but nothing changed on ER8411 end (ONT state was OK).

The only device I did not physically reboot was the ER8411 only. These details let me think of a software issue and not to ISP/line anomaly and let me confirm this:

 

Checking the debug logs downloaded from ER8411 (just before removing it from controller), syslog.log is reporting an additional message:

 

2024-06-10 12:11:56 pppoe_client[7484]: <5> 08214206 WAN4: PPPoE session is over
2024-06-10 12:11:56 pppoe_client[7484]: <5> 08214212 [OMADA]WAN4: PPPoE connection was disconnected by peer. (AC-MAC=MAC_ADDR, Session-ID=0x0548)
2024-06-10 12:11:56 ppp[7484]: <4> 02313107 [OMADA]<null>: PPPOE PPPOE_USERNAME failed to connect to the server because IPCP negotiation failed and local IPv4 addresses duplicate.
2024-06-10 12:11:55 ppp[7484]: <5> 02313103 [OMADA]WAN4: PPPOE_USERNAME authenticated.
2024-06-10 12:11:55 ppp[7484]: <5> 02313101 [OMADA]WAN4: LCP negotiation succeeded. (MTU=1492, AUTH=<null>)
2024-06-10 12:11:55 pppoe_client[7484]: <5> 08214211 [OMADA]WAN4: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0548)
2024-06-10 12:11:52 pppoe_client[6282]: <5> 08214202 WAN4: PPPoE began connecting automatically.

 

###

 

2024-06-10 12:24:48 pppoe_client[28082]: <5> 08214211 [OMADA]WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:45 pppoe_client[26839]: <5> 08214202 WAN1: PPPoE began connecting automatically.
2024-06-10 12:24:33 pppoe_client[24702]: <5> 08214206 WAN1: PPPoE session is over
2024-06-10 12:24:33 pppoe_client[24702]: <5> 08214212 [OMADA]WAN1: PPPoE connection was disconnected by peer. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:33 ppp[24702]: <4> 02313107 [OMADA]<null>: PPPOE PPPOE_USERNAME failed to connect to the server because IPCP negotiation failed and local IPv4 addresses duplicate.
2024-06-10 12:24:33 ppp[24702]: <5> 02313103 [OMADA]WAN1: PPPOE_USERNAME authenticated.
2024-06-10 12:24:32 ppp[24702]: <5> 02313101 [OMADA]WAN1: LCP negotiation succeeded. (MTU=1492, AUTH=<null>)
2024-06-10 12:24:32 pppoe_client[24702]: <5> 08214211 [OMADA]WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:30 pppoe_client[23724]: <5> 08214202 WAN1: PPPoE began connecting automatically.
2024-06-10 12:24:17 pppoe_client[16986]: <5> 08214206 WAN1: PPPoE session is over
2024-06-10 12:24:17 ppp[16986]: <4> 02313122 [OMADA]WAN1: LCP sending CONFIG-REQUEST timeout.

 

MAC_ADDR / PPPOE_USERNAME are censored by me for posting the comment here in the forum.

 

I have one more question, my ONT is the default gateway once the PPPOE connection is established, the default gateway IP is 192.168.100.1. I setup a MGMT VLAN weeks ago with VLAN ID 100 and the ER8411 has IP 192.168.100.254. This could cause conflicts being the ONT and another VLAN on the same subnet or it is correctly segregated/virtualized at Omada level? Otherwise, the system should prevent using the same IPs or provide an error message:

 

/**************************************************************************************************/
/*                                            vnete ifconfig                                           */
/**************************************************************************************************/
 
kern0.1   Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.1.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1458111 errors:0 dropped:310 overruns:0 frame:0
          TX packets:684744 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:109193077 (104.1 MiB)  TX bytes:56727782 (54.0 MiB)
kern0.100 Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.100.254  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2600 (2.5 KiB)
kern0.200 Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.200.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2562 (2.5 KiB)
kern0.99  Link encap:Ethernet  HWaddr XXX  
          inet addr:192.168.99.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2762 (2.6 KiB)
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:680 (680.0 B)  TX bytes:680 (680.0 B)
veth1     Link encap:Ethernet  HWaddr MAC_ADDR_USED_BY_ER8411_FOR_PPPOE  
          inet addr:169.254.11.22  Bcast:0.0.0.0  Mask:255.255.255.252 <--- Fallback?
          inet6 addr: fe80::IPV6_ADDR2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:682325 errors:0 dropped:0 overruns:0 frame:0
          TX packets:775696 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:56391692 (53.7 MiB)  TX bytes:61942817 (59.0 MiB)

 

I'll dig a bit on my available logs and let you know in case of further finding.

  1  
  1  
#3
Options
Re:ER8411 Loss of WAN connectivity after power outage-Solution
2024-06-12 03:48:06 - last edited 2024-06-12 03:48:15

Hi @Bianco8 

Thanks for posting in our business forum.

Bianco8 wrote

  @Clive_A 

Clive_A wrote

Hi @Bianco8 

Thanks for posting in our business forum.

I cannot say this is a problem with the router.

As long as the PPPoE is sent out, it is not up to the router to determine whether to connect or disconnect. It is your ISP to determine that.

Bianco8 wrote

  
SFP+ WAN1: PPPoE connection was disconnected by peer. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:38    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:24:33    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:50    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:23:08    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
Jun 10, 2024 08:22:26    

Gateway PPPoE Module Information    
SFP+ WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=08-b2-58-40-fa-98, Session-ID=0x00d0)
 

 

This log should not be a problem with us at all as it is disconnected by the peer server. The router should just send the discovery and wait for the peer server to allow its connection.

Understanding PPPoE Interaction and PPPoE PADI Timeout


And, based on your log,

 

(Did you manually censor the MAC address?)

 

The same thing happened to the SFP+ WAN as well. I think it should be a line issue or the ISP. The only way to find this out is to Wireshark during the down time and see if the ISP actually sends the LCP response.

So far I cannot judge it to be a problem with the router.

Many thanks for your prompt reply.

I saw the peer disconnect message, which led me to think about ISP issues. 

I forgot to add some context: ER8411 is connected to a Nokia XGS-PON ONT 10G (provided by ISP).

After several hours of outage, I took a look on the setup and saw that the ONT lights were reporting nominal status. So I connected 2 different routers and PPPoE went up immediately. As soon as I plugged the ER8411, PPPOE was stuck down with such messages. I also physically rebooted the ONT but nothing changed on ER8411 end (ONT state was OK).

The only device I did not physically reboot was the ER8411 only. These details let me think of a software issue and not to ISP/line anomaly and let me confirm this:

 

Checking the debug logs downloaded from ER8411 (just before removing it from controller), syslog.log is reporting an additional message:

 

2024-06-10 12:11:56 pppoe_client[7484]: <5> 08214206 WAN4: PPPoE session is over
2024-06-10 12:11:56 pppoe_client[7484]: <5> 08214212 [OMADA]WAN4: PPPoE connection was disconnected by peer. (AC-MAC=MAC_ADDR, Session-ID=0x0548)
2024-06-10 12:11:56 ppp[7484]: <4> 02313107 [OMADA]<null>: PPPOE PPPOE_USERNAME failed to connect to the server because IPCP negotiation failed and local IPv4 addresses duplicate.
2024-06-10 12:11:55 ppp[7484]: <5> 02313103 [OMADA]WAN4: PPPOE_USERNAME authenticated.
2024-06-10 12:11:55 ppp[7484]: <5> 02313101 [OMADA]WAN4: LCP negotiation succeeded. (MTU=1492, AUTH=<null>)
2024-06-10 12:11:55 pppoe_client[7484]: <5> 08214211 [OMADA]WAN4: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0548)
2024-06-10 12:11:52 pppoe_client[6282]: <5> 08214202 WAN4: PPPoE began connecting automatically.

 

###

 

2024-06-10 12:24:48 pppoe_client[28082]: <5> 08214211 [OMADA]WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:45 pppoe_client[26839]: <5> 08214202 WAN1: PPPoE began connecting automatically.
2024-06-10 12:24:33 pppoe_client[24702]: <5> 08214206 WAN1: PPPoE session is over
2024-06-10 12:24:33 pppoe_client[24702]: <5> 08214212 [OMADA]WAN1: PPPoE connection was disconnected by peer. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:33 ppp[24702]: <4> 02313107 [OMADA]<null>: PPPOE PPPOE_USERNAME failed to connect to the server because IPCP negotiation failed and local IPv4 addresses duplicate.
2024-06-10 12:24:33 ppp[24702]: <5> 02313103 [OMADA]WAN1: PPPOE_USERNAME authenticated.
2024-06-10 12:24:32 ppp[24702]: <5> 02313101 [OMADA]WAN1: LCP negotiation succeeded. (MTU=1492, AUTH=<null>)
2024-06-10 12:24:32 pppoe_client[24702]: <5> 08214211 [OMADA]WAN1: PPPoE Discovery phase negotiation succeeded. (AC-MAC=MAC_ADDR, Session-ID=0x0590)
2024-06-10 12:24:30 pppoe_client[23724]: <5> 08214202 WAN1: PPPoE began connecting automatically.
2024-06-10 12:24:17 pppoe_client[16986]: <5> 08214206 WAN1: PPPoE session is over
2024-06-10 12:24:17 ppp[16986]: <4> 02313122 [OMADA]WAN1: LCP sending CONFIG-REQUEST timeout.

 

MAC_ADDR / PPPOE_USERNAME are censored by me for posting the comment here in the forum.

 

I have one more question, my ONT is the default gateway once the PPPOE connection is established, the default gateway IP is 192.168.100.1. I setup a MGMT VLAN weeks ago with VLAN ID 100 and the ER8411 has IP 192.168.100.254. This could cause conflicts being the ONT and another VLAN on the same subnet or it is correctly segregated/virtualized at Omada level? Otherwise, the system should prevent using the same IPs or provide an error message:

 

/**************************************************************************************************/
/*                                            vnete ifconfig                                           */
/**************************************************************************************************/
 
kern0.1   Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.1.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1458111 errors:0 dropped:310 overruns:0 frame:0
          TX packets:684744 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:109193077 (104.1 MiB)  TX bytes:56727782 (54.0 MiB)
kern0.100 Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.100.254  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2600 (2.5 KiB)
kern0.200 Link encap:Ethernet  HWaddr MAC_ADDR1  
          inet addr:192.168.200.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2562 (2.5 KiB)
kern0.99  Link encap:Ethernet  HWaddr XXX  
          inet addr:192.168.99.1  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: IPV6_ADDR1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2762 (2.6 KiB)
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:680 (680.0 B)  TX bytes:680 (680.0 B)
veth1     Link encap:Ethernet  HWaddr MAC_ADDR_USED_BY_ER8411_FOR_PPPOE  
          inet addr:169.254.11.22  Bcast:0.0.0.0  Mask:255.255.255.252 <--- Fallback?
          inet6 addr: fe80::IPV6_ADDR2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:682325 errors:0 dropped:0 overruns:0 frame:0
          TX packets:775696 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:56391692 (53.7 MiB)  TX bytes:61942817 (59.0 MiB)

 

I'll dig a bit on my available logs and let you know in case of further finding.

Oh. I thought you were using a public IP address on the WAN but well, it is not.

See the highlighted part, the IP was duplicated and that was the reason why the router rejected the connection.

 

And the system log indeed shows you that the IP was duplicated in your highlighted yellow parts.

Seems to be resolved now. You should avoid the WAN and LAN subnet overlapping.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update! ☛Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
Recommended Solution
  1  
  1  
#4
Options
Re:ER8411 Loss of WAN connectivity after power outage
2024-06-12 07:07:35

  @Clive_A 

 

Hello Clive,

 

I have a public IPv4 address for my WAN, but I know for sure that the ONT is using a default 192.168.100.1 IP ( hack-gpon.[org]/xgs/ont-nokia-xs-010x-q/) .

This setup was present since couple of weeks, so I don't understand why the issue happened only when I lost PPPOE connectivity and mostly, why the issue disappeared following a remove/reset and re-adopting the ER8411 with the same configuration as before.

 

Thank you,

  0  
  0  
#5
Options
Re:ER8411 Loss of WAN connectivity after power outage
2024-06-12 07:22:15

Hi @Bianco8 

Thanks for posting in our business forum.

Bianco8 wrote

  @Clive_A 

 

Hello Clive,

 

I have a public IPv4 address for my WAN, but I know for sure that the ONT is using a default 192.168.100.1 IP ( hack-gpon.[org]/xgs/ont-nokia-xs-010x-q/) .

This setup was present since couple of weeks, so I don't understand why the issue happened only when I lost PPPOE connectivity and mostly, why the issue disappeared following a remove/reset and re-adopting the ER8411 with the same configuration as before.

 

Thank you,

Then try to reboot both modem and router at the same time and see if you experience the same issue. This simulates your power outage. Same thing and try again with the controller adopted the router.

As you said this was fine at first but later you created the VLAN which duplicates the IP like the log writes.

And dev read your log and told me that they think the same thing that the previous log shows connection denial, it should come from your ISP based on the log.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update! ☛Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
  1  
  1  
#6
Options
Re:ER8411 Loss of WAN connectivity after power outage
2024-06-12 07:58:44

  @Clive_A 

Thanks for your feedback Clive, I don't want to argue further and I have been misleading earlier: my internal/management VLAN 100 (192.168.100.0/24, which ER8411 had 192.168.100.254) was configured and UP since 2 weeks.

The power outage happened two days ago when the VLAN configuration was present already, and was still present after reset/re-adopt of the Omada. Please also note that I don't see any 192.168.100.0/24 IP address configured on my ER8411 WAN interfaces (currently UP with public IPv4, the line does never provide a NAT private IPv4 address).

 

I don't agree with the ISP PPPoE reset/timeout as PPPoE went up immediately on multiple devices when ER8411 was unable to bring it up. I also spoofed the WAN MAC to the MAC of another device which brought UP PPPoE session immediately.

 

I'm pretty sure that the issue would have fixed with an hardware reboot (i.e. unplugging both power supplies on the ER8411) would have recovered the situation, since the soft-reboot via Omada GUI did not restored the service. Otherwise, we're facing some weird software issue as the WAN ONT does not provide a 192.168.100.0/24 IP (no DHCP server) and I don't see why the ER8411 would get a fallback IP address due to duplicate IPs.

This is clear: the configuration was the same before and after the issue, the only restorative action was a forget/reset and then re-adopt of ER8411. 

I agree that the IP address duplicate might occur and should prevented (I moved my MGMT VLAN elsewhere so 100 is free) but:

 

  • veth1 / the VIF for PPPoE should be properly segregated as it's a virtual interface on WAN edge independently by IP subnets used on both edges, at least this is my expectation for a business grade product
  • Duplicate IPs/collision should have occurred as soon as I configured the VLAN100 in the first place since the default gateway for the WAN interface is fixed at 192.168.100.1. I I would expect an informative message since I'm configuring an IP already used (if not properly segregated).
  • In regards of this, the dump of the route table before reset the ER8411 shows:

default via 169.254.11.21 dev veth1 
169.254.11.20/30 dev veth1  proto kernel  scope link  src 169.254.11.22 
192.168.1.0/24 dev kern0.1  proto kernel  scope link  src 192.168.1.1 
192.168.99.0/24 dev kern0.99  proto kernel  scope link  src 192.168.99.1 
192.168.100.0/24 dev kern0.100  proto kernel  scope link  src 192.168.100.254 
192.168.200.0/24 dev kern0.200  proto kernel  scope link  src 192.168.200.1

 

Now, after removing my MGMT VLAN100 I get a similar output:

 

default via 169.254.11.21 dev veth1 
169.254.11.20/30 dev veth1  proto kernel  scope link  src 169.254.11.22 
192.168.1.0/24 dev kern0.1  proto kernel  scope link  src 192.168.1.1 
192.168.8.0/24 dev kern0.8  proto kernel  scope link  src 192.168.8.1 
192.168.99.0/24 dev kern0.99  proto kernel  scope link  src 192.168.99.1 
192.168.200.0/24 dev kern0.200  proto kernel  scope link  src 192.168.200.1

 

So I now see that 169.254.11.20/30 is not a fallback IP but the actual subnet used by ER8411 to communicate with the ONT.

This is confirmed via local ping:

 

root@deb2:~# ping 169.254.11.21
PING 169.254.11.21 (169.254.11.21) 56(84) bytes of data.
64 bytes from 169.254.11.21: icmp_seq=1 ttl=63 time=0.451 ms
64 bytes from 169.254.11.21: icmp_seq=2 ttl=63 time=0.500 ms
^C
--- 169.254.11.21 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1064ms
rtt min/avg/max/mdev = 0.451/0.475/0.500/0.024 ms
root@deb2:~# ping 169.254.11.22
PING 169.254.11.22 (169.254.11.22) 56(84) bytes of data.
64 bytes from 169.254.11.22: icmp_seq=1 ttl=64 time=0.402 ms
64 bytes from 169.254.11.22: icmp_seq=2 ttl=64 time=0.316 ms
^C

 

I don't see any reference on the route table for 192.168.100.1 (ONT IP) but I do confirm it is the peer IP used by PPPOE WAN:

PPPoE xxx connection on [WAN1] is established successfully. (Local IP:xxx.xxx.xxx.xxx, peer:192.168.100.1, DNS1:xxx, DNS2:xxx.)

 

Maybe since PPPOE was UP when I configured internal VLAN100, it did not changed the running config and then at the next boot it configured VLAN100 first and then later PPPOE resulting in 192.168.100.0/24 already in use? Despite I had no configuration for 192.168.100.1 specifically.

  1  
  1  
#7
Options
Re:ER8411 Loss of WAN connectivity after power outage
2024-06-12 08:04:07

 @Bianco8

VLAN 4094 and Invisible Routing Table 169.254.X.Y

I don't wanna argue over this either.

Best Regards! If you are new to the forum, please read: Howto - A Guide to Use Forum Effectively. Read Before You Post. Look for a model? Search your model NOW Official and Beta firmware. NEW features! Subscribe for the latest update! ☛Download Beta Here☚ ☛ ★ Configuration Guide ★ ☚ ☛ ★ Knowledge Base ★ ☚ ☛ ★ Troubleshooting ★ ☚ ● Be kind and nice. ● Stay on the topic. ● Post details. ● Search first. ● Please don't take it for granted. ● No email confidentiality should be violated. ● S/N, MAC, and your true public IP should be mosaiced.
  1  
  1  
#8
Options

Information

Helpful: 0

Views: 819

Replies: 7

Tags

WAN Setup Network Connectivity Highlighted Thread
Related Articles
ER8411 & DHCP WAN ?
515 0
Adding firewall between WAN and ER8411 - will this work?
1027 0
ER8411 - Dual WAN with faillover, primary WAN isn't prioritized after reconnection
2821 1
cannot get WAN on ER8411 to accept a DHCP response
505 0
ER8411 Cannot change PVID and Link Speeds of SFP Ports
751 0
ER8411 Pre-purchase questions.
401 0
About Us
Press
Copyright © TP-Link Systems Inc. All rights reserved.