massive problems with LACP

massive problems with LACP

massive problems with LACP
massive problems with LACP
a week ago
Model: TL-SX3008F  
Hardware Version: V1
Firmware Version: 1.27

Dear community,

 

maybe you can help find the problem here. I'm running out of ideas.
I have two SX3008F and want them to be connected via LACP on Port 7 and Port 8 with 10GB fibre.
I started connecting Port 7 on both switches and configured the LAG1 for Port 7 and 8. Switch1 as active and switch2 as passiv. Link speed is set to manual 10Gbps / full duplex on both switches and spanning tree is active with the default settings.

With only one cable connected, all is working fine.
As soon as I connect the second cable, I have massive timeouts between these two switches and of cource for all connected devices.

Please can someone help me?

Many thanks in advance.

Kind regards,
Joerg

  0      
  0      
#1
Options
10 Reply
Re:massive problems with LACP
Thursday

  @Jokie74 

Do you have the output of the LACP from CLI?

I assume the LACP is not effective. 

Disable any loop detection or STP.


The behavior you described is clearly a problem with the connection loop. The system somehow thinks the second cable is causing the loop. 

  0  
  0  
#2
Options
Re:massive problems with LACP
Thursday

  @Clive_A 
Thanks for getting into this. 
I don't have a CLI output, I used the Omada Controller Software to configure it.
Now I deactivated spanning tree and loopback detection and it even got worsier. With "spanning tree" I had 1 out of 4 ping losts, now I have 8 out of 10 lost.
How can I get a CLI output?

kind regards,
Joerg  

  0  
  0  
#3
Options
Re:massive problems with LACP
Thursday

This is the config of switch 1:

 

!SX3008F
#
vlan 1
 name "System-VLAN"
#
vlan 254
#
hostname "Wohnen"
location ""
contact-info ""
serial_port baud_rate 38400
ip dns-address primary 192.168.0.254
#
#
system-time ntp UTC+01:00 ntp 12    
system-time dst date Apr 6 02:00 2025 Oct 26 02:00 2025 60
#
#
sdm prefer omada
#
#
#
no protocol-vlan template 1

no protocol-vlan template 1

no protocol-vlan template 1
                                      
no protocol-vlan template 1

no protocol-vlan template 1

#
no dot1x handshake
#
user name jokie74 privilege admin secret 5 #########
telnet disable
no service reset-disable
#
#
no ip ssh server 
#
#
spanning-tree
spanning-tree mode rstp
#
no snmp-server
#
ip http server
#
no voice vlan oui 00:01:E3
no voice vlan oui 00:03:6B
no voice vlan oui 00:12:43
no voice vlan oui 00:0F:E2
no voice vlan oui 00:60:B9
no voice vlan oui 00:D0:1E
no voice vlan oui 00:E0:75
no voice vlan oui 00:E0:BB
no voice vlan oui 00:04:0D
no voice vlan oui 00:1B:4F
no voice vlan oui 00:04:13
#
lldp
#
ipv6 routing
#
profile ip id 482871780 ip 0.0.0.0/0
profile ipv6 id 1 ipv6 ::/0
profile network id 1204030299 vid 1 ip 192.168.0.1/24
profile network id 80780757 vid 254

#
access-list mode blacklist
#
port-channel load-balance src-dst-ip
loopback-detection
#
#
no boot autoinstall auto-save
no boot autoinstall auto-reboot
#
auto-voip
#
#
no controller cloud-based
cloud-firmware upgrade auto-check
interface port-channel 1
  description "LAG1"
  speed 10000
  switchport general allowed vlan 254 tagged
  no spanning-tree
#
interface vlan 1
  ip address 192.168.0.252 255.255.255.0 gateway 192.168.0.254
  no ipv6 enable
#
interface vlan 254
  ip address-alloc dhcp
  no ipv6 enable
#
interface ten-gigabitEthernet 1/0/1
  description "Port1"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/2
  description "Port2"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/3
  description "Port3"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/4
  description "Port4"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/5
  description "Port5"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/6
  description "Port6"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/7
  description "Port7"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
  channel-group 1 mode active
#
interface ten-gigabitEthernet 1/0/8
  description "Port8"
  switchport general allowed vlan 254 tagged
  no spanning-tree
  
  lldp med-status
  channel-group 1 mode active
  loopback-detection config process-mode port-based recovery-mode auto
  loopback-detection
#
end

  0  
  0  
#4
Options
Re:massive problems with LACP
Friday

  @Jokie74 

Jokie74 wrote

This is the config of switch 1:

 

!SX3008F
#
vlan 1
 name "System-VLAN"
#
vlan 254
#
hostname "Wohnen"
location ""
contact-info ""
serial_port baud_rate 38400
ip dns-address primary 192.168.0.254
#
#
system-time ntp UTC+01:00 ntp 12    
system-time dst date Apr 6 02:00 2025 Oct 26 02:00 2025 60
#
#
sdm prefer omada
#
#
#
no protocol-vlan template 1

no protocol-vlan template 1

no protocol-vlan template 1
                                      
no protocol-vlan template 1

no protocol-vlan template 1

#
no dot1x handshake
#
user name jokie74 privilege admin secret 5 #########
telnet disable
no service reset-disable
#
#
no ip ssh server 
#
#
spanning-tree
spanning-tree mode rstp
#
no snmp-server
#
ip http server
#
no voice vlan oui 00:01:E3
no voice vlan oui 00:03:6B
no voice vlan oui 00:12:43
no voice vlan oui 00:0F:E2
no voice vlan oui 00:60:B9
no voice vlan oui 00:D0:1E
no voice vlan oui 00:E0:75
no voice vlan oui 00:E0:BB
no voice vlan oui 00:04:0D
no voice vlan oui 00:1B:4F
no voice vlan oui 00:04:13
#
lldp
#
ipv6 routing
#
profile ip id 482871780 ip 0.0.0.0/0
profile ipv6 id 1 ipv6 ::/0
profile network id 1204030299 vid 1 ip 192.168.0.1/24
profile network id 80780757 vid 254

#
access-list mode blacklist
#
port-channel load-balance src-dst-ip
loopback-detection
#
#
no boot autoinstall auto-save
no boot autoinstall auto-reboot
#
auto-voip
#
#
no controller cloud-based
cloud-firmware upgrade auto-check
interface port-channel 1
  description "LAG1"
  speed 10000
  switchport general allowed vlan 254 tagged
  no spanning-tree
#
interface vlan 1
  ip address 192.168.0.252 255.255.255.0 gateway 192.168.0.254
  no ipv6 enable
#
interface vlan 254
  ip address-alloc dhcp
  no ipv6 enable
#
interface ten-gigabitEthernet 1/0/1
  description "Port1"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/2
  description "Port2"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/3
  description "Port3"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/4
  description "Port4"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/5
  description "Port5"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/6
  description "Port6"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
#
interface ten-gigabitEthernet 1/0/7
  description "Port7"
  switchport general allowed vlan 254 tagged
  spanning-tree
  
  lldp med-status
  channel-group 1 mode active
#
interface ten-gigabitEthernet 1/0/8
  description "Port8"
  switchport general allowed vlan 254 tagged
  no spanning-tree
  
  lldp med-status
  channel-group 1 mode active
  loopback-detection config process-mode port-based recovery-mode auto
  loopback-detection
#
end

Try this command: 

show etherchannel group_id detail

 

  0  
  0  
#5
Options
Re:massive problems with LACP
Saturday - last edited Saturday

  @Clive_A 

sorry for my late reply. I was a bit busy last days. I unplugged port 7 because of the instability.
Here is the output as you requested:

switch1# show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.

 

Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SA      Down      32768         0x1       0       0x7     0x45

Te1/0/8   SA      Up        32768         0x1       0xb3    0x8     0x3d

 

Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SP      0            0000.0000.0000       0       0x0     0

Te1/0/8   SP      32768        d844.89ab.118f       0xb1    0x8     0x3c

 

switch2# show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.


Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SP      Down      32768         0x1       0       0x7     0x44

Te1/0/8   SP      Up        32768         0x1       0xb1    0x8     0x3c


Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SP      0            0000.0000.0000       0       0x0     0

Te1/0/8   SA      32768        d844.89ab.117d       0xb3    0x8     0x3d


I hope you can read it, it is difficult to post this, because the format is deleted here.

kind regards,
Joerg

  0  
  0  
#6
Options
Re:massive problems with LACP
Monday

  @Jokie74 

Jokie74 wrote

  @Clive_A 

sorry for my late reply. I was a bit busy last days. I unplugged port 7 because of the instability.
Here is the output as you requested:

switch1# show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.

 

Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SA      Down      32768         0x1       0       0x7     0x45

Te1/0/8   SA      Up        32768         0x1       0xb3    0x8     0x3d

 

Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SP      0            0000.0000.0000       0       0x0     0

Te1/0/8   SP      32768        d844.89ab.118f       0xb1    0x8     0x3c

 

switch2# show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.


Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SP      Down      32768         0x1       0       0x7     0x44

Te1/0/8   SP      Up        32768         0x1       0xb1    0x8     0x3c


Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SP      0            0000.0000.0000       0       0x0     0

Te1/0/8   SA      32768        d844.89ab.117d       0xb3    0x8     0x3d


I hope you can read it, it is difficult to post this, because the format is deleted here.

kind regards,
Joerg

I want to see the output when #7 was connected. 

The hex shows the connection is down. 

 

Is it possible that you have a loop somewhere else on the two switches? 

The full diagram, did you examine it?

 

I think you can remove all other cables and leave the two switches, connect them, and test if the LACP works or not.

Wireshark would be great for the analysis, and it is LACP. You can see the abnormality from the packets. 

  1  
  1  
#7
Options
Re:massive problems with LACP
Monday

Dear  @Clive_A ,

I'm not able to upload a network diagramm here, so I will post the code for mermaidchart:

graph TD

    subgraph Router

        Internet["🌐<br>Internet"]

    end

 

    subgraph "Switch 1 | Clients & Uplink"

        Router["Router<br><b>2.5G RJ45</b>"]

        Switch1["<b>Switch 1</b><br>TP-Link SX3008F"]

        AP["WIFI 7 AP<br><b>2.5G RJ45</b>"]

        TV["TV<br><b>1G RJ45</b>"]

        PC1["PC 1<br><b>2.5G RJ45</b>"]

        PC2["PC 2<br><b>2.5G RJ45</b>"]

    end

 

    subgraph "Switch 2 | Server & Storage"

        Switch2["<b>Switch 2</b><br>TP-Link SX3008F"]

        NAS["NAS<br><b>10G SFP+</b>"]

        Proxmox["Proxmox Server<br>(5x Virtuell Hosts)<br><b>10G SFP+</b>"]

    end

 

    Internet --- Router

    Router -- "2.5G CAT6" --> Switch1

    Switch1 -- "2.5G CAT6" --> AP

    Switch1 -- "1G CAT6" --> TV

    Switch1 -- "2.5G CAT6" --> PC1

    Switch1 -- "2.5G CAT6" --> PC2

 

    Switch1 <== "<b>20G LACP Backbone</b><br>(2x 10G Fiber)" ==> Switch2

 

    Switch2 -- "10G Fiber" --> NAS

    Switch2 -- "10G Fiber" --> Proxmox

 

    classDef switch fill:#f9f,stroke:#333,stroke-width:2px;

    class Switch1,Switch2 switch

I will plug in port seven again when I'm home and send you the output of the command again.

Many thanks for your help.

kind regards,
Joerg

  0  
  0  
#8
Options
Re:massive problems with LACP
Monday

Hello  @Clive_A 

 

Here is the output of the command with port 7 plugged in:

 

Keller#show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

                -------------------

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.

 

Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SP      Up        32768         0x1       0xb1    0x7     0x4

Te1/0/8   SP      Up        32768         0x1       0xb1    0x8     0x3c

 

Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SA      32768        d844.89ab.117d       0xb3    0x7     0xc5

Te1/0/8   SA      32768        d844.89ab.117d       0xb3    0x8     0x3d

 

 

Wohnen#show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

                -------------------

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.

 

Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SA      Up        32768         0x1       0xb3    0x7     0x5

Te1/0/8   SA      Up        32768         0x1       0xb3    0x8     0x3d

 

Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SP      32768        d844.89ab.118f       0xb1    0x7     0xc

Te1/0/8   SP      32768        d844.89ab.118f       0xb1    0x8     0x3c

 

Kind regards,
Joerg

  0  
  0  
#9
Options
Re:massive problems with LACP
Tuesday

  @Jokie74 

Jokie74 wrote

Hello  @Clive_A 

 

Here is the output of the command with port 7 plugged in:

 

Keller#show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

                -------------------

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.

 

Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SP      Up        32768         0x1       0xb1    0x7     0x4

Te1/0/8   SP      Up        32768         0x1       0xb1    0x8     0x3c

 

Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SA      32768        d844.89ab.117d       0xb3    0x7     0xc5

Te1/0/8   SA      32768        d844.89ab.117d       0xb3    0x8     0x3d

 

 

Wohnen#show etherchannel 1 detail

Group state = L2

Ports: 2  MaxPorts = 16

Protocol:   LACP

                Ports in the group:

                -------------------

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.

        A - Device is in active mode.        P - Device is in passive mode.

 

Local information:

                            LACP port     Admin     Oper    Port    Port

Port      Flags   State     Priority      Key       Key     Number  State

Te1/0/7   SA      Up        32768         0x1       0xb3    0x7     0x5

Te1/0/8   SA      Up        32768         0x1       0xb3    0x8     0x3d

 

Partner's information:

                  LACP port                         Oper    Port    Port

Port      Flags   Priority     Dev ID               Key     Number  State

Te1/0/7   SP      32768        d844.89ab.118f       0xb1    0x7     0xc

Te1/0/8   SP      32768        d844.89ab.118f       0xb1    0x8     0x3c

 

Kind regards,
Joerg

Port 7 is not linked. The output hex is showing it is not connected. It shows up but it is not linked. I cannot tell anything else without Wireshark to see how LACP sends the packets. 

 

What module and fiber do you use? 

I mentioned other steps in the previous reply. Did you follow them? 

Is your firmware 1.20.7 on both switches? 

You have turned off the STP and checked your network diagram, right? 

What if you set the link speed to auto, is there a difference? 

Do you use SRC DST MAC and IP as the hash? What does the hash part look like?

  0  
  0  
#10
Options
Re:massive problems with LACP
19 hours ago - last edited 19 hours ago

Hello @Clive_A .
Yes, I tried the settings you posted a few days ago, but it got worse.
There are two sfp+ 10Gbe Modules in each switch connected via 20 Meter FTTH fibre cables.
Each cable alone is working fine without any problems at all. So, I don't know if this still could be the source of my troubles.
Firmware on both switches is 1.20.7, but the problems also existed in older firmware levels.
I reached out to TP-Link support a few month ago, but even they wouldn't really get into it.

 

I will set the LAG to 10 Gbe / full duplex and test again.

For the hash algorythm I tried all options without any difference.
Many thanks to you.

kind regards,
Joerg

  0  
  0  
#11
Options