TL-SX3016F blocks traffic

TL-SX3016F blocks traffic

TL-SX3016F blocks traffic
TL-SX3016F blocks traffic
Monday
Model: TL-SX3016F  
Hardware Version: V1
Firmware Version: 1.20.2 Build 20240807 Rel.64539

Hello,

 

I had an issue with a TL-SX3016F.

This is my schema.

1 x TL-SX3016F

2 x Mikrotik CCR1036-8G-2S+ connected to TL-SX3016F via 10 Gbps DAC cable

First CCR1036-8G-2S+ is the main and run routerOs 6.48.6. Second CCR1036-8G-2S+ is a neighbor and run routerOs 6.49.18

All devices are installed in air-conditioned datacenter.

To the TL-SX3016F are connected several fiber links, both 10 Gbps and 1 Gbps, to other Mikrotik routers in other sites. All those Mikrotik routers had routerOs 6.48.6.

All Mikrotik router run OSPF.
 

System was up since 300 days and suddenly, on 25th August, OSPF went down on all routers at 10.30AM for 5 minutes and then another time at 12.00AM for 5 minutes.

I rebooted main CCR1036-8G-2S+, thinking about an OSPF problem, and I added some static backup routes on routers.

The same issue appears twice on 27h August at he same time (more o less) and traffic didn't pass even with the static routes.

I upgraded main CCR1036-8G-2S+ to routerOs 6.49.18 thinking about some bug or firmware mismatch.

On 28th August early morning I also upgraded all other neighbor Mikrotik routers to 6.49.18 to have all OSPF routers with same firmware.

On 28th August night (about 10.30 PM) OSPF and traffic goes down.

During the night, on 29th August at 1.00 AM I rebooted TL-SX3016F and since that time I had no more problems. Now I'm still monitoring system.

 

I honestly don't think it's a routerOs problem because even with the upgrade the issue occured.

I also don't think it's a problem of neighbor routers because all OSPF went down at the same time.

Maybe a main CCR1036-8G-2S+ issue? It could be but:

- a single reboot didn't help => issue remains

- a routerOs upgrade didn't help => issue remains

- a switch reboot seems help...

 

I add another point.

One year ago I had similar issue on the same system (with less neighbor routers but same schema).

Suddenly traffic stops passing many time in few days. Thinking about a switch problem I changed the switch and everything worked good until now.

Old switch was TL-SX3016F bought together with that now installed but with previous firmware version 1.0.

 

Due to this I thought about an hardware o firmware problem occurred with the old switch but now, one year after, with a similar problem I think that on both cases something happened to both switches. I don't know if firmware issue or if something happens after many days of uptime or other.

 

I hope you can help me.

 

Thanks

 

Best regards

 

Andrea

  0      
  0      
#1
Options
5 Reply
Re:TL-SX3016F blocks traffic
Tuesday

  @AR79 

Based on the description, I don't think this is a problem with the switch. 
At least, no obvious evidence or details indicating this.

  1  
  1  
#2
Options
Re:TL-SX3016F blocks traffic
Tuesday
Maybe there is no evidence that is a switch issue but a problem with switch could explain what happened. I need help to confirm or deny the hypothesis.
  0  
  0  
#3
Options
Re:TL-SX3016F blocks traffic
Wednesday

  @AR79 

AR79 wrote

Maybe there is no evidence that is a switch issue but a problem with switch could explain what happened. I need help to confirm or deny the hypothesis.

The switch doesn't support OSPF. So, the OSPF ends at the router side when it connects to the switch. 

I am not sure if that's the reason, but if you say that it had been working fine for over 300 days, and it suddenly died, somehow. I think that's a different story. 

 

It's hard to judge without Wireshark or some kind of system log from the router side. 

Or you can also read the logs from the switch if anything indicates some kind of block on the switch. e.g. STP? 

  0  
  0  
#4
Options
Re:TL-SX3016F blocks traffic
Wednesday - last edited Wednesday

I don't want enable OSPF on switch, just switch let pass OSPF.

 

In router log I saw only that OSPF went down and then up so it isn't possible understand if there was an internal issue (not reported in logs) or if switch blocked traffic.

The result would be the same.

I'm trying to go deeper in router analysis to find more logs and see what happens when OSPF goes down.

Besides logs, in traffic graphs, I saw traffic peaks on router ports when issue occurred.

I thought a loop but there shouldn't be any loop because switch logically separates port with different VLAN.

Unless something happens to the switch.

 

On switch sides there were no logs when issue occurred.

STP is never been enabled.

  0  
  0  
#5
Options
Re:TL-SX3016F blocks traffic
Yesterday

Is there a newer firmware versione than 1.20.2 Build 20240807 Rel.64539 ?

If yes, where could I download it?

  0  
  0  
#6
Options