10G fiber link change to 1G after PC reboot
I have recently found out that the 10G link on my PC is stuck on 1G speed after reboot. I am using Mellanox Connectx-3 Pro NIC on both my NAS and PC. The problem only seems to to happen to my windows PC.
The configuration on the switch seem to be set to the correct speed:
However, you can see the speed also shows 1G on the switch side as well:
I will only be able to recoery the error state by reboot the switch itself. I tried disable/enable the port on both side, and it does nothing. I tried the following which also does not help:
- remove LAG
- remove VLAN
- disable QoS on the my NIC
- diable flow control
after reboot the switch this is what I can see:
The switch is also showing the right speed.
I alwasy thought that fiber is single speed, and you generally will not be able to change it on the software, hence is it quite odd I start to see these issues recently.
- Copy Link
- Subscribe
- Bookmark
- Report Inappropriate Content
Hi @AlanHell
Thanks for posting in our business forum.
So I cannot rule out if this is a problem with your NIC. If this does not happen to any other SFP+, it might be the NIC compatibility issue.
Besides, it is on PCIE. I am not sure if that's related.
These parameters you disabled do not relate to the link speed/negotiation.
You probably should check if there is a new firmware to the NIC you have. So it cannot rule out the software issue on the NIC.
- Copy Link
- Report Inappropriate Content
@Clive_A That was my initial though. However, I have the exact same NIC on my Linux machine and NAS, none of them has the same issue. Both of them are using PCI-e slot on the mother board. The only difference is the OS, the working one has Linux base system. I have tried to update the firmwre on the NIC to the neweset one, and it does not resolve the issue either.
The reason I suspect the switch might has something to do with it is because reseting it fix it. It seems that whenever I reboot my PC, the swtich somehow negotiated a different speed for the same link. I have to reboot the switch to reinitialized the link back to 10G. It is very strange as I never see fiber link running on different speed before.
- Copy Link
- Report Inappropriate Content
Hi @AlanHell
Thanks for posting in our business forum.
AlanHell wrote
@Clive_A That was my initial though. However, I have the exact same NIC on my Linux machine and NAS, none of them has the same issue. Both of them are using PCI-e slot on the mother board. The only difference is the OS, the working one has Linux base system. I have tried to update the firmwre on the NIC to the neweset one, and it does not resolve the issue either.
The reason I suspect the switch might has something to do with it is because reseting it fix it. It seems that whenever I reboot my PC, the swtich somehow negotiated a different speed for the same link. I have to reboot the switch to reinitialized the link back to 10G. It is very strange as I never see fiber link running on different speed before.
Huh, you mean this never happened before with TP-Link switch until you tried a Linux machine on the switch?
Have you tried a different module? It really seems to be an issue with the compatibility.
If you force the port to be 10Gbps, will it work?
- Copy Link
- Report Inappropriate Content
@Clive_A On the contratry, I never have issuse with LInux and TP-Link. It is when I try to use Windows 10, it stars to have issues. And the problem only happen when I reboot the PC side, and will only go away if I then reboot the switch.
I cannot force the speed to 10G as it is an fiber link, TP-Link does not allow you to set your own speed, and my NIC on my PC side also does not have speed setting avalible either.
- Copy Link
- Report Inappropriate Content
Hi @AlanHell
Thanks for posting in our business forum.
AlanHell wrote
@Clive_A On the contratry, I never have issuse with LInux and TP-Link. It is when I try to use Windows 10, it stars to have issues. And the problem only happen when I reboot the PC side, and will only go away if I then reboot the switch.
I cannot force the speed to 10G as it is an fiber link, TP-Link does not allow you to set your own speed, and my NIC on my PC side also does not have speed setting avalible either.
I think Windows allows you to force it. Did you do a research on it?
Moreover, you can do a simple test to verify if it is an issue with the NIC(PCIE).
So, disconnect the fiber from your PC. Boot it up and it enters the desktop, connect the fiber, and check if the link is auto to 10Gbps.
If yes, that means there is no problem with the PC when it stabilizes. It could be a problem with the bootup process and PCIE. Because when your PC and motherboard(PCIE) fully function, it stays 10Gbps. It only falls to 1Gbps when you connect them together and PC boots up. So it might be what I say in the previous sentence.
Or your PC boots up, it is stuck at 1Gbps, you reconnect the fiber, not reboot the switch, it can auto to 10Gbps, then it still points to your PCIE bootup.
BTW, what's the PCIE gen used by your NIC? Motherboard name? Which slot do you use your NIC? It seems to be 4 lanes?
- Copy Link
- Report Inappropriate Content
@Clive_A Well, as I said, it is an Fiber SPF+ transceiver, so auto negotiation will not work (I think it is an L1 protocol that only works for copper link). This means I will not be able to force speed, the speed setup will only show up if I plug in a copper transceiver.
'
As for the link down test. I have try to unplug the fiber cable during while reboot my PC, and only plug it in when my windows is fully up. The link still stuck at 1Gbps. It seems that only when I reboot the switch, the speed will be back to normal. I have matched SFPs on all my 10G slots on switch and across all my PCs. Only this Windows 10 PC is given me trouble. And I only see this after update the firmware of the switch recently. However, I also upgrade my WIndows 10 PC recently as well. Hence I am not sure which one is at fault right now.
I used a second PCI-e x 16 slot on my mother board. I believe it is gen 3. My motherboard is MSI MEG Z390. I think I should have enough PCI-e Lanes for this NIC. In the end, I am able to run 10G on both port just fine after reseting my swtich.
- Copy Link
- Report Inappropriate Content
Hi @AlanHell
Thanks for posting in our business forum.
AlanHell wrote
@Clive_A Well, as I said, it is an Fiber SPF+ transceiver, so auto negotiation will not work (I think it is an L1 protocol that only works for copper link). This means I will not be able to force speed, the speed setup will only show up if I plug in a copper transceiver.
'
As for the link down test. I have try to unplug the fiber cable during while reboot my PC, and only plug it in when my windows is fully up. The link still stuck at 1Gbps. It seems that only when I reboot the switch, the speed will be back to normal. I have matched SFPs on all my 10G slots on switch and across all my PCs. Only this Windows 10 PC is given me trouble. And I only see this after update the firmware of the switch recently. However, I also upgrade my WIndows 10 PC recently as well. Hence I am not sure which one is at fault right now.
I used a second PCI-e x 16 slot on my mother board. I believe it is gen 3. My motherboard is MSI MEG Z390. I think I should have enough PCI-e Lanes for this NIC. In the end, I am able to run 10G on both port just fine after reseting my swtich.
You mean a complete reset and fixed it and there is no issue with it at all now? Is resolved?
- Copy Link
- Report Inappropriate Content
No, I mean, everytime when I reboot my PC, my PC will be stuck in 1G. To fix it, I will need to reboot my switch to get it back on 10G.
- Copy Link
- Report Inappropriate Content
Information
Helpful: 0
Views: 595
Replies: 8
Voters 0
No one has voted for it yet.