Issues with flapping SFP config

Routers
,
1

I have noticed some flapping in the SFP mode configuration. It has been quiet for some time and then for whatever reason starts to flap frequently, then it may be quiet again. This is a status output from my custom log tracker:

==============================
 route_swd_logmon STATUS
==============================
State: RUNNING (pid 342)
Total flaps: 4431
Last 20 events:
Dec 14 18:24:57.955 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to 10gbase_r
Dec 14 18:25:16.409 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_fiber
Dec 14 18:25:32.077 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_baset
Dec 14 18:25:52.751 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_plus
Dec 14 18:26:06.412 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to 10gbase_r
Dec 14 18:26:26.912 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_fiber
Dec 14 18:26:37.573 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_baset
Dec 14 18:26:54.236 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_plus
Dec 14 18:27:10.900 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to 10gbase_r
Dec 14 18:27:26.347 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_fiber
Dec 14 18:27:43.014 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_baset
Dec 14 18:27:55.673 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_plus
Dec 14 18:28:08.335 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to 10gbase_r
Dec 14 18:28:22.778 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_fiber
Dec 14 18:28:38.438 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_baset
Dec 14 18:28:53.103 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_plus
Dec 14 18:29:10.764 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to 10gbase_r
Dec 14 18:29:22.212 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_fiber
Dec 14 18:29:39.877 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_baset
Dec 14 18:29:51.543 01-Route10-Rack daemon.alert route-swd: sfp0: set mode to sgmii_plus==============================

What kind of issue is this? Is it an SFP transceiver on the verge of failing, bad connection with the ISP, issue with the SFP in the Route10, firmware issue, or something else?

2

Do you have the transceiver set to the respective mode it’s operating on or auto?

3

It’s set to Default, via the Route10 SFP port, which should default to Auto (?!).

I also tried forcing it to 1Gbps (it is an SFP transceiver for a 1Gbps ISP plan), but then lost all connection to the internet and to the network devices from the cloud controller. I managed to reconfigure via UI+CLI to make it load the Default/Auto setting again. Then it was quiet for quite some time, and then started flapping one or two days ago again.

4

What is the brand/model transceiver that you’re using?

5

Skylane, 1310/1550, 20km, 100/1000Mb, SFP.

It used to go into the stock media converter, the Inteno XG6846, provided by the fiber installer. Now connected to the Route10 directly.

6

Ok. Too many variables here.

Firstly, is this the same transceiver your ISP provided you?

Second, assuming the answer is yes to the above, does your ISP allow for this methodology or concept of bypassing their gateway (Inteno XG6846) ?

Third, assuming the answer is yes to the above, when transitioning over to the Route10, I’d imagine you unseated the fiber and transceiver independently from the ISP gateway device. Was the same fiber used to reconnect, or did you replace with a longer fiber to extend to the route10?

Let’s start there.

7

Yes, the very same transceiver.

No idea, I didn’t think that was a thing…

So, I replaced the fiber cable, with another longer version with the same specification. The original and the replacement both have pre-terminated connectors (in both ends)

I have checked the signal strength, and got perfect result.

8

I noticed the fiber module you are using is rated for 20km? Is this a 20km fiber run or, in a home? If it is in a home, the link may be dropping to too hot of a laser signal (laser overload).

9

This is an FTTx application. It’s likely a bi-di sfp with 20k reach from the premise to the OLT at the central office.

10

If there is no link, and the port is configured with defaults, the firmware will cycle through 1 Gbps and 10 Gbps modes (on random intervals, since transceiver firmwares will also randomly try different speeds) until it obtains a link. If you force 1 Gbps / 10 Gbps it will stick to the desired mode.

11

So, actually, the link is up and running, but it cycles through the configurations anyway.

Setting to what I thought was the natural choice, 1Gbps, broke the connection/link, altogether.

And, I guess, 10G should not be considered either as the tranceiver is rated 100/1000 mbps.

12

What does ethtool on the port say when it is cycling?

13

Oh, my, oh, my. I may have had a major brain meltdown here. :face_with_peeking_eye:

I almost forgot I have another transceiver in the left SFP port, which is not used and I plugged it with a silicon lid. It may not be a wild guess that it is this one that is repatedly cyling it’s configuration. I got the impression that it started quite a while after rebooting the device (hours or even a day later), so I didn’t connect the dots. Also, got lost in the numbering direction for Lx, ethX, etc.

I will remove the unused transceiver and monitor the logs.

2 Likes