I’ve tried a few DAC cables to connect my Route 10 to a Mikrotik CRS309-1G-8S+ switch.
The Mikrotik doesn’t to see any “Link Partner Advertising” rates/speeds advertised to the SFP interface and therefore doesn’t create any link, Auto and 10 BaseCR dont work, but forcing to “1G baseT full” does show a link up time. but only at 1Gbps so not useful…
I can see the Route 10 correctly identifies the DAC cables, one example below, FS.com cables also detected fine with model info shown etc, but no link is established.
e.g: SFP Make: OEM OUI: 004020 Model: SFP-H10GB-CU2MRev: 03 Serial: XXXXXXXXX Date: 190712
Cables are known working as were working before the were unplugged from a Mikrotik CRS device, and connected to the Route 10 SFP port.
As a follow up, I’m also unclear on how to enable Jumbo frames on just the SPF port where my 10 Gbit network will be running.
Any thoughts or next steps to test?
Running of cloud controller and latest firmware on both.
Per above, the other side of the SPF DAC cable is a Mikrotik CRS309-1G-8S+ switch.
Happy to share via PM any diagnostic data, would prefer to not invite for now.
Route10 WAN1 - ISP via ethernet
Route 10 LAN 3 (non-poe port) connected to POE 24 port switch and rest of devices working fine.
Route 10 SFP (LAN4) → CRS309-1G-8S+ → x devices connected to 10Gb switch, but doesn’t work…
WAN is on WAN1, and is running DHCP with VLAN10
Default IP range is 192.168.2.0/24 (LAN3 uses this)
I have the 10Gbit running on 10.10.0.0/16. - trying to run on the SFPs (tried both same result).
I’ve had to put the 10Gbit on a VLAN (100) as can’t seem to give specific interfaces an IP range.
I have a 200 VLAN for wifi kids network (can ignore for now, but exains why 200 vlan is in the below)
I have tried using WAN2 port, changing from “WAN” to “Standard” mode, in case it was a faulty SFP
Yes I can reboot either side and the 10Gbps link will come back up.
I can also set the Route10 SFP interface to Auto and get a link at 10Gbps now.
On the Mikrotik switch, it still doesn’t seem to see the Route10 is “Link Partner Advertising”, but does seem to work now
Wondering if the DAC cables advertise a mode that Route10 doesn’t know what to do with or handle ‘10G BaseCR’? vs the Optics which advertise ‘10G baseSR LR’
Are you familiar with how to pull up the terminal within the controller? From the Network tab, Shift+Click on the Route10’s name.
From there, issue ethtool -I eth5 (that’s a capital i). Copy and paste the output here, please.
Of course, the DAC must support the same negotiations as the Route10 or you would never get link. At least the command will tell you what the Route10 as well as the other side of the link’s options are (from the Route10’s perspective).
As requested @Alta-Matt_v2
Looks to be advertising baseT? Instead of other options for an SFP?
I read elsewhere someone when using a SFP module with RJ45 instead of Fibre or DAC had no issues, wonder if this is part of the reason?
So maybe need an update to support more modes?
~ # ethtool -I eth5
Settings for eth5:
Supported ports: [ FIBRE ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: MII
PHYAD: 31
Transceiver: external
Auto-negotiation: off
Link detected: yes
Did you also want the same output when a DAC is installed?
And now with a 1Gbps RJ45 SFP and Cat6 cable:
‘Auto’ syncs to 1Gbps, forcing either 1Gbps or 5GBps do create a link, both only syncs at 1Gbps (per below).
Link detected: yes
root@Route10:~# ethtool -I eth4
Settings for eth4:
Supported ports: [ FIBRE ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 30
Transceiver: external
Auto-negotiation: off
Link detected: yes
Really starting to look like there is some hardcoding of the “Supported” and “Advertised” link modes, instead of querying? what the SFP or DAC actually supports to using those.
Be good to get this fixed or keen to see if others using the SFP ports are getting different results.
A lot to go through, but from ethtool, the main things we’re looking for are that both the supported link modes and the advertised link modes include the desired speed. We’re also looking to make sure the Transceiver field says “external”.
I can confirm the Route10 supports 10GBASE-SR. If it didn’t I wouldn’t be able to send you this message lol, I’m actively using it.
Based on your output with the DAC installed, it looks like it simply isn’t supported.
There’s another option to force 10gbps via terminal. I can do it or provide some instruction on how to do it, your choice.
This one is a little perplexing. You used a 1gbps SFP, so I would expect it to max out at 1gbps.
Thanks.
Happy to try the commands if you can share them with me and will be useful to confirm compatibility issues with the DAC cables or something else.
My curiosity with the 1Gbps SFP was that 1,2.5,5 & 10 Gbps are all reported as “Supported and Advertised” when the SFP simply can’t do 2.5,5 or 10 Gbps., so jumped to conclusion all 4 are just always being pumped out as “Supported and Advertised”
And agree it supports and works with 10GBASE-SR as thats the optics I used to finally get the 10Gbps uplink, but was wondering if that was ‘luck’ more than anything else given ehat ethtools reports.
First, I really want to emphasize when messing around with hard set link speeds, there’s always the possibility that link may not return. If that’s the case, accessibility may be cut off, so it’s a good idea to have a backup plan. In this case, if we down the SFP link, you should be able to plug in to one of the Ethernet ports and undo the changes.
Now, SSH in to the Route10 vi /cfg/sfpX.txt where X is 0 for LAN4, and 1 for WAN2
Put 10gbase_r in that file. If you’re unfamiliar with vim, you’ll need to hit the letter i to edit. To save, do Esc followed by :wq followed by Enter/Return
Then reboot
Plug in the DAC and let’s see if it links up. If it’s still unsuccessful, SSH back into the Route10, rm /cfg/sfpX.txt again replacing X with the appropriate number, and reboot to revert the setting.