Hi Folks,
I’m experiencing what appears to be a port-specific hardware acceleration/forwarding issue on my Route10, which became apparent immediately after the router updated to firmware 1.5f in the early hours of 21st August.
Issue
LAN port 4 (L3) is now limited to approximately 900–1,000 Mbps WAN download throughput, despite negotiating at 2.5 Gbps.
LAN ports 2 (L1) and 3 (L2), using the same devices, cables and Speedtest server, achieve approximately 1.6–1.7 Gbps.
The problem follows physical port 4, rather than the connected device.
Testing performed
My internet connection delivers approximately 1.7 Gbps downstream / 105 Mbps upstream.
Route10’s internal Speedtest:
-
Download: ~1,695 Mbps
-
Upload: ~105 Mbps
Mac connected to a known-good LAN port:
-
Download: ~1,682 Mbps
-
Upload: ~106 Mbps
When the same Mac and Ethernet connection are moved to L4:
-
Download: ~895 Mbps
-
Upload remains approximately 100 Mbps
I initially discovered the problem through my VMware ESXi server.
ESXi is connected using an Intel I225-LM 2.5GbE NIC and negotiates at 2,500 Mbps full duplex.
With ESXi connected to L4:
-
Ubuntu VM: ~940–1,000 Mbps
-
Debian VM: ~940 Mbps
Moving the same ESXi server/cable to a known-good Route10 LAN port immediately results in:
- Ubuntu VM: ~1,688 Mbps
No changes were made to ESXi or the VM between these tests.
I have also tested L3, which behaves normally like L2 and achieves full throughput. The issue therefore appears isolated to L4.
Route10 CPU behaviour
There is a significant difference in Route10 CPU utilisation depending on which physical LAN port carries the traffic.
During a Speedtest through L4 at approximately 900 Mbps:
-
Software IRQ (
sirq): ~44% -
ksoftirqd/0: ~16% -
ksoftirqd/1: ~10% -
ksoftirqd/2: ~13% -
Suricata: ~26%
-
CPU idle: ~29%
During the equivalent test through L2 at approximately 1.7 Gbps:
-
Software IRQ: ~1%
-
ksoftirqd: approximately 0% -
Suricata: ~1%
-
CPU idle: ~97%
This makes it appear that traffic through L4 may be falling back to software packet processing rather than using the hardware-accelerated forwarding path.
Port configuration
L2, L3 and L4 are configured equivalently.
L4 reports a 2.5 Gbps link, so this does not appear to be a 1 GbE autonegotiation issue.
There are no configured:
-
Client bandwidth limits
-
WAN download/upload limits
-
CAKE bandwidth limits
-
ESXi traffic-shaping policies
I have also:
-
Reset/reapplied the configuration for L4
-
Rebooted the Route10
Neither changed the behaviour.
Firmware
The issue became apparent immediately following the automatic update to 1.5f.
Given that L2 and L3 continue to achieve approximately 1.7 Gbps with very little CPU utilisation, while L4 is limited to approximately 900 Mbps and generates very high software-interrupt/packet-processing load, could you please investigate whether there is a known issue in 1.5f affecting hardware acceleration/flow offloading on individual LAN ports?
For the time being I have moved my ESXi server from L4 to L3, where full ~1.7 Gbps WAN performance has been restored.
Thanks