Severe packet loss on fxp interfaces with the new bridging code...
dillon at apollo.backplane.com
Fri Dec 23 10:06:21 PST 2005
:On an SMP machine without any of the new mpsafe sysctls active, bridging
:over fxp interfaces barely works. Maximum bandwidth is somewhere around
:100K/s, and I am seeing up to 80% packet loss. (as measured with a
:simultaneous scp and ping) After a bit of detective work, I have found
:the location of these mysteriously vanishing packets, but I do not
:understand the true nature of the problem.
Ok, I've run some tests. I haven't been able to reproduce any
packet loss testing between nv0 and em0. I have one of my shuttle
SMP boxes (test28) set up to bridge to test29:
(p.s. if you want to submit a patch to fix the output packet count
too that would be great!)
test28# netstat -I bridge0 -in 1
input (bridge0) output
packets errs bytes packets errs bytes colls
29392 0 2880416 0 0 0 0
27542 0 2699116 0 0 0 0
25886 0 2536828 0 0 0 0
office1# ping -f test29
PING test29.backplane.com (18.104.22.168): 56 data bytes
--- test29.backplane.com ping statistics ---
110597 packets transmitted, 110589 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.080/0.254/10.473/0.074 ms
On your ping test, make sure that you have set 'net.inet.icmp.icmplim'
to some high number on the target box so it doesn't throw away the
Now, of course, neither of these is using the fxp driver. But I
haven't found anything wrong in particular with the fxp driver so
also check your mbuf clusters:
test28# sysctl -a | fgrep nmb
Maybe your kern.ipc.nmbclusters is too low. If so, set it in
/boot/loader.conf (it has to be set at boot-time, not afterwords).
If that turns out to be the problem the only thing I'll have to do
is nudge the defaults.
More information about the Kernel