Turn on adaptive MPSAFE for network threads, MPSAFE for IP and ARP

Sepherosa Ziehau sepherosa at gmail.com
Mon Mar 9 18:28:34 PDT 2009

On Tue, Mar 10, 2009 at 12:28 AM, Matthias Schmidt
<matthias at dragonflybsd.org> wrote:
> Hi Sephe,
> * Sepherosa Ziehau wrote:
>> Hi all,
>> I plan to set following tunables by default:
>> net.netisr.mpsafe_thread="1"
>> net.inet.udp.mpsafe_thread="1"
>> net.inet.tcp.mpsafe_thread="1"
>> net.link.ether.inet.arp_mpsafe="1"
>> net.inet.ip.mpsafe="1"
>> The first three enables adaptive MPSAFE for network threads.
>> The last two enable IP and ARP MPSAFE.
>> This gives us BGL free ip forwarding path and BGL free bridge
>> forwarding path.  pf is still under BGL though.
> Do you have any statistics regarding network performance?  Is there a
> "visible" difference?

On my Phenom9550 (2GB memory) w/ dual port 82571EB, one direction
forwarding, packets even spreaded to each core.  INVARIANTS is turned
on in the kernel config (I don't think it makes much sense to run a
system without INVARIANTS).

WIthout ipfw, BGL, ip forwarding:
~370Kpps - ~380Kpps

Without ipfw, BGL free, ip forwarding:

With ipfw default pass rule, BGL free, ip/bridge forwarding:

With ipfw 10 no matching rules + default pass rule, BGL free,
ip/bridge forwarding:

Without ipfw, BGL free, ip fast forwarding (ipflow):
~650Kpps - ~670Kpps

Without ipfw, BGL free, bridging forwarding:

For pure forwarding (i.e. no firewalling), the major bottle neck is on
transmit path (as I have measured, if you used spinlock on ifq, the
whole forwarding could be choked).  Hopefully ongoing multi queue work
could make the situation much better.

Best Regards,

Live Free or Die

More information about the Kernel mailing list