Turn on adaptive MPSAFE for network threads, MPSAFE for IP and ARP

[Matthew Dillon]

:On my Phenom9550 (2GB memory) w/ dual port 82571EB, one direction
:forwarding, packets even spreaded to each core.  INVARIANTS is turned
:on in the kernel config (I don't think it makes much sense to run a
:system without INVARIANTS).
:
:...
:
:For pure forwarding (i.e. no firewalling), the major bottle neck is on
:transmit path (as I have measured, if you used spinlock on ifq, the
:whole forwarding could be choked).  Hopefully ongoing multi queue work
:could make the situation much better.
:
:Best Regards,
:sephe

    I wonder, would it make sense to make all ifq interactions within
    the kernel multi-cpu (one per cpu) even if the paricular hardware
    does not have multiple queues?  It seems to me it would be fairly easy
    to do using bsfl() on a cpumask_t to locate cpus with non-empty queues.

    The kernel could then implement a protected entry point into the device
    driver to run the queue(s).  On any given packet heading out the interface
    the kernel would poll the spin lock and enter the device driver if it
    is able to get it.  If the spin lock cannot be acquired the device
    driver has already been entered into by another cpu and will pick up the
    packet on the per-cpu queue.

					-Matt
					Matthew Dillon 
					<dillon at backplane.com>