Turn on adaptive MPSAFE for network threads, MPSAFE for IP and ARP
dillon at apollo.backplane.com
Tue Mar 10 17:24:14 PDT 2009
:On my Phenom9550 (2GB memory) w/ dual port 82571EB, one direction
:forwarding, packets even spreaded to each core. INVARIANTS is turned
:on in the kernel config (I don't think it makes much sense to run a
:system without INVARIANTS).
:For pure forwarding (i.e. no firewalling), the major bottle neck is on
:transmit path (as I have measured, if you used spinlock on ifq, the
:whole forwarding could be choked). Hopefully ongoing multi queue work
:could make the situation much better.
I wonder, would it make sense to make all ifq interactions within
the kernel multi-cpu (one per cpu) even if the paricular hardware
does not have multiple queues? It seems to me it would be fairly easy
to do using bsfl() on a cpumask_t to locate cpus with non-empty queues.
The kernel could then implement a protected entry point into the device
driver to run the queue(s). On any given packet heading out the interface
the kernel would poll the spin lock and enter the device driver if it
is able to get it. If the spin lock cannot be acquired the device
driver has already been entered into by another cpu and will pick up the
packet on the per-cpu queue.
<dillon at backplane.com>
More information about the Kernel