PF & SMP
Matthew Dillon
dillon at apollo.backplane.com
Fri Sep 22 09:47:54 PDT 2006
:How much time is really wasted where one cpu is doing nothing but
:waiting on one kthread to complete nowadays?
:
://jonas
It only effects parallelism for cpu-heavy operations within the kernel.
So, for example, a computer doing nothing but routing packets between
two interfaces would be effected and a computer doing very tcp-heavy
web server operations from data cached in memory would be effected
but a computer doing user-centric stuff, such as serving mail or
running a web-server with a heavy-weight backend (CGIs, java,
database accesses), or doing lots of disk I/O, would not be effected
as much.
You can monitor big giant lock contention by observing the
'lwkt.mplock_contention_count' sysctl. This counter is incremented
every time the LWKT scheduler has to spin due to wanting to schedule
a thread that requires the MP lock but not being able to obtain the
MP lock because another cpu is holding it. Each tick of the counter
represents an amount of time wasted that is based on the speed of the
cpu. I would guess around 100ns per tick on a modern cpu.
As part of my parallel buildworld test I record the contention
count delta for each -j 8 build. I typically get numbers in the
800-900 million range. x 100 ns pertick == about 100 seconds of wasted
of cpu (for both cpus together) for a buildworld that takes 1450 seconds,
and dividing by the two cpus, that probably represents a real-world
loss in build time of between 4% and 7% over what it could hav achieved.
That is a very rough estimate, I haven't actually measured what a 'tick'
is... it could be less then 100ns but I don't think it is more.
A kernel-centric test, such as someone blasting TCP between two machines
but not otherwise doing any userland processing of the data, will result
in far greater contention. Such tests are a good way to test potential
contention, but don't really represent real-world effects.
The biggest effect MP contention has in a real-world system is
probably related to applications that communicate over local pipes
or unix domain sockets... for example, a web server with a MYSQL backend.
In this case being able to make the pipe and localhost socket code
MP safe would reap large rewards vs making the rest of the kernel MP safe.
Making thing like the disk I/O path MP safe doesn't generaly reap large
rewards because the disk I/O itself, rather than the cpu, represents the
bottleneck.
-Matt
More information about the Users
mailing list