pipe structure and KVM caching patch

Matthew Dillon dillon at apollo.backplane.com
Mon Feb 16 23:49:49 PST 2004


    I've done a first pass at improving pipe creation and teardown latencies
    and reducing kmem operations (which are nasty on SMP systems).  I added
    a per-cpu cache of pipe structures and I retain VM object and kmem
    mappings in the cached pipe structures.  Everything can be monitored
    and some params modified via the kern.pipe sysctl.  It is untuned but
    still represents a good first-order solution.  The cache eats around 1MB
    of KVM (virtual, not real memory) per cpu under heavy load.

    I haven't had any problems testing it so far but more testing can't
    hurt.  The patch can be retrieved at:

	fetch http://apollo.backplane.com/DFlyMisc/pipe01.patch

    My intention is to commit the pipe patch on thursday, in 3 days. 
    Robert Watson first mentioned the KVM caching issue for pipes in
    FBsd & DFly.  KVM allocations and deallocations are expensive on UP
    systems and even MORE expensive on SMP systems due to TLB shootdown
    requirements.

    -

    The cpu synchronization and PMap TLB shootdown patch will be going in 
    tomorrow (tuesday).  I've gotten positive reports from David and a few 
    other people, and it has been running on Leaf without incident, so I am
    fairly comfortable with its robustness and I need to move onto 
    stage2 (the VM side of the SMP TLB shootdown race).  I've CC'd Alan and
    Tor but it's up to them to decide how much of it to port to FreeBSD.

					-Matt
					Matthew Dillon 
					<dillon at xxxxxxxxxxxxx>





More information about the Kernel mailing list