Interactive performance regression since 2.0 (patch #2)

[Petr Janda]

Hi Matt,
Just out of interest while talking about CPU schedulers. There was a project 
for the GSoC to write new scheduler using the Stride algorithm. Is this going 
to be committed?

Petr

On Wed, 17 Dec 2008 17:28:52 Matthew Dillon wrote:
>     Ok, here's a new patch to try:
>
> 	fetch http://apollo.backplane.com/DFlyMisc/sched02.patch
>
>     This turned into a three-day marathon run but I'm really happy with
>     the results.
>
>     The old scheduler had a huge number of issues.  I can't count the
> number of issues I came across.  I had to substantially rewrite the logic.
>
>     The new user scheduler does not depend on the helper threads when
>     switching under load.  It uses a very cool algorithm whereby threads
>     trying to return back into user mode 'bid' for the single user mode
>     slot (per-cpu).  So if thread A bids the best priority so far,
>     then switches to thread B (also trying to exit to user mode) which
>     bids a better priority, thread B will directly deschedule thread A
>     and take over the bid.  The scheduler helper is only used to kick
>     idle cpus to pick up the threads that lost the bid.  This results in
>     highly optimal operation.
>
>     I also found an issue with MP lock contention on SMP boxes.  If the
>     LWKT scheduler could not acquire the MP lock it would continue looking
>     for threads that didn't need it.  Sounds great, but what was happening
>     was that the scheduler was skipping important kernel threads needing
>     the lock and scheduling cpu-intensive user threads, and then not
>     rescheduling for a whole tick.  I created two fixes for this issue
>     controllable live with a sysctl.
>
>     lwkt.chain_mplock=0		Causes the LWKT scheduler to refuse to
> 				schedule a user thread if kernel threads
> 				exist which need the MP lock but couldn't
> 				get it.
>
>     lwkt.chain_mplock=1		Causes the LWKT scheduler to schedule the
> 				user thread but adds logic to rel_mplock()
> 				to attempt to IPI some other cpu needing
> 				the MP lock, creating an event that allows
> 				that other cpu to reschedule out of the user
> 				thread.
>
>     I intend to commit this on Friday baring problems, so please test both
>     modes.
>
>     So far on my desktop I have tested lwkt.chain_mplock=0 with excellent
>     results.
>
> 					-Matt
> 					Matthew Dillon
> 					<dillon at backplane.com>