Testing needed - HAMMER PFS exports via NFS

[YONETANI Tomokazu]

On Sun, Sep 28, 2008 at 01:59:38AM +0900, YONETANI Tomokazu wrote:
> I also tweaked nata driver(using sys_cputimer->count()) and found that
> 
> - `indefinite wait!' message is displayed when spin_lock_wr() is called
>   from ata_start(), to lock state_mtx
> - ata_interrupt takes too long while it's holding state_mtx when it calls
>   ch->hw.end_transaction. digging further, it's callout_stop() right
>   below the `end_finished' label in ata_end_transaction() that is taking
>   long time (> sys_cputimer->freq)
> 
> I'll try diving into callout_stop() later.

It turned out that the while loop in lwkt_wait_ipiq() performs so many
iterations (18000000~) in such a case.  It's called from callout_stop()
when the caller's CPU isn't the same as that of the caller of callout_reset(),
in which case callout_stop() must issue a synchronous IPI.

lwkt_wait_ipiq():
				       :
            cpu_enable_intr();
            while ((int)(ip->ip_xindex - seq) < 0) {
                crit_enter();
                lwkt_process_ipiq();
                crit_exit();
				       :
	    }

Since the only caller of lwkt_wait_ipiq() is callout_stop(), I don't know
if the number is extreme or not, but usually the number of iterations in
this loop stays very low, and it doesn't seem to reach 5000 during
`make -j2 buildworld' on my Athlon64x2 box.  The number depends on the
hardware, of course, but even on VMWare, the upper bound of the iterations
in the normal case is hundred times lower than the abnormal case.
Assuming that the values returned by sys_cputimer->count() are not skewed,
it takes about one second (that is, slightly above sys_cputimer->freq)
to return from lwkt_wait_ipiq(), when `indefinite wait!' message is shown.