possible lockup when ptrace()'ing multi-lwp process

[Nicolas Thery]

Hello,

I observed a lockup while debugging a multi-lwp process with gdb.  Gdb is stuck
in wait4() even though the debuggee is stopped (SSTOP).  Here is my
analysis.

proc.p_nstopped counts how many lwps are stopped in the current process.  When
p_nstopped reaches p_nthreads, the parent process is woken up if blocked in
wait4(2).

When the process is unstopped (proc_unstop()), LWP_WSTOP is reset for all
lwps and all LSSTOP'ed lwps transition to LSSLEEP because setrunnable() is
called.

When the first lwp of the unstopped process runs again, p_nstopped ==
p_nthreads-1 (because p_nstopped is decremented when the lwp wakes up in
tstop()).

If this lwp calls proc_stop() before the other lwps in the process get a chance
to run and there are at least 2 other lwps, p_nstopped > p_nthreads because
proc_stop() increments p_nstopped for each LSSLEEP lwp unless LWP_WSTOP is set.
As a consequence, the wake-up test at the end of proc_stop() fails and gdb
hangs.

The suggested fix uses the LWP_WSTOP bit to ensure p_nstopped is incremented
and decremented exactly once per lwp.  Here is a minimal patch doing this.

There may be other related issues but I need to spend more time studying the
code to confirm or infirm my suspicions.

Cheers,
Nicolas

Index: src/sys/kern/kern_sig.c
===================================================================
--- src.orig/sys/kern/kern_sig.c	2007-08-03 20:10:24.000000000 +0200
+++ src/sys/kern/kern_sig.c	2007-08-03 20:14:45.000000000 +0200
@@ -1358,6 +1358,7 @@
 			 */
 			if (lp->lwp_flag & LWP_WSTOP) {
 				--p->p_nstopped;
+				lp->lwp_flag &= ~LWP_WSTOP;
 			} else {
 				if (bootverbose)
 					kprintf("proc_unstop: lwp %d/%d sleeping, not stopped\n",
@@ -1370,7 +1371,6 @@
 			break;

 		}
-		lp->lwp_flag &= ~LWP_WSTOP;
 	}
 	crit_exit();
 }
Index: src/sys/kern/kern_synch.c
===================================================================
--- src.orig/sys/kern/kern_synch.c	2007-08-03 20:10:24.000000000 +0200
+++ src/sys/kern/kern_synch.c	2007-08-03 20:15:46.000000000 +0200
@@ -1013,6 +1013,7 @@
 	}
 	tsleep(lp->lwp_proc, 0, "stop", 0);
 	p->p_nstopped--;
+	lp->lwp_flag &= ~LWP_WSTOP;
 	crit_exit();
 }