[patch] clear direction flag for signal handlers

[Aggelos Economopoulos]

On Sunday 09 March 2008, Aggelos Economopoulos wrote:
> On Saturday 08 March 2008, Simon 'corecode' Schubert wrote:
> > Aggelos Economopoulos wrote:
> > > gcc-4.3 assumes the direction flag is clear on function entry as
> > > specified by the i386 abi. Ensure that is the case when running
> > > a signal handler.
> > > 
> > > Linux-kernel discussion with gcc people starts here:
> > > http://article.gmane.org/gmane.linux.kernel/650279
> > > 
> > > Index: platform/pc32/i386/machdep.c
> > 
> > You might want to pull this up to vkernel as well.
> 
> And to amd64 as well, otherwise this might stay unnoticed for a long time.
> Wait. Did I even fix regular i386? Wouldn't be surprised if I've sent an
> empty patch...

swildner@ pointed out that there's yet another path that ends up running
a signal handler (for linux binaries). Updated patch follows.

gcc-4.3 assumes the direction flag is clear on function entry as
specified by the i386 abi. Ensure that is the case when running
a signal handler.

Linux-kernel discussion with gcc people starts here:
http://article.gmane.org/gmane.linux.kernel/650279

Index: platform/pc32/i386/machdep.c
===================================================================
retrieving revision 1.129
diff -u -r1.129 machdep.c
--- platform/pc32/i386/machdep.c
+++ platform/pc32/i386/machdep.c
@@ -515,7 +515,13 @@
 
 	regs->tf_esp = (int)sfp;
 	regs->tf_eip = PS_STRINGS - *(p->p_sysent->sv_szsigcode);
-	regs->tf_eflags &= ~PSL_T;
+
+	/*
+	 * i386 abi specifies that the direction flag must be cleared
+	 * on function entry
+	 */
+	regs->tf_eflags &= ~(PSL_T|PSL_D);
+
 	regs->tf_cs = _ucodesel;
 	regs->tf_ds = _udatasel;
 	regs->tf_es = _udatasel;
Index: platform/vkernel/i386/cpu_regs.c
===================================================================
retrieving revision 1.24
diff -u -r1.24 cpu_regs.c
--- platform/vkernel/i386/cpu_regs.c
+++ platform/vkernel/i386/cpu_regs.c
@@ -325,7 +325,13 @@
 
 	regs->tf_esp = (int)sfp;
 	regs->tf_eip = PS_STRINGS - *(p->p_sysent->sv_szsigcode);
-	regs->tf_eflags &= ~PSL_T;
+
+	/*
+	 * i386 abi specifies that the direction flag must be cleared
+	 * on function entry
+	 */
+	regs->tf_eflags &= ~(PSL_T|PSL_D);
+
 	regs->tf_cs = _ucodesel;
 	regs->tf_ds = _udatasel;
 	regs->tf_es = _udatasel;
Index: platform/pc64/amd64/cpu_regs.c
===================================================================
retrieving revision 1.4
diff -u -r1.4 cpu_regs.c
--- platform/pc64/amd64/cpu_regs.c
+++ platform/pc64/amd64/cpu_regs.c
@@ -327,7 +327,13 @@
 
 	regs->tf_rsp = (int)sfp;
 	regs->tf_rip = PS_STRINGS - *(p->p_sysent->sv_szsigcode);
-	regs->tf_rflags &= ~PSL_T;
+
+	/*
+	 * amd64 abi specifies that the direction flag must be cleared
+	 * on function entry
+	 */
+	regs->tf_rflags &= ~(PSL_T|PSL_D);
+
 	regs->tf_cs = _ucodesel;
 	/* regs->tf_ds = _udatasel;
 	regs->tf_es = _udatasel; */
Index: emulation/linux/i386/linux_sysvec.c
===================================================================
retrieving revision 1.29
diff -u -r1.29 linux_sysvec.c
--- emulation/linux/i386/linux_sysvec.c
+++ emulation/linux/i386/linux_sysvec.c
@@ -369,7 +369,13 @@
 	regs->tf_esp = (int)fp;
 	regs->tf_eip = PS_STRINGS - *(p->p_sysent->sv_szsigcode) + 
 	    linux_sznonrtsigcode;
-	regs->tf_eflags &= ~(PSL_T | PSL_VM);
+
+	/*
+	 * i386 abi specifies that the direction flag must be cleared
+	 * on function entry
+	 */
+	regs->tf_eflags &= ~(PSL_T | PSL_VM | PSL_D);
+
 	regs->tf_cs = _ucodesel;
 	regs->tf_ds = _udatasel;
 	regs->tf_es = _udatasel;
@@ -503,7 +509,13 @@
 	 */
 	regs->tf_esp = (int)fp;
 	regs->tf_eip = PS_STRINGS - *(p->p_sysent->sv_szsigcode);
-	regs->tf_eflags &= ~(PSL_T | PSL_VM);
+
+	/*
+	 * i386 abi specifies that the direction flag must be cleared
+	 * on function entry
+	 */
+	regs->tf_eflags &= ~(PSL_T | PSL_VM | PSL_D);
+
 	regs->tf_cs = _ucodesel;
 	regs->tf_ds = _udatasel;
 	regs->tf_es = _udatasel;