Question regarding IPIQs

[Christiano F. Haesbaert]

Ping, if I used the wrong list please point me to the correct one.



On 22 August 2013 17:22, Christiano F. Haesbaert <haesbaert at haesbaert.org>wrote:

> Hi there,
>
> I've been studying dragonfly for a while now and this is something I
> could never figure out, I'm pretty sure I'm missing something and I'd
> appreciate if someone could give me some pointers.
>
> Suppose the following cenario.
>
> Cenario A [A]
> cpu0: on whichever context
> cpu0: spin_lock(&a) (now in a critical section)
> cpu0: wakeup(ident1) (ident1 has a sleeper on cpu1)
> cpu0: lwkt_send_ipiq3(cpu1, wakeup(ident1))
>
> But ipiq for cpu1 is full, so I enable interrupts, although I'm still on
> a critical section, process my own ipiq and spin until cpu1 queue is not
> full, but at this point I'm still holding spin_lock a.
>
> This is the code I'm referring to, lwkt_ipiq:221-229
>
>         while (ip->ip_windex - ip->ip_rindex > MAXCPUFIFO / 4) {
>             if (atomic_poll_acquire_int(&target->gd_npoll)) {
>                 logipiq(cpu_send, func, arg1, arg2, gd, target);
>                 cpu_send_ipiq(target->gd_cpuid);
>             }
>             KKASSERT(ip->ip_windex - ip->ip_rindex != MAXCPUFIFO - 1);
>             lwkt_process_ipiq();
>             cpu_pause();
>         }
>
> cpu1: ipiq is full and is running an ithread, so it's in a critical
> section.
> cpu1: spin_lock(&a);
> cpu1: ??deadlocked??
>
> cpu1 will never process its own ipiq, therefore cpu0 will never make
> progress,
> since cpu0 holds spin_lock a, cpu1 will never make progress as well.
>
> Would this imply that a code that may generate an ipi down the stack, may
> never
> hold a spinlock ? I understand it is very likely that there is no current
> path
> that does a wakeup holding a spinlock.
>
> Furthermore, in this cenario you end up processing your own ipiq even if
> you
> were already on a critical section, can't the ipiq callbacks actually race
> against the code that lead to
> wakeup()->lwkt_sent_ipiq3()->lwkt_process_ipiq() ?
>
> Cenario [B]
> Still regarding the paragraph above, what if both cpu0 and cpu1 have their
> ipiqs
> full _and_ come from a code in a critical section, as in:
>
> cpu0: cpu0 own ipiq is full
> cpu0: crit_enter()
> cpu0: crit_enter()
> cpu0: wakeup(ident1) (ident1 has a sleeper on cpu1)
> ...
>
> cpu1: cpu1 own ipiq is full
> cpu1: crit_enter()
> cpu1: crit_enter()
> cpu1: wakeup(ident0) (ident0 has a sleeper on cpu0)
> ...
>
> Both realize the other is full and process their own queues, but they
> break the
> atomicity required by the two calls to crit_enter() earlier
>
> What am I missing ? (Sorry for the noise).
>
> Thanks
>
> --
> Christiano Farina HAESBAERT
> Do NOT send me html mail.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dragonflybsd.org/pipermail/kernel/attachments/20130828/21e617dd/attachment-0003.htm>