messaging questions...

[Matthew Dillon]

:So, in essence, we are differentiating between a blocking call [there 
:is no blocking call per-se] and
:whether the receiver can process the request in a blocking manner.  If 
:it requires blocking processing
:then we complete the request asynchronously and tell the caller to move 
:on.
:
:Essentially everything is "non-blocking" from the users perspective due 
:to the ability to queue work
:up internally on a thread's work queue.  That thread is not interacting 
:with users directly though I would
:assume except maybe to deal with the reply port?
:
:How am I doing? :)
:
:This is a bit different from message passing as I am used to... but 
:then I work with MPI all the time.

    Yes, in a nutshell.  Of course, there's some flexibility there.  For
    example, in order to make incremental commits DragonFly's current
    syscall messaging actually will block (because we haven't rewritten the
    kernel syscall backend to deal with the sync/async nature of the
    messaging API).  But eventually it will follow the semantics correctly.

    Also, even if a target could execute something synchronously without
    blocking it might decide to queue it anyway due to the complexity
    of the operation.

    So now you know what the target port's mp_beginmsg() function does, going
    back to the front end API lwkt_domsg() and lwkt_sendmsg()... these 
    functions abstract the interface to the client so the client does not
    have to worry about whether something returns EASYNC or not.

    lwkt_domsg() looks like this.  It basically always returns synchrously
    and is allowed to block.  If it sees an async result from the target
    port function it will then wait for the message to be replied
    synchronously, so EASYNC is never returned to the caller.

int
lwkt_domsg(lwkt_port_t port, lwkt_msg_t msg)
{
    int error;

    msg->ms_flags &= ~(MSGF_ASYNC | MSGF_REPLY | MSGF_QUEUED);
    msg->ms_reply_port = &curthread->td_msgport;
    msg->ms_abortreq = 0;
    if ((error = lwkt_beginmsg(port, msg)) == EASYNC) {
        error = lwkt_waitmsg(msg);
    }
    return(error);
}

    lwkt_sendmsg() looks like this.   The caller is always able to assume
    EASYNC when they call lwkt_sendmsg().  If the target port function in
    fact is able to run the message synchrously, lwkt_sendmsg() will 'fake'
    asynchronous operation by replying the message itself.

void
lwkt_sendmsg(lwkt_port_t port, lwkt_msg_t msg)
{  
    int error;
 
    msg->ms_flags |= MSGF_ASYNC;
    msg->ms_flags &= ~(MSGF_REPLY | MSGF_QUEUED);
    msg->ms_reply_port = &curthread->td_msgport;
    msg->ms_abortreq = 0;
    if ((error = lwkt_beginmsg(port, msg)) != EASYNC) {
        lwkt_replymsg(msg, error);
    }
}


    The MSGF_ASYNC flag in the above functions serves only as a 'hint' to
    the target port's mp_beginmsg function.  The target port is free to
    return a synchronous or asynchronous result code regardless of what
    the client has requested.  The client uses the lwkt_domsg() and 
    lwkt_sendmsg() API to properly abstract the client's wishes and to get
    a deterministic 'always asynchronous' or 'always synchronous' result
    independant of how the target port actually decides to deal with the
    request.

						-Matt