Speeding up networking, worth a read.

[Matthew Dillon]

:Simon 'corecode' Schubert wrote:
:
:> ...we don't feel comfortable with a tcp stack in userland...
:
:Could you explain in basic bonehead terms why not?  I've long
:been a fan of the microkernel which runs *everything* in
:userland.
:
:But (as my wife would quickly tell you) I'm an idiot  :o)
:Is your objection based on performance, or security, or --
:what?

    Well, what they are talking about is different from microkernel type
    architectures.  They are talking about moving all the protocol code
    into the same context as the user program that is making the
    connection.  The entire performance benefit comes from not having
    to context switch into the kernel and back out again.  There is some
    argument for cache locality of reference, but personally speaking
    I think there are plenty of other ways to localize data accesses
    that do not require running everything in one context (such as what
    we've done with our threaded TCP protocol implementation).

    In otherwords, you lose the protection between functional subsystems.
    Now, its true that everything running in the kernel is also running in
    a single context (the kernel context), but I would argue that the
    danger here is that you are suddenly asking application programmers to
    debug the TCP stack, and kernel/protocol programmers to debug a
    massive user application when things go wrong. 

    It would be disaster.  It would become extremely difficult to track down
    bugs.   And because of that I think its an ultra bad idea.  It is not
    worth the ~1 uS savings per system call.  It just isn't.
  
    In a traditional microkernel architecture things like the TCP stack would
    run in their own processes and, indeed, this WOULD improve stability
    simply by exposing bugs more quickly due to the protection space 
    separation that each subsystem enjoys.  I think this is a fine idea *IF*
    your sole goal is to have an ultra reliable system rather then an
    ultra fast system.  Most people need a mix of the two, with emphasis
    on performance, which is why microkernel architectures don't fare so
    well in the real world.

    There are also issues with distributability.  On a UP system or a 2-way
    SMP system, or in situations where you have many forked processes running,
    you might get some performance gains from localizing the TCP protocol
    stack to the application making the connection.  But who is to say that
    that situation is going to persist as we continue to move forwards?  If
    SUN has their way, we'll be running on 256-way machines in the household
    20 years from now and all of that serialized protocol code will destroy
    performance rather then improve it.

					-Matt
					Matthew Dillon 
					<dillon at xxxxxxxxxxxxx>