load balancing (was: Re: re(4) update)

[Joerg Sonnenberger]

On Mon, Oct 30, 2006 at 11:22:53AM -0800, Matthew Dillon wrote:
>     I personally believe that this was one of the original reasons why
>     zero-copy was implemented.  It was largely superceeded by sendfile()
>     which is zero-copy by design, to the point where I do not personally
>     believe that there is any good argument for a zero-copy VM solution
>     for userland OTHER then sendfile().

This is also the reason for one of the most often violated rules of web
programming -- separation of static from dynamic content. Esp. high-load
servers often benefit from it as the static content is usually just
driven by bandwidth and not CPU. Doing load balancing for a static
server is therefore often enough just not worth it.

The problem with proxy architectures is of course the second copying,
esp. kernel -> userland -> kernel, which can put a bunch of additional
load on frontend machines. Using approaches similar to state-full
firewall handling e.g. of FTP can be used for this as well, e.g. make it
a transparent proxy. This is a lot more work though. I'm not aware of
any such Open Source solution though. It shouldn't be hard to do that
e.g. in a small kernel module though.

Joerg