Fwd: load balancing

[Miguel Figueiredo Mascarenhas Sousa Filipe]

Hi,

On 10/31/06, Joerg Sonnenberger <joerg at xxxxxxxxxxxxxxxxx> wrote:
 On Tue, Oct 31, 2006 at 12:24:05AM +0100, Simon 'corecode' Schubert wrote:
> Joerg Sonnenberger wrote:
> >The problem with proxy architectures is of course the second copying,
> >esp. kernel -> userland -> kernel, which can put a bunch of additional
> >load on frontend machines. Using approaches similar to state-full
> >firewall handling e.g. of FTP can be used for this as well, e.g. make it
> >a transparent proxy. This is a lot more work though. I'm not aware of
> >any such Open Source solution though. It shouldn't be hard to do that
> >e.g. in a small kernel module though.
>
> are you thinking of a way to bond two file descriptors together?  that
 > would be nice.  first you do all header processing and whatnot, and then
> you just say
>
> bond_fd(server_fd /* read side */, client_fd /* write side */);
That and/or a generalised sendfile. That would be more in the line of
sendfile(src, dst, size), whre src can be a socket as well. Both is
useful for a number of situations to avoid unncessary round trips.


check out the new splice() and tee() syscalls on linux, they allow for
zero-copy data movement...
http://kerneltrap.org/node/6505

quoting from Linus Torvalds explanation:

"
Now, the advantage of splice()/tee() is that you can do zero-copy movement
of data, and unlike sendfile() you can do it on _arbitrary_ data (and, as
shown by "tee()", it's more than just sending the data to somebody else:
you can duplicate the data and choose to forward it to two or more
different users - for things like logging etc).
So while sendfile() can send files (surprise surprise), splice() really is
a general "read/write in user space" and then some, so you can forward
data from one socket to another, without ever copying it into user space.
Or, rather than just a boring socket->socket forwarding, you could, for
example, forward data that comes from a MPEG-4 hardware encoder, and tee()
it to duplicate the stream, and write one of the streams to disk, and the
other one to a socket for a real-time broadcast. Again, all without
actually physically copying it around in memory.
"


Best Regards,

--
Miguel Sousa Filipe