What is DF aimed at?

[Dmitri Nikulin]

On 6/5/06, Danial Thom <danial_thom at xxxxxxxxx> wrote:


--- Matthew Dillon <dillon at xxxxxxxxxxxxxxxxxxxx>
wrote:
> :Uh, how do you get that? Clustering implies
> :networking, and Matt has repeatedly stated
> that
> :he doesn't really care about network
> performance.
> :
> :And clustering implies servers, which Matt has
> :recently and repeatedly stated aren't his
> focal
> :point. I don't see how you can have one as a
> goal
> :and not the other. Clustering required hightly
> :efficient networking first and foremost.
> :
> :DT
>
>     Er, I said no such thing.  You apparently
> did not read my
>     posting(s) very carefully.
You said you only wanted "very good networking"
and that if you wanted to push traffic at the
bleeding edge you should get dedicated hardware
solutions from cisco.
You know optimizing networking isn't such a bad
thing. You can have your cake and eat it too.
Having stellar networking performance will not
hurt your project, nor is it a "waste of time".
It would make your OS much more attractive to a
much wider user base. Even Intel gave in and
decided to build a CPU to win the benchmarks.
People want the best. No-one is going to notice
if you're 3rd best, but everyone will notice if
you're #1.
People use Linux and it's far from the best in security, stability and
for many cases even performance. They use it because it does a lot and
is marketable. Done. Quality doesn't matter otherwise it would never
have left the garage.
Even so, optimizing every last possible drop from the network stack is
*not* compatible with the goals of this project. For example, if you
understand the LWKT system and Matt's presentation/emails regarding
the way socket threads are distributed on multi-processor systems,
you'll note that they're split by port and bound to that CPU for their
lifetime. This means that load balancing is not as optimal as
possible, since actual load is not factored in. However, work
aggregation is a lot more successful, because migrations are costly.
Also, the system itself is near lockless and, as far as localised
network stacks go, impressively optimal already.
Since getting proper load balancing of the threads in would be counter
to DragonFly's very architecture, and since that optimization itself
has significant downsides and can actually make a pitifully small
positive difference, there's no point optimizing to that extreme. This
is an example, I'm sure Matt could conjure many more cases where the
extra optimization just isn't worth it, but he has better things to
do.
I don't know about "stellar" here. Let's wait until more of the kernel
is MPSAFE, including the network stack, and do a bench set against a
few instances of FreeBSD and Linux. I'd be more than happy to try it
out, I've got some em (Intel gigabit) cards and an X2 4400+ and that's
a nice start. You can try it on your millions of dollars worth of 10GE
machines. Contribute to the project! You have money and obviously a
lot of spare time, so help the project out instead of insulting its
developers. That'll be a good deed and you may realise just how great
this community is when you're not perceived as an ass bandit.
 -- Dmitri Nikulin