DragonFlyBSD Project Update - colo upgrade, future trends

[Tim Darby]

I like the web forum idea and I agree with the comments about HAMMER2
needing a HAMMER1-like mirroring capability soon-ish.

Tim


On Mon, Jul 22, 2019 at 1:56 PM Matthew Dillon <dillon at backplane.com> wrote:

> For the last week I've been testing out a replacement for Monster, our
> 48-core opteron server.  The project will be removing Monster from the colo
> in a week or two and replacing it with three machines which together will
> use half the power that Monster did alone.
>
> The goal is to clear out a little power budget in the colo and to really
> beef-up our package-building capabilities to reduce the turn-around time
> needed to test ports syncs and updates to the binary package system.
> Currently we use two blades to do most of the building, plus monster
> sometimes.  The blades take almost a week (120 hours+) to do a full synth
> run and monster takes around 27.5 hours.  But we need to do three bulk
> builds more or less at the same time... one for the release branch, one for
> the development branch, and one for staging updates.  It just takes too
> long and its been gnawing at me for a little while.
>
> Well, Zen 2 to the rescue!  These new CPUs can take ECC, there's actually
> an IPMI mobo available, and they are fast as hell and cheap for what we get.
>
> The new machines will be two 3900X based servers, plus a dual-xeon system
> that I already had at home.   The 3900X's can each do a full synth run in
> 24.5 hours and the Xeon can do it in around 31 hours.  Monster will be
> retired.  And the crazy thing about this?  Monster burns 1000W going full
> bore.  Each of the 3900X servers burns 160W and the Xeon burns 200W.  In
> otherwords, we are replacing 1000W with only 520W and getting roughly 6x
> the performance efficiency in the upgrade.  This tell you just how much
> more power-efficient machines have become in the last 9 years or so.
>
> This upgrade will allow us to do full builds for both release and dev in
> roughly one day instead of seven days, and do it without interfering with
> staging work that might be happening at the same time.
>
> --
>
> Future trends - DragonFlyBSD has reached a bit of a cross-roads.  With
> most of the SMP work now essentially complete across the entire system the
> main project focus is now on supplying reliable binary ports for release
> and developer branches, DRM  (GPU) support and other UI elements to keep
> DragonFlyBSD relevant on workstations, and continuing Filesystem work on
> HAMMER2 to get multi-device and clustering going.
>
> John Marino (marino) pioneered the 'synth' system for dports and continues
> to help out, but his focus is on RavenPorts for now.  Rimvydas Jasinskas
> (zrj) and Antonio Huete Jimenez (tuxillo) have done a huge amount of work
> bringing dports back into operational form and getting the FreeBSD ports
> sync stuff working better.    DragonFlyBSD is still using synth, and will
> probably remain on synth for the foreseeable future, though there is always
> some discussion about how best to move dports forwards.  It's an excellent
> build platform for us.
>
> Francois Tigeot (ftigeot) has done a ton of work taking DRM up to
> Linux-4.7.10 and this has worked very well for Intel iGPUs.  We are now
> finally starting to dive into Linux's 'amdgpu' subsystem which is much
> older, in order to modernize our AMD support (which is still deficient).
>  Numerous other people have spent a considerable amount of time helping
> test GPU support and tracking down bugs.  The work is ongoing.
>
> I apologize for only writing everyone's names in plain ascii :-)
>
> Right now HAMMER2 makes for an excellent single-device filesystem
> (extremely well given that it supports writable snapshots and compression
> out of the box), but it remains deficient when it comes to expandable
> storage, multiple devices, and clustering.  This will be active work for me
> but honestly the amdgpu support has to come first so it's still going to be
> a long-haul for HAMMER2.
>
> The mailing lists are not seeing much if any activity any more.  This is
> more a generational issue... people kinda prefer web-based forums these
> days and younger generations do not use mailing lists at all for group
> stuff (not really).  Even the devs almost universally use IRC and not
> mailing lists for discussions now (its kinda too bad that we don't have a
> permanent irc log stored on DFly servers for posterity).  So we are looking
> into potentially shifting user interaction to a web-based forum, perhaps
> this year, and retiring the mailing lists, leaving just an archive for the
> mailing list.  Possibly sometime this year, so look for action on that
> upcoming.
>
> Our other main developers: Sascha Wildner (swildner), who keeps the tree
> in tip-top shape and catches numerous issues, Justin Sherrill (JustinS) who
> handles our distribution rolls and the DragonFly and general BSD blog on
> our home page, Sepherosa Ziehau (sephe) who focuses on the network
> subsystem, Tomohiro Kusumi who helps with HAMMER and HAMMER2, Peter Avalos,
> and more.  I will list more down below.
>
> --
>
> It's hard to say how the future will develop.  There are only three
> open-source operating systems in the entire world that really pull it
> together on having a complete, modern, SMP kernel:  Linux, DragonFlyBSD,
> and FreeBSD.  And that's it.  We also have NetBSD and OpenBSD and I'd kinda
> like to know what their plans are, because the future is clearly going not
> only multi-core, but many-core.  For everything.  But as I like to say, for
> SMP there are only three at the moment.  One can't dispute that Linux has
> nearly all the eyeballs, and DragonFly has very few.  But OpenSource tends
> to live on forever and algorithms never die... I think there is a place for
> all of these projects and there really aren't any alternatives if you want
> a sparkling clean system that doesn't have too many layers of abstraction.
> At the current Juncture DragonFlyBSD is doing well and there are no plans
> to slow down or stop.
>
> There are many other developers who help out with DragonFlyBSD on a
> regular basis, or drop in from time to time, as well as past developers who
> did an awful lot of work.   For this I am going to run the names out of the
> git log in alphabetical order, so I don't miss anyone (hopefully).  And to
> 'User' and 'Charlie Root'... we will never know who you were, but the party
> is still going!
>
> -Matt
>
> Aaron LI
> Adam Hoka
> Adam Sakareassen
> Adrian Chadd
> Aggelos Economopoulos
> Alex Hornung
> Alexander Kuleshov
> Alexander Polakov
> Alexandre Perrin
> Antonio Huete
> Antonio Huete Jimenez
> Antonio Nikishaev
> Aycan iRiCAN
> Ben Woolley
> Bill Yuan
> Brad Hoffman
> Brills Peng
> Charlie Root
> Chris Pressey
> Chris Turner
> Chris Turner
> Chris Wilson
> Christian Groessler
> Constantine A. Murenin
> Daniel Bilik
> Dave Hayes
> David P. Reese
> David Rhodus
> David Shao
> David Xu
> Diederik de Groot
> Dimitris Papastamos
> Dylan Reinhold
> Ed Schouten
> Eirik Nygaard
> Eitan Adler
> Francis GUDIN
> Franco Fichtner
> Fran\xc3\xa7ois Tigeot
> Gregory Neil Shapiro
> Gwenio
> Hasso Tepper
> Hidetoshi Shimokawa
> Hiroki Sato
> Hiten Pandya
> Ilya Dryomov
> Imre Vadasz
> Imre Vad\xc3\xa1sz
> Imre Vad\xc3\xa1sz
> Jan Lentfer
> Jan Sucan
> Javier Alc\xc3\xa1zar
> Jean-S\xc3\xa9bastien P\xc3\xa9dron
> Jeffrey Hsu
> Jeremy C. Reed
> Jeroen Ruigrok/asmodai
> Joe Talbott
> Joerg Sonnenberger
> Johannes Hofmann
> John Marino
> Jordan Gordeev
> Joris Giovannangeli
> Justin C. Sherrill
> Levente Kurusa
> Liam J. Foy
> Lubos Boucek
> Magliano Andrea
> Markus Pfeiffer
> Matt Dillon
> Matteo Cypriani
> Matthew Dillon
> Matthias Rampke
> Matthias Schmidt
> Maurizio Lombardi
> Max Herrgard
> Max Herrg\xc3\xa5rd
> Max Okumoto
> Maxim Ag
> Michael Neumann
> Michael Neumann
> Michael Neumann
> Mihai Carabas
> Nicolas Thery
> Nicolas Thery
> Nolan Lum
> Noritoshi Demizu
> Nuno Antunes
> Nuno Antunes
> Peeter
> Peeter Must
> Peter Avalos
> Pierre-Alain TORET
> Robert Garrett
> Robin Hahling
> Rui Paulo
> Rumko
> Samuel J. Greear
> Sascha Wildner
> Scott Ullrich
> Sepherosa Ziehau
> Simon 'corecode' Schubert
> Simon Arlott
> Simon Schubert
> Simon Schubert
> Stathis Kamperis
> Sylvestre Gallon
> Thomas E. Spanjaard
> Thomas Nikolajsen
> Tim
> Tim Bisson
> Tobias Heilig
> Tomasz Konojacki
> Tomohiro Kusumi
> Ulrich Sp\xc3\xb6rlein
> User
> Venkatesh Srinivas
> Victor Balada Diaz
> Vishesh Yadav
> Walter Sheets
> YONETANI Tomokazu
> Yellow Rabbit
> Yonghong Yan
> Zach Crownover
> b86
> dumbbell
> glebius
> hrs
> jkim
> minux
> rnoland
> sinetek
> zrj
> \xc3\x81kos Kov\xc3\xa1cs
>
> -Matt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dragonflybsd.org/pipermail/users/attachments/20190729/342d5dba/attachment-0003.htm>