DragonFlyBSD Project Update - colo upgrade, future trends

[Damian Vicino]

Hi,
I think I was following this list for around 10 years. I'm not actively
posting anymore, but still reading every email. Many of the emails here are
Master classes in engineering.  And many discussion threads are very
interesting to follow.
I never contributed code to the project, but I did help on testing drivers
and a lot of advocacy arranging thing to get some devs to talk in
conferences, or presenting talks about how we used hammerfs in my job in
sysadmin events.
I would miss this list a lot if gone.
Thank you for all the great work you do.
Damián



On Tue, Jul 23, 2019, 22:32 Naoya Sugioka <naoya.sugioka at gmail.com> wrote:

> Very long time since I have sent some patches to the mailing list. Even my
> name is not on the list, but I still recall my small contribution. I am not
> actively posting, but am reading almost all the email conversation and then
> cerebrating progress of DragonflyBSD for more than ten years. I am a happy
> users of 5.x release branch.
>
> There are many users like me, a kind of stealth mode. I hope new system
> still has a slim chance to join as same moratorium style.
>
> Thank you very much for your efforts.
> -Naoya
>
>
>
> On Mon, Jul 22, 2019, 1:56 PM Matthew Dillon <dillon at backplane.com> wrote:
>
>> For the last week I've been testing out a replacement for Monster, our
>> 48-core opteron server.  The project will be removing Monster from the colo
>> in a week or two and replacing it with three machines which together will
>> use half the power that Monster did alone.
>>
>> The goal is to clear out a little power budget in the colo and to really
>> beef-up our package-building capabilities to reduce the turn-around time
>> needed to test ports syncs and updates to the binary package system.
>> Currently we use two blades to do most of the building, plus monster
>> sometimes.  The blades take almost a week (120 hours+) to do a full synth
>> run and monster takes around 27.5 hours.  But we need to do three bulk
>> builds more or less at the same time... one for the release branch, one for
>> the development branch, and one for staging updates.  It just takes too
>> long and its been gnawing at me for a little while.
>>
>> Well, Zen 2 to the rescue!  These new CPUs can take ECC, there's actually
>> an IPMI mobo available, and they are fast as hell and cheap for what we get.
>>
>> The new machines will be two 3900X based servers, plus a dual-xeon system
>> that I already had at home.   The 3900X's can each do a full synth run in
>> 24.5 hours and the Xeon can do it in around 31 hours.  Monster will be
>> retired.  And the crazy thing about this?  Monster burns 1000W going full
>> bore.  Each of the 3900X servers burns 160W and the Xeon burns 200W.  In
>> otherwords, we are replacing 1000W with only 520W and getting roughly 6x
>> the performance efficiency in the upgrade.  This tell you just how much
>> more power-efficient machines have become in the last 9 years or so.
>>
>> This upgrade will allow us to do full builds for both release and dev in
>> roughly one day instead of seven days, and do it without interfering with
>> staging work that might be happening at the same time.
>>
>> --
>>
>> Future trends - DragonFlyBSD has reached a bit of a cross-roads.  With
>> most of the SMP work now essentially complete across the entire system the
>> main project focus is now on supplying reliable binary ports for release
>> and developer branches, DRM  (GPU) support and other UI elements to keep
>> DragonFlyBSD relevant on workstations, and continuing Filesystem work on
>> HAMMER2 to get multi-device and clustering going.
>>
>> John Marino (marino) pioneered the 'synth' system for dports and
>> continues to help out, but his focus is on RavenPorts for now.  Rimvydas
>> Jasinskas  (zrj) and Antonio Huete Jimenez (tuxillo) have done a huge
>> amount of work bringing dports back into operational form and getting the
>> FreeBSD ports sync stuff working better.    DragonFlyBSD is still using
>> synth, and will probably remain on synth for the foreseeable future, though
>> there is always some discussion about how best to move dports forwards.
>> It's an excellent build platform for us.
>>
>> Francois Tigeot (ftigeot) has done a ton of work taking DRM up to
>> Linux-4.7.10 and this has worked very well for Intel iGPUs.  We are now
>> finally starting to dive into Linux's 'amdgpu' subsystem which is much
>> older, in order to modernize our AMD support (which is still deficient).
>>  Numerous other people have spent a considerable amount of time helping
>> test GPU support and tracking down bugs.  The work is ongoing.
>>
>> I apologize for only writing everyone's names in plain ascii :-)
>>
>> Right now HAMMER2 makes for an excellent single-device filesystem
>> (extremely well given that it supports writable snapshots and compression
>> out of the box), but it remains deficient when it comes to expandable
>> storage, multiple devices, and clustering.  This will be active work for me
>> but honestly the amdgpu support has to come first so it's still going to be
>> a long-haul for HAMMER2.
>>
>> The mailing lists are not seeing much if any activity any more.  This is
>> more a generational issue... people kinda prefer web-based forums these
>> days and younger generations do not use mailing lists at all for group
>> stuff (not really).  Even the devs almost universally use IRC and not
>> mailing lists for discussions now (its kinda too bad that we don't have a
>> permanent irc log stored on DFly servers for posterity).  So we are looking
>> into potentially shifting user interaction to a web-based forum, perhaps
>> this year, and retiring the mailing lists, leaving just an archive for the
>> mailing list.  Possibly sometime this year, so look for action on that
>> upcoming.
>>
>> Our other main developers: Sascha Wildner (swildner), who keeps the tree
>> in tip-top shape and catches numerous issues, Justin Sherrill (JustinS) who
>> handles our distribution rolls and the DragonFly and general BSD blog on
>> our home page, Sepherosa Ziehau (sephe) who focuses on the network
>> subsystem, Tomohiro Kusumi who helps with HAMMER and HAMMER2, Peter Avalos,
>> and more.  I will list more down below.
>>
>> --
>>
>> It's hard to say how the future will develop.  There are only three
>> open-source operating systems in the entire world that really pull it
>> together on having a complete, modern, SMP kernel:  Linux, DragonFlyBSD,
>> and FreeBSD.  And that's it.  We also have NetBSD and OpenBSD and I'd kinda
>> like to know what their plans are, because the future is clearly going not
>> only multi-core, but many-core.  For everything.  But as I like to say, for
>> SMP there are only three at the moment.  One can't dispute that Linux has
>> nearly all the eyeballs, and DragonFly has very few.  But OpenSource tends
>> to live on forever and algorithms never die... I think there is a place for
>> all of these projects and there really aren't any alternatives if you want
>> a sparkling clean system that doesn't have too many layers of abstraction.
>> At the current Juncture DragonFlyBSD is doing well and there are no plans
>> to slow down or stop.
>>
>> There are many other developers who help out with DragonFlyBSD on a
>> regular basis, or drop in from time to time, as well as past developers who
>> did an awful lot of work.   For this I am going to run the names out of the
>> git log in alphabetical order, so I don't miss anyone (hopefully).  And to
>> 'User' and 'Charlie Root'... we will never know who you were, but the party
>> is still going!
>>
>> -Matt
>>
>> Aaron LI
>> Adam Hoka
>> Adam Sakareassen
>> Adrian Chadd
>> Aggelos Economopoulos
>> Alex Hornung
>> Alexander Kuleshov
>> Alexander Polakov
>> Alexandre Perrin
>> Antonio Huete
>> Antonio Huete Jimenez
>> Antonio Nikishaev
>> Aycan iRiCAN
>> Ben Woolley
>> Bill Yuan
>> Brad Hoffman
>> Brills Peng
>> Charlie Root
>> Chris Pressey
>> Chris Turner
>> Chris Turner
>> Chris Wilson
>> Christian Groessler
>> Constantine A. Murenin
>> Daniel Bilik
>> Dave Hayes
>> David P. Reese
>> David Rhodus
>> David Shao
>> David Xu
>> Diederik de Groot
>> Dimitris Papastamos
>> Dylan Reinhold
>> Ed Schouten
>> Eirik Nygaard
>> Eitan Adler
>> Francis GUDIN
>> Franco Fichtner
>> Fran\xc3\xa7ois Tigeot
>> Gregory Neil Shapiro
>> Gwenio
>> Hasso Tepper
>> Hidetoshi Shimokawa
>> Hiroki Sato
>> Hiten Pandya
>> Ilya Dryomov
>> Imre Vadasz
>> Imre Vad\xc3\xa1sz
>> Imre Vad\xc3\xa1sz
>> Jan Lentfer
>> Jan Sucan
>> Javier Alc\xc3\xa1zar
>> Jean-S\xc3\xa9bastien P\xc3\xa9dron
>> Jeffrey Hsu
>> Jeremy C. Reed
>> Jeroen Ruigrok/asmodai
>> Joe Talbott
>> Joerg Sonnenberger
>> Johannes Hofmann
>> John Marino
>> Jordan Gordeev
>> Joris Giovannangeli
>> Justin C. Sherrill
>> Levente Kurusa
>> Liam J. Foy
>> Lubos Boucek
>> Magliano Andrea
>> Markus Pfeiffer
>> Matt Dillon
>> Matteo Cypriani
>> Matthew Dillon
>> Matthias Rampke
>> Matthias Schmidt
>> Maurizio Lombardi
>> Max Herrgard
>> Max Herrg\xc3\xa5rd
>> Max Okumoto
>> Maxim Ag
>> Michael Neumann
>> Michael Neumann
>> Michael Neumann
>> Mihai Carabas
>> Nicolas Thery
>> Nicolas Thery
>> Nolan Lum
>> Noritoshi Demizu
>> Nuno Antunes
>> Nuno Antunes
>> Peeter
>> Peeter Must
>> Peter Avalos
>> Pierre-Alain TORET
>> Robert Garrett
>> Robin Hahling
>> Rui Paulo
>> Rumko
>> Samuel J. Greear
>> Sascha Wildner
>> Scott Ullrich
>> Sepherosa Ziehau
>> Simon 'corecode' Schubert
>> Simon Arlott
>> Simon Schubert
>> Simon Schubert
>> Stathis Kamperis
>> Sylvestre Gallon
>> Thomas E. Spanjaard
>> Thomas Nikolajsen
>> Tim
>> Tim Bisson
>> Tobias Heilig
>> Tomasz Konojacki
>> Tomohiro Kusumi
>> Ulrich Sp\xc3\xb6rlein
>> User
>> Venkatesh Srinivas
>> Victor Balada Diaz
>> Vishesh Yadav
>> Walter Sheets
>> YONETANI Tomokazu
>> Yellow Rabbit
>> Yonghong Yan
>> Zach Crownover
>> b86
>> dumbbell
>> glebius
>> hrs
>> jkim
>> minux
>> rnoland
>> sinetek
>> zrj
>> \xc3\x81kos Kov\xc3\xa1cs
>>
>> -Matt
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dragonflybsd.org/pipermail/users/attachments/20190724/11ce4d4a/attachment-0003.htm>