Checkpointing (Was Re: Initial impressions of DFly)

[Allan Fields]

On Wed, Jun 30, 2004 at 10:20:11PM +0100, Hiten Pandya wrote:
> Jonathan Fosburgh wrote:
> >After another, um, heated discussion, on -current@, I finally decided to 
> >use the other half of my harddrive here at work and install DFly.  I did 
> >the route of installing FBSD 4.10 and then a source upgrade and it worked 
> >wonderfully.  So far, all I can say is "WOW." :) I have been running 

Actually, I'm thinking of making my laptop the DFBSD test host.  I figure,
it's pretty much Free at the moment (running FreeBSD-stable) as
long as I have a machine to SSH in with.

> 	[ ... snipping ... ]
> 
> 	You should be able to get around the cups-base problem because
> 	we allow the system name to be overriden because most of the
> 	scripts query using 'uname' and not 'sysctl', so something like
> 	this should get you through (on csh):
> 
> 	# setenv uname_S DragonFlyBSD
> 	(or FreeBSD, whatever floats the boat)

I think I've heard some objection to the DFBSD label before.  As a
casual DragonFly observer at the moment, I was just wondering what
might be less desirable about such a designation: Shorter than
NetBSD though perhaps too close to FBSD, what with it having a
common suffix / substring?

The *BSD O/S regex grows larger:
/( O(pen)?
 | Pico
 | Echo
 | N(et)?
 | F(ree)?
 | DF(ly)?? | DragonFly
 ) BSD      | D(ragonFly|arwin) /i
# Reader exercise: the *BSD glob

The use of DragonFly is currently most popular according to Google,
but what of the lazy typists among us?  Also see why DFBSD fits
so nicely into the pattern. ;) -- Should Darwin match?

Right, anyway...


List of things I specifically find interesting about DragonFly
project at the moment:

  *     VFS changes (interesting to see what approaches are being
	taken in this line of development)

  *	Message passing: alternatives to traditional syscalls

  *	Process Checkpointing Support:

	Although I am aware of the drawbacks to any such approach,
	I give thumbs up for work going forward on this type of
	facility.  Unfortunately solutions at any one layer alone
	will likely not suffice in conclusively addressing issues
	of persistence, whether it be:
	-  hardware level battery/disk-backed persistent memory
	-  hardware level (BIOS supported APM sleep, ACPI sleep
	   states)
	-  system level ("hibernation") full-system-image persistence
	-  kernel supported transparent process checkpointing
	-  process level checkpointing (library/API supported process
	   image persistence)
	-  application level checkpointing (through explicit use
	   of persistent object stores and fine-grained state
	   management)
	-  traditional file/database backed data serialization
	   (coupled w/ manual state checkpoints/resumption points)

	It is possible for the more sticky issues to get resolved
	with continued effort on maturing process checkpointing
	mechanisms and supporting infrastructure.  Full session
	support and persistent networking being two of the long
	standing barriers to adoption.

	I would be interested to hear what if any benefit could be
	gained from the FreeBSD Rio/Vista research which aimed to
	provide discount checkpointing in a transaction based
	persistence framework.  Unfortunately the latest work dates
	back to FreeBSD-2.2.7.  I've experimented with it
	(FreeBSD-2.2.7-Rio), but not coaxed it into working reliably.
	I'm not aware of any other serious effort on the FreeBSD
	side.  [ http://www.eecs.umich.edu/Rio/ ]

	Therefore, I conclude that any and all work on persistence
	under BSD (including "another" persistence scheme) is
	beneficial in the, otherwise resultant, absolute void.  Ideally,
	work should be coordinated such that all BSDs are made to
	benefit if at possible, but this appears challenging.
	Perhaps it would be possible to further such research goals
	on a platform such as DragonFly to demonstrate the effectiveness
	of the approach employed and then other projects would be
	willing to adopt such features.

	I'd imagine that this work ties into SGE work as well..
	Reading the Mosix pages, it appears Linux process checkpointing
	may play an important role in some distributed computing
	applications.  Though such tools could be designed explicitly
	to utilize a manual checkpointing scheme and may currently
	employ a work-block/batched approach, true process migration
	isn't with-out it's merits, even outside of such applications.

	An often overlooked application to process-level persistence
	is fault-tolerance.  It might be possible to have a process
	survive an otherwise fatal system panic and/or hardware
	failure.  If coupled with effective session support it could
	provide an inexpensive solution which is otherwise difficult
	to implement at the scope of the program or in hardware.

	For instance, ACPI support is at this point not entirely
	functional on all systems and doesn't address distributed
	systems/migration.  On the other end of the spectrum, most
	popular windowing environments implement session management
	in such a way that the granularity of state management
	leaves much to be desired and which places the onus on the
	program to implement appropriate supporting infrastructure.
	Complex libraries such as KDE's handle some aspects of state
	persistence for an application in the base classes but this
	is just very basic window state, etc.

	In many cases reassembling and synchronizing the state of
	multiple cooperating process may prove problematic absent
	a kernel-level facility which can capture the state of each
	process, including system states.

	Coordinating persistence at multiple layers of implementation
	is potentially important to provide desired behaviour.  It
	might be beneficial to model the best parts of XSMP-like
	behaviour (which X apps currently underutilized) using
	signals to allow for a higher-level of coordination between
	persistence layers (kernel->userspace communication).
	Perhaps a signal such as "checkpoint pending" could result
	in a program having a chance to (where necessary):
	-  notify clients;
	-  stop packets from being queued;
	-  finish atomic operations and flush changes to disk;
	-  preserve any state necessary for clean resumption.

	On the existence of prior literature on the topic: yes,
	it's important to see what has been concluded already, but
	I'm all for (re)?creating what is needed today, by addressing
	the topic in a practical (pragmatic) light in addition to
	any theoretical considerations which factor into the problem.
	[ http://citeseer.ist.psu.edu/?q=process+checkpoint ]

	I get the feeling many commercial and specific implementations
	exist but can't gather the usefulness to making BSD processes
	persistent/distributed/fault-tolerant.

	No matter which angle this problem is approached from, it
	is a persistent issue in need of proper resolution.  The
	answer isn't to leave it unaddressed (the current broken
	state of affairs) simply because there are unresolved
	implementation details (complications) and numerous
	architectural consideration which will need to be thoroughly
	investigated anew (possibly including the creation of
	further supporting interfaces for subsystems on which a
	process relies on during it's lifetime.)  Rather, persistence
	in my opinion is a central design consideration rather than
	an after-thought.

	Hoping that BIOS supported sleep will solve all system-level
	persistence, isn't portable, hasn't proven reliable and
	will only lead to stagnation in an important and overlooked
	area of system architecture (IMHO). [ Note: that by portability
	I am referring to implementation portability rather than
	process migration in heterogenous environments. ]


> 	That should do the trick boss! :-p
> 
> 		-Hiten
> 		hmp at xxxxxxxxxxxxx

-- 
 Allan Fields, AFRSL - http://afields.ca
 2D4F 6806 D307 0889 6125  C31D F745 0D72 39B4 5541
Attachment:
pgp00000.pgp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgp00000.pgp
Type: application/octet-stream
Size: 187 bytes
Desc: "Description: PGP signature"
URL: <http://lists.dragonflybsd.org/pipermail/kernel/attachments/20040701/ee295826/attachment-0015.obj>