Unionfs etc Re: Packaging

[Sean Hogan]

Okay, I started experimenting and very quickly got stuck.  It appears 
that I can't perform multiple union mounts on the one mount point. 
Don't know if this is expected behaviour.

I'm still using FreeBSD (5.2.1-RELEASE to be precise) but I'm presuming 
the following holds true for DF.

I would like to create a jail on a complex union mount:
	top layer (and mount point) is:
            /.mnt/share/jails/test  (just an empty dir)
	next layer is: /work/FreeBSD/5.2.1-RELEASE/install
	bottom layer is: /usr/local
So I try the following:
su-2.05b# mount_unionfs -b /work/FreeBSD/5.2.1-RELEASE/install \ 		 
/.mnt/share/jails/test
su-2.05b# mount_unionfs -b /usr/local \
    /.mnt/share/jails/test
mount_unionfs: /usr/local: Operation not supported

I *can* mount /usr/local under /work/FreeBSD/5.2.1-RELEASE/install,
and then mount that under /.mnt/share/jails/test,
but that pollutes /work/FreeBSD/5.2.1-RELEASE/install.
Is this expected behaviour?  Is it relavent?
cheers,
SDH


Matthew Dillon wrote:

:# mkdir /tmp/a
:# mount_union /tmp/a /usr/local
:
:works fine and if I create a file in /usr/local it gets "really"
:created in /tmp and is only visible there after umounting /usr/local
:-- just what I wanted.  However,
:
:# mount_union /tmp/a /
:says :mount_union: /tmp/a (/usr/tmp/a) and / (/) are not distinct paths
:
:Is there a way around this?  I ask because I'd rather not assume
:anything about PREFIX, and I'd like this to be usable for
:badly-behaved installers that put stuff into other places like /etc,
    Well, /tmp/a on top of / creates a recursion within unionfs, which is
    why it is not allowed.  Actually, the more interesting mount is:
    mount_union -b / /tmp/a

    Which would create an image of the entire filesystem in /tmp/a, 
suitable
    for chroot'ing into /tmp/a.   But this is not currently allowed 
either.

    If you (or someone) would like to fix this, I think it's a fairly 
simple
    matter to have unionfs detect the recursion (e.g. /tmp/a/tmp/a) and
    disallow access to it.

    This would make unionfs a *very* powerful tool.


(lurk mode off)
Well, I have no right to say anything at this point, but this discussion 
lies close to my heart so I might add some of the things I've been 
thinking about.

I was hoping to write up an organized and critiqued summary of my 
thoughts.... but it hasn't happened in the last two weeks so I see no 
reason it will happen in the next two.  So... this is messy and 
full-of-holes.

Ever since I've been using FreeBSD I've been intrigued by the potential 
of jails and union mounts.   And I've been perplexed by the 
schizophrenic directory layout:  eg 
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
and it is worse for configuration files.

I would like to set up virtual machines via union mounts and jails, such 
that inside each machine you have precisely the programs, libraries, 
config files that you need.  And nothing else.

I want to have no doubts about where in the directory tree something 
can/should be found.
That is... libraries are in /lib, config files in /etc, and so on.

I think that as much of the base install as possible should be relocated 
to the ports/packages system.

I imagine that building a package would involve:

    Creating a jailed process on top of a multi-layered union mount
        - top level is empty (to begin) .
            path in the base system might be:
            /pkg/<name>-<version>-<port_rev>-<ENV?>
        - next level has source and makefiles, etc for the port.
            path in the base system might be:
            /src/<name>-....
        - lower levels get dependencies mounted as appropriate.
            path in the base system might be:
            /pkg/<dep-name>-....
           
    If the port has dependencies, need to request the base-system to
    mount the dependencies on lower layers, possibly building them
    first.
    
    Build the package.  Hmmm, maybe the top level shouldn't be
    mounted until after this stage.

    Install the package.  PREFIX=/
    Now every thing gets installed in /bin /sbin /lib /etc.   Yay.
    In the base-system they appear under /pkg/<name>-..../bin
        and so on.
    Now all packages are ready to be union mounted into
    our virtual machine.  Yay.
The above process is still too simplistic (as well as being 
full-of-holes) becos we would sometimes want only the libraries from a 
package in our virtual machine.

A virtual machine is now a jail on top of a multi-layered union mount.
    Top level is empty (to begin)
        path in base-system might be:
        /jails/<hostname>
    Lower layers are from the /pkg/<name>-.... tree as desired
    Configuration changes happen in /etc of the top layer,
    and affect this VM only (can't even be accessed by other VMs).
    Command line options to set a special location for config file
    lose their relavence.  World peace ensues.
I suspect that the root of the base-system should be static binaries 
(maybe in a boot-time loaded mfs) and that everything else can (should?) 
be dynamically linked.

Okay.  A bit of a dog's breakfast, I know.
I'm a big believer in "the proof of the pudding..." so if it is too hard 
to understand without some sort of prototype implementation (or if it is 
all just a piece of crap) then it is probably not worth responding...

It probably requires a lot of attention to union mount and other VFS 
voodoo.  It might be slow or resource intensive... my gut-feeling is 
otherwise but I have no basis for this.

Anyway... it's said.
cheers,
SDH