variant symlinks vs VFS, and microkernels vs error kernels

[Chris Pressey]

Sorry for starting a new thread, but the original one was getting pretty
deeply nested, and this combines a couple of issues.

(While reading this, keep in mind this is an academic discussion.  In
the end, Matt is the 800lb gorilla here, and if he wants variant
symlinks in the DragonFlyBSD kernel, then by gum it'll have 'em.  But I
wouldn't feel right letting that happen without presenting the case for
alternatives first.)

On 02 Oct 2003 03:35:41 GMT
Sander Vesik <sander at xxxxxxxxxxxxxxxxxxx> wrote:
> Chris Pressey <cpressey at xxxxxxxxxxxxxxx> wrote:
> > On Wed, 1 Oct 2003 17:47:44 -0600
> > Mike Porter <mupi at xxxxxxxxx> wrote:
> > 
> >> As I said before, I don't see the point/need to make a program
> >> 'unavailable' to a user (in any sense except that typing, for
> >example,> 'gcc --version' will result in a different value).
> > 
> > What if I have users that I don't want to run gcc at all?  Granted,
> > today I would set up groups and make gcc group-executable only - but
> > this VFS-viewfs way seems much more elegant, because they wouldn't
> > even have to know gcc exists.
> 
> If a variant symlink uses an unset variuable then the lookup probably
> fails.

Not really what I was talking about, though... they can still go digging
around for what the symlink points to, and launch it with the full path
when they find it.

On Thu, 2 Oct 2003 02:29:19 -0600
Mike Porter <mupi at xxxxxxxxx> wrote:
> On Wednesday 01 October 2003 08:57 pm, Chris Pressey wrote:
> > What if I have users that I don't want to run gcc at all?  Granted,
> > today I would set up groups and make gcc group-executable only - but
> > this VFS-viewfs way seems much more elegant, because they wouldn't
> > even have to know gcc exists.
> 
> I wasn't considering this.  however, for those users, even under VFS,
> all they have to do is 'install' the port (however we define install)
> and they get it back.

But if they can't see 'install' either... :)

> It really isn't any different using symlinks, except...if you have 
> already installed all the versions, you can use gids to prevent
> execution of all installed versions.  VFS could (it seems) simply
> create a local copy of gcc, in the user's home directory, if
> necessary, bypassing any restrictions (OK, I guess if the user is
> putting gcc in their home directory anyway, it is a moot point, they
> would be able to do THAT regardless of vfs or symlinks.

Well - if they don't already have a C compiler, they can't build it from
source.  I suppose they could find a gcc binary for DragonFlyBSD out on
the net somewhere - but how about if I require binaries on my system to
be cryptographically signed?

The point is that I see VFS as being very *different* from variant
symlinks.  Variant symlinks are yet another aliasing mechanism, like
regular symlinks or 'use.perl', while the VFS would genuinely abstract
the underlying filesystem layout away from the tools that use it.

I tend to favour cleaner solutions, even if they're more work.  I also
prefer abstractions that have real teeth (ironically, because I've been
bitten more often by toothless ones...)  I think the solution can still
be worked towards in a piecemeal way, I'm just skeptical about this
particular piece of meal.

I'm quite willing to take the performance hit, too.  Matt has suggested
that, with his namecache niftiness, it will be negligible anyway.

> > > To me, the idea of a program being unavailable
> > > means that no matter what I do as the user, I will never see/know
> > > that the program is installed.  This to me is overkill.
> >
> > It IS overkill, for package management.  But it's not just for
> > package management, right?  Done correctly, it could unify a number
> > of disparate mechanisms currently in place.  chroot, for one.
> 
> That's why ultimately doing both is a good idea.  VFS certainly has
> its place, and will work well for a lot things.  variant symlinks will
> do a lot of the same things (not all) and should be easier to put in,
> heck, even I might be able to do it, if I can ever find time (although
> with my skills, I would almost certainly break something first <(}: ).
>  As matt said, it will address  maybe 85% of the cases for VFS, and be
>  easier to put in, should cost less (in terms of performance), and
>  otherwise just seems a good idea. 

Where's the fire, though?

As Matt also wrote: "correct choice of features", not "just laying on
hacks"!

Perhaps it's a little unjust to call variant symlinks a "hack"; to be
more polite, I should say that they strike me as one of those
"incremental improvements" that would be happily adopted by FreeBSD 5.x,
*if* people could agree on it.  It's an "easy, low impact" change that
would provide "immediate benefits" in a "pragmatic way" and doesn't
"make the kernel *too* much bigger and more complicated..."

Which brings me to the other point:

On Thu, 2 Oct 2003 10:29:33 -0700 (PDT)
Matthew Dillon <dillon at xxxxxxxxxxxxxxxxxxxx> wrote:

>     The environment is in user space, which means that namei/lookup
>     would have to go accessing user memory, and the environment is
>     also not indexed so lookups would be slow.  And, even worse, the
>     environment*IS* duplicated in each process.

It is my distinct feeling that namei/lookup *should be in user space*.

(From there, the problem of indexing the environment into a hash table
and making it an overlay onto, rather than a full copy of, a parent
environment, can be adressed.)

Ideally the kernel should be made *simpler*, **not** more complicated.

Forget "microkernel" - it's a buzzword.  But don't forget a *smaller*
kernel.  Smaller kernels panic less because there's less to go wrong.

Specifically, I find myself agreeing very much with the sentiment that
the kernel should be an "error kernel" - *only* the code which *must*
be correct under all circumstances.  That means it shouldn't be doing
parse-y stuff with plenty of edge cases like resolving filenames - much
less, variable expansion!

Now, maybe not everybody feels this way - regardless, one of the
features of DFBSD that I've definately heard Matt mention is the
eventual ease of migrating code into or out of the kernel.  IIRC,
specifically the idea of running a VFS mount in userspace.  A quick grep
through /home/dcvs/dragonsrc/sys/kern shows 52 calls to namei() - 43 of
which are in vfs_*.c files - so VFS is AFAICT namei's biggest consumer. 
If VFS can come out of the kernel, then it stands to reason that it
ought to be relatively painless to take out namei/lookup too.

This all suggests to me (who am by no means a kernel hacker but who have
been around the design block a few times) that putting variant symlinks
into namei() tomorrow, primarily for the sake of package management it
seems, would be a premature step in the wrong long-term direction.

Thank you for your time.

-Chris