Preliminary restructuring layout (was Re: sys/ tree re-structuring proposal)

[Julian Elischer]

Matthew Dillon wrote:
     Well, excellent, it looks like I didn't mess things up too badly :-)

     I'm going to be working on the module build today but I thought I would
     also explain some of things that were done including some that might
     look a little wierd.
Additional cleanups everyone should do to reduce confusion:

    # remove random non-archive garbage from your sys/ tree and then
    # cvs update in order to allow cvs to remove dead directories.
    # (using the suggested 'update -Pd' line in your ~/.cvsrc)
    #
    cd /usr/src/sys
    find . -name '*.bak' -delete
    find . -name '*.orig' -delete
    cvs update
Obvious changes:

    netproto/{atalk,atm,ipsec,ipx,key,natm,ncp,ns,smb}

	Generally replaces top level directories netatalk, netatm, etc.  This
	change went fairly smoothly and I think it is an improvement.
    net/{accf_data,accf_http,bridge,disc,dummynet,ef,faith,gif,gre,i4b,ip6fw,
	 ip_mroute,ipfilter,ipfw,ppp,ppp_layer,sl,sppp,stf,tap,tun,vlan}
[Warning.. my opinion only]

Since you already have a 'net' I would have put net proto as net/proto
"All networking is under 'net'" sounds more consitent than
"All networking is under net and netproto".
Also the two accf modules should be under net/accf/...
in fact I'm not sure that teh accept filters shouldn't be under
net/proto/tcp/accf

things that are IP based should be under ip somewhere I think rather than
under 'net' directly..
	Generally move 'internal' network interfaces and partitionable
	infrastructure to subdirectories.


Ok but more hierarchy can make things clearer..

	net/ipfw and net/ip6fw are experiments.  In particular, ip6fw is
	experimentally moved from netinet6.  I haven't moved other things
	from netinet6 because I really don't know how good an idea it is
	yet.
	i4b was the most complex thing to clean up, particularly note that
	the architecture-specific portions of i4b were moved from i386/include
	to net/i4b/include/i386.  This was done as an experiment to see how
	well things could be organizde with architecture-specific pieces broken
	out and associated with their subsystem rather then jumbled together
	in one directory.  It required creating an 'i4b_machine' directory
	in /usr/include in order for programs to access machine-dependant
	portions of i4b (since <machine/BLAH> no longer has those files).
    dev/* -> dev/{agp,atm,crypto,disk,drm,hfa,misc,netif,powermng,raid,serial,
		sound,streams,usbmisc,video}/*
Im assuming that you went ahead and put the scsi adapters under 'disk'..
I still disagree with this.. they should be under dev/scsi.. possibly even
bus/scsi
	Moved the device drivers from the jumbled mess under dev/ to sub
	categories.

    *fs,miscfs/* -> vfs/{coda,deadfs,fdesc,fifofs,gnu,hpfs,isofs,mfs,msdosfs,
			nfs,ntfs,nullfs,nwfs,portal,procfs,smbfs,specfs,
			ufs,umapfs,union}
	All the VFS subsystems have been moved to the vfs/ directory.  mfs is
	no longer under ufs and ufs and ffs have been combined together and
	is just named ufs.
yay (ish)

you are undoing the work that Kirk did to separate ufs and ffs.
when (if) lfs comes back, where does it go? (since it was also 'ufs').

	I really liked how this came out.

    emulation/{ibcs2,linux,posix4,svr4}

	I moved the random top level emulations and ibcs2 (which was under
	i386) into emulation.  It may have been a mistake to move posix4 to
	emulation and I might move that back to the top level.
	Eventually emulation/ will be converted to a userland middle layer
	as previously discussed, so it was important to partition it into
	its own subdirectory.
	Architecture specific files were moved under their appropriate 
	emulation subdirectory.  e.g. instead of i386/linux we have
	emulation/linux/i386 for linux-specific i386-specific files.

    bus/{cam,canbus,eisa,firewire,iicbus,isa,mca,pccard,pci,ppbus,smbus,usb}

	All the bus architectures were separated out and moved into their
	own subdirectory.
	I may have made a mistake moving cam to bus/cam.  It's hard to say 
	what 'cam' is but it seems to be referenced in enough disparate 
	places that leaving it at the top level is probably the better choice.

    modules/* -> associated directories

	All the module Makefile's have been moved to their associated
	subsystems.  The 'modules' directory hierarchy has been deleted.
	The Makefile's are currently named 'Makefile.module' just so I know
	what I have to go in and fix.  They are not part of the build system
	yet.
    netgraph/*.h -> netgraph/<subsystem>/*.h

	When I was reorganizing netgraph I had to reincorporate the
	modules/ Makefiles into it which is why I moved the subsystems
	into their own subdirectories.  I actually didn't mind all the
	subsystems being at the toplevel (in netgraph/*) but I didn't see
	an easy way to incorporate the modules Makefile's that way.
	This turned out to be a real hassle for /usr/include file generation
	since there are a lot of netgraph related userland utilities.  It
	might make sense to undo this part of the move but for now I am 
	going to concentrate on making the modules build work.


I have always wante to have Makefiles near the code for the modules.
netgraph can 'stand' these changes.. as for /usr/include..
well, there aren't THAT many userland utilities..
Inobvious changes:

    By removing the use of -I- I changed many of the absolute #include file
    paths that were being used to reference header files in the same 
    directory as the source file into relative #include file paths using
    quotes, e.g. "blah.h" instead of <a/b/c/blah.h>.  This generally made
    it a whole lot easier to move subsystems around and I also believe it
    is a good idea just on principle.  This also makes inter-subsystem
    dependancies far more apparent in the codebase.  I did not universally
    'fix' all the #include's for locality of reference because it is an
    aweful lot of work, but I made a good dent.

    However, 'config' generates header files for devices named <DEVNAME>.h
    in the compile/ directory, which conflicts with similarly named header
    files in the source tree.  Originally the source code used "blah.h"
    to get at the compile/ directory header file and <a/b/c/d/blah.h> to
    get at the source-version.



    To solve this I changed 'config' to generate use_<DEVNAME>.h instead of
    <DEVNAME>.h.  I think this change will be well received because it really
    makes things clear in the source modules and it fits well with the
    opt_<SUBSYSTEMNAME>.h files that config already generations for options.
Why not put them in a subdirectory..
. .compile/GENERIC/opt/blah.
It would help clean up the build directory..
    /usr/include generation required considerable work.  Anyone making further
    changes to /usr/src/include/Makefile should run them by me before
    comitting as there are some gotchas (which I documented in the Makefile).
    I added a lot of 'compatibility' softlinks to reduce the amount of work
    that had to be done in /usr/src.  Please examine your newly generated
    /usr/include to see the new organization.
Many ecxported .h fiels don't need to be exported.
Phk was at one stage (I think it was him) trying to work on a way to 
indicate which .h files should be expoerted and which should not.


    UFS used to be broken up into ufs/ufs, ufs/ffs, and there was also ufs/mfs.
    Now we just have vfs/mfs and vfs/ufs, with all the files that were in
    ufs/ufs and ufs/ffs now in vfs/ufs.  There were no conflicts in making
    this change and its about time that we stopped abstracting ffs as a layer
    on top of ufs, at least in regards to source file organization.
Have you asked Kirk about this? what about ifs?


Final Note:

    None of the changes are set in stone and I am sure we will be doing 
    additional reorganization work, especially in net/, netinet/, and
    netinet6/.  However, it isn't easy to reorganize files due to the huge
    number of #include dependancies so I am going to let things settle for
    at least a month, maybe longer, before any additional major rewiring.

						-Matt