cvsup

[Simon 'corecode' Schubert]

Vincent Stemen wrote:
On 2008-01-15, Simon 'corecode' Schubert <corecode at fs.ei.tum.de> wrote:
Yes.  There are a couple of mirrors which serve the cvs repo via rsync. 
For instance use chlamydia.fs.ei.tum.de.  There are more, I guess.
Thanks.  

I just checked the list, but most of the sites say they only mirror
_Daily snapshots and official ISOs_.  The rsync link to the one you
mentioned, *chlamydia.fs.ei.tum.de*, does take you to a web page that
says they have the cvsup collections, but they do not provide the full
path to access them via rsync.
I guess you're expected to find out with rsync yourself :)

You know that you can get a listing with rsync?

sweatshorts % rsync chlamydia.fs.ei.tum.de:: 
               dflysnap        DragonFlyBSD daily snapshots
dragonfly-cvs   DragonFly CVS repository
freebsd-cvs     FreeBSD CVS repository
netbsd-cvs      NetBSD CVS repository
openbsd-cvs     OpenBSD CVS repository
sweatshorts % rsync chlamydia.fs.ei.tum.de::dragonfly-cvs 
               drwxr-xr-x         512 2007/05/07 23:38:22 .
-rwxrwxr-x         321 2003/07/15 06:17:23 fixme
drwxrwxr-x        1536 2008/01/16 12:33:43 CVSROOT
drwxrwxr-x         512 2006/06/28 22:24:29 dfports
drwxrwxr-x         512 2007/04/06 22:03:43 doc
drwxrwxr-x         512 2007/12/21 06:39:21 site
drwxrwxr-x         512 2004/06/18 13:23:48 src.sys.alpha.old
drwxrwxr-x        1024 2008/01/14 13:39:27 src

Rsync is very slow when you tag a repo, because all the files change, but 
only one line.  It's not really optimized for this task.  But this doesn't 
happen very often, though.
That is interesting.  The rsync docs claim to be very fast and say this

    The rsync remote-update protocol allows rsync to transfer just the dif-
    ferences between two sets of files across the network connection, using an
    efficient  checksum-search  algorithm ...
Well, of course in general it is fast.  But CVS repos are a special case, 
consisting of hundreds of thousands of small RCS text files, which only 
get one line added when the repo is tagged.  A general solution is bound 
to perform worse than a special case solution.

I ran a test transfering 8 text files over the internet that are each about
450K and the performance seems pretty good when I changed just one line in each
file.  In my test I told rsync to use ssh, so the transfer was also encrypted.
CVS repos are different:  Imagine 100000 files with an average size of 
4-8KB.  I think rsync has to do 100000 network roundtrips to negotiate on 
the changesets to be sent.

Total bytes sent: 31162
Total bytes received: 26842
Considering the fact that you only changed one line per file, that's quite 
a lot.  Plus it doesn't say how many round trips it was needing.

As you can see, it took 9 seconds for a full download, but only less than 1.5
seconds to update them.  That seems reasonably fast to me.  Is cvsup really
faster than that?  I am sceptical that it could be much faster.
Yes, cvsup is way faster in such a case.  Maybe not necessarily for 8 
files, but for 800 for sure.  It pipelines communication, so that there is 
no need to wait for immediate replies, thus saving network roundtrip times.

cheers
  simon
--
Serve - BSD     +++  RENT this banner advert  +++    ASCII Ribbon   /"\
Work - Mac      +++  space for low €€€ NOW!1  +++      Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \