cpdup work heads-up

Matthew Dillon dillon at apollo.backplane.com
Fri Apr 11 11:02:54 PDT 2008

    Ok, I think I've bashed cpdup's new features into shape on HEAD.
    Beware that the updated cpdup must be running on both sides of the
    link to use the new feature and there is no endian conversion.

    The new feature is '-pN', e.g. -p16, which parallelizes operations
    when the source and/or destination is a remote host specification.
    This should significantly speed up operations over slow links, and
    even over fast links.

    I originally didn't intend to implement this sort of feature but
    at the moment my offsite backup box has 100ms of latency due to a
    routing snafu well beyond my control.  Needless to say, 10 files
    checked per second is a bit too slow.

    One final item not yet solved is that apparently cpdup's individual
    write()'s to its pipe to ssh is resulting in ssh (which turns off nagle)
    to send out one TCP packet per request.  haven't found a way to tell
    ssh to leave nagle on for the batch link, since interactive response is
    not really needed when using the -pN option.  This is resulting in
    fairly expensive and unnecessary packet overhead.  If anyone has any
    ideas on how to fix ssh I'm all ears.  I'd rather not gang the writes
    in cpdup, it would be kinda messy to do that.

    Once this has been tested well enough I will MFC it to 1.12.


More information about the Users mailing list