cross-device copying / USB improvements

[benny]

Matthew Dillon wrote:

> 
>     There still needs to be some work done in the that regard,
>     like to try a few times and then discard the buffer.  There's
>     a problem though in that the buffers represent modified meta-data
>     that is synchronized with the mount.  If the kernel throws the
>     buffers away the filesystem mount could become even more confused
>     then just getting I/O errors.
This sounds to me like the umount command is waiting for the buffer to 
flush while the buffer is not given up if the mount is in no proper 
state. Is this what you are saying?

My naive approach is to let a forced umount blindly clear any related
buffers.

> :
> :This would be a cool way to "recover" from accidently interrupted
> :connections or unreliable devices.
> 
>     Yah, but I don't think it is in the cards.  There is no way
>     for the kernel to know what buffers actually made it to the
>     device or not.
Well, I think it doesn't have to - in conjunction with the rsync based
changes to cp and mv (BTW I am only taking about storage devices, hard
disks, usb-sticks and the like in my posting, should have made this clear).

My naive idea is to let cp/mv simply resume by starting a new transfer 
with only the differences, something like this

-> cp starts 
-> device unplugged 
-> triggers umount -f, buffers cleared, sigwait to cp, locks mount point
-> wait some secs 
-> device plugged 
-> triggers remount under same mount point 
   (if not possible: error, give up, release mount point)
-> sighup (or some other signal) to cp 
-> cp resumes. i.e. starts copying the same things again, this time 
   behaving like rsync (I guess rsync solves incomplete transfers by 
   some kind of checksum / hashing)

> 
>     They are all good ideas :-)  Just not so easy to implement.
Yeah, that was my fear :-). But since you are digging very deep anyway, it
could make a nice (sub-)project. Isn't the dragonfly project about proving
great ideas by cool implementations? Just to stimulate you ;-)

I had the impression that the reason why USB still isn't getting better in
FreeBSD is that nobody didn't really understood the the deeper internals
and since you just had fixed a lot of issues I thought you are in the topic
anyway and there could be a chance to fix the last remaining issues.

But if somebody who just wrote a filesystem tells me that this is not easy
to implement, this sounds like there is no chance to get somebody on such
work anytime soon...



Cheers,
Benny



> 
> -Matt

-- 
---------------------------------------------------------------------------------------------------