faster rsync of huge directories

[Tom Rosenfeld]

On Mon, Apr 12, 2010 at 10:04 AM, Vitaly <linux at karasik.org> wrote:

> 2010/4/12 Tom Rosenfeld <trosen at bezeqint.net>
> >
> > Hi,
> >
> > I am a great fan of rsync for copying filesystems. However I now have a
> filesystem which is several hundred gigabytes and apparently has a lot of
> small files. I have been running rsync all night and it still did not start
> copying as it is still building the file list.
> > Is there any way to get it to start copying as it goes. Or do any of you
> have a better tool?
> >
>
> Are both servers in the same LAN? IMHO, your problem in network band
> witchbetween source and destination.
> I have ~4M files, ~800GB - rsync is very fast in the same LAN (1Gb),
> and slowly for remote destination.
>
> Regards,
> Vitaly
>

I am not even using a lan. It is disk to disk. I have ~16M files ~900GB.
rsync has been running about 18 hours and has indexed over 8 million files,
but still did not copy even one.

Thanks,
-tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cs.huji.ac.il/pipermail/linux-il/attachments/20100412/7f5af1b5/attachment.html>