Rsync throwing errors but files appear to be fine

I am doing a massive 200TB transfer with rsync. I used a shell script I found off the internet to spawn a few processes but here is the relevant rsync command itself:

        rsync --recursive 
              --whole-file 
              --inplace 
              --sparse 
              --no-compress 
              --max-alloc=8GiB 
              --size-only 
              --human-readable 
              --info=progress2 
              --log-file="$rsync_file_basename.log" 
              --log-file-format="%o=%-7'''b | total=%-7'''l [%i] => %f%L" 
           "/mnt/disk${disk_id}/$share_name/$share_subdir/" 
           "$target_path" 
        >> "$rsync_file_basename.out"

The transfer is going pretty well but in watching the logs, I see two kinds of errors sporadically pop up.

mostly this:

rsync: [receiver] ftruncate failed on
"/mnt/remotes/TS140_stuff/pictures/family_1/IMG345.jpg": Resource
temporarily unavailable (11)

But also a few of these:

rsync: [receiver] write failed on
"/mnt/remotes/TS140_stuff/shows/ep1.mp4": Resource temporarily
unavailable (11) rsync error: error in file IO (code 11) at
receiver.c(380) [receiver=3.2.7]
rsync: [sender] write error: Broken pipe (32) rsync error: error in
file IO (code 11) at io.c(1700) [sender=3.2.7]

Now, the target directory is an SMB mount and this is a lot of data so I expected some things like this.

But the reason for this post is that I can see the files at the target destination and they look fine. Does rsync have some kind of retry logic that it’s using based on the command?

Asked By: john cs

||

rsync doesn’t really do retries. However there’s nothing stopping you wrapping it in a loop:

#!/bin/sh
k=1 ss=1
while [ "$k" -le 5 ] 
do
    echo "Attempt $k" >&2
    rsync -rt ... &&
        ss=0 &&
        break
    sleep 60
    k=$((k+1))
done

if [ "$ss" -ne 0 ]
then
    echo "ERROR: rsync repeatedly failed" >&2
    exit 1
fi

Make sure you use the --times (-t) flag, otherwise this loop will keep repeating until $k runs out. Do you understand what the flags you’ve chosen actually do? They’re a strange set, and I’d recommend removing most of them:

rsync -rtS
    --info=progress2 
    --log-file="$rsync_file_basename.log" 
    --log-file-format="%o=%-7'''b | total=%-7'''l [%i] => %f%L" 
    "/mnt/disk$disk_id/$share_name/$share_subdir/" 
    "$target_path"

Finally, if you can access the server that provides the SMB share directly, you will gain a considerable speed improvement for (at least) retries by using rsync in its network mode:

rsync -rtSz local_path/ remote_host:remote_path 
Answered By: Chris Davies
Categories: Answers Tags: , , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.