Can I watch the progress of a `sync` operation?
I’ve copied a large file to a USB disk mounted on a Linux system with async. This returns to a command prompt relatively quickly, but when I type sync
, of course, it all has to go to disk, and that takes a long time.
I understand that it’s going to be slow, but is there somewhere where I can watch a counter go down to zero? Watching buffers
in top
doesn’t help.
You can look at the /sys/block/<device>/stat
file for the appropriate device while you’re syncing. The 9th column will indicate the number of in-flight requests on the device, which should go down to zero when the sync is done.
Don’t know of a way to translate that to a number of bytes, but it should give you a rough idea of how much “stuff” is still pending.
See the stat.txt file in the kernel documentation for a bit more information. (There’s also an inflight
file in that directory on my system which looks like it could contain read and write in-flight requests, but I can’t find docs for that.)
Looking at /proc/meminfo
will show the Dirty
number shrinking over time as all the data spools out; some of it may spill into Writeback
as well. That will be a summary against all devices, but in the cases where one device on the system is much slower than the rest you’ll usually end up where everything in that queue is related to it. You’ll probably find the Dirty
number large when you start and the sync
finishes about the same time it approaches 0. Try this to get an interactive display:
watch -d grep -e Dirty: -e Writeback: /proc/meminfo
With regular disks I can normally ignore Writeback
, but I’m not sure if it’s involved more often in the USB transfer path. If it just bounces up and down without a clear trend to it, you can probably just look at the Dirty
number.
By using Greg’s answer, you can simply have sync
run in background while displaying the state of the Dirty block in memory.
To achieve this, simply run this command:
sync & watch -n 1 grep -e Dirty: /proc/meminfo
This will call sync
in the background while executing watch
in the front. When the sync
command will have finished (around when the size of the Dirty block has reached 0), you will have an output that looks like this :
1] + 27260 done sync
This means that the command has finished and you can kill the watch
command with Ctrl+C.
You can watch current block device I/O traffic with nmon, as in:
NMON=ld nmon -s1
(this preselects display of load graph and disk graph with 1 second refresh time.. this can also be set by starting nmon
without parameters, than pressing l
, d
, -
to toggle graphs and reduce refresh time from default 2s .. +
of course would increase refresh delay.)
I wrote a script for this based on the answers here:
watchSync
#!/bin/sh
# Output the `Dirty` section of `/proc/meminfo`
report() {
local dirty="$(grep -w Dirty: /proc/meminfo | cut -d ':' -f 2 | tr -d ' ')"
echo -n -e "e[2KrSyncing ${dirty}... "
}
report
# Start syncing
sync &
SYNC_PID=$!
# Give a short sleep in case there's not much
sleep 0.125
# While sync is running, report
while ps -p $SYNC_PID > /dev/null 2>&1; do
report
sleep 1
done
echo "Done!"
repr () {
if [[ $# -gt 0 ]]; then
local result="$(printf -- '%q ' "$@")"
printf -- '%s' "${result::-1}"
fi
return $?
}
sync () {
local args
if [[ $- = *i* ]]; then
args="$(repr "$@")"
bash -sc "$(shopt -p); $(declare -f); ${FUNCNAME[0]} $args"
return $?
fi
local initial_value
local percentage
local backspace=''
local i
command sync "$@" &
initial_value="$(grep -e 'Dirty:' /proc/meminfo | awk '{print $2}')"
if [[ $initial_value = 0 ]]; then initial_value=1; fi
sleep 0.1
while ps -p $! > /dev/null; do
percentage="$(((($initial_value-$(grep -e 'Dirty:' /proc/meminfo | awk '{print $2}'))*100)/$initial_value))"
percentage="$(( $percentage < 0 ? 0 : $percentage ))%"
printf -- '%b' "$backspace$percentage"
backspace=''
for (( i=1; i<=${#percentage}; i++ )); do backspace+='b b'; done
sleep 1
done
printf -- '%b' "$backspace"
return $(wait $!)
}
Based on the other answers I made this over-bloated one-liner that tracks summary progress as well as device-specific progress:
# Create the command
watchSync() {
watch -n1 'grep -E "(Dirty|Write)" /proc/meminfo; echo; ls /sys/block/ | while read device; do awk "{ print "$device: " $9 }" "/sys/block/$device/stat"; done'
}
# Run the command
watchSync
The output looks like this:
Dirty: 848956 kB
Writeback: 125948 kB
WritebackTmp: 0 kB
dm-0: 0
dm-1: 0
loop0: 0
loop1: 0
loop2: 0
loop3: 0
loop4: 0
loop5: 0
loop6: 0
loop7: 0
nvme0n1: 0
sda: 0
sdb: 0
sdc: 124
The above tells me that 1) I have a few 100,000 kBytes that need writing, and 2) the device being written to is sdc
. Sync is complete when, in this case, sdc
and Writeback
hit zero (this will happen at the same time).