Scrub is the continuous half of "zero errors"
Run zpool scrub on the production pool and on the replica. Keep the finished line that says repaired 0B with 0 errors. That line is not a guest that booted. Boot a clone on other storage before you call the copy clean.
Scrub the pool. Then boot a copy. zpool scrub is the half you can run while the guest stays up. It reads every allocated block and checks the checksum. A finished line that says repaired 0B with 0 errors is not a guest that started.
The 0 in 3-2-1-1-0 is a restore that was checked. A scrub is the checksum half of that check. The boot is the other half. Test a virtual machine restore.
What does zpool scrub check?
zpool scrub tank starts a scrub of the pool named tank. The argument is a pool, not a dataset. zpool scrub tank/vm/db is the wrong name. One scrub covers every guest on that pool. It does not cover a second pool, and it does not cross to the replica host.
A normal scrub reads the allocated blocks and verifies each checksum. On a mirror, raidz, or draid, ZFS rewrites a block whose checksum fails when another copy is still good. A pool with one disk has nothing to copy from. The scrub can tell you the block is bad. It cannot rebuild it.
A resilver reads blocks ZFS already knows are stale, such as a disk you attached or replaced. A scrub reads the pool to find silent errors. If zpool status says resilvered, that line is a disk rebuild, not a scrub. ZFS runs one of those at a time, and a scrub will not start while a resilver is still going.
zpool status tank is the result. While the scrub runs, the scan line says scrub in progress. New writes can push that percentage past 100%, so wait for the finished line. zpool status -x skips a healthy pool and will not show you the scrub date.
What does the finished line mean?
The clean line names the pool, then scrub repaired 0B, a duration, with 0 errors, and a date. Under the config, READ, WRITE, and CKSUM are 0, the state is ONLINE, and the footer says No known data errors. Write that line down. “Zero errors”, with none of that, is not a record.
repaired 64K with 0 errors is a different event. The scrub found bad blocks and reconstructed them. The uncorrectable count is 0, and a disk still returned a bad checksum. Note which device shows a CKSUM count before you run zpool clear. Clear wipes the counters. It does not replace the disk. Keep the scan line in the note, then replace the disk that failed.
with 2 errors means at least one block could not be repaired. zpool status -v lists damaged files when it can. If the dataset is encrypted and the key is not loaded, that list can omit the file name. Stop treating that pool as the copy you would boot. Restore a snapshot from before the damage, from the other pool. A snapshot is not a backup.
A paused scrub is not a finished scrub. zpool scrub -p tank pauses. A reboot or an export leaves it paused, including after the next import. zpool scrub tank resumes it from the last point saved to disk, in the same mode. zpool scrub -s tank stops it. A stopped scrub has no repaired line to trust.
A scrub that does not read the whole pool is not this check. -S and -E limit which blocks, by the date they were created. -e only visits files that already have errors. -C continues from the last saved transaction group. The weekly job is zpool scrub with none of those flags.
Do you scrub the replica?
Yes. The production scan line says nothing about the other host. zpool scrub backup on the replica is a second job. Read that status too.
A scrub does not tell you whether the snapshot you need is still on the replica. The replica keeps snapshots the source is no longer allowed to expire. The age of the newest snapshot that arrived is a different number. The recovery point objective is the time since the last successful send.
On a systemd host, OpenZFS ships zfs-scrub-weekly@tank.timer and zfs-scrub-monthly@tank.timer. Enable the timer for that pool, on each host. The timer starts the scrub. It does not read the result. The next day, run zpool status on both pools. If the scan line still says in progress or paused, you do not have this week’s check. ZFS will not run a second scrub on top of the first, and a large pool can still be running when the next week starts.
Do you load the encryption key?
Not for an ordinary scrub. Checksums are verified without the key. A replica that received a raw send can be scrubbed while the key stays on the host you trust to read the guest. One dataset per virtual machine is the job, the quota, and the key.
zpool scrub -t is a thorough scrub. It also decrypts and decompresses, which catches the rare case where the checksum matches and decryption still fails. That pass needs the keys loaded. If a key is missing, or you unload it during the scrub, those blocks fall back to an ordinary scrub. Do not load the replica key just to run -t. Ordinary scrub is the check that matches a raw replica. If you want the thorough pass, load the key on a host you already allow to read the guest, and run -t there.
What does a clean scrub miss?
A scrub checks that the blocks on disk are the blocks ZFS wrote. It does not check that those blocks are the guest you meant to keep.
A guest that overwrote its files with valid new data still checksums. The scrub passes. The snapshot you sent is the point you can return to, and only if that snapshot left this pool.
A scrub does not freeze the guest. It does not ask a database to finish a transaction. A clean pool can hold a torn write from hours ago. Make application-consistent virtual machine backups.
A scrub does not prove the copy left the building. Keep an off-site virtual machine backup. A tool that only checksums a backup file does not read the live zvol. If the disk is a zvol on this pool, the scrub is that read. It is still not a boot.
How do you prove the guest boots?
Pick the snapshot you mean to keep. Clone or receive it onto storage that is not this pool. New name, new UUID, NIC on a bridge with no uplink. Boot it. A record written before the snapshot has to be there. A record written after it has to be absent. A cleanroom is a clone on a bridge with no uplink. Instant recovery is a clone you can promote.
Write the pool names, both scan lines, the dates on those lines, the repaired sizes, the error counts, the snapshot name, and the boot note. A scrub from last month does not replace a boot you did not do.
Check the scan line, then boot
These scripts build fixtures under a temporary directory. They do not run zpool and they do not boot a guest. The first accepts a finished scrub of two different pools only when each scan line says repaired 0B with 0 errors, the pool is ONLINE, the device counters are 0, and the footer has no known data errors. It rejects a scrub still running, a paused scrub, a resilver, a repair, an error count, a dataset name, a degraded pool, and the same pool named twice.
#!/usr/bin/env bash
set -euo pipefail
root="$(mktemp -d)"
trap 'rm -rf "$root"' EXIT
field() {
local file="$1"
local key="$2"
local line
while IFS= read -r line || [[ -n "$line" ]]; do
if [[ "$line" == "${key}="* ]]; then
printf '%s\n' "${line#"${key}="}"
return 0
fi
done < "$file"
printf 'missing %s in %s\n' "$key" "$file" >&2
return 1
}
is_uint() {
[[ "$1" =~ ^[0-9]+$ ]]
}
accept_scan() {
local scan="$1"
[[ "$scan" == "scrub repaired 0B "* ]] || return 1
[[ "$scan" == *" with 0 errors on "* ]] || return 1
[[ "$scan" != *"in progress"* ]] || return 1
[[ "$scan" != *"paused"* ]] || return 1
}
accept_pool() {
local file="$1"
local pool scan state errors cksum read_n write_n
pool="$(field "$file" pool)"
scan="$(field "$file" scan)"
state="$(field "$file" state)"
errors="$(field "$file" errors)"
cksum="$(field "$file" cksum)"
read_n="$(field "$file" read)"
write_n="$(field "$file" write)"
[[ -n "$pool" && "$pool" != */* ]] || return 1
[[ "$state" == "ONLINE" ]] || return 1
[[ "$errors" == "No known data errors" ]] || return 1
is_uint "$cksum" || return 1
is_uint "$read_n" || return 1
is_uint "$write_n" || return 1
[[ "$cksum" -eq 0 && "$read_n" -eq 0 && "$write_n" -eq 0 ]] || return 1
accept_scan "$scan"
}
accept_pair() {
local prod="$1"
local replica="$2"
local prod_name replica_name
prod_name="$(field "$prod" pool)"
replica_name="$(field "$replica" pool)"
[[ -n "$prod_name" && -n "$replica_name" && "$prod_name" != "$replica_name" ]] || return 1
[[ "$(field "$prod" role)" == "production" ]] || return 1
[[ "$(field "$replica" role)" == "replica" ]] || return 1
accept_pool "$prod" || return 1
accept_pool "$replica" || return 1
}
write_pool() {
local dest="$1"
local role="$2"
local pool="$3"
local scan="$4"
local state="$5"
local errors="$6"
local cksum="$7"
local read_n="$8"
local write_n="$9"
cat > "$dest" <<EOF
role=${role}
pool=${pool}
scan=${scan}
state=${state}
errors=${errors}
cksum=${cksum}
read=${read_n}
write=${write_n}
EOF
}
must_fail() {
local label="$1"
shift
if "$@"; then
printf 'checker accepted %s\n' "$label" >&2
exit 1
fi
}
clean_scan="scrub repaired 0B in 00:12:34 with 0 errors on Thu Nov 12 02:00:00 2026"
mkdir -p "$root"
write_pool "${root}/prod-good" production tank "$clean_scan" ONLINE "No known data errors" 0 0 0
write_pool "${root}/rep-good" replica backup "$clean_scan" ONLINE "No known data errors" 0 0 0
accept_pair "${root}/prod-good" "${root}/rep-good"
write_pool "${root}/in-progress" production tank "scrub in progress since Thu Nov 12 01:00:00 2026" ONLINE "No known data errors" 0 0 0
must_fail "a scrub still in progress" accept_pair "${root}/in-progress" "${root}/rep-good"
write_pool "${root}/paused" production tank "scrub paused since Thu Nov 12 01:00:00 2026" ONLINE "No known data errors" 0 0 0
must_fail "a paused scrub" accept_pair "${root}/paused" "${root}/rep-good"
write_pool "${root}/resilver" production tank "resilvered 1.2T in 01:00:00 with 0 errors on Thu Nov 12 02:00:00 2026" ONLINE "No known data errors" 0 0 0
must_fail "a resilver" accept_pair "${root}/resilver" "${root}/rep-good"
write_pool "${root}/repaired" production tank "scrub repaired 64K in 00:12:34 with 0 errors on Thu Nov 12 02:00:00 2026" ONLINE "No known data errors" 0 0 0
must_fail "a scrub that repaired blocks" accept_pair "${root}/repaired" "${root}/rep-good"
write_pool "${root}/uncorrectable" production tank "scrub repaired 0B in 00:12:34 with 2 errors on Thu Nov 12 02:00:00 2026" ONLINE "Permanent errors have been detected" 2 0 0
must_fail "a scrub with uncorrectable errors" accept_pair "${root}/uncorrectable" "${root}/rep-good"
write_pool "${root}/dataset" production tank/vm/db "$clean_scan" ONLINE "No known data errors" 0 0 0
must_fail "a dataset name" accept_pair "${root}/dataset" "${root}/rep-good"
write_pool "${root}/degraded" production tank "$clean_scan" DEGRADED "No known data errors" 0 0 0
must_fail "a degraded pool" accept_pair "${root}/degraded" "${root}/rep-good"
write_pool "${root}/same" replica tank "$clean_scan" ONLINE "No known data errors" 0 0 0
must_fail "the same pool named twice" accept_pair "${root}/prod-good" "${root}/same"
write_pool "${root}/none" production tank "none requested" ONLINE "No known data errors" 0 0 0
must_fail "a pool that was never scrubbed" accept_pair "${root}/none" "${root}/rep-good"
printf 'Scrub line check passed.\n'
printf 'Proves: the good fixture is two different ONLINE pools, each with a finished scrub that repaired 0B with 0 errors, zero device counters, and no known data errors.\n'
printf 'Does not prove: zpool scrub ran, a disk was read, or a guest boots from a snapshot.\n'The second accepts a note only when both scan lines are that clean line and the boot fields name a different guest, other storage, an isolated network, no uplink, a record older than the snapshot, and a destroyed test VM. A scrub-only note fails. The note is a fixture. Running the script does not start a domain.
#!/usr/bin/env bash
set -euo pipefail
root="$(mktemp -d)"
trap 'rm -rf "$root"' EXIT
field() {
local file="$1"
local key="$2"
local line
while IFS= read -r line || [[ -n "$line" ]]; do
if [[ "$line" == "${key}="* ]]; then
printf '%s\n' "${line#"${key}="}"
return 0
fi
done < "$file"
printf 'missing %s in %s\n' "$key" "$file" >&2
return 1
}
is_uint() {
[[ "$1" =~ ^[0-9]+$ ]]
}
accept_scan() {
local scan="$1"
[[ "$scan" == "scrub repaired 0B "* ]] || return 1
[[ "$scan" == *" with 0 errors on "* ]] || return 1
[[ "$scan" != *"in progress"* ]] || return 1
[[ "$scan" != *"paused"* ]] || return 1
}
accept_note() {
local file="$1"
local prod_pool replica_pool prod_scan replica_scan
local source_vm test_vm source_storage test_storage
local production_network test_network uplink booted app
local record snapshot destroyed
prod_pool="$(field "$file" prod_pool)"
replica_pool="$(field "$file" replica_pool)"
prod_scan="$(field "$file" prod_scan)"
replica_scan="$(field "$file" replica_scan)"
source_vm="$(field "$file" source_vm)"
test_vm="$(field "$file" test_vm)"
source_storage="$(field "$file" source_storage)"
test_storage="$(field "$file" test_storage)"
production_network="$(field "$file" production_network)"
test_network="$(field "$file" test_network)"
uplink="$(field "$file" uplink)"
booted="$(field "$file" booted)"
app="$(field "$file" application_started)"
record="$(field "$file" record_epoch)"
snapshot="$(field "$file" snapshot_epoch)"
destroyed="$(field "$file" test_vm_destroyed)"
[[ -n "$prod_pool" && "$prod_pool" != */* ]] || return 1
[[ -n "$replica_pool" && "$replica_pool" != */* ]] || return 1
[[ "$prod_pool" != "$replica_pool" ]] || return 1
accept_scan "$prod_scan" || return 1
accept_scan "$replica_scan" || return 1
[[ "$booted" == "yes" ]] || return 1
[[ "$app" == "yes" ]] || return 1
[[ "$uplink" == "no" ]] || return 1
[[ "$destroyed" == "yes" ]] || return 1
[[ -n "$test_vm" && "$test_vm" != "$source_vm" ]] || return 1
[[ -n "$test_storage" && "$test_storage" != "$source_storage" ]] || return 1
[[ -n "$test_network" && "$test_network" != "$production_network" ]] || return 1
is_uint "$record" || return 1
is_uint "$snapshot" || return 1
[[ "$((10#$record))" -lt "$((10#$snapshot))" ]] || return 1
}
write_note() {
local dest="$1"
local prod_scan="$2"
local replica_scan="$3"
local test_vm="$4"
local test_storage="$5"
local test_network="$6"
local uplink="$7"
local booted="$8"
local app="$9"
local record="${10}"
local snapshot="${11}"
local destroyed="${12}"
cat > "$dest" <<EOF
prod_pool=tank
replica_pool=backup
prod_scan=${prod_scan}
replica_scan=${replica_scan}
source_vm=db-01
test_vm=${test_vm}
source_storage=tank
test_storage=${test_storage}
production_network=prod-app
test_network=${test_network}
uplink=${uplink}
booted=${booted}
application_started=${app}
record_epoch=${record}
snapshot_epoch=${snapshot}
test_vm_destroyed=${destroyed}
EOF
}
must_fail() {
local label="$1"
shift
if "$@"; then
printf 'checker accepted %s\n' "$label" >&2
exit 1
fi
}
clean_scan="scrub repaired 0B in 00:12:34 with 0 errors on Thu Nov 12 02:00:00 2026"
running_scan="scrub in progress since Thu Nov 12 01:00:00 2026"
repaired_scan="scrub repaired 64K in 00:12:34 with 0 errors on Thu Nov 12 02:00:00 2026"
good="${root}/good.note"
scrub_only="${root}/scrub-only.note"
replica_running="${root}/replica-running.note"
repaired="${root}/repaired.note"
uplink_yes="${root}/uplink.note"
same_net="${root}/same-net.note"
late="${root}/late-record.note"
same_name="${root}/same-name.note"
same_disk="${root}/same-storage.note"
kept="${root}/vm-kept.note"
write_note "$good" "$clean_scan" "$clean_scan" db-01-restore-test recovery-pool restore-isolated no yes yes 1712610000 1712700000 yes
write_note "$scrub_only" "$clean_scan" "$clean_scan" db-01-restore-test recovery-pool restore-isolated no no no 1712610000 1712700000 yes
write_note "$replica_running" "$clean_scan" "$running_scan" db-01-restore-test recovery-pool restore-isolated no yes yes 1712610000 1712700000 yes
write_note "$repaired" "$repaired_scan" "$clean_scan" db-01-restore-test recovery-pool restore-isolated no yes yes 1712610000 1712700000 yes
write_note "$uplink_yes" "$clean_scan" "$clean_scan" db-01-restore-test recovery-pool restore-isolated yes yes yes 1712610000 1712700000 yes
write_note "$same_net" "$clean_scan" "$clean_scan" db-01-restore-test recovery-pool prod-app no yes yes 1712610000 1712700000 yes
write_note "$late" "$clean_scan" "$clean_scan" db-01-restore-test recovery-pool restore-isolated no yes yes 1712700000 1712700000 yes
write_note "$same_name" "$clean_scan" "$clean_scan" db-01 recovery-pool restore-isolated no yes yes 1712610000 1712700000 yes
write_note "$same_disk" "$clean_scan" "$clean_scan" db-01-restore-test tank restore-isolated no yes yes 1712610000 1712700000 yes
write_note "$kept" "$clean_scan" "$clean_scan" db-01-restore-test recovery-pool restore-isolated no yes yes 1712610000 1712700000 no
accept_note "$good"
for bad in "$scrub_only" "$replica_running" "$repaired" "$uplink_yes" "$same_net" "$late" "$same_name" "$same_disk" "$kept"; do
must_fail "$(basename "$bad")" accept_note "$bad"
done
printf 'Scrub and boot note check passed.\n'
printf 'Proves: the good fixture has a clean finished scrub on two pools, a distinct guest and storage and network, no uplink, a boot, an application, an older record, and a destroyed test VM.\n'
printf 'Does not prove: zpool scrub ran, a domain was defined, or a guest booted on an isolated bridge.\n'Then boot the guest. A scan line can describe a scrub that never happened. These checks do not prove a restore.