How to monitor backups (restic, borg, pg_dump)

Monitor backups with a heartbeat that pings only when the backup command exits successfully, plus checks on the result: the newest backup's age and size, a periodic repository check (restic check, borg check), and a scheduled restore drill that pings its own heartbeat. A backup you have never restored is a hope, not a backup.

Updated 3 October 2026 · by PuzariCloud · commands checked against current releases

What can go wrong with backups (restic, borg, pg_dump)

The job stopped running. A changed crontab, a moved server or an expired cloud credential, and the last good backup quietly gets older every day.
pg_dump fails but the pipeline succeeds. In pg_dump app | gzip > app.sql.gz the exit status is gzip's. Without set -o pipefail a failed dump leaves a tiny, valid gzip file and a green job.
Repository locked or full. restic leaves a stale lock after a killed run (restic unlock clears it); borg refuses to run while a lock is held; the destination runs out of space.
Corrupt or partial data. Bit rot, an interrupted upload or a bug: you only find out when restic check, borg check or a restore runs.
Never restored. Missing tables, wrong encryption key, no access to the bucket from a new machine. Restore problems are found during the outage unless you rehearse.

The signals that matter, with alert thresholds

Signal	Alert when	Why it matters
Success ping	late by more than the grace period	No good backup today.
Exit code	non-zero	The tool reported failure.
Age of newest backup	> 26 h for a daily backup	Independent of the job's own reporting.
Size of newest backup	below a floor, or < 50% of the previous	Empty or truncated dump.
Repository check	non-zero, weekly	Corruption.
Restore drill	late or failed, weekly or monthly	Recoverability, the thing that matters.
Free space on the destination	≥ 85% used	Next run fails.

Thresholds are starting points. Tune them after a week of data: an alert nobody acts on should be raised or removed.

How to check backups (restic, borg, pg_dump) by hand

restic: latest snapshot and an integrity check

restic snapshots --latest 1
restic check

borg: latest archive and a check

borg list --last 1 /srv/borg/repo
borg check /srv/borg/repo

pg_dump custom format: is the archive readable?

pg_restore --list /backups/app.dump | head

gzip'd SQL dump: is the file intact?

gzip -t /backups/app.sql.gz && echo ok

Age and size of the newest file

ls -lt --time-style=+%F_%T /backups | head -3

How to watch backups (restic, borg, pg_dump) with Vigil

In New monitor create a Heartbeat per backup job with its cron schedule and timezone. Use set -euo pipefail in the script, ping https://puzaricloud.in/ping/<token>/start at the beginning and https://puzaricloud.in/ping/<token>/$? at the end, and set a maximum run time.
POST the tail of the log with the final ping (tail -50 backup.log | curl -fsS -m 10 --data-binary @- https://puzaricloud.in/ping/<token>/$?) so the failure reason is in the alert.
Add the host under Servers and run the one-line installer it shows (curl -fsSL https://puzaricloud.in/install-server-agent.sh | sh -s -- --token <token>). The agent is one Python file with no dependencies; CPU, memory, disk, inode and load rules start working with the first report. Add the age and size commands below with limits (backup_age_s = 93600 for a daily job, backup_too_small = 1). These check the result, independently of what the script reports.
Schedule restic check or borg check weekly, and a restore drill that restores into a scratch database and runs a sanity query, each with its own heartbeat.
This is how Vigil watches its own 6-hourly database backups: a heartbeat per run and a weekly restore drill on Sundays.

/etc/vigil-agent.json (add to the existing keys)

{
  "commands": {
    "backup_age_s": "echo $(( $(date +%s) - $(stat -c %Y /backups/latest.dump) ))",
    "backup_too_small": "[ $(stat -c %s /backups/latest.dump) -lt 1000000 ] && echo 1 || echo 0"
  }
}

Every alert goes to the channels you pick: email, Slack, Telegram, Discord, Microsoft Teams, Pushover, webhooks or on-call paging. Failures are confirmed by a second check before anyone is woken up.

backups (restic, borg, pg_dump) monitoring FAQ

Isn't the exit code enough?

It tells you the tool thinks it succeeded. Age and size checks catch the job that stopped running, the pipeline that hid an error, and the dump of an empty database after a bad migration.

How often should I test a restore?

Weekly for anything you could not rebuild, monthly at the least. Automate it and give it its own heartbeat; a drill that has to be remembered stops happening.

Should the backup alert go to the same channel as uptime alerts?

Usually to the same people, but a missed backup is rarely urgent at 3 AM. Many teams route backup heartbeats to email or Slack and keep phone push for outages.