How to monitor Sidekiq
Monitor Sidekiq by watching queue size per queue, the size of the retry and dead sets, whether the Sidekiq processes are running, and whether scheduled jobs complete. All of it is in Redis, so redis-cli shows it by hand. Vigil reads the same keys with agent commands, heartbeats scheduled jobs and collects exceptions through the Sentry SDK.
What can go wrong with Sidekiq
- Backlog. A slow dependency or a burst of jobs makes queues grow; users wait for emails, webhooks and exports.
- Retries hide failures. A failing job is retried 25 times with exponential back-off over about three weeks by default. Nothing looks broken while the retry set grows.
- Dead set fills up. After the last retry a job moves to the dead set, which keeps at most 10,000 jobs for 6 months by default, then drops them for good.
- Processes stopped. A deploy that did not restart Sidekiq, or an OOM kill, and nothing consumes the queues.
- Redis trouble. Sidekiq is only as healthy as its Redis: memory full with
noevictionmeans enqueues fail; an eviction policy means jobs vanish.
The signals that matter, with alert thresholds
| Signal | Alert when | Why it matters |
|---|---|---|
| Queue size per queue | > a few minutes of throughput | Backlog. |
| Retry set size | rising steadily | A job class is failing. |
| Dead set size | any increase | Jobs have given up. |
| Sidekiq processes | fewer than expected | No consumers. |
| Scheduled job success ping | late | Cron-style job not running. |
| Redis memory | ≥ 80% of maxmemory | Enqueue failures next. |
Thresholds are starting points. Tune them after a week of data: an alert nobody acts on should be raised or removed.
How to check Sidekiq by hand
redis-cli smembers queues redis-cli llen queue:default
redis-cli zcard retry redis-cli zcard dead redis-cli zcard schedule
ps aux | grep '[s]idekiq'
bin/rails runner 'puts Sidekiq::Queue.new("default").latency'
If Redis uses a namespace or a non-default database, add -n <db> and the prefix to the key names.
How to watch Sidekiq with Vigil
- Add the host under Servers and run the one-line installer it shows (
curl -fsSL https://puzaricloud.in/install-server-agent.sh | sh -s -- --token <token>). The agent is one Python file with no dependencies; CPU, memory, disk, inode and load rules start working with the first report. Add the Sidekiq systemd unit to the services to watch, andsidekiqto the processes that must be running. - Add the queue commands below with limits, e.g.
sidekiq_default = 1000,sidekiq_retry = 200,sidekiq_dead = 1(alert on the first dead job, raise it later if needed). - Add the Redis instance as a database to watch memory and evictions; set its hit-rate minimum to 0, since a job queue is not a cache.
- For sidekiq-cron or sidekiq-scheduler jobs that matter, create a Heartbeat in New monitor with the same cron expression and ping it at the end of
perform. - Create a project under Errors and use its DSN with
sentry-ruby+sentry-sidekiq; failures arrive grouped by job class with arguments and the retry count.
{
"commands": {
"sidekiq_default": "redis-cli llen queue:default",
"sidekiq_retry": "redis-cli zcard retry",
"sidekiq_dead": "redis-cli zcard dead"
}
}
Every alert goes to the channels you pick: email, Slack, Telegram, Discord, Microsoft Teams, Pushover, webhooks or on-call paging. Failures are confirmed by a second check before anyone is woken up.
Sidekiq monitoring FAQ
Is the Sidekiq Web UI enough?
It shows the same numbers, but only when someone looks. Alerts on the retry and dead sets catch a failing job class on the first day instead of when a customer asks.
Why alert on the first dead job?
A dead job has failed every retry for weeks; it is real work that did not happen. If a job class is expected to die sometimes, raise the limit for that set.
Do I need Sidekiq Pro or Enterprise?
No. Everything here reads standard Redis keys that open-source Sidekiq uses.