How to monitor MySQL and MariaDB
Monitor MySQL or MariaDB by checking that it answers (mysqladmin ping or a TCP check on 3306), then watching Threads_connected against max_connections, whether both replica threads are running, replica lag, the InnoDB buffer pool hit rate and free disk. Vigil's agent reads all of it through the mysql client and alerts on thresholds.
What can go wrong with MySQL and MariaDB
- Too many connections. Clients get
ERROR 1040 (HY000): Too many connectionsonceThreads_connectedreachesmax_connections(151 by default). Sleeping connections from an application pool with no idle timeout are the usual cause. - Replication stops on an error. A duplicate key (error 1062) or a missing row stops the SQL thread.
Seconds_Behind_Sourcethen showsNULL, not a big number, so a lag-only alert stays silent while the replica falls further behind. - Binary logs fill the disk. Binlogs are kept for
binlog_expire_logs_seconds(30 days by default in MySQL 8.0). A write-heavy week or a replica that stopped reading can fill the data volume, and MySQL stops accepting writes. - Purge lag from long transactions. An open transaction stops InnoDB purging old row versions. The history list length climbs into the millions and every query gets slower.
- Buffer pool too small. When the working set no longer fits in
innodb_buffer_pool_size, reads go to disk and latency jumps without any error.
The signals that matter, with alert thresholds
| Signal | Alert when | Why it matters |
|---|---|---|
| mysqladmin ping | fails twice in a row | Users are affected now. |
| Threads_connected / max_connections | ≥ 80% warn, ≥ 90% page | Room to react before ERROR 1040. |
| Replica IO and SQL threads | either one = No | The only reliable sign of stopped replication. |
| Seconds_Behind_Source | > 30 s | Stale reads and data loss on failover. |
| Buffer pool hit rate | < 99% on OLTP; alert < 95% sustained | Working set spills to disk. |
| History list length | > 1,000,000 | Purge is blocked by a long transaction. |
| Slow queries per interval | a sudden step up | Usually a missing index after a deploy. |
| Disk used on the data volume | ≥ 85% | Binlogs, temp tables and the redo log need headroom. |
Thresholds are starting points. Tune them after a week of data: an alert nobody acts on should be raised or removed.
How to check MySQL and MariaDB by hand
mysqladmin ping mysqladmin status
mysql -e "SHOW GLOBAL STATUS LIKE 'Threads_connected'; SHOW VARIABLES LIKE 'max_connections'"
mysql -e "SHOW REPLICA STATUS\G" | grep -E 'Replica_IO_Running|Replica_SQL_Running|Seconds_Behind_Source|Last_SQL_Error'
mysql -e "SHOW ENGINE INNODB STATUS\G" | grep -i 'history list length'
mysql -e "SELECT trx_id, trx_started, trx_mysql_thread_id FROM information_schema.innodb_trx ORDER BY trx_started LIMIT 5"
How to watch MySQL and MariaDB with Vigil
- Add the host under Servers and run the one-line installer it shows (
curl -fsSL https://puzaricloud.in/install-server-agent.sh | sh -s -- --token <token>). The agent is one Python file with no dependencies; CPU, memory, disk, inode and load rules start working with the first report. - Create a monitoring user:
CREATE USER 'vigil'@'localhost' IDENTIFIED BY '…'; GRANT PROCESS, REPLICATION CLIENT ON *.* TO 'vigil'@'localhost'; GRANT SELECT ON performance_schema.* TO 'vigil'@'localhost';and add it underdatabasesin/etc/vigil-agent.json(below). The password is passed inMYSQL_PWD, never on the command line. Test withsudo vigil-server-agent --db-test. - The agent reads
SHOW GLOBAL STATUS, replica status, buffer pool hit rate, slow queries and deadlocks per interval; with performance_schema on, the Databases tab lists the top statements by total time. Default rules: 90% of max connections, 90% hit rate, 30 s lag. - Add the replica-threads command from the example with the limit
replica_threads_stopped = 1, so a stopped SQL thread pages even when lag shows NULL. - Add
mysql(ormariadb) to the systemd services to watch, and, if the port is reachable from outside, a TCP port monitor on 3306 from New monitor.
{
"databases": [
{"name": "main", "dsn": "mysql://vigil:secret@127.0.0.1:3306/app"}
],
"commands": {
"replica_threads_stopped": "mysql -N -e \"SHOW REPLICA STATUS\\G\" | grep -c 'Running: No' || true"
}
}
Every alert goes to the channels you pick: email, Slack, Telegram, Discord, Microsoft Teams, Pushover, webhooks or on-call paging. Failures are confirmed by a second check before anyone is woken up.
MySQL and MariaDB monitoring FAQ
Why does Seconds_Behind_Source show NULL?
Because the SQL thread (or the IO thread) is not running, usually after a replication error shown in Last_SQL_Error. NULL means "unknown", not "zero", so always alert on the thread state as well as on lag.
Does the same setup work for MariaDB?
Yes. The agent accepts mariadb:// as well as mysql:// and falls back to SHOW SLAVE STATUS on versions without SHOW REPLICA STATUS.
What privileges does monitoring need?
PROCESS for InnoDB status and other sessions, REPLICATION CLIENT for replica status, and SELECT on performance_schema if you want the top-statements table. No write privileges.