0021: Back up databases as logical dumps
On this page
Status: accepted
Context
People want their local databases backed up daily, kept for a few days, and restored one database at a time. Copying a running server's data folder isn't consistent unless the server stops, and it restores a whole server, not one database. Short Leash already installs each server's own client tools beside it.
Decision
- Format. Each database is backed up on its own with the server's tools from Short Leash's install, over TCP on 127.0.0.1 as
root:- MySQL:
mysqldump --single-transaction --routines --triggers --events --hex-blob --no-tablespaces, gzipped to.sql.gz. - PostgreSQL:
pg_dump -Fc(already compressed), as.dump.
- MySQL:
- Layout.
<folder>/<service>/<database>/<YYYY-MM-DD_HHMMSS>-<reason>.<ext>, with the time in UTC. The folder is the list: there's no index to fall out of step. A backup is written under a hidden temporary name and renamed when complete. Folders are0700. - Schedule. Daily at a local time (03:00 by default), checked every minute. A run missed while the computer slept or Short Leash wasn't running happens at the next check. A service that's off is skipped, never started, and is backed up once it runs.
- Keeping. Backups older than the keep setting (7 days by default) are deleted after each full run. A database's newest backup is always kept.
- Restore. Replacing a database first takes a "Before restore" backup of it, then drops and recreates it and loads the backup. Restoring into a new name leaves the original alone and refuses a name that's taken.
- Scope. MySQL and PostgreSQL only. Valkey, Memcached, Meilisearch and RustFS hold caches, indexes and files rather than app data, so they aren't backed up.
Consequences
- Backups are portable, open in any tool, and restore one database without touching the rest.
- Large databases take longer to back up and restore than copying files would.
- A PostgreSQL backup from a newer major version may not restore into an older server;
pg_restoresays so. - Backups hold your data in plain form. Moving the folder somewhere that syncs is the person's choice.