Notify about backup with mail

README.md

@@ -1,38 +1,23 @@
 # backup-agent
 
-Backup and disaster-recovery tooling for Linux servers: [Borg](https://borgbackup.readthedocs.io/)
+Backup and disaster-recovery tooling for a Linux server: [Borg](https://borgbackup.readthedocs.io/)
-backing up a target directory tree — including a MariaDB database running in
+backing up `/home/srv/files/content` — including a MariaDB database
-Docker — with an offsite mirror on Scaleway S3 via `rclone`.
+running in Docker — with an offsite mirror on Scaleway S3 via `rclone`.
 
 MariaDB is never stopped during backup: `dump_db.sh` takes a
 transactionally-consistent logical dump (`mysqldump --single-transaction`)
-while the container keeps running, and the container's raw data directory is
+while the container keeps running, and the container's raw data directory
-excluded from the archive entirely (a `.nobackup` marker file), so only the
+is excluded from the archive entirely (a `.nobackup` marker file), so only
-logical dump ever gets backed up. Zero DB downtime.
+the logical dump ever gets backed up. Zero DB downtime.
 
 ## Components
 
 | File | Purpose |
 |---|---|
-| `configs/` | Per-deployment config files (one per server). Each sets `TARGET`, `REPO`, `DB_CONTAINER`, etc. |
 | `borg-backup.sh` | Daily backup orchestrator (run from cron): dump → archive → prune → compact → integrity check → offsite sync. |
 | `dump_db.sh` | Per-database `mysqldump`/`mariadb-dump`, atomic staging/swap. Invoked by `borg-backup.sh`. |
 | `restore.sh` | Recovery CLI: `full` (disaster recovery), `db <name>` (single database), `file <path>` (single file/dir), `--list-archives`. Every mode supports `--dry-run`. |
 
-## Multi-server use
-
-Both scripts are deployment-agnostic. Point them at a config with `BACKUP_CONF`:
-
-```bash
-# backup
-BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf /opt/backup-agent/borg-backup.sh
-
-# restore
-BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf /opt/backup-agent/restore.sh --list-archives
-```
-
-See `configs/` for existing deployments and `RUNBOOK.md §2` for deployment steps.
-
 ## Quickstart
 
 ```bash
@@ -52,16 +37,16 @@ Day to day:
 
 ## Encryption
 
-The Borg repos on current deployments are **unencrypted** by deliberate operator
+The Borg repo at `/home/srv/files/backups/borg-2025` is **unencrypted** by
-choice — `borg-backup.sh` will keep printing a warning about it on every run,
+deliberate choice on this deployment — `borg-backup.sh` will keep printing
-which is expected. See `RUNBOOK.md` if you want to switch to an encrypted repo.
+a warning about it on every run, which is expected. See `RUNBOOK.md` if
+you want to switch to an encrypted repo.
 
 ## Requirements
 
-`borg`, `docker`, `rclone`, `flock`, a `mysql`/`mariadb` client. When
+`borg`, `docker`, `rclone`, `flock`, a `mysql`/`mariadb` client — see
-`DB_CONTAINER` is empty in the config, `docker` is not required. Targets
+`REQUIRED_CMDS` in `borg-backup.sh`. Targets Linux; `flock(1)` doesn't
-Linux; `flock(1)` doesn't exist on macOS, so these scripts won't run as-is
+exist on macOS, so these scripts won't run as-is on a Mac.
-on a Mac.
 
 ## License
 

RUNBOOK.md

@@ -3,131 +3,99 @@
 Covers `borg-backup.sh` (daily backup), `dump_db.sh` (MariaDB logical
 dumps, invoked by the backup script), and `restore.sh` (recovery).
 
-All deployment-specific values (`TARGET`, `REPO`, `DB_CONTAINER`, etc.) live in
+## 1. One-Time Setup
-a config file under `configs/`. Both scripts source it automatically — see
-**§2** for how to point them at the right one.
 
----
+Run once, by hand, on the server:
 
-## 1. One-Time Setup (per server)
+1. Initialize the encrypted Borg repo:
+   ```bash
+   mkdir -p /home/srv/files/backups
+   borg init --encryption=repokey-blake2 /home/srv/files/backups/borg-2025
+   ```
+2. Create the passphrase file used by both backup and restore:
+   ```bash
+   echo 'your-strong-passphrase' > /root/.borg-passphrase
+   chmod 600 /root/.borg-passphrase
+   ```
+3. Create the MariaDB `backup` user used by `dump_db.sh` (read-only, no
+   stop/lock of the server required thanks to `--single-transaction`):
+   ```sql
+   CREATE USER 'backup'@'%' IDENTIFIED BY 'a-strong-password';
+   GRANT SELECT, LOCK TABLES, SHOW VIEW, TRIGGER, PROCESS, RELOAD ON *.* TO 'backup'@'%';
+   ```
+   ```bash
+   echo 'a-strong-password' > /root/.mariadb-backup.pw
+   chmod 600 /root/.mariadb-backup.pw
+   ```
+4. Create the root password file used only by `restore.sh` (restore needs
+   CREATE/DROP privileges the `backup` user does not have):
+   ```bash
+   echo 'the-mariadb-root-password' > /root/.mariadb-root.pw
+   chmod 600 /root/.mariadb-root.pw
+   ```
+5. Exclude MariaDB's raw data directory from the archive. Find the
+   directory bind-mounted into the container as its datadir and drop a
+   marker file in it:
+   ```bash
+   touch /home/srv/files/content/mariadb/data/.nobackup
+   ```
+   This is what allows backups to run with the container up: only the
+   logical dump under `mariadb/dump/` is ever archived or restored from.
+6. Configure the `scaleway` rclone remote:
+   ```bash
+   rclone config
+   # create a remote named "scaleway", type S3, matching your Scaleway
+   # Object Storage credentials and region
+   ```
+7. Set up email notifications (optional — skip this if you don't want mail):
+   Install a mail transport agent (choose one):
+   ```bash
+   # Option A: mailutils (Debian/Ubuntu) — includes /usr/bin/mail
+   apt install -y mailutils
 
-Run once, by hand, on the server. Substitute values from your config file
+   # Option B: msmtp + mailutils (lightweight, no full MTA)
-(`configs/<deployment>.conf`). The examples below use shell variables sourced
+   apt install -y msmtp msmtp-mta mailutils
-from it:
+   ```
+   Configure the recipient address. Either set `MAIL_TO` in the environment
+   (`/etc/environment` or the cron file itself) or edit `borg-backup.sh`:
+   ```
+   MAIL_TO="${MAIL_TO:-you@example.com}"
+   ```
+   If using `msmtp`, configure `/etc/msmtprc`:
+   ```
+   # /etc/msmtprc — example for Gmail SMTP
+   defaults
+   auth           on
+   tls            on
+   tls_trust_file /etc/ssl/certs/ca-certificates.crt
 
-```bash
+   account        default
-source /opt/backup-agent/configs/nexusvoice.conf   # adapt path
+   host           smtp.gmail.com
-```
+   port           587
-
+   from           backup-sender@gmail.com
-### 1.1 Initialize the Borg repo
+   user           backup-sender@gmail.com
-
+   password       your-app-password
-```bash
+   ```
-mkdir -p "$(dirname "$REPO")"
+   The script auto-detects which command is available (`mail` → `sendmail` → `msmtp`)
-borg init --encryption=repokey-blake2 "$REPO"
+   and falls back silently if none is present — notifications are never fatal.
-# If running unencrypted by deliberate choice:
-# borg init --encryption=none "$REPO"
-```
-
-### 1.2 Passphrase file (backup and restore both read it)
-
-```bash
-echo 'your-strong-passphrase' > "$BORG_PASSPHRASE_FILE"
-chmod 600 "$BORG_PASSPHRASE_FILE"
-```
-
-Skip this step only if running without encryption and without a passphrase.
-
-### 1.3 MariaDB `backup` user (used by `dump_db.sh`)
-
-Read-only; `--single-transaction` makes the dump consistent without stopping
-the container.
-
-```sql
-CREATE USER 'backup'@'%' IDENTIFIED BY 'a-strong-password';
-GRANT SELECT, LOCK TABLES, SHOW VIEW, TRIGGER, PROCESS, RELOAD ON *.* TO 'backup'@'%';
-```
-
-```bash
-echo 'a-strong-password' > /root/.mariadb-backup.pw
-chmod 600 /root/.mariadb-backup.pw
-```
-
-### 1.4 MariaDB root password file (used only by `restore.sh`)
-
-Restore needs CREATE/DROP privileges the `backup` user does not have.
-
-```bash
-echo 'the-mariadb-root-password' > "$ROOT_PASSWORD_FILE"
-chmod 600 "$ROOT_PASSWORD_FILE"
-```
-
-### 1.5 Exclude the MariaDB raw data directory
-
-Find the host directory bind-mounted into the container as its datadir and
-drop a marker file in it. Borg skips any directory that contains `.nobackup`
-(via `--exclude-if-present`), so live InnoDB files are never read:
-
-```bash
-touch "${TARGET}/mariadb/data/.nobackup"
-```
-
-This is what allows backups to run with the container up: only the logical
-dump under `${DUMP_SUBDIR}/` is ever archived or restored.
-
-### 1.6 Configure the Scaleway rclone remote
-
-```bash
-rclone config
-# Create a remote named "scaleway", type S3, with your Scaleway
-# Object Storage credentials and region.
-```
-
----
 
 ## 2. Deploying the Scripts
 
-### Single deployment
+Copy `borg-backup.sh`, `dump_db.sh`, and `restore.sh` to `/opt/backup-agent/`
-
+(this exact path is what `borg-backup.sh` invokes for `dump_db.sh`). Make
-Copy `borg-backup.sh`, `dump_db.sh`, `restore.sh`, and the `configs/` directory
+all three executable:
-to `/opt/backup-agent/`. Symlink the relevant config as `backup.conf` beside
-the scripts, or pass `BACKUP_CONF` explicitly in the cron entry.
 
 ```bash
-chmod +x /opt/backup-agent/borg-backup.sh \
+chmod +x /opt/backup-agent/borg-backup.sh /opt/backup-agent/restore.sh
-         /opt/backup-agent/restore.sh \
+chmod +x /opt/backup-agent/dump_db.sh
-         /opt/backup-agent/dump_db.sh
-
-# Option A — symlink (scripts auto-discover backup.conf beside them):
-ln -s /opt/backup-agent/configs/srv.conf /opt/backup-agent/backup.conf
-
-# Option B — explicit BACKUP_CONF in the cron entry (see §3).
 ```
 
-### Multiple deployments on different servers
-
-Each server gets its own config. Deploy the same three scripts to
-`/opt/backup-agent/` on each server; point each cron entry at its config via
-`BACKUP_CONF`:
-
-```
-# /etc/cron.d/borg-backup  (nexusvoice server)
-BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf
-30 2 * * * root /opt/backup-agent/borg-backup.sh >> /var/log/borg/cron.log 2>&1
-```
-
-Before the first run, verify `configs/nexusvoice.conf`:
-- `TARGET` — current path; update if files move (e.g. `/home/acid/nexusvoice` → `/opt/nexusvoice`)
-- `DB_CONTAINER` — confirm the MariaDB container name on that server
-- `REPO` — must not overlap with any other deployment's repo
-
----
-
 ## 3. Scheduling
 
 Add a cron entry to run the backup daily, off-peak:
-
 ```
-# /etc/cron.d/borg-backup  (srv — default config via symlink)
+# /etc/cron.d/borg-backup
+MAIL_TO=you@example.com
 30 2 * * * root /opt/backup-agent/borg-backup.sh >> /var/log/borg/cron.log 2>&1
 ```
 
@@ -138,139 +106,128 @@ ls -lt /var/log/borg/backup-*.log | head -1   # latest log file
 tail -50 /var/log/borg/backup-*.log           # inspect it
 ```
 
----
-
 ## 4. Day-2 Operations
 
-Source your config first to get the right `BORG_REPO`, `BORG_PASSCOMMAND`, etc.:
+List archives:
 
 ```bash
-source /opt/backup-agent/configs/nexusvoice.conf   # adapt
-export BORG_REPO="$REPO"
-export BORG_PASSCOMMAND="cat $BORG_PASSPHRASE_FILE"
-```
-
-Then standard borg commands work without extra flags:
-
-```bash
-# List archives
 ./restore.sh --list-archives
-# or directly:
-borg list
-
-# Repo size and health
-borg info
-
-# Rotate the passphrase
-# Note: only re-encrypts the key, not the archive data; old passphrase
-# still needed for archives created before the change until fully migrated.
-borg key change-passphrase
-
-# Break a stale lockfile (backup or restore aborted mid-run)
-borg break-lock
 ```
 
----
+Check repo size and health:
+
+```bash
+BORG_PASSCOMMAND="cat /root/.borg-passphrase" borg info /home/srv/files/backups/borg-2025
+```
+
+Rotate the passphrase (creates a new key, re-encrypts nothing — old
+archives still need the old passphrase to read, so keep both until fully
+migrated):
+
+```bash
+BORG_PASSCOMMAND="cat /root/.borg-passphrase" borg key change-passphrase /home/srv/files/backups/borg-2025
+```
+
+Stale lockfile (backup or restore aborted mid-run and left the repo
+locked):
+
+```bash
+BORG_PASSCOMMAND="cat /root/.borg-passphrase" borg break-lock /home/srv/files/backups/borg-2025
+```
 
 ## 5. Recovery Procedures
 
-All `restore.sh` commands accept:
+All `restore.sh` commands accept `--dry-run` to preview exactly what would
-- `--dry-run` — preview exactly what would happen without touching anything
+happen without touching anything, and `--archive NAME` to target a
-- `--archive NAME` — target a specific archive instead of the latest (get names via `--list-archives`)
+specific archive instead of the latest (see archive names via
-
+`--list-archives`). Root DB credentials come from `MYSQL_ROOT_PASSWORD` in
-Root DB credentials come from `MYSQL_ROOT_PASSWORD` in the environment if set,
+the environment if set, otherwise from `/root/.mariadb-root.pw` — set
-otherwise from the `ROOT_PASSWORD_FILE` in your config (default
+whichever is more convenient for how you're invoking it. Restore logs go
-`/root/.mariadb-root.pw`). Restore logs go to `/var/log/borg/restore-*.log`;
+to `/var/log/borg/restore-*.log` (the `RESTORE_LOGDIR` environment
-`RESTORE_LOGDIR` overrides the directory (useful for testing).
+variable overrides the directory, mainly useful for testing). `full` and
-
+`db` share `borg-backup.sh`'s lockfile, so a restore refuses to start
-`restore.sh` shares `borg-backup.sh`'s lockfile (`LOCKFILE` in the config),
+while the nightly backup is mid-run (and vice versa) rather than racing
-so a restore refuses to start while the nightly backup is mid-run and vice
+it.
-versa.
-
-For all `restore.sh` commands, select the deployment with `BACKUP_CONF`:
-
-```bash
-export BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf
-```
-
-Or run without it if `backup.conf` is already symlinked beside the script.
 
 ### 5.1 Full disaster recovery (new or wiped server)
 
-Use when the whole server/container is gone and you're rebuilding from scratch.
+Use when the whole server/container is gone and you're rebuilding from
+scratch.
 
 ```bash
 # 1. Reinstall borg, docker, and the mariadb container image/compose file
-#    (not covered by restore.sh — this is infra provisioning).
+#    (not covered by restore.sh - this is infra provisioning).
 # 2. Restore the passphrase file (from your password manager / secondary
-#    backup — it is NOT stored in the repo it protects) to the path set in
+#    backup - it is NOT stored in the repo it protects) to
-#    BORG_PASSPHRASE_FILE, and the root DB password to ROOT_PASSWORD_FILE.
+#    /root/.borg-passphrase, and the root DB password to
+#    /root/.mariadb-root.pw.
 # 3. Preview:
 ./restore.sh full --dry-run
-# 4. Run for real. --force is only required if $TARGET already has data in it
+# 4. Run for real. --force is only required if /home/srv/files/content
-#    (e.g. a stale mount); omit it on a genuinely empty/fresh server:
+#    already has data in it (e.g. a stale mount); omit it on a genuinely
+#    empty/fresh server:
 ./restore.sh full --force
 ```
 
-This extracts the full content tree, starts the DB container and waits for it
+This extracts the full content tree from the archive, starts the
-to report healthy, then restores every database dump (users/grants first).
+`mariadb` container and waits for it to report healthy, then restores
+every database dump (users/grants first) — the container must be running
+before any of the dump restores, which is why it starts first.
 
 **Verify afterward:**
-- `docker ps` shows the DB container running and healthy.
+- `docker ps` shows `mariadb` running and healthy.
 - The application responds normally.
 - Spot-check row counts on a couple of tables against what you'd expect.
 
 ### 5.2 Single database restore
 
-Use when one database got corrupted or someone ran a bad migration/query —
+Use when one database got corrupted or someone ran a bad migration/query
-this **drops and recreates** that database.
+against it — this **drops and recreates** that database.
 
 ```bash
 ./restore.sh db shopdb --dry-run     # preview
-./restore.sh db shopdb               # prompts: type "shopdb" to confirm
+./restore.sh db shopdb                # prompts: type "shopdb" to confirm
 ```
 
 Non-interactive (e.g. scripted from a monitoring alert): add `--yes` to
 skip the typed confirmation.
 
-**Verify afterward:** connect to the database and check the tables/row counts
+**Verify afterward:** connect to the database and check the tables/row
-you expect.
+counts you expect.
 
 ### 5.3 Single file/directory restore
 
-Use for accidental deletion or to inspect an old version — never touches the
+Use for accidental deletion of a file, or to inspect an old version — this
-running database or container.
+never touches the running database or container.
 
 ```bash
-./restore.sh file path/relative/to/target/some-file.txt --dest /tmp/recovered
+./restore.sh file path/relative/to/content/some-file.txt --dest /tmp/recovered
 ```
 
-The final location of the recovered item is printed at the end. Borg
+The final location of the recovered item is printed at the end (it lands
-preserves the absolute path it was archived with, so the file lands under
+under `/tmp/recovered/home/srv/files/content/...` — Borg preserves the
-`/tmp/recovered/<TARGET>/...`.
+absolute path it was archived with).
-
----
 
 ## 6. Restore Drill Cadence
 
-Quarterly, run a real `full` extract into a scratch directory to confirm
+Quarterly, run a real `full` restore into a scratch directory (not
-backups are actually usable.
+`/home/srv/files/content`) to confirm backups are actually usable:
 
-`borg extract` always extracts into the current directory; the archive name
+`borg extract` always extracts into the current directory (there's no
-must come from `borg list --short` (plain `borg list` prints a formatted
+`--destination` flag — this is why `restore.sh` itself `cd`s into the
-line, not a bare name). Source your config to get the right values:
+destination before extracting), and the archive name must come from
+`borg list --short` (plain `borg list` prints a formatted line, not a bare
+name), so:
 
 ```bash
-source /opt/backup-agent/configs/nexusvoice.conf   # adapt
-export BORG_REPO="$REPO"
-export BORG_PASSCOMMAND="cat $BORG_PASSPHRASE_FILE"
-
 mkdir -p /tmp/restore-drill && cd /tmp/restore-drill
+export BORG_REPO=/home/srv/files/backups/borg-2025
+export BORG_PASSCOMMAND="cat /root/.borg-passphrase"
 LATEST=$(borg list --short | tail -1)
 borg extract --lock-wait 600 "::$LATEST"
 ```
 
-Confirm the dump files under `${DUMP_SUBDIR}/` are present, non-empty, and
+Confirm the dump files under `mariadb/dump/` are present, non-empty, and
-importable (`mysql -u root -p < mariadb/dump/somedb.sql` against a throwaway
+importable (`mysql -u root -p < mariadb/dump/somedb.sql` against a
-MariaDB container). Log the drill date and outcome somewhere durable (e.g. a
+throwaway MariaDB container). Log the drill date and outcome somewhere
-team wiki page).
+durable (e.g. a team wiki page).

borg-backup.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 # =============================================================================
-# Borg backup: $TARGET + MariaDB  ->  local borg repo  ->  Scaleway S3 mirror
+# Borg backup: /content + MariaDB  ->  local repo  ->  Scaleway S3 mirror
 # =============================================================================
 # CHANGES vs. original:
 #   • Encryption enabled (repokey-blake2 via BORG_PASSCOMMAND)
@@ -12,7 +12,6 @@
 #     excluded from the archive (drop a .nobackup file in it - see
 #     --exclude-if-present below) so no live InnoDB file is ever copied.
 #     Zero DB downtime during backup.
-#   • Config split into per-deployment backup.conf (see configs/)
 # =============================================================================
 
 set -euo pipefail
@@ -21,31 +20,48 @@ export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 umask 077
 
 # ========================= CONFIGURATION =========================
-# All deployment-specific constants live in a config file.
-# Default: backup.conf next to this script.
-# Override: BACKUP_CONF=/path/to/other.conf borg-backup.sh
-# See configs/ for per-deployment examples.
 
-_SCRIPT_DIR="$(cd "$(dirname "$(readlink -f "$0")")" && pwd)"
+NAME="borg-2025"
-BACKUP_CONF="${BACKUP_CONF:-${_SCRIPT_DIR}/backup.conf}"
+REPO="/home/srv/files/backups/$NAME"
-[[ -r "$BACKUP_CONF" ]] || { echo "ERROR: config not found: $BACKUP_CONF"; exit 1; }
+TARGET="/home/srv/files/content"
-# shellcheck source=/dev/null
-source "$BACKUP_CONF"
 
-# Derived values (override in config only if you need a non-standard layout)
+# dump_db.sh lives with the rest of this toolkit, not inside $TARGET - its
-# dump_db.sh lives with this toolkit; DUMP_DIR is passed explicitly so dumps
+# own default dump dir is relative to wherever IT lives, so DUMP_DIR must be
-# land inside $TARGET where borg can see them.
+# passed explicitly (below) to keep dumps inside $TARGET where borg can see them.
-export DUMP_DIR="${DUMP_DIR:-${TARGET}/${DUMP_SUBDIR}}"
+DUMP_SCRIPT="/opt/backup-agent/dump_db.sh"
+export DUMP_DIR="${TARGET}/mariadb/dump"
 
-# docker is only required when DB_CONTAINER is set
+REPO_MOUNT=""
-if [[ -n "${DB_CONTAINER:-}" ]]; then
+
-    REQUIRED_CMDS=(borg rclone docker timeout flock date find)
+LOGDIR="/var/log/borg"
-else
+LOG_RETENTION_DAYS=90
-    REQUIRED_CMDS=(borg rclone timeout flock date find)
+
-fi
+RCLONE_REMOTE="scaleway"
+RCLONE_PATH="par-backup-1/$NAME"
+RCLONE_MAX_DELETE=200
+
+LOCKFILE="/var/lock/borg-backup.lock"
+
+DB_CONTAINER="mariadb"
+DB_START_TIMEOUT=180
+
+CREATE_TIMEOUT="6h"
+PRUNE_TIMEOUT="2h"
+SYNC_TIMEOUT="12h"
+CHECK_TIMEOUT="4h"
+
+# Passphrase file: chmod 600, owned by the backup user.
+# Create it with:  echo 'your-strong-passphrase' > /root/.borg-passphrase
+BORG_PASSPHRASE_FILE="${BORG_PASSPHRASE_FILE:-/root/.borg-passphrase}"
+
+REQUIRED_CMDS=(borg rclone docker timeout flock date find)
 
 # =================================================================
 
+# Email notification (set to your address to receive backup summary).
+# Leave empty to skip notifications. The script auto-detects mail/sendmail/msmtp.
+MAIL_TO="${MAIL_TO:-}"
+
 SELF="$(readlink -f "$0")"
 mkdir -p "$LOGDIR"
 LOGFILE="${BORG_BACKUP_LOGFILE:-$LOGDIR/backup-$(date +%Y-%m-%d-%H%M%S).log}"
@@ -77,6 +93,68 @@ run_cmd() {
     "$@"
 }
 
+# Detect available mail transport. Returns the command name or empty string.
+find_mail_cmd() {
+    for cmd in mail sendmail msmtp; do
+        command -v "$cmd" >/dev/null 2>&1 && { echo "$cmd"; return 0; }
+    done
+    return 1
+}
+
+# Send an email notification about the backup result.
+# Called from cleanup() so the log file is complete.
+# Non-fatal — mail failures do not affect the backup exit code.
+send_notification() {
+    local exit_code=$1 status started ended
+    [[ -n "$MAIL_TO" ]] || return 0
+
+    local mail_cmd
+    mail_cmd="$(find_mail_cmd)" || { log "Notification skipped: no mail command found"; return 0; }
+
+    if [ "$exit_code" -eq 0 ]; then
+        status="SUCCESS"
+    else
+        status="FAILED (exit $exit_code)"
+    fi
+
+    started="$(head -1 "$LOGFILE" 2>/dev/null | sed -n 's/^\[\(.*\)\].*/\1/p' || echo "?")"
+    ended="$(date '+%F %T')"
+
+    # Build message body with headers and log tail.
+    {
+        echo "Subject: Backup $status - $(hostname -s)"
+        echo "To: $MAIL_TO"
+        echo ""
+        echo "===== Backup Summary ====="
+        echo "Host:    $(hostname -f)"
+        echo "Archive: $ARCHIVE"
+        echo "Status:  $status"
+        echo "Started: $started"
+        echo "Ended:   $ended"
+        echo "Log:     $LOGFILE"
+        echo ""
+        echo "----- Last 80 lines of backup log -----"
+        tail -80 "$LOGFILE" 2>/dev/null || echo "(log unavailable)"
+    } | case "$mail_cmd" in
+        mail)
+            mail -s "Backup $status - $(hostname -s)" "$MAIL_TO"
+            ;;
+        sendmail)
+            sendmail "$MAIL_TO"
+            ;;
+        msmtp)
+            msmtp "$MAIL_TO"
+            ;;
+    esac
+
+    local rc=$?
+    if [ "$rc" -eq 0 ]; then
+        log "Notification sent to $MAIL_TO (via $mail_cmd)"
+    else
+        log "WARNING: notification via $mail_cmd failed (exit $rc)"
+    fi
+}
+
 container_running() {
     [[ "$(docker inspect -f '{{.State.Running}}' "$1" 2>/dev/null || echo false)" == "true" ]]
 }
@@ -116,7 +194,7 @@ cleanup() {
 
     # We never stop the container ourselves anymore, but if it crashed for
     # an unrelated reason during the backup window, try to bring it back.
-    if [[ -n "${DB_CONTAINER:-}" ]] && ! container_running "$DB_CONTAINER"; then
+    if ! container_running "$DB_CONTAINER"; then
         step "Cleanup: $DB_CONTAINER is down, attempting restart"
         start_db || { log "CRITICAL: $DB_CONTAINER is DOWN - manual action required"
                       [ "$exit_code" -eq 0 ] && exit_code=1; }
@@ -131,6 +209,7 @@ cleanup() {
         log "=== Backup FAILED (exit $exit_code) ==="
     fi
     log "Full log: $LOGFILE"
+    send_notification "$exit_code"
 }
 
 # Returns 0 if the repo is encrypted, 1 if not.
@@ -180,10 +259,8 @@ preflight() {
         return 1
     fi
 
-    if [[ -n "${DB_CONTAINER:-}" ]]; then
+    container_running "$DB_CONTAINER" \
-        container_running "$DB_CONTAINER" \
+        || { log "ERROR: $DB_CONTAINER is not running before we start"; return 1; }
-            || { log "ERROR: $DB_CONTAINER is not running before we start"; return 1; }
-    fi
 
     log "Preflight OK"
 }
@@ -214,24 +291,20 @@ preflight
 # --- 1. Logical dumps (container stays up the whole time) ---------------
 step "Step 1: MariaDB dumps"
 
-if [[ -n "${DB_CONTAINER:-}" ]]; then
+if [[ ! -x "$DUMP_SCRIPT" ]]; then
-    if [[ ! -x "$DUMP_SCRIPT" ]]; then
+    log "ERROR: dump script missing or not executable: $DUMP_SCRIPT"
-        log "ERROR: dump script missing or not executable: $DUMP_SCRIPT"
+    exit 1
-        exit 1
+fi
-    fi
+run_cmd "$DUMP_SCRIPT"
-    run_cmd "$DUMP_SCRIPT"
 
-    if [[ -d "$DUMP_DIR" ]]; then
+if [[ -d "$DUMP_DIR" ]]; then
-        fresh=$(find "$DUMP_DIR" -type f -size +1k -mmin -60 | wc -l)
+    fresh=$(find "$DUMP_DIR" -type f -size +1k -mmin -60 | wc -l)
-        empty=$(find "$DUMP_DIR" -type f -size -1k -mmin -60 | wc -l)
+    empty=$(find "$DUMP_DIR" -type f -size -1k -mmin -60 | wc -l)
-        log "Dumps: $fresh fresh non-trivial file(s), $empty suspiciously small"
+    log "Dumps: $fresh fresh non-trivial file(s), $empty suspiciously small"
-        (( fresh > 0 )) || { log "ERROR: no usable dumps produced"; exit 1; }
+    (( fresh > 0 )) || { log "ERROR: no usable dumps produced"; exit 1; }
-        (( empty == 0 )) || log "WARNING: $empty near-empty dump file(s) - check $DUMP_DIR"
+    (( empty == 0 )) || log "WARNING: $empty near-empty dump file(s) - check $DUMP_DIR"
-    else
-        log "WARNING: dump directory not found: $DUMP_DIR"
-    fi
 else
-    log "DB_CONTAINER not set - skipping database dump"
+    log "WARNING: dump directory not found: $DUMP_DIR"
 fi
 
 # --- 2. Create the archive -----------------------------------------------
@@ -242,10 +315,8 @@ fi
 # are excluded too - they churn on every request, so backing them up adds
 # noise and dedup overhead for no recovery value.
 step "Step 2: Creating archive $ARCHIVE"
-if [[ -n "${DB_CONTAINER:-}" ]]; then
+container_running "$DB_CONTAINER" \
-    container_running "$DB_CONTAINER" \
+    || { log "ERROR: $DB_CONTAINER is not running - refusing to archive"; exit 1; }
-        || { log "ERROR: $DB_CONTAINER is not running - refusing to archive"; exit 1; }
-fi
 run_cmd timeout --signal=INT --kill-after=120s "$CREATE_TIMEOUT" \
     borg create \
         --lock-wait 600 \

configs/chaudron.conf

@@ -1,39 +0,0 @@
-# =============================================================================
-# Deployment config: srv (primary server — /home/srv/files/content)
-# Used by: borg-backup.sh, restore.sh
-# Deploy: BACKUP_CONF=/opt/backup-agent/configs/srv.conf /opt/backup-agent/borg-backup.sh
-#         (or symlink configs/srv.conf -> ../backup.conf next to the scripts)
-# =============================================================================
-
-# === Repo & target ===
-NAME="chaudron"
-REPO="/home/srv/files/backups/${NAME}"
-TARGET="/home/srv/files/content"
-
-# === Database (Docker) ===
-# Set DB_CONTAINER="" to skip all DB dump steps entirely.
-DB_CONTAINER="mariadb"
-DUMP_SUBDIR="mariadb/dump"    # relative to $TARGET; dump_db.sh writes here
-DUMP_SCRIPT="/opt/backup-agent/dump_db.sh"
-
-# === Rclone offsite mirror ===
-RCLONE_REMOTE="scaleway"
-RCLONE_PATH="cyanet-backups-prod/${NAME}"
-RCLONE_MAX_DELETE=200
-
-# === Credentials ===
-BORG_PASSPHRASE_FILE="/root/.borg-passphrase"
-ROOT_PASSWORD_FILE="/root/.mariadb-root.pw"   # restore only; needs CREATE/DROP
-
-# === Paths ===
-REPO_MOUNT=""                          # if non-empty, checked as a mountpoint
-LOCKFILE="/var/lock/borg-backup.lock"
-LOGDIR="/var/log/borg"
-LOG_RETENTION_DAYS=90
-
-# === Timeouts ===
-DB_START_TIMEOUT=180
-CREATE_TIMEOUT="6h"
-PRUNE_TIMEOUT="2h"
-SYNC_TIMEOUT="12h"
-CHECK_TIMEOUT="4h"

configs/nexusvoice.conf

@@ -1,38 +0,0 @@
-# =============================================================================
-# Deployment config: nexusvoice (/home/acid/nexusvoice — may move to /opt/nexusvoice)
-# Used by: borg-backup.sh, restore.sh
-# Deploy: BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf /opt/backup-agent/borg-backup.sh
-# =============================================================================
-
-# === Repo & target ===
-NAME="nexusvoice"
-REPO="/opt/backups/${NAME}"          # adjust if you want the repo elsewhere
-TARGET="/home/acid/nexusvoice"             # update to /opt/nexusvoice when files move
-
-# === Database (Docker) ===
-# Set DB_CONTAINER="" to skip all DB dump steps entirely.
-DB_CONTAINER="mariadb"                     # TODO: confirm container name on this server
-DUMP_SUBDIR="mariadb/dump"                 # relative to $TARGET; dump_db.sh writes here
-DUMP_SCRIPT="/home/acid/nexusvoice/mariadb/dump_db.sh"
-
-# === Rclone offsite mirror ===
-RCLONE_REMOTE="scaleway"
-RCLONE_PATH="cyanet-backups-prod/cilaos"      # different directory from srv in same bucket
-RCLONE_MAX_DELETE=200
-
-# === Credentials ===
-BORG_PASSPHRASE_FILE="/root/.borg-passphrase"
-ROOT_PASSWORD_FILE="/root/.mariadb-root.pw"   # restore only; needs CREATE/DROP
-
-# === Paths ===
-REPO_MOUNT=""                          # if non-empty, checked as a mountpoint
-LOCKFILE="/var/lock/borg-backup.lock"
-LOGDIR="/var/log/borg"
-LOG_RETENTION_DAYS=90
-
-# === Timeouts ===
-DB_START_TIMEOUT=180
-CREATE_TIMEOUT="6h"
-PRUNE_TIMEOUT="2h"
-SYNC_TIMEOUT="12h"
-CHECK_TIMEOUT="4h"

restore.sh

@@ -14,26 +14,25 @@ export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH"
 umask 077
 
 # ========================= CONFIGURATION =========================
-# All deployment-specific constants live in a config file.
-# Default: backup.conf next to this script.
-# Override: BACKUP_CONF=/path/to/other.conf restore.sh <cmd>
-# See configs/ for per-deployment examples.
 
-_SCRIPT_DIR="$(cd "$(dirname "$(readlink -f "$0")")" && pwd)"
+NAME="borg-2025"
-BACKUP_CONF="${BACKUP_CONF:-${_SCRIPT_DIR}/backup.conf}"
+REPO="/home/srv/files/backups/$NAME"
-[[ -r "$BACKUP_CONF" ]] || { echo "ERROR: config not found: $BACKUP_CONF"; exit 1; }
+TARGET="/home/srv/files/content"
-# shellcheck source=/dev/null
-source "$BACKUP_CONF"
-
-# Derived values
 ARCHIVE_TARGET_PATH="${TARGET#/}"
 
+DB_CONTAINER="mariadb"
+DB_START_TIMEOUT=180
+
+BORG_PASSPHRASE_FILE="${BORG_PASSPHRASE_FILE:-/root/.borg-passphrase}"
+ROOT_PASSWORD_FILE="${ROOT_PASSWORD_FILE:-/root/.mariadb-root.pw}"
+DUMP_SUBDIR="mariadb/dump"
+
 # Same lockfile borg-backup.sh takes (via flock -n 9) before touching $TARGET
 # or the repo, so a restore and the nightly backup cron job can never run
 # concurrently against each other.
 LOCKFILE="${LOCKFILE:-/var/lock/borg-backup.lock}"
 
-LOGDIR="${RESTORE_LOGDIR:-${LOGDIR:-/var/log/borg}}"
+LOGDIR="${RESTORE_LOGDIR:-/var/log/borg}"
 mkdir -p "$LOGDIR" 2>/dev/null || LOGDIR="/tmp"
 LOGFILE="$LOGDIR/restore-$(date +%Y-%m-%d-%H%M%S).log"