feat: extract deployment config into configs/ for multi-server reuse
Both borg-backup.sh and restore.sh now source a config file instead of
hardcoding paths and container names. Default is backup.conf next to the
script; override with BACKUP_CONF=/path/to/other.conf.
- configs/srv.conf — current server (unchanged behaviour)
- configs/nexusvoice.conf — nexusvoice server (/home/acid/nexusvoice,
separate Scaleway path par-backup-1/nexusvoice)
- DB steps in borg-backup.sh are now guarded on [[ -n "$DB_CONTAINER" ]]
so a config with DB_CONTAINER="" skips the dump entirely
- RUNBOOK.md §2 documents single and multi-deployment patterns
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3rQSEidP6Y61kaCtxjVV1
This commit is contained in:
+26
-4
@@ -50,21 +50,43 @@ Run once, by hand, on the server:
|
|||||||
|
|
||||||
## 2. Deploying the Scripts
|
## 2. Deploying the Scripts
|
||||||
|
|
||||||
Copy `borg-backup.sh`, `dump_db.sh`, and `restore.sh` to `/opt/backup-agent/`
|
### Single deployment (default)
|
||||||
(this exact path is what `borg-backup.sh` invokes for `dump_db.sh`). Make
|
|
||||||
all three executable:
|
Copy `borg-backup.sh`, `dump_db.sh`, `restore.sh`, and your chosen config
|
||||||
|
from `configs/` to `/opt/backup-agent/`. Symlink the config as `backup.conf`
|
||||||
|
next to the scripts, or set `BACKUP_CONF` in the cron entry:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
chmod +x /opt/backup-agent/borg-backup.sh /opt/backup-agent/restore.sh
|
chmod +x /opt/backup-agent/borg-backup.sh /opt/backup-agent/restore.sh
|
||||||
chmod +x /opt/backup-agent/dump_db.sh
|
chmod +x /opt/backup-agent/dump_db.sh
|
||||||
|
# Option A — symlink (scripts auto-discover backup.conf beside them):
|
||||||
|
ln -s /opt/backup-agent/configs/srv.conf /opt/backup-agent/backup.conf
|
||||||
|
# Option B — explicit env var in the cron entry (see §3).
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Multiple deployments on different servers
|
||||||
|
|
||||||
|
Each server gets its own config file. Deploy the same three scripts to
|
||||||
|
`/opt/backup-agent/` on each server. Point each server's cron entry at its
|
||||||
|
config via `BACKUP_CONF`:
|
||||||
|
|
||||||
|
```
|
||||||
|
# /etc/cron.d/borg-backup (nexusvoice server)
|
||||||
|
BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf
|
||||||
|
30 2 * * * root /opt/backup-agent/borg-backup.sh >> /var/log/borg/cron.log 2>&1
|
||||||
|
```
|
||||||
|
|
||||||
|
Before running, verify/update `configs/nexusvoice.conf`:
|
||||||
|
- `TARGET` — confirm `/home/acid/nexusvoice` (or `/opt/nexusvoice` after the move)
|
||||||
|
- `DB_CONTAINER` — confirm the MariaDB container name on that server
|
||||||
|
- `REPO` — must not overlap with the srv repo; uses separate Scaleway path
|
||||||
|
|
||||||
## 3. Scheduling
|
## 3. Scheduling
|
||||||
|
|
||||||
Add a cron entry to run the backup daily, off-peak:
|
Add a cron entry to run the backup daily, off-peak:
|
||||||
|
|
||||||
```
|
```
|
||||||
# /etc/cron.d/borg-backup
|
# /etc/cron.d/borg-backup (srv — default config via symlink)
|
||||||
30 2 * * * root /opt/backup-agent/borg-backup.sh >> /var/log/borg/cron.log 2>&1
|
30 2 * * * root /opt/backup-agent/borg-backup.sh >> /var/log/borg/cron.log 2>&1
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+46
-50
@@ -1,6 +1,6 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
# Borg backup: /content + MariaDB -> local repo -> Scaleway S3 mirror
|
# Borg backup: $TARGET + MariaDB -> local borg repo -> Scaleway S3 mirror
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
# CHANGES vs. original:
|
# CHANGES vs. original:
|
||||||
# • Encryption enabled (repokey-blake2 via BORG_PASSCOMMAND)
|
# • Encryption enabled (repokey-blake2 via BORG_PASSCOMMAND)
|
||||||
@@ -12,6 +12,7 @@
|
|||||||
# excluded from the archive (drop a .nobackup file in it - see
|
# excluded from the archive (drop a .nobackup file in it - see
|
||||||
# --exclude-if-present below) so no live InnoDB file is ever copied.
|
# --exclude-if-present below) so no live InnoDB file is ever copied.
|
||||||
# Zero DB downtime during backup.
|
# Zero DB downtime during backup.
|
||||||
|
# • Config split into per-deployment backup.conf (see configs/)
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
@@ -20,41 +21,28 @@ export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
|||||||
umask 077
|
umask 077
|
||||||
|
|
||||||
# ========================= CONFIGURATION =========================
|
# ========================= CONFIGURATION =========================
|
||||||
|
# All deployment-specific constants live in a config file.
|
||||||
|
# Default: backup.conf next to this script.
|
||||||
|
# Override: BACKUP_CONF=/path/to/other.conf borg-backup.sh
|
||||||
|
# See configs/ for per-deployment examples.
|
||||||
|
|
||||||
NAME="borg-2025"
|
_SCRIPT_DIR="$(cd "$(dirname "$(readlink -f "$0")")" && pwd)"
|
||||||
REPO="/home/srv/files/backups/$NAME"
|
BACKUP_CONF="${BACKUP_CONF:-${_SCRIPT_DIR}/backup.conf}"
|
||||||
TARGET="/home/srv/files/content"
|
[[ -r "$BACKUP_CONF" ]] || { echo "ERROR: config not found: $BACKUP_CONF"; exit 1; }
|
||||||
|
# shellcheck source=/dev/null
|
||||||
|
source "$BACKUP_CONF"
|
||||||
|
|
||||||
# dump_db.sh lives with the rest of this toolkit, not inside $TARGET - its
|
# Derived values (override in config only if you need a non-standard layout)
|
||||||
# own default dump dir is relative to wherever IT lives, so DUMP_DIR must be
|
# dump_db.sh lives with this toolkit; DUMP_DIR is passed explicitly so dumps
|
||||||
# passed explicitly (below) to keep dumps inside $TARGET where borg can see them.
|
# land inside $TARGET where borg can see them.
|
||||||
DUMP_SCRIPT="/opt/backup-agent/dump_db.sh"
|
export DUMP_DIR="${DUMP_DIR:-${TARGET}/${DUMP_SUBDIR}}"
|
||||||
export DUMP_DIR="${TARGET}/mariadb/dump"
|
|
||||||
|
|
||||||
REPO_MOUNT=""
|
# docker is only required when DB_CONTAINER is set
|
||||||
|
if [[ -n "${DB_CONTAINER:-}" ]]; then
|
||||||
LOGDIR="/var/log/borg"
|
REQUIRED_CMDS=(borg rclone docker timeout flock date find)
|
||||||
LOG_RETENTION_DAYS=90
|
else
|
||||||
|
REQUIRED_CMDS=(borg rclone timeout flock date find)
|
||||||
RCLONE_REMOTE="scaleway"
|
fi
|
||||||
RCLONE_PATH="par-backup-1/$NAME"
|
|
||||||
RCLONE_MAX_DELETE=200
|
|
||||||
|
|
||||||
LOCKFILE="/var/lock/borg-backup.lock"
|
|
||||||
|
|
||||||
DB_CONTAINER="mariadb"
|
|
||||||
DB_START_TIMEOUT=180
|
|
||||||
|
|
||||||
CREATE_TIMEOUT="6h"
|
|
||||||
PRUNE_TIMEOUT="2h"
|
|
||||||
SYNC_TIMEOUT="12h"
|
|
||||||
CHECK_TIMEOUT="4h"
|
|
||||||
|
|
||||||
# Passphrase file: chmod 600, owned by the backup user.
|
|
||||||
# Create it with: echo 'your-strong-passphrase' > /root/.borg-passphrase
|
|
||||||
BORG_PASSPHRASE_FILE="${BORG_PASSPHRASE_FILE:-/root/.borg-passphrase}"
|
|
||||||
|
|
||||||
REQUIRED_CMDS=(borg rclone docker timeout flock date find)
|
|
||||||
|
|
||||||
# =================================================================
|
# =================================================================
|
||||||
|
|
||||||
@@ -128,7 +116,7 @@ cleanup() {
|
|||||||
|
|
||||||
# We never stop the container ourselves anymore, but if it crashed for
|
# We never stop the container ourselves anymore, but if it crashed for
|
||||||
# an unrelated reason during the backup window, try to bring it back.
|
# an unrelated reason during the backup window, try to bring it back.
|
||||||
if ! container_running "$DB_CONTAINER"; then
|
if [[ -n "${DB_CONTAINER:-}" ]] && ! container_running "$DB_CONTAINER"; then
|
||||||
step "Cleanup: $DB_CONTAINER is down, attempting restart"
|
step "Cleanup: $DB_CONTAINER is down, attempting restart"
|
||||||
start_db || { log "CRITICAL: $DB_CONTAINER is DOWN - manual action required"
|
start_db || { log "CRITICAL: $DB_CONTAINER is DOWN - manual action required"
|
||||||
[ "$exit_code" -eq 0 ] && exit_code=1; }
|
[ "$exit_code" -eq 0 ] && exit_code=1; }
|
||||||
@@ -192,8 +180,10 @@ preflight() {
|
|||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
container_running "$DB_CONTAINER" \
|
if [[ -n "${DB_CONTAINER:-}" ]]; then
|
||||||
|| { log "ERROR: $DB_CONTAINER is not running before we start"; return 1; }
|
container_running "$DB_CONTAINER" \
|
||||||
|
|| { log "ERROR: $DB_CONTAINER is not running before we start"; return 1; }
|
||||||
|
fi
|
||||||
|
|
||||||
log "Preflight OK"
|
log "Preflight OK"
|
||||||
}
|
}
|
||||||
@@ -224,20 +214,24 @@ preflight
|
|||||||
# --- 1. Logical dumps (container stays up the whole time) ---------------
|
# --- 1. Logical dumps (container stays up the whole time) ---------------
|
||||||
step "Step 1: MariaDB dumps"
|
step "Step 1: MariaDB dumps"
|
||||||
|
|
||||||
if [[ ! -x "$DUMP_SCRIPT" ]]; then
|
if [[ -n "${DB_CONTAINER:-}" ]]; then
|
||||||
log "ERROR: dump script missing or not executable: $DUMP_SCRIPT"
|
if [[ ! -x "$DUMP_SCRIPT" ]]; then
|
||||||
exit 1
|
log "ERROR: dump script missing or not executable: $DUMP_SCRIPT"
|
||||||
fi
|
exit 1
|
||||||
run_cmd "$DUMP_SCRIPT"
|
fi
|
||||||
|
run_cmd "$DUMP_SCRIPT"
|
||||||
|
|
||||||
if [[ -d "$DUMP_DIR" ]]; then
|
if [[ -d "$DUMP_DIR" ]]; then
|
||||||
fresh=$(find "$DUMP_DIR" -type f -size +1k -mmin -60 | wc -l)
|
fresh=$(find "$DUMP_DIR" -type f -size +1k -mmin -60 | wc -l)
|
||||||
empty=$(find "$DUMP_DIR" -type f -size -1k -mmin -60 | wc -l)
|
empty=$(find "$DUMP_DIR" -type f -size -1k -mmin -60 | wc -l)
|
||||||
log "Dumps: $fresh fresh non-trivial file(s), $empty suspiciously small"
|
log "Dumps: $fresh fresh non-trivial file(s), $empty suspiciously small"
|
||||||
(( fresh > 0 )) || { log "ERROR: no usable dumps produced"; exit 1; }
|
(( fresh > 0 )) || { log "ERROR: no usable dumps produced"; exit 1; }
|
||||||
(( empty == 0 )) || log "WARNING: $empty near-empty dump file(s) - check $DUMP_DIR"
|
(( empty == 0 )) || log "WARNING: $empty near-empty dump file(s) - check $DUMP_DIR"
|
||||||
|
else
|
||||||
|
log "WARNING: dump directory not found: $DUMP_DIR"
|
||||||
|
fi
|
||||||
else
|
else
|
||||||
log "WARNING: dump directory not found: $DUMP_DIR"
|
log "DB_CONTAINER not set - skipping database dump"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# --- 2. Create the archive -----------------------------------------------
|
# --- 2. Create the archive -----------------------------------------------
|
||||||
@@ -248,8 +242,10 @@ fi
|
|||||||
# are excluded too - they churn on every request, so backing them up adds
|
# are excluded too - they churn on every request, so backing them up adds
|
||||||
# noise and dedup overhead for no recovery value.
|
# noise and dedup overhead for no recovery value.
|
||||||
step "Step 2: Creating archive $ARCHIVE"
|
step "Step 2: Creating archive $ARCHIVE"
|
||||||
container_running "$DB_CONTAINER" \
|
if [[ -n "${DB_CONTAINER:-}" ]]; then
|
||||||
|| { log "ERROR: $DB_CONTAINER is not running - refusing to archive"; exit 1; }
|
container_running "$DB_CONTAINER" \
|
||||||
|
|| { log "ERROR: $DB_CONTAINER is not running - refusing to archive"; exit 1; }
|
||||||
|
fi
|
||||||
run_cmd timeout --signal=INT --kill-after=120s "$CREATE_TIMEOUT" \
|
run_cmd timeout --signal=INT --kill-after=120s "$CREATE_TIMEOUT" \
|
||||||
borg create \
|
borg create \
|
||||||
--lock-wait 600 \
|
--lock-wait 600 \
|
||||||
|
|||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# =============================================================================
|
||||||
|
# Deployment config: nexusvoice (/home/acid/nexusvoice — may move to /opt/nexusvoice)
|
||||||
|
# Used by: borg-backup.sh, restore.sh
|
||||||
|
# Deploy: BACKUP_CONF=/opt/backup-agent/configs/nexusvoice.conf /opt/backup-agent/borg-backup.sh
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
# === Repo & target ===
|
||||||
|
NAME="nexusvoice-2025"
|
||||||
|
REPO="/home/acid/backups/${NAME}" # adjust if you want the repo elsewhere
|
||||||
|
TARGET="/home/acid/nexusvoice" # update to /opt/nexusvoice when files move
|
||||||
|
|
||||||
|
# === Database (Docker) ===
|
||||||
|
# Set DB_CONTAINER="" to skip all DB dump steps entirely.
|
||||||
|
DB_CONTAINER="mariadb" # TODO: confirm container name on this server
|
||||||
|
DUMP_SUBDIR="mariadb/dump" # relative to $TARGET; dump_db.sh writes here
|
||||||
|
DUMP_SCRIPT="/opt/backup-agent/dump_db.sh"
|
||||||
|
|
||||||
|
# === Rclone offsite mirror ===
|
||||||
|
RCLONE_REMOTE="scaleway"
|
||||||
|
RCLONE_PATH="par-backup-1/nexusvoice" # different directory from srv in same bucket
|
||||||
|
RCLONE_MAX_DELETE=200
|
||||||
|
|
||||||
|
# === Credentials ===
|
||||||
|
BORG_PASSPHRASE_FILE="/root/.borg-passphrase"
|
||||||
|
ROOT_PASSWORD_FILE="/root/.mariadb-root.pw" # restore only; needs CREATE/DROP
|
||||||
|
|
||||||
|
# === Paths ===
|
||||||
|
REPO_MOUNT="" # if non-empty, checked as a mountpoint
|
||||||
|
LOCKFILE="/var/lock/borg-backup.lock"
|
||||||
|
LOGDIR="/var/log/borg"
|
||||||
|
LOG_RETENTION_DAYS=90
|
||||||
|
|
||||||
|
# === Timeouts ===
|
||||||
|
DB_START_TIMEOUT=180
|
||||||
|
CREATE_TIMEOUT="6h"
|
||||||
|
PRUNE_TIMEOUT="2h"
|
||||||
|
SYNC_TIMEOUT="12h"
|
||||||
|
CHECK_TIMEOUT="4h"
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# =============================================================================
|
||||||
|
# Deployment config: srv (primary server — /home/srv/files/content)
|
||||||
|
# Used by: borg-backup.sh, restore.sh
|
||||||
|
# Deploy: BACKUP_CONF=/opt/backup-agent/configs/srv.conf /opt/backup-agent/borg-backup.sh
|
||||||
|
# (or symlink configs/srv.conf -> ../backup.conf next to the scripts)
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
# === Repo & target ===
|
||||||
|
NAME="borg-2025"
|
||||||
|
REPO="/home/srv/files/backups/${NAME}"
|
||||||
|
TARGET="/home/srv/files/content"
|
||||||
|
|
||||||
|
# === Database (Docker) ===
|
||||||
|
# Set DB_CONTAINER="" to skip all DB dump steps entirely.
|
||||||
|
DB_CONTAINER="mariadb"
|
||||||
|
DUMP_SUBDIR="mariadb/dump" # relative to $TARGET; dump_db.sh writes here
|
||||||
|
DUMP_SCRIPT="/opt/backup-agent/dump_db.sh"
|
||||||
|
|
||||||
|
# === Rclone offsite mirror ===
|
||||||
|
RCLONE_REMOTE="scaleway"
|
||||||
|
RCLONE_PATH="par-backup-1/${NAME}"
|
||||||
|
RCLONE_MAX_DELETE=200
|
||||||
|
|
||||||
|
# === Credentials ===
|
||||||
|
BORG_PASSPHRASE_FILE="/root/.borg-passphrase"
|
||||||
|
ROOT_PASSWORD_FILE="/root/.mariadb-root.pw" # restore only; needs CREATE/DROP
|
||||||
|
|
||||||
|
# === Paths ===
|
||||||
|
REPO_MOUNT="" # if non-empty, checked as a mountpoint
|
||||||
|
LOCKFILE="/var/lock/borg-backup.lock"
|
||||||
|
LOGDIR="/var/log/borg"
|
||||||
|
LOG_RETENTION_DAYS=90
|
||||||
|
|
||||||
|
# === Timeouts ===
|
||||||
|
DB_START_TIMEOUT=180
|
||||||
|
CREATE_TIMEOUT="6h"
|
||||||
|
PRUNE_TIMEOUT="2h"
|
||||||
|
SYNC_TIMEOUT="12h"
|
||||||
|
CHECK_TIMEOUT="4h"
|
||||||
+12
-11
@@ -14,25 +14,26 @@ export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH"
|
|||||||
umask 077
|
umask 077
|
||||||
|
|
||||||
# ========================= CONFIGURATION =========================
|
# ========================= CONFIGURATION =========================
|
||||||
|
# All deployment-specific constants live in a config file.
|
||||||
|
# Default: backup.conf next to this script.
|
||||||
|
# Override: BACKUP_CONF=/path/to/other.conf restore.sh <cmd>
|
||||||
|
# See configs/ for per-deployment examples.
|
||||||
|
|
||||||
NAME="borg-2025"
|
_SCRIPT_DIR="$(cd "$(dirname "$(readlink -f "$0")")" && pwd)"
|
||||||
REPO="/home/srv/files/backups/$NAME"
|
BACKUP_CONF="${BACKUP_CONF:-${_SCRIPT_DIR}/backup.conf}"
|
||||||
TARGET="/home/srv/files/content"
|
[[ -r "$BACKUP_CONF" ]] || { echo "ERROR: config not found: $BACKUP_CONF"; exit 1; }
|
||||||
|
# shellcheck source=/dev/null
|
||||||
|
source "$BACKUP_CONF"
|
||||||
|
|
||||||
|
# Derived values
|
||||||
ARCHIVE_TARGET_PATH="${TARGET#/}"
|
ARCHIVE_TARGET_PATH="${TARGET#/}"
|
||||||
|
|
||||||
DB_CONTAINER="mariadb"
|
|
||||||
DB_START_TIMEOUT=180
|
|
||||||
|
|
||||||
BORG_PASSPHRASE_FILE="${BORG_PASSPHRASE_FILE:-/root/.borg-passphrase}"
|
|
||||||
ROOT_PASSWORD_FILE="${ROOT_PASSWORD_FILE:-/root/.mariadb-root.pw}"
|
|
||||||
DUMP_SUBDIR="mariadb/dump"
|
|
||||||
|
|
||||||
# Same lockfile borg-backup.sh takes (via flock -n 9) before touching $TARGET
|
# Same lockfile borg-backup.sh takes (via flock -n 9) before touching $TARGET
|
||||||
# or the repo, so a restore and the nightly backup cron job can never run
|
# or the repo, so a restore and the nightly backup cron job can never run
|
||||||
# concurrently against each other.
|
# concurrently against each other.
|
||||||
LOCKFILE="${LOCKFILE:-/var/lock/borg-backup.lock}"
|
LOCKFILE="${LOCKFILE:-/var/lock/borg-backup.lock}"
|
||||||
|
|
||||||
LOGDIR="${RESTORE_LOGDIR:-/var/log/borg}"
|
LOGDIR="${RESTORE_LOGDIR:-${LOGDIR:-/var/log/borg}}"
|
||||||
mkdir -p "$LOGDIR" 2>/dev/null || LOGDIR="/tmp"
|
mkdir -p "$LOGDIR" 2>/dev/null || LOGDIR="/tmp"
|
||||||
LOGFILE="$LOGDIR/restore-$(date +%Y-%m-%d-%H%M%S).log"
|
LOGFILE="$LOGDIR/restore-$(date +%Y-%m-%d-%H%M%S).log"
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user