Backups#
Backups for local systems#
Backups are performed using borgbackup
The warehouse is located on voile
NixOS#
A nixos module exists under modules/nixos/borg.
Darwin#
The darwin module was custom built and is under modules/darwin/borg.
Adding a host#
- In this repo, run
just borg-generate-secrets <hostname> - This will create a repo key and passphrase in
secrets.yamlvia sops - Commit and push the secrets
- Run
just sops-update-hoststo re-encrypt for all keys - Add the host to borgwarehouse
- Configure the host
...
tsunaminoai.borg = {
enable = true;
repo = "8a95a28d";
};
...
- Deploy
Schedule#
Darwin hosts run the backup at 03:00 local time via launchd (StartCalendarInterval with Hour = 3, Minute = 0). NixOS hosts run daily via the borgmatic systemd timer (upstream OnCalendar=daily with RandomizedDelaySec=3h), firing at a randomized time between midnight and 03:00.
Status and storage-usage checks are pushed to borgwarehouse every 2 hours by systemd oneshot units + timers (borgwarehouse-cron-status / borgwarehouse-cron-storage, OnCalendar = *-*-* 00/2:05:00; enabled per-host with tsunaminoai.borg.enableCron, currently only ereshkigal). The units POST to BorgWarehouse ≥3.6’s /api/v1/cron/{status,storage} endpoints authenticated by the CRONJOB_KEY (sops borgwarehouse/cronjob-key — a different key from the read-only account API key, sops borgwarehouse/api-key). A non-2xx response fails the unit, so a broken push pages via SystemdUnitFailed → ntfy instead of failing silently like the old crontab did. BorgWarehouse handles backup-failure and low-storage alerting.
Restoring from Backup#
See Restoring a Host from Backup for the full procedure, including how to handle sops key rotation after a machine wipe.
Backups for MS365#
Backups for all of the sc2.in domain are performed daily by the Active Backup MS365 package running on voile.
The portal for backups is located at Active Backup for Microsoft 365 Portal
