Borg2 backup with rclone
I recently started to figure out how to backup with Borg2 to cloud storage. Can be achieved conveniently since Borg2 2.0.0b11. At the time of this writing, I used 2.0.0b14. This post is simply a rambling on my backup journey to implement this in a semi-automated way on my Turris Omnia router.
A while ago I purchased quite some TB of storage at pCloud, a small Swiss cloud storage provider. Only recently, I found out that you can use rclone to push files and data to the cloud in an rsync-like fashion.
Since I wanted to use that approach to backup files from my router, I first figured ways to install it with opkg. I followed the Turris docs to switch to a more recent feed/branch “Here be Lions” but noticed quickly, that I actually needed to install Borg2. What most distributions package nowadays is Borg (verison 1), which is the stable release. Borg2 is a breaking change (incompatible, but runs the new feature that supports rclone).
Therefore, I decided to build a small Debian 12 lxc container and run Borg inside the container. That approach worked well.
To mount a host path inside the container, I used the following modification of the lxc config file:
# Mount host ssd
lxc.mount.entry = /srv host-srv none bind,create=dir 0 0
This will mount the /srv directory of the Turris host to the container, so the Borg process can backup files from that path.
The rclone setup with pCloud was straight forward, I simply needed to confirm the login from my laptop with a browser, because the Turris cannot connect via browser to confirm the login on pCloud. Easy!
The next step was configuring the Borg backup repository on pCloud.
My borg configuration points to the backup folder on pCloud:
# ~/.config/borg/config
BORG_REPO='rclone:pcloud:backup-folder'
BORG_PASSPHRASE='***'
PATTERNSFILE='/root/.config/borg/patterns.lst'
I also maintain a patterns file, but won't go into details here.
borg repo-create --encryption=repokey-chacha20-poly1305
⚠️ Note that the repo-create and also a lot of other commands in Borg2 are similar, but different from the Borg v1 commands. A little bit confusing when reading the docs, but you'll get the hang of it..
I chose to use chacha20-poly1305 encryption mode, because that combination was suggested to me as the fastest algorithm on my Turris. You can find out the speed of the different supported algorithms by running:
borg benchmark cpu
This is yet another convenient feature of Borg2.
Lastly, I started the backup using a Systemd timer and some variation of borg create combined with another prune command to prune old backups.
So far, so good. I'm happy that I finally found a suitable solution for my backup. The process seems quite slow though. In my create borg command I included the --compression lz4 (which should be Speedy Gonzales), but more than 8h for the first backup? Common..
I found a small signal trick that allowed me to check the current state of the upload process. This will show me the current file of the upload process:
kill -s USR1 $(pidof python) && journalctl -eu borg-backup | tail -1
I could also contribute a small improvement in the Borg docs regarding the compilation of the dependencies required for Borg2 🎉 so the hassle was worth it.
For automation of the entire procedure, I created a small packer build script that can rebuild the lxc container from scratch whenever needed. Essentially, it contains the commands to install the prerequisites for Borg on Debian followed by the installation with Pip in an virtual environment and the configuration with rclone.
Edit 2024-12-17
Turris Omnia Borg benchmark:
Chunkers =======================================================
buzhash,19,23,21,4095 1GB 12.052s
fixed,1048576 1GB 2.655s
Non-cryptographic checksums / hashes ===========================
xxh64 1GB 2.394s
crc32 (zlib) 1GB 2.970s
Cryptographic hashes / MACs ====================================
hmac-sha256 1GB 10.785s
blake2b-256 1GB 24.264s
Encryption =====================================================
aes-256-ctr-hmac-sha256 1GB 39.176s
aes-256-ctr-blake2b 1GB 51.622s
aes-256-ocb 1GB 34.483s
chacha20-poly1305 1GB 12.335s
KDFs (slow is GOOD, use argon2!) ===============================
pbkdf2 5 1.969s
argon2 5 7.606s
Compression ====================================================
lz4 0.1GB 0.239s
zstd,1 0.1GB 0.739s
zstd,3 0.1GB 0.923s
zstd,5 0.1GB 16.528s
zstd,10 0.1GB 26.171s
zstd,16 0.1GB 51.617s
zstd,22 0.1GB 64.239s
zlib,0 0.1GB 0.703s
zlib,6 0.1GB 15.758s
zlib,9 0.1GB 16.217s
lzma,0 0.1GB 88.872s
lzma,6 0.1GB 114.931s
lzma,9 0.1GB 96.051s
msgpack ========================================================
msgpack 100k Items 0.818s
For comparison, on my x230, where I would choose blake2 for hashing (blake2-chacha20-poly1305):
Chunkers =======================================================
buzhash,19,23,21,4095 1GB 1.360s
fixed,1048576 1GB 0.134s
Non-cryptographic checksums / hashes ===========================
xxh64 1GB 0.097s
crc32 (zlib) 1GB 0.406s
Cryptographic hashes / MACs ====================================
hmac-sha256 1GB 3.474s
blake2b-256 1GB 1.980s
Encryption =====================================================
aes-256-ctr-hmac-sha256 1GB 3.901s
aes-256-ctr-blake2b 1GB 3.938s
aes-256-ocb 1GB 0.572s
chacha20-poly1305 1GB 1.251s
KDFs (slow is GOOD, use argon2!) ===============================
pbkdf2 5 0.362s
argon2 5 0.434s
Compression ====================================================
lz4 0.1GB 0.022s
zstd,1 0.1GB 0.041s
zstd,3 0.1GB 0.062s
zstd,5 0.1GB 0.104s
zstd,10 0.1GB 0.188s
zstd,16 0.1GB 13.794s
zstd,22 0.1GB 14.795s
zlib,0 0.1GB 0.067s
zlib,6 0.1GB 2.735s
zlib,9 0.1GB 2.740s
lzma,0 0.1GB 18.819s
lzma,6 0.1GB 36.049s
lzma,9 0.1GB 30.293s
msgpack ========================================================
msgpack 100k Items 0.258s