about summary refs log tree commit diff
path: root/users/wpcarro/website/blog/posts/restic.md
diff options
context:
space:
mode:
authorWilliam Carroll <wpcarro@gmail.com>2022-07-01T03·50-0700
committerclbot <clbot@tvl.fyi>2022-07-01T03·53+0000
commitaaa0f92c45c58ccf96ac238fa618c4cfd4b884ce (patch)
treedc4e774ca6b36886d94c350edf6ec2c85898d60d /users/wpcarro/website/blog/posts/restic.md
parent2ec0d3611960b163a7052e8554ba065f3c89a8cc (diff)
feat(wpcarro/blog): Create short post about restic r/4268
Read the blog post for more information :)

Change-Id: I46a2f8ec9dc0a6546062e645f34186aa9195c70f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/5907
Reviewed-by: wpcarro <wpcarro@gmail.com>
Autosubmit: wpcarro <wpcarro@gmail.com>
Tested-by: BuildkiteCI
Diffstat (limited to 'users/wpcarro/website/blog/posts/restic.md')
-rw-r--r--users/wpcarro/website/blog/posts/restic.md91
1 files changed, 91 insertions, 0 deletions
diff --git a/users/wpcarro/website/blog/posts/restic.md b/users/wpcarro/website/blog/posts/restic.md
new file mode 100644
index 000000000000..4af1fab3682f
--- /dev/null
+++ b/users/wpcarro/website/blog/posts/restic.md
@@ -0,0 +1,91 @@
+Continuing along the trend that [Profpatsch][2] recently inspired in me: writing
+short notes to myself instead of fully fledged blog posts aimed at some
+unknowable audience. Today we're looking at how I burned myself by only
+*partially* RTFD.
+
+## Background
+
+I recently started using [restic][4] and NixOS thanks to the help of [TVL's
+`restic.nix` module][1]. I setup `1x/h` backups to [MinIO][3] (S3-compatible
+storage) for just a handful of `/var/lib` directories (`~9GiB` total), but after
+a few days MinIO reported that my bucket size was `O(100GiB)`!
+
+> What's going on?
+> -- me
+
+```shell
+$ restic stats
+repository 763bfe37 opened successfully, password is correct
+scanning...
+Stats in restore-size mode:
+Snapshots processed:   175
+   Total File Count:   31369384
+         Total Size:   21.027 GiB
+```
+
+> Wait: 20GiB... wat?
+> -- me (moments later)
+
+Maybe we're snapshotting our MinIO buckets, and that's contributing to our
+bucket size. Checking the logs proved that `restic` was backing-up `1.5GiB/h`,
+which supported MinIO's reports.
+
+> Ah maybe `restic stats` isn't reporting what I *think* it's reporting...
+> -- me (again)
+
+Let's consult Le Docs:
+
+```shell
+$ restic stats -h
+
+The "stats" command walks one or multiple snapshots in a repository
+and accumulates statistics about the data stored therein. It reports
+on the number of unique files and their sizes, according to one of
+the counting modes as given by the --mode flag.
+
+It operates on all snapshots matching the selection criteria or all
+snapshots if nothing is specified. The special snapshot ID "latest"
+is also supported. Some modes make more sense over
+just a single snapshot, while others are useful across all snapshots,
+depending on what you are trying to calculate.
+
+[to be continued]
+```
+
+This is where I stopped reading (the first time). But then I returned a second
+time as I was running low on theories...
+
+```shell
+[continued]
+
+The modes are:
+
+* restore-size: (default) Counts the size of the restored files.
+* files-by-contents: Counts total size of files, where a file is
+   considered unique if it has unique contents.
+* raw-data: Counts the size of blobs in the repository, regardless of
+  how many files reference them.
+* blobs-per-file: A combination of files-by-contents and raw-data.
+```
+
+Bingo: `--mode=raw-data` **not** `--mode=restore-size`.
+
+## Solution
+
+```shell
+$ restic stats --mode=raw-data
+repository 763bfe37 opened successfully, password is correct
+scanning...
+Stats in raw-data mode:
+Snapshots processed:   175
+   Total Blob Count:   710988
+         Total Size:   303.216 GiB
+```
+
+> Ah... the world agrees again.
+> -- me
+
+[1]: https://cs.tvl.fyi/depot@2ec0d3611960b163a7052e8554ba065f3c89a8cc/-/blob/ops/modules/restic.nix?L9
+[2]: https://github.com/profpatsch
+[3]: https://min.io/
+[4]: https://restic.net/