blob: 175fe04d7084d4e36a18614605fdae2b1982f9b3 (
plain) (
tree)
|
|
# Caching in Nixery
This page gives a quick overview over the caching done by Nixery. All cache data
is written to Nixery's storage bucket and is based on deterministic identifiers
or content-addressing, meaning that cache entries under the same key *never
change*.
## Manifests
Manifests of builds are cached at `$BUCKET/manifests/$KEY`. The effect of this
cache is that multiple instances of Nixery do not need to rebuild the same
manifest from scratch.
Since the manifest cache is populated only *after* layers are uploaded, Nixery
can immediately return the manifest to its clients without needing to check
whether layers have been uploaded already.
`$KEY` is generated by creating a SHA1 hash of the requested content of a
manifest plus the package source specification.
Manifests are *only* cached if the package source specification is *not* a
moving target.
Manifest caching *only* applies in the following cases:
* package source specification is a specific git commit
* package source specification is a specific NixOS/nixpkgs commit
Manifest caching *never* applies in the following cases:
* package source specification is a local file path (i.e. `NIXERY_PKGS_PATH`)
* package source specification is a NixOS channel (e.g. `NIXERY_CHANNEL=nixos-19.03`)
* package source specification is a git branch or tag (e.g. `staging`, `master` or `latest`)
It is thus always preferable to request images from a fully-pinned package
source.
Manifests can be removed from the manifest cache without negative consequences.
## Layer tarballs
Layer tarballs are the files that Nixery clients retrieve from the storage
bucket to download an image.
They are stored content-addressably at `$BUCKET/layers/$SHA256HASH` and layer
requests sent to Nixery will redirect directly to this storage location.
The effect of this cache is that Nixery does not need to upload identical layers
repeatedly. When Nixery notices that a layer already exists in GCS, it will use
the object metadata to compare its MD5-hash with the locally computed one and
skip uploading.
Removing layers from the cache is *potentially problematic* if there are cached
manifests or layer builds referencing those layers.
To clean up layers, a user must ensure that no other cached resources still
reference these layers.
## Layer builds
Layer builds are cached at `$BUCKET/builds/$HASH`, where `$HASH` is a SHA1 of
the Nix store paths included in the layer.
The content of the cached entries is a JSON-object that contains the MD5 and
SHA256 hashes of the built layer.
The effect of this cache is that different instances of Nixery will not build,
hash and upload layers that have identical contents across different instances.
Layer builds can be removed from the cache without negative consequences.
|
