| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This introduces a new Nix derivation that, given an attribute set of
layer hashes mapped to store paths, will create a layer tarball for
each of the store paths.
This is going to be used by the builder to create layers that are not
present in the cache.
Relates to #50.
|
|
|
|
This creates a cache key which can be used to check if a layer has
already been built.
|
|
Refactors the layer grouping package (which previously compiled to a
separate binary) to expose the layer grouping logic via a function
instead.
This is the next step towards creating layers inside of the server
component instead of in Nix.
Relates to #50.
|
|
|
|
Simplifies the wrapper script used to invoke Nix builds from Nixery to
just contain the essentials, since the layer grouping logic is moving
into the server itself.
|
|
This is the first step towards a more granular build process where
some of the build responsibility moves into the server component.
Rather than assembling all layers inside of Nix, it will only create
the symlink forest and return information about the runtime paths
required by the image.
The server is then responsible for grouping these paths into layers,
and assembling the layers themselves.
Relates to #50.
|
|
This makes CI build the group-layers tool (and cache it to Cachix!)
|
|
|
|
Fixes two launch script compatibility issues with other container
runtimes (such as gvisor):
* don't fail if /tmp already exists
* don't fail if the environment becomes unset
|
|
Instead of dumping all Nix output as one at the end of the build
process, stream it live as the lines come in.
This is a lot more useful for debugging stuff like where manifest
retrievals get stuck.
|
|
|
|
This is useful when running Nixery locally.
|
|
This version matches the updated popularity URL.
|
|
|
|
The change has been upstreamed in Nixpkgs.
|
|
|
|
Caches manifests under `manifests/$cacheKey` in the GCS bucket and
introduces two-tiered retrieval of manifests from the caches (local
first, bucket second).
There is some cleanup to be done in this code, but the initial version
works.
|
|
Use the PackageSource.CacheKey function introduced in the previous
commit to determine the key at which a manifest should be cached in
the local cache.
Due to this change, manifests for moving target sources are no longer
cached and the recency threshold logic has been removed.
|
|
Introduces three new types representing each of the possible package
sources and moves the logic for specifying the package source to the
server.
Concrete changes:
* Determining whether a specified git reference is a commit vs. a
branch/tag is now done in the server, and is done more precisely by
using a regular expression.
* Package sources now have a new `CacheKey` function which can be used
to retrieve a key under which a build manifest can be cached *if*
the package source is not a moving target (i.e. a full git commit
hash of either nixpkgs or a private repository).
This function is not yet used.
* Users *must* now specify a package source, Nixery no longer defaults
to anything and will fail to launch if no source is configured.
|
|
|
|
Adds a NIX_TIMEOUT environment variable which can be set to a number
of seconds that is the maximum allowed time each Nix builder can run.
By default this is set to 60 seconds, which should be plenty for most
use-cases as Nixery is not expected to be performing builds of
uncached binaries in most production cases.
Currently the errors Nix throws on a build timeout are not separated
from other types of errors, meaning that users will see a generic 500
server error in case of a timeout.
This fixes #47
|
|
Explicitly refer to where things come from, and also don't import
dockerTools as it is no longer used for anything.
|
|
Moves the relevant parts of the customisation layer construction from
dockerTools.mkCustomisationLayer into the Nixery code base.
The version in dockerTools builds additional files (including via
hashing of potentially large files) which are not required when
serving an image over the registry protocol.
|
|
This package is used by a variety of programs that users may want to
embed into Nixery in addition, for example cachix, but those packages
don't refer to it explicitly.
|
|
The new version of the document has syntactic fixes that render pipes
in code blocks in tables correctly across dialects.
Fixes #44
|
|
Thanks to clever!
|
|
This is the same commit for which Nixery has popularity data, but that
isn't particularly relevant.
|
|
This makes it possible to inject additional programs (e.g. Cachix)
into a Nixery container.
|
|
|
|
This is required by git in cases where Nixery is configured with a
custom git repository.
I've also added a shell back into the image to make debugging a
running Nixery easier. It turns out some of the dependencies already
pull in bash anyways, so this is just surfacing it to $PATH.
|
|
Before this change, Nixery would pass on the image name unmodified to
Nix which would lead it to cache-bust the manifest and configuration
layers for images that are content-identical but have different
package ordering.
This fixes #38.
|
|
|
|
These probably should be part of every container image by default, but
adding it to the "shell" convenience name probably is our best bet for
now.
|
|
|
|
|
|
|
|
This test, after performing the usual Nixery build, loads the built
image into Docker, runs it, pulls an image from Nixery and runs that
image.
To make this work, there is some configuration on the Travis side.
Most importantly, the following environment variables have special
values:
* `GOOGLE_KEY`: This is set to a base64-encoded service account key to
be used in the test.
* `GCS_SIGNING_PEM`: This is set to a base64-encoded signing key (in
PEM) that is used for signing URLs.
Both of these are available to all branches in the Nixery repository.
|
|
|
|
They grow up so fast :')
|
|
Implements a cache that keeps track of:
a) Manifests that have already been built (for up to 6 hours)
b) Layers that have already been seen (and uploaded to GCS)
This significantly speeds up response times for images that are full
or partial matches with previous images served by an instance.
|
|
This module is going to get more complex as the implementation of #32
progresses.
|
|
Some upcoming changes might require the Nix build to be split into
multiple separate nix-build invocations of different expressions, thus
splitting this out is useful.
It also fixes an issue where `build-image/default.nix` might be called
in an environment where no Nix channels are configured.
|
|
ALl the ones except for build-image.nix are considered trivial. On the
latter, nixfmt makes some useful changes but by-and-large it is not
ready for that code yet.
|
|
Removes usage of the old layering algorithm and replaces it with the
new one.
Apart from the new layer layout this means that each layer is now
built in a separate derivation, which hopefully leads to better
cacheability.
|
|
The issue is described in detail in a comment in
`build-image/default.nix`, please read it.
|
|
|
|
Adds the script used to generate the popularity information for all of
nixpkgs.
The README lists the (currently somewhat rough) usage instructions.
|
|
Instead of requiring the server component to be made aware of the
location of the Nix builder via environment variables, this commit
introduces a wrapper script for the builder that can simply exist on
the builders $PATH.
This is one step towards a slightly nicer out-of-the-box experience
when using `nix-build -A nixery-bin`.
|
|
|
