about summary refs log tree commit diff
path: root/tools/nixery/builder (follow)
AgeCommit message (Collapse)AuthorFilesLines
2022-09-19 r/4924 fix(nixery): Avoid race when the same image is fetched in paralleltalyz1-42/+50
Remove a race condition which appears when uploadHashLayer is called with the same key from multiple threads simultaneously. This can easily happen when the same image path is requested by multiple clients at the same time. When it does, a 500 status is returned and the following error message is logged: { "context": { "filePath": "github.com/google/nixery/builder/builder.go", "lineNumber": 440, "functionName": "github.com/google/nixery/builder.uploadHashLayer" }, "error": "rename /var/lib/nixery/staging/<hash> /var/lib/nixery/layers/<hash>: no such file or directory", "eventTime": "...", "layer": "<hash>", "message": "failed to move layer from staging", ... } To solve this issue, introduce a mutex keyed on the uploaded hash and move all layer caching into uploadHashLayer. This could additionally provide a small performance benefit when an already built image is requested and NIXERY_PKGS_PATH is set, since symlink layers and config layers are now also cached. Change-Id: I50788a7ec7940cb5e5760f244692e361019a9bb7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6695 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-05-23 r/4105 refactor(nixery): Extract layering logic into separate packageVincent Ambo3-358/+8
This will be required for making a standalone, Nixery-style image builder function usable from Nix. Change-Id: I5e36348bd4c32d249d56f6628cd046916691319f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5601 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-04-20 r/3982 chore(nixery): Housekeeping for depot compatibilityVincent Ambo5-65/+10
Cleans up a whole bunch of things I wanted to get out of the door right away: * depot internal references to //third_party/nixery have been replaced with //tools/nixery * cleaned up files from Github * fixed SPDX & Copyright headers * code formatting and inclusion in //tools/depotfmt checks Change-Id: Iea79f0fdf3aa04f71741d4f4032f88605ae415bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/5486 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su>
2021-12-27 feat: set SSL_CERT_FILE and provide a CmdJérôme Petazzoni1-1/+9
Two minor "quality of life" improvements: - automatically set SSL_CERT_FILE environment variable, so that programs relying on OpenSSL for certificate validation can actually validate certificates (the certificates are included no matter what since we add the "cacert" package to all iamges) - if the requested image includes an interactive shell (e.g. if it includes the "shell" metapackage), set the image Cmd to "bash", which allows to execute "docker run nixery.dev/shell" and get a shell) I'm happy to split this PR in two if you'd like, but since both features touch the Config structure and are rather small, I thought it would make sense to bundle them together.
2020-10-29 feat(storage): Add support for content-types (GCS only)Vincent Ambo2-3/+3
Extends storage.Persist to accept a Content-Type argument, which in the GCS backend is persisted with the object to ensure that the object is served back with this content-type. This is not yet implemented for the filesystem backend, where the parameter is simply ignored. This should help in the case of clients which expect the returned objects to have content-types set when, for example, fetching layers by digest.
2020-01-19 fix(builder): Fix minor logging switcharooVincent Ambo1-1/+1
2019-11-27 refactor: Reshuffle file structure for better code layoutVincent Ambo5-0/+1357
This gets rid of the package called "server" and instead moves everything into the project root, such that Go actually builds us a binary called `nixery`. This is the first step towards factoring out CLI-based functionality for Nixery.