about summary refs log tree commit diff
path: root/tvix/shell.nix
AgeCommit message (Collapse)AuthorFilesLines
2024-10-01 r/8744 feat(tvix/build), add OciBuildService, the old wayFlorian Klink1-0/+2
This is just patchset 10 of CL10855, before the color_eyre changes, rebased to the tvix_castore api. Change-Id: If4b42412ff8568058908cda971ad7d6f2d9f9b7b --- This provides a build service invoking runc. It can be used by using the `oci://$path_to_some_tempdir` builder URL for now. For now, it can be tested as such: ``` BUILD_SERVICE_ADDR=oci://$PWD/bundles target/debug/tvix let pkgs = (import <nixpkgs> {}); in builtins.readDir pkgs.perl ``` readDir is to actually trigger IO into the store path (which triggers the builds). For now it fails due to missing reference scanning (see followup CLs). Change-Id: I09b40e410114ce69966a41a0e3c33281b859e443 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12526 Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-09-05 r/8653 chore(3p/sources): bump channels & overlays (2024-09-01)Vincent Ambo1-7/+0
Included changes: * users/aspen: explicitly use open-source nvidia driver This now has to be specified explicitly, otherwise evaluation fails with an error. * users/aspen: nixfmt -> nixfmt-classic * users/aspen: fixes for renamed packages & options * users/tazjin: fixes for renamed packages & options * 3p/overlays: remove cbtemulator patch (merged upstream) * tvix/shell: remove unnecessary patches (merged upstream) * 3p/rust-crates: mark libgit2_sys as broken * users/Profpatsch: mark git-db as broken * 3p/overlays: pick `mypaint` from stable channel * tvix: fix comments that clippy doesn't like anymore * tvix/glue: disable a misfiring clippy lint (applying its suggestion breaks code below) Change-Id: I6d3fc027694bbe7425a2d25dc53d65467a44f3b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12403 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Reviewed-by: Profpatsch <mail@profpatsch.de> Autosubmit: tazjin <tazjin@tvl.su>
2024-06-16 r/8286 refactor(tvix/store): drop FUSE multithread workaround for DarwinFlorian Klink1-0/+13
The underlying issue in macFUSE has been fixed in https://github.com/osxfuse/osxfuse/issues/974. Bump our `macfuse` in nixpkgs to a version containing the fix. This can be removed while our nixpkgs pin is bumped past https://github.com/NixOS/nixpkgs/pull/320197. Change-Id: Ia0e644fb13198e45018b0a218647ef211acf4df1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11824 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-06-14 r/8268 feat(tvix/docs): add d2 supportFlorian Klink1-0/+2
This provides a nicer syntax to draw some diagrams. Change-Id: I2ab580e61399a45f01ed422ee29382e860753ffa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11807 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev>
2024-06-14 r/8267 feat(tvix/docs): add mdbook-admonishFlorian Klink1-0/+1
This provides a plugin for callouts. It needs to have additional CSS file added to `book.toml`, which can be (re-)generated by `mdbook-admonish install`. See https://github.com/tommilligan/mdbook-admonish/issues/171#issuecomment-2009166079 for more context. Use it by adding a warning one to the architecture document. Change-Id: I75c9a33d00acb603c6da10d3f9ce3485731c1672 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11805 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev>
2024-05-03 r/8070 test(tvix/[ca]store): enable bigtable only with "integration" featureFlorian Klink1-2/+0
The emulator and bigtable client are quite big. Remove them from the default //tvix:shell. Put the tests behind a `integration` feature flag, and add a variant with that enabled to CI, and drop the bigtable tools from //tvix:shell. Change-Id: Ie042097a0d6fc26542faa96c139b77298ccb160a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11582 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7908 fix(tvix): apply cbtemulator patch in tvix viewFlorian Klink1-1/+10
This tells josh to include the patch from third_party/overlays/patches/cbtemulator-uds.patch at nixpkgs/cbtemulator-uds.patch. We then use the `additionalOverlays` feature introduced earlier, and pass it an overlay adding the patch to `cbtemulator`. This only gets applied if tvix/shell.nix is not called with a pkgs argument, which is the case in the tvix view. Change-Id: I7bc5543460ddd30325ab0998e891c15fcaf8bcfe Reviewed-on: https://cl.tvl.fyi/c/depot/+/11408 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-04-13 r/7907 feat(3p/nixpkgs): add additionalOverlays optional argFlorian Klink1-0/+1
This allows applying additional overlays to nixpkgs before instantiating it. Change-Id: Iee486086e13bb73e6bd20a817b1106c3cd99c935 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11407 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-09 r/7881 feat(tvix/castore/directory): add bigtable backendFlorian Klink1-0/+2
This adds a Directory service using https://cloud.google.com/bigtable/docs/ as a K/V store. Directory (closures) are put in individual keys. We don't do any bucketed upload of directory closures (yet), as castore/ fs does query individually, does not request recursively (and buffers). This will be addressed by store composition at some point. Change-Id: I7fada45bf386a78b7ec93be38c5f03879a2a6e22 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11212 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-03-23 r/7764 feat(tvix): add mdbook to shellFlorian Klink1-0/+3
With this, you can `mdbook build` inside the docs/ directory. Change-Id: I97df12ed839e3c7a8065563c039436e1a830281e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11243 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-18 r/7408 chore(tvix/shell): add `cargo-expand` to debug proc-macroRyan Lahfa1-0/+1
I lost a lot of hope and had to read the source code of `quote!`, `cargo expand` was invaluable in this adventure. We should keep it IMHO. Change-Id: Icfb4c80d413602f2bdc6deab0d595183825d88ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/10635 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-29 r/7269 feat(tvix/shell): add grpcurl to shellFlorian Klink1-0/+1
This is a bit more useful than evans when sending over the same message over and over again. Change-Id: I600b6b9f591c0c963c5d270005aa1cc84d2a0770 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10453 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-12-12 r/7161 feat(tvix/glue): add nixpkgs eval benchmarkFlorian Klink1-0/+9
This adds a criterion.rs-based testbench into tvix-glue. It can be invoked by running `cargo bench` from inside the `tvix-glue` crate. `target/criterion/report/index.html` contains nice graphs. It's able to diff against the previous run, so you can invoke `cargo bench` before and after a certain change to reason about the impact in evaluation performance. Currently, we need to create a bunch of Evaluator resources inside the benchmark loop itself, which is a bit annoying, as it leaks into the things we benchmark. This should become better with b/262. Fixes b/322. Change-Id: I91656a308887baa1d459ed54d58baae919a4aaf2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10245 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-11 r/7144 feat(tvix): Benchmark nixpkgs eval with hyperfineAspen Smith1-0/+1
Add some hyperfine benchmarks to Tvix's windtunnel benchmark script for evaluating the outPath of hello and a cross-compiled hello. Change-Id: I9d76e5ce0a3fd7d9c125c36c5fced675b660a8a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10248 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi>
2023-11-03 r/6929 fix(tvix): fix `cargo test` for `nix_oracle`Florian Klink1-0/+1
As described in https://b.tvl.fyi/issues/313, the `nix_oracle` tests currently fail if run without Nix 2.3 in `$PATH`. This is not a problem for people running Nix 2.3 on their systems, and neither is it for the tests running inside Nix derivations itself (as we hardcode Nix 2.3 there), but for interactive `cargo test` runs, either using `mg shell //tvix:shell`, or `nix-shell` (from the tvix workspace) it currently is broken. Adding Nix 2.3 to the shell fixes that. Change-Id: Idbe01aa92d63eb1a3dab5797b9be5eba1e8d7f7a Reviewed-on: https://cl.tvl.fyi/c/depot/+/9907 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-10-16 r/6840 feat(tvix): add go to shellFlorian Klink1-0/+1
That's needed to build nar-bridge, as well as things in tvix/*store/ protos. Change-Id: If5f71b6c528a27689faa3c5ab3489cc528f9e6e5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9759 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-10-16 r/6836 fix(tvix/shell): fix cargo build on MacOSFlorian Klink1-4/+6
It looks like we need Security Framework in the shell to be able to imperatively run `cargo build`. Change-Id: Ia5df9052bafca3610d4ed235b180b812d33d0d15 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9756 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2023-10-07 r/6721 refactor(tvix,views/tvix): move shell into separate fileFlorian Klink1-0/+33
So far, we provided a custom `default.nix` in the root of the tvix josh workspace, which re-defined the shell attribute from `tvix/default.nix`. Some of the recent fixes, e.g. the MacOS-specific additions to the list of dependencies however didn't get ported over to this file, and in general, it's quite annoying to have two different places for these things. Initially I explored the idea of moving this default.nix file to a default-depot.nix file in the josh worktree only, and then "polyfill" some of the dependencies, or set up readTree in the josh workspace too, but it turned out to pull in too many dependencies to be worth the effort (nix.sparseTree, tools.depotfmt, crate2nix overlay, third_party.gitignoreSource). I now took a different approach - moving the definition of the `shell` attribute from `tvix/default.nix` to its own `shell.nix` file, which is imported from `tvix/default.nix` in regular depot usecases. Josh workspace consumers only see the `shell.nix`, which can be used in a self-contained fashion, the other `default.nix` is gone entirely, and we update the workspace file to also not show `tvix/default.nix` at the root either, so running `nix-shell` and then `cargo build` should still work. Change-Id: I6cb54d45d150c597612530ba44bc578f9d7f9120 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9556 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2022-12-15 r/5421 refactor(tvix): provide a combined //tvix:shell mkShell targetVincent Ambo1-17/+0
Manually maintained shell target based on the previous "loose" `shell.nix` file. We might want to have a function that combines the dependencies of all the targets automatically, but at a quick glance that was actually non-trivial so I'm leaving it as an exercise for someone else. Change-Id: I74754940088f1b58e3b6754fb782470c80ea4292 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7540 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-04 r/5385 feat(tvix/store): build rust proto filesFlorian Klink1-0/+2
This uses prost-build to build the proto files. Change-Id: Ia99fcfa4a19c741683cf28a45202e1d9211f9131 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7286 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-11-11 r/5277 feat(tvix/eval): move shell.nix and .envrc to //tvixFlorian Klink1-0/+15
Both //tvix/eval and //tvix/nix_cli have need to for rust tooling available in $PATH. Move this one level up, so it's accessible in all subdirectories. Change-Id: I0763bbe9cefdc962f3a8f86c51e8f67cde8b4b04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7248 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-08-12 r/4419 chore(tvix): move nix-store CLI scaffolding to subfolderVincent Ambo1-11/+0
For some reason a top-level Rust project ended up in this location, which is incompatible with the actual project structure that's being prepared for merge right now. Change-Id: I9d919ad72fc7e4e4d8cbb9899e7f8d90fa7ca87a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6060 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2022-01-31 r/3723 style: format entire depot with nixpkgs-fmtVincent Ambo1-2/+3
This CL can be used to compare the style of nixpkgs-fmt against other formatters (nixpkgs, alejandra). Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: cynthia <cynthia@tvl.fyi> Reviewed-by: edef <edef@edef.eu> Reviewed-by: eta <tvl@eta.st> Reviewed-by: grfn <grfn@gws.fyi>
2022-01-06 r/3521 feat(tvix): set up cargo rust projectProfpatsch1-0/+10
First steps for baba Change-Id: Id6a68c5630cb85f280f4dcc7b2acf10c02454fd6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4732 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>