about summary refs log tree commit diff
path: root/tools (unfollow)
AgeCommit message (Collapse)AuthorFilesLines
2023-01-18 r/5688 feat(corp/data-import): insert OpenCorpora data into SQLiteVincent Ambo2-9/+155
This is an initial and kind of dumb table structure, but there's some massaging that needs to be done before this makes more sense. Change-Id: I441288b684ef86be507099bcc4ebf984598789c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7861 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18 r/5687 feat(wpcarro/slx): Render transactionsWilliam Carroll3-3/+250
Wire-up clientside slx with HTML. Change-Id: Ieef517b47fae8d1af67bb0c7fcb7eae853f138e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7832 Reviewed-by: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
2023-01-18 r/5686 feat(wpcarro/website): Prefer docker imageWilliam Carroll1-2/+36
Google Cloud Run uses images to define services, so: ```shell $ mg build :image $ docker load <./result $ docker tag website:latest gcr.io/wpcarros-infrastructure/website:latest $ docker push gcr.io/wpcarros-infrastructure/website:latest ``` And then restart the service with `:latest`. TODO: Figure-out some ~sane CI solution (maybe personal Buildkite). Change-Id: I5734e3344779552aba7e0478321ba99610204e29 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7735 Reviewed-by: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
2023-01-18 r/5685 feat(wpcarro/diogenes): Delete diogenesWilliam Carroll7-207/+0
Trying to prune my monthly GCP bill, which is ~$60. Will run my website as a stateless Google Cloud Run service and see if that'll help. I still need to figure out what to do with my Quassel instance... Change-Id: I934b55029f14132af74cabde5e0ddb9e2d3bb933 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7734 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com>
2023-01-18 r/5684 feat(corp/data-import): parse lemmas from OpenCorpora dumpVincent Ambo2-14/+135
Change-Id: I1e4efcfc8e555f61578b563411d5e6ed9590d8e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7860 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18 r/5683 feat(corp/russian/data-import): new OpenCorpora data import toolVincent Ambo6-0/+829
Adds the beginning of a tool which can import OpenCorpora data into a SQLite database. This is quite a lot of toil and there's probably a better way to do this, but overall becoming this intimately familiar with the data structures is quite helpful for understanding what I can/can't do with only this dataset. Change-Id: Ieab33a8ce07ea4ac87917b9c8132226bbc6523b1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7859 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-17 r/5682 feat(wpcarro/slx): Support filtering by dateWilliam Carroll1-10/+60
Queries like `before:"03/05/2020` Change-Id: I34b06f38c8f3abeed9ad6824cd365065a1fe8d2b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7831 Reviewed-by: wpcarro <wpcarro@gmail.com> Autosubmit: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
2023-01-17 r/5681 feat(wpcarro/slx): Match against all columnsWilliam Carroll2-8/+57
With `MATCH_ALL` type. Change-Id: I2fe537c2b277a86be5c04a27da088f31fe7eb09e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7817 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com>
2023-01-17 r/5680 feat(wpcarro/slx): Simple Select impl'd in JSWilliam Carroll2-0/+282
Proof-of-concept implementation of "simple select" to write browser apps that can filter lists with labels like: ``` author:Hemingway title:For\sWhom.+ ``` Change-Id: I6c4cc65fe6319c9230fdc1e983462864acdc6aef Reviewed-on: https://cl.tvl.fyi/c/depot/+/7812 Reviewed-by: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
2023-01-17 r/5679 fix(tazjin/tverskoy): disable systemd-oomdVincent Ambo1-0/+5
This seems to have been turned on in ~November/December 2022, and is wreaking havoc on my system usability as it keeps killing my X session as soon as any kind of load happens on the system. This can be as little as accidentally playing two YouTube videos (as each YouTube tab can take 3-5GiB of RAM), or trying to process a dataset locally. I'm not sure if this is the culprit, but it sure seems like it. Change-Id: Id742b4506262cc362c9fd3f2575aea23e5c092a7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7858 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-17 r/5678 chore(tazjin/predlozhnik): move to //corpVincent Ambo9-1/+10
This is currently hosted by the company, and I'm assigning my copyright to the company, which also runs an ad placement on the page. Note that the NixOS module for hosting it has not been moved yet. Change-Id: Iba9e1cab9370faa79e43c3344fbfbbbabead50b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7857 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-17 r/5677 refactor(tvix/cli): consistently assert type unity in known_pathsVincent Ambo1-46/+41
No situation should be allowed in which a path is inserted into known_paths with different types twice, which we previously enforced only for some path types. Change-Id: I8cb47d4b29c0aab3c58694f8b590e131deba7043 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7843 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5676 refactor(tvix/eval): non-hacky suspended native thunksVincent Ambo1-55/+35
Instead of having a representation of suspended native thunks that involves constructing a fake code chunk, make these thunks a first-class part of the internal thunk representation. The previous code was not that simple to understand, and actually contained a critical bug which could lead to Tvix crashes. This version fixes the particular instance of that bug, but instead uncovers another (b/238) which can still lead to Tvix crashes. Fixes: b/237. Change-Id: I771d03864084d63953bdbb518fec94487481f839 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7750 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5675 refactor(tvix/eval): remove `Box` in new_suspended_nativeVincent Ambo3-7/+5
This is unnecessary, Rc already provides all the boxing we need. Change-Id: I08cf0939c48da43f04c847526c7e5dae5336d528 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7749 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-17 r/5674 feat(tvix/eval): add builtins to builtinsVincent Ambo3-2/+34
This is a somewhat terrifying hack that enables us to support `builtins.builtins`, by running a "fake compilation" inside of a suspended native thunk that can resolve the weak pointer to the globals. With this implementation, the thunk at `builtins.builtins` actually resolves to the "real" `builtins` (verified with a new test). This is kind of ugly, and it's something users shouldn't use, but bubbling a warning out of this is difficult at the moment due to a little bit of trickery with how the spans in suspended native thunks work (they don't) (see b/237, b/238) Change-Id: I67d0e93246dd5b279c960aeda00402031aa12af3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7748 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5673 feat(tvix/cli): add replacement strings tracking to KnownPathsVincent Ambo1-0/+29
Replacement strings are some weird internal feature of Nix that is required for calculating derivation hashes. We need to track these like other paths, as they need to be re-used on builds with dependencies on values from previous builds. Change-Id: Ie955b3fb5ae3685cfadfbe4d06ea6b5e219590c7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7828 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5672 feat(tvix/cli): track known plain paths in NixCompatIOVincent Ambo3-27/+45
When adding things to a C++ Nix store, ensure that the path is tracked in the tracker. Since the mechanism for propagating the tracker instance isn't finalised yet, I've opted to take an Rc<RefCell> parameter for it. How exactly that ends up there is going to become clear in the next commits, but for now it's just instantiated in main with Default::default. Change-Id: I90f0b44f2d4f292dedc98ff1aa39041d279b61fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/7833 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5671 refactor(tvix/cli): reference scanner owns all the stringsVincent Ambo2-50/+29
This gets very complex very quickly otherwise, as all the construction paths for a reference scanner and all the access patterns for the KnownPaths structure are not yet fully understood. Change-Id: Ibadf1f18b476695f3c286fc6896ae557760edf63 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7827 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5670 refactor(tvix/value): use proptest strategies from imbl crateVincent Ambo7-77/+211
Instead of going through Vec/BTreeMap for generating our internal types, use the proptest strategies from imbl. The one thing I couldn't figure out in the previous implementation is where the ranges/sizes of generated collections came from. The strategies in proptest use different types (Range, with an unknown default value, and SizeRange with 0..100). I've opted to specify 0..100 directly, but we can probably make it configurable. Change-Id: I749bc4c703fe424099240cab822b1642e5216361 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7791 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5669 feat(tvix/cli): add known_paths moduleVincent Ambo3-5/+143
This module implements types used to track the set of known paths in the context of an evaluation. These are used to determine the build references of a derivation. Change-Id: I81e15ae33632784e699128916485751613b231a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7816 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-16 r/5668 refactor(tvix/derivation): expose proper ValidateDerivationErrorFlorian Klink7-30/+109
Use proper errors, instead of anyhow. Change-Id: I6db14c72a6319b389b0136aac7b84f50a30fb366 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7847 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 r/5667 feat(tvix/derivation): make input_sources a BTreeSetFlorian Klink3-14/+5
These need to be sorted anyways, so let's use the correct data structure for it. Change-Id: I009c9989d7647dc1df716170f3680c981db6e4b2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7846 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 r/5666 chore(tvix/derivation): drop TODOs related to stripping outputsFlorian Klink1-2/+0
Different to the go-nix implementation, we don't have a custom "writing mode" where output paths are omitted from output, but we simply run the serialization with these fields unpopulated (during construction). This means, there's no need for a custom writing mode that strips outputs, so we can drop the comments. Change-Id: Ic0aafd4e34e0294603490cfce2b1aef4085ff34b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7845 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-16 r/5665 feat(tvix/eval): add error variant for threading through errorsVincent Ambo1-2/+17
This variant is required for external builtins (which in our case includes `derivation`) to thread through reasonable error messages. This has some potential for improvement, but it's an improvement over the status quo of panicking in the external builtins when no appropriate error is available. Change-Id: I7e4bdb0a156c7717092dde30aa4785192182dc66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7841 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-16 r/5664