refactor(tvix/eval): streamline construction of globals/builtins r/5581

Previously the construction of globals (a compiler-only concept) and
builtins (a (now) user-facing API) was intermingled between multiple
different modules, and kind of difficult to understand.

The complexity of this had grown in large part due to the
implementation of `builtins.import`, which required the notorious
"knot-tying" trick using Rc::new_cyclic (see cl/7097) for constructing
the set of globals.

As part of the new `Evaluation` API users should have the ability to
bring their own builtins, and control explicitly whether or not impure
builtins are available (regardless of whether they're compiled in or
not).

To streamline the construction and allow the new API features to work,
this commit restructures things by making these changes:

1. The `tvix_eval::builtins` module is now only responsible for
   exporting sets of builtins. It no longer has any knowledge of
   whether or not certain sets (e.g. only pure, or pure+impure) are
   enabled, and it has no control over which builtins are globally
   available (this is now handled in the compiler).

2. The compiler module is now responsible for both constructing the
   final attribute set of builtins from the set of builtins supplied
   by a user, as well as for populating its globals (that is
   identifiers which are available at the top-level scope).

3. The `Evaluation` API now carries a `builtins` field which is
   populated with the pure builtins by default, and can be extended by
   users.

4. The `import` feature has been moved into the compiler, as a
   special case. In general, builtins no longer have the ability to
   reference the "fix point" of the globals set.

This should not change any functionality, and in fact preserves minor
differences between Tvix/Nix that we already had (such as
`builtins.builtins` not existing).

Change-Id: Icdf5dd50eb81eb9260d89269d6e08b1e67811a2c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7738
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>

1 files changed, 31 insertions, 4 deletions

diff --git a/tvix/eval/src/lib.rs b/tvix/eval/src/lib.rs
index a467a1884c..fa76aca56b 100644
--- a/tvix/eval/src/lib.rs
+++ b/tvix/eval/src/lib.rs
@@ -12,7 +12,7 @@
 //! These features are optional and the API of this crate exposes functionality
 //! for controlling how they work.
 
-mod builtins;
+pub mod builtins;
 mod chunk;
 mod compiler;
 mod errors;
@@ -41,7 +41,6 @@ use std::str::FromStr;
 use std::sync::Arc;
 
 // Re-export the public interface used by other crates.
-pub use crate::builtins::global_builtins;
 pub use crate::compiler::{compile, prepare_globals};
 pub use crate::errors::{Error, ErrorKind, EvalResult};
 pub use crate::io::{DummyIO, EvalIO, FileType};
@@ -83,12 +82,24 @@ pub struct Evaluation<'code, 'co, 'ro> {
     /// Top-level file reference for this code inside the source map.
     file: Arc<codemap::File>,
 
+    /// Set of all builtins that should be available during the
+    /// evaluation.
+    ///
+    /// This defaults to all pure builtins. Users might want to add
+    /// the set of impure builtins, or other custom builtins.
+    pub builtins: Vec<(&'static str, Value)>,
+
     /// Implementation of file-IO to use during evaluation, e.g. for
     /// impure builtins.
     ///
     /// Defaults to [`DummyIO`] if not set explicitly.
     pub io_handle: Box<dyn EvalIO>,
 
+    /// Determines whether the `import` builtin should be made
+    /// available. Note that this depends on the `io_handle` being
+    /// able to read the files specified as arguments to `import`.
+    pub enable_import: bool,
+
     /// (optional) Nix search path, e.g. the value of `NIX_PATH` used
     /// for resolving items on the search path (such as `<nixpkgs>`).
     pub nix_path: Option<String>,
@@ -134,18 +145,34 @@ impl<'code, 'co, 'ro> Evaluation<'code, 'co, 'ro> {
 
         let file = source_map.add_file(location_str, code.into());
 
+        let mut builtins = builtins::pure_builtins();
+        builtins.extend(builtins::placeholders()); // these are temporary
+
         Evaluation {
             code,
             location,
             source_map,
             file,
+            builtins,
             io_handle: Box::new(DummyIO {}),
+            enable_import: false,
             nix_path: None,
             compiler_observer: None,
             runtime_observer: None,
         }
     }
 
+    #[cfg(feature = "impure")]
+    /// Initialise an `Evaluation` for the given snippet, with all
+    /// impure features turned on by default.
+    pub fn new_impure(code: &'code str, location: Option<PathBuf>) -> Self {
+        let mut eval = Self::new(code, location);
+        eval.enable_import = true;
+        eval.builtins.extend(builtins::impure_builtins());
+        eval.io_handle = Box::new(StdIO);
+        eval
+    }
+
     /// Clone the reference to the contained source code map. This is used after
     /// an evaluation for pretty error printing.
     pub fn source_map(&self) -> SourceCode {
@@ -173,8 +200,8 @@ impl<'code, 'co, 'ro> Evaluation<'code, 'co, 'ro> {
         // access to the parsed expression.
         result.expr = parsed.tree().expr();
 
-        let builtins =
-            crate::compiler::prepare_globals(Box::new(global_builtins(self.source_map())));
+        let source = self.source_map();
+        let builtins = crate::compiler::prepare_globals(self.builtins, source, self.enable_import);
 
         let mut noop_observer = observer::NoOpObserver::default();
         let compiler_observer = self.compiler_observer.take().unwrap_or(&mut noop_observer);