BinaryCacheStore: Optionally write a NAR listing

The store parameter "write-nar-listing=1" will cause BinaryCacheStore
to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’
added to the binary cache. This file contains an XZ-compressed JSON
file describing the contents of the NAR, excluding the contents of
regular files.

E.g.

  {
    "version": 1,
    "root": {
      "type": "directory",
      "entries": {
        "lib": {
          "type": "directory",
          "entries": {
            "Mcrt1.o": {
              "type": "regular",
              "size": 1288
            },
            "Scrt1.o": {
              "type": "regular",
              "size": 3920
            },
          }
        }
      }
      ...
    }
  }

(The actual file has no indentation.)

This is intended to speed up the NixOS channels programs index
generator [1], since fetching gazillions of large NARs from
cache.nixos.org is currently a bottleneck for updating the regular
(non-small) channel.

[1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc

1 files changed, 7 insertions, 1 deletions

diff --git a/src/libstore/binary-cache-store.hh b/src/libstore/binary-cache-store.hh
index 574696cf3a..333cf08562 100644
--- a/src/libstore/binary-cache-store.hh
+++ b/src/libstore/binary-cache-store.hh
@@ -19,12 +19,16 @@ private:
 
     std::string compression;
 
+    bool writeNARListing;
+
 protected:
 
     BinaryCacheStore(const Params & params);
 
     [[noreturn]] void notImpl();
 
+public:
+
     virtual bool fileExists(const std::string & path) = 0;
 
     virtual void upsertFile(const std::string & path, const std::string & data) = 0;
@@ -37,6 +41,8 @@ protected:
 
     std::shared_ptr<std::string> getFile(const std::string & path);
 
+protected:
+
     bool wantMassQuery_ = false;
     int priority = 50;
 
@@ -86,7 +92,7 @@ public:
 
     bool wantMassQuery() override { return wantMassQuery_; }
 
-    void addToStore(const ValidPathInfo & info, const std::string & nar,
+    void addToStore(const ValidPathInfo & info, const ref<std::string> & nar,
         bool repair = false, bool dontCheckSigs = false) override;
 
     Path addToStore(const string & name, const Path & srcPath,