fix(tests/substitution): adapt to new module

LICENSE

@@ -0,0 +1,274 @@
+European Union Public Licence
+V. 1.2
+
+EUPL © the European Union 2007, 2016
+
+This European Union Public Licence (the ‘EUPL’) applies to the Work (as
+defined below) which is provided under the terms of this Licence. Any use of
+the Work, other than as authorised under this Licence is prohibited (to the
+extent such use is covered by a right of the copyright holder of the Work).
+
+The Work is provided under the terms of this Licence when the Licensor (as
+defined below) has placed the following notice immediately following the
+copyright notice for the Work: “Licensed under the EUPL”, or has expressed by
+any other means his willingness to license under the EUPL.
+
+1. Definitions
+
+In this Licence, the following terms have the following meaning:
+— ‘The Licence’: this Licence.
+— ‘The Original Work’: the work or software distributed or communicated by the
+  ‘Licensor under this Licence, available as Source Code and also as
+  ‘Executable Code as the case may be.
+— ‘Derivative Works’: the works or software that could be created by the
+  ‘Licensee, based upon the Original Work or modifications thereof. This
+  ‘Licence does not define the extent of modification or dependence on the
+  ‘Original Work required in order to classify a work as a Derivative Work;
+  ‘this extent is determined by copyright law applicable in the country
+  ‘mentioned in Article 15.
+— ‘The Work’: the Original Work or its Derivative Works.
+— ‘The Source Code’: the human-readable form of the Work which is the most
+  convenient for people to study and modify.
+
+— ‘The Executable Code’: any code which has generally been compiled and which
+  is meant to be interpreted by a computer as a program.
+— ‘The Licensor’: the natural or legal person that distributes or communicates
+  the Work under the Licence.
+— ‘Contributor(s)’: any natural or legal person who modifies the Work under
+  the Licence, or otherwise contributes to the creation of a Derivative Work.
+— ‘The Licensee’ or ‘You’: any natural or legal person who makes any usage of
+  the Work under the terms of the Licence.
+— ‘Distribution’ or ‘Communication’: any act of selling, giving, lending,
+  renting, distributing, communicating, transmitting, or otherwise making
+  available, online or offline, copies of the Work or providing access to its
+  essential functionalities at the disposal of any other natural or legal
+  person.
+
+2. Scope of the rights granted by the Licence
+
+The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
+sublicensable licence to do the following, for the duration of copyright
+vested in the Original Work:
+
+— use the Work in any circumstance and for all usage,
+— reproduce the Work,
+— modify the Work, and make Derivative Works based upon the Work,
+— communicate to the public, including the right to make available or display
+  the Work or copies thereof to the public and perform publicly, as the case
+  may be, the Work,
+— distribute the Work or copies thereof,
+— lend and rent the Work or copies thereof,
+— sublicense rights in the Work or copies thereof.
+
+Those rights can be exercised on any media, supports and formats, whether now
+known or later invented, as far as the applicable law permits so.
+
+In the countries where moral rights apply, the Licensor waives his right to
+exercise his moral right to the extent allowed by law in order to make
+effective the licence of the economic rights here above listed.
+
+The Licensor grants to the Licensee royalty-free, non-exclusive usage rights
+to any patents held by the Licensor, to the extent necessary to make use of
+the rights granted on the Work under this Licence.
+
+3. Communication of the Source Code
+
+The Licensor may provide the Work either in its Source Code form, or as
+Executable Code. If the Work is provided as Executable Code, the Licensor
+provides in addition a machine-readable copy of the Source Code of the Work
+along with each copy of the Work that the Licensor distributes or indicates,
+in a notice following the copyright notice attached to the Work, a repository
+where the Source Code is easily and freely accessible for as long as the
+Licensor continues to distribute or communicate the Work.
+
+4. Limitations on copyright
+
+Nothing in this Licence is intended to deprive the Licensee of the benefits
+from any exception or limitation to the exclusive rights of the rights owners
+in the Work, of the exhaustion of those rights or of other applicable
+limitations thereto.
+
+5. Obligations of the Licensee
+
+The grant of the rights mentioned above is subject to some restrictions and
+obligations imposed on the Licensee. Those obligations are the following:
+
+Attribution right: The Licensee shall keep intact all copyright, patent or
+trademarks notices and all notices that refer to the Licence and to the
+disclaimer of warranties. The Licensee must include a copy of such notices and
+a copy of the Licence with every copy of the Work he/she distributes or
+communicates. The Licensee must cause any Derivative Work to carry prominent
+notices stating that the Work has been modified and the date of modification.
+
+Copyleft clause: If the Licensee distributes or communicates copies of the
+Original Works or Derivative Works, this Distribution or Communication will be
+done under the terms of this Licence or of a later version of this Licence
+unless the Original Work is expressly distributed only under this version of
+the Licence — for example by communicating ‘EUPL v. 1.2 only’. The Licensee
+(becoming Licensor) cannot offer or impose any additional terms or conditions
+on the Work or Derivative Work that alter or restrict the terms of the
+Licence.
+
+Compatibility clause: If the Licensee Distributes or Communicates Derivative
+Works or copies thereof based upon both the Work and another work licensed
+under a Compatible Licence, this Distribution or Communication can be done
+under the terms of this Compatible Licence. For the sake of this clause,
+‘Compatible Licence’ refers to the licences listed in the appendix attached to
+this Licence. Should the Licensee's obligations under the Compatible Licence
+conflict with his/her obligations under this Licence, the obligations of the
+Compatible Licence shall prevail.
+
+Provision of Source Code: When distributing or communicating copies of the
+Work, the Licensee will provide a machine-readable copy of the Source Code or
+indicate a repository where this Source will be easily and freely available
+for as long as the Licensee continues to distribute or communicate the Work.
+
+Legal Protection: This Licence does not grant permission to use the trade
+names, trademarks, service marks, or names of the Licensor, except as required
+for reasonable and customary use in describing the origin of the Work and
+reproducing the content of the copyright notice.
+
+6. Chain of Authorship
+
+The original Licensor warrants that the copyright in the Original Work granted
+hereunder is owned by him/her or licensed to him/her and that he/she has the
+power and authority to grant the Licence.
+
+Each Contributor warrants that the copyright in the modifications he/she
+brings to the Work are owned by him/her or licensed to him/her and that he/she
+has the power and authority to grant the Licence.
+
+Each time You accept the Licence, the original Licensor and subsequent
+Contributors grant You a licence to their contributions to the Work, under the
+terms of this Licence.
+
+7. Disclaimer of Warranty
+
+The Work is a work in progress, which is continuously improved by numerous
+Contributors. It is not a finished work and may therefore contain defects or
+‘bugs’ inherent to this type of development.
+
+For the above reason, the Work is provided under the Licence on an ‘as is’
+basis and without warranties of any kind concerning the Work, including
+without limitation merchantability, fitness for a particular purpose, absence
+of defects or errors, accuracy, non-infringement of intellectual property
+rights other than copyright as stated in Article 6 of this Licence.
+
+This disclaimer of warranty is an essential part of the Licence and a
+condition for the grant of any rights to the Work.
+
+8. Disclaimer of Liability
+
+Except in the cases of wilful misconduct or damages directly caused to natural
+persons, the Licensor will in no event be liable for any direct or indirect,
+material or moral, damages of any kind, arising out of the Licence or of the
+use of the Work, including without limitation, damages for loss of goodwill,
+work stoppage, computer failure or malfunction, loss of data or any commercial
+damage, even if the Licensor has been advised of the possibility of such
+damage. However, the Licensor will be liable under statutory product liability
+laws as far such laws apply to the Work.
+
+9. Additional agreements
+
+While distributing the Work, You may choose to conclude an additional
+agreement, defining obligations or services consistent with this Licence.
+However, if accepting obligations, You may act only on your own behalf and on
+your sole responsibility, not on behalf of the original Licensor or any other
+Contributor, and only if You agree to indemnify, defend, and hold each
+Contributor harmless for any liability incurred by, or claims asserted against
+such Contributor by the fact You have accepted any warranty or additional
+liability.
+
+10. Acceptance of the Licence
+
+The provisions of this Licence can be accepted by clicking on an icon ‘I
+agree’ placed under the bottom of a window displaying the text of this Licence
+or by affirming consent in any other similar way, in accordance with the rules
+of applicable law. Clicking on that icon indicates your clear and irrevocable
+acceptance of this Licence and all of its terms and conditions.
+
+Similarly, you irrevocably accept this Licence and all of its terms and
+conditions by exercising any rights granted to You by Article 2 of this
+Licence, such as the use of the Work, the creation by You of a Derivative Work
+or the Distribution or Communication by You of the Work or copies thereof.
+
+11. Information to the public
+
+In case of any Distribution or Communication of the Work by means of
+electronic communication by You (for example, by offering to download the Work
+from a remote location) the distribution channel or media (for example, a
+website) must at least provide to the public the information requested by the
+applicable law regarding the Licensor, the Licence and the way it may be
+accessible, concluded, stored and reproduced by the Licensee.
+
+12. Termination of the Licence
+
+The Licence and the rights granted hereunder will terminate automatically upon
+any breach by the Licensee of the terms of the Licence. Such a termination
+will not terminate the licences of any person who has received the Work from
+the Licensee under the Licence, provided such persons remain in full
+compliance with the Licence.
+
+13. Miscellaneous
+
+Without prejudice of Article 9 above, the Licence represents the complete
+agreement between the Parties as to the Work.
+
+If any provision of the Licence is invalid or unenforceable under applicable
+law, this will not affect the validity or enforceability of the Licence as a
+whole. Such provision will be construed or reformed so as necessary to make it
+valid and enforceable.
+
+The European Commission may publish other linguistic versions or new versions
+of this Licence or updated versions of the Appendix, so far this is required
+and reasonable, without reducing the scope of the rights granted by the
+Licence. New versions of the Licence will be published with a unique version
+number.
+
+All linguistic versions of this Licence, approved by the European Commission,
+have identical value. Parties can take advantage of the linguistic version of
+their choice.
+
+14. Jurisdiction
+
+Without prejudice to specific agreement between parties,
+— any litigation resulting from the interpretation of this License, arising
+  between the European Union institutions, bodies, offices or agencies, as a
+  Licensor, and any Licensee, will be subject to the jurisdiction of the Court
+  of Justice of the European Union, as laid down in article 272 of the Treaty
+  on the Functioning of the European Union,
+— any litigation arising between other parties and resulting from the
+  interpretation of this License, will be subject to the exclusive
+  jurisdiction of the competent court where the Licensor resides or conducts
+  its primary business.
+
+15. Applicable Law
+
+Without prejudice to specific agreement between parties,
+— this Licence shall be governed by the law of the European Union Member State
+  where the Licensor has his seat, resides or has his registered office,
+— this licence shall be governed by Belgian law if the Licensor has no seat,
+  residence or registered office inside a European Union Member State.
+
+Appendix
+
+‘Compatible Licences’ according to Article 5 EUPL are:
+— GNU General Public License (GPL) v. 2, v. 3
+— GNU Affero General Public License (AGPL) v. 3
+— Open Software License (OSL) v. 2.1, v. 3.0
+— Eclipse Public License (EPL) v. 1.0
+— CeCILL v. 2.0, v. 2.1
+— Mozilla Public Licence (MPL) v. 2
+— GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
+— Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
+  works other than software
+— European Union Public Licence (EUPL) v. 1.1, v. 1.2
+— Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or
+  Strong Reciprocity (LiLiQ-R+)
+
+— The European Commission may update this Appendix to later versions of the
+  above licences without producing a new version of the EUPL, as long as they
+  provide the rights granted in Article 2 of this Licence and protect the
+  covered Source Code from exclusive appropriation.
+— All other changes or additions to this Appendix require the production of a
+  new EUPL version.

modules/default.nix

@@ -6,6 +6,7 @@
 }:
 let
   cfg = config.services.tvix-binary-cache;
+  settingsFormat = pkgs.formats.toml { };
   systemdHardening = {
     PrivateDevices = true;
     PrivateTmp = true;
@@ -13,7 +14,7 @@ let
     ProtectKernelTunables = true;
     RestrictSUIDSGID = true;
 
-    ProtectSystem = "strict";
+    #ProtectSystem = "strict";
     ProtectKernelLogs = true;
     ProtectProc = "invisible";
     PrivateUsers = true;
@@ -28,61 +29,78 @@ in
   options = {
     services.tvix-binary-cache = {
       enable = lib.mkEnableOption "BinaryCache using tvix ca-store";
-      blob-service-addr = lib.mkOption {
-        type = lib.types.str;
-        default = "objectstore+file://%S/tvix-castore/blobs.object-store";
-        description = ''
-          `blob-service-addr` option for the mutualized content addressed storage.
-        '';
-      };
-      directory-service-addr = lib.mkOption {
-        type = lib.types.str;
-        default = "sled://%S/tvix-castore/directories.sled";
-        description = ''
-          `directory-service-addr` option for the mutualized content addressed storage.
-        '';
+      castoreDir = lib.mkOption {
+        type = lib.types.nullOr lib.types.str;
+        default = "/var/lib/castore";
       };
       caches = lib.mkOption {
         type = lib.types.attrsOf (
           lib.types.submodule (
-            { name, ... }:
+            { name, ... }@cacheAttrs:
             {
-              options = {
-                port = lib.mkOption {
-                  type = lib.types.port;
-                  default = 9000;
+              config =
+                let
+                  common-composition = {
+                    blobservices.default = {
+                      type = "objectstore";
+                      object_store_url = "file://${cfg.castoreDir}/blobs.object-store";
+                      object_store_options = { };
                     };
+                    directoryservices = {
+                      objectstore = {
+                        type = "objectstore";
+                        object_store_url = "file://${cfg.castoreDir}/directories.object-store";
+                        object_store_options = { };
+                      };
+                      memory = {
+                        type = "memory";
+                      };
+                      cache = {
+                        type = "cache";
+                        near = "memory";
+                        far = "objectstore";
+                      };
+                      default = {
+                        type = "router";
+                        writes = "objectstore";
+                        reads = "cache";
+                      };
+                    };
+                  };
+                in
+                {
+                  nar-bridge-composition = lib.recursiveUpdate common-composition {
+                    pathinfoservices.default = {
+                      type = "grpc";
+                      url = "grpc+http://${cacheAttrs.config.grpcListenAddress}";
+                    };
+                  };
+                  tvix-daemon-composition = lib.recursiveUpdate common-composition {
+                    pathinfoservices.default = {
+                      type = "sled";
+                      is_temporary = false;
+                      path = "/var/lib/tvix-daemon-${name}/pathinfos.sled";
+                    };
+                  };
+                };
+
+              options = {
+                grpcListenAddress = lib.mkOption { type = lib.types.str; };
+                narBridgeListenAddress = lib.mkOption { type = lib.types.str; };
+                nar-bridge-composition = lib.mkOption { inherit (settingsFormat) type; };
+                tvix-daemon-composition = lib.mkOption { inherit (settingsFormat) type; };
                 name = lib.mkOption {
                   type = lib.types.str;
                   description = "Name of the cache";
                   default = name;
                   defaultText = lib.literalMD "Defaults to attribute name in services.tvix-binary-cache.caches";
                 };
-                path-info-service-addr = lib.mkOption {
-                  type = with lib.types; str;
-                  description = "Path info service path";
-                  default = "sled://%S/%N/pathinfo.sled";
-                };
                 remote-path-info-service-addr = lib.mkOption {
                   type = with lib.types; nullOr str;
                   description = "Upstream cache to substitute from if nothing in ";
                   example = "nix+https://cache.nixos.org?trusted-public-keys=cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=";
                   default = null;
                 };
-                blob-service-addr = lib.mkOption {
-                  type = with lib.types; nullOr str;
-                  default = null;
-                  description = ''
-                    Use a specific blob service and do not use the mutualized one.
-                  '';
-                };
-                directory-service-addr = lib.mkOption {
-                  type = with lib.types; nullOr str;
-                  default = null;
-                  description = ''
-                    Use a specific directory address and do not use the mutualized one.
-                  '';
-                };
               };
             }
           )
@@ -93,75 +111,48 @@ in
 
   config = lib.mkIf cfg.enable {
 
-    environment.systemPackages = [ pkgs.tvix-store ];
+    environment.systemPackages = [ pkgs.tvix ];
+    users.users.tvix-castore = {
+      isSystemUser = true;
+      group = "tvix-castore";
+    };
+    users.groups.tvix-castore = { };
+
+    systemd.tmpfiles.rules = [ "d ${cfg.castoreDir} 770 root tvix-castore -" ];
 
     systemd.services = lib.mkMerge (
-      (lib.singleton {
-        tvix-castore = {
-          wants = [ "tvix-castore.service" ];
-          after = [ "tvix-castore.service" ];
+      (lib.singleton { })
+      ++ (lib.mapAttrsToList (name: cache: {
+        "tvix-daemon-${cache.name}" = {
           environment = {
-            BLOB_SERVICE_ADDR = cfg.blob-service-addr;
-            DIRECTORY_SERVICE_ADDR = cfg.directory-service-addr;
-            PATH_INFO_SERVICE_ADDR = "sled://%S/tvix-castore/pathinfo.sled"; # Unused but probably needed
+            EXPERIMENTAL_STORE_COMPOSITION = settingsFormat.generate "Config.toml" cache.tvix-daemon-composition;
           };
           serviceConfig = {
-            ExecStart = "${pkgs.tvix-store}/bin/tvix-store --otlp=false daemon --listen-address=\"%t/tvix-castore/socket\"";
-            DynamicUser = true;
-            User = "tvix-binary-cache";
-            StateDirectory = "tvix-castore";
-            RuntimeDirectory = "tvix-castore";
+            UMask = "007";
+            #ExecStart = "${pkgs.tvix-store}/bin/tvix-store --otlp=false daemon --listen-address=\"%t/tvix-castore/socket\"";
+            ExecStart = "${pkgs.tvix}/bin/tvix-store --otlp=false daemon --listen-address=\"${cache.grpcListenAddress}\"";
+            StateDirectory = "tvix-daemon-${cache.name}";
+            RuntimeDirectory = "tvix-daemon-${cache.name}";
+            User = "tvix-castore";
+            Group = "tvix-castore";
           } // systemdHardening;
-
         };
-      })
-      ++ (lib.mapAttrsToList (
-        name: cfg:
-        let
-          unitName = "tvix-store-${cfg.name}";
-        in
-        {
-          "nar-bridge-${cfg.name}" = {
-            wants = [ "tvix-store-${cfg.name}.service" ];
+        "narbridge-${cache.name}" = {
           wantedBy = [ "multi-user.target" ];
-            after = [ "tvix-store-${cfg.name}.service" ];
-            serviceConfig = rec {
-              ExecStart = "${lib.getExe pkgs.nar-bridge-go} --otlp=false --listen-addr=\"[::1]:${builtins.toString cfg.port}\" --store-addr=\"unix://%t/${unitName}/socket\"";
-
-              DynamicUser = true;
-              User = "tvix-binary-cache";
-            } // systemdHardening;
-          };
-          ${unitName} = {
-            wants = [ "tvix-castore.service" ];
-            after = [ "tvix-castore.service" ];
+          wants = [ "tvix-daemon-${cache.name}.service" ];
+          after = [ "tvix-daemon-${cache.name}.service" ];
           environment = {
-              BLOB_SERVICE_ADDR =
-                if cfg.blob-service-addr != null then
-                  cfg.blob-service-addr
-                else
-                  "grpc+unix://%t/tvix-castore/socket";
-              DIRECTORY_SERVICE_ADDR =
-                if cfg.directory-service-addr != null then
-                  cfg.directory-service-addr
-                else
-                  "grpc+unix://%t/tvix-castore/socket";
-              PATH_INFO_SERVICE_ADDR = cfg.path-info-service-addr;
-              REMOTE_PATH_INFO_SERVICE_ADDR = lib.mkIf (
-                cfg.remote-path-info-service-addr != null
-              ) cfg.remote-path-info-service-addr;
+            EXPERIMENTAL_STORE_COMPOSITION = settingsFormat.generate "Config.toml" cache.nar-bridge-composition;
           };
           serviceConfig = {
-              ExecStart = "${pkgs.tvix-store}/bin/tvix-store --otlp=false daemon --listen-address=\"%t/${unitName}/socket\"";
-              DynamicUser = true;
-              User = "tvix-binary-cache";
-              StateDirectory = unitName;
-              RuntimeDirectory = unitName;
+            ExecStart = "${pkgs.tvix}/bin/nar-bridge --otlp=false --listen-address=\"${cache.narBridgeListenAddress}\"";
+            User = "tvix-castore";
+            Group = "tvix-castore";
+            RuntimeDirectory = "narbridge-${cache.name}";
           } // systemdHardening;
         };
 
-        }
-      ) cfg.caches)
+      }) cfg.caches)
     );
 
   };

modules/nginx.nix

@@ -21,12 +21,11 @@
       enable = true;
       recommendedProxySettings = true;
       virtualHosts.${config.services.tvix-binary-cache.nginx.host} = {
-        default = true;
         locations = lib.mkMerge (
           lib.mapAttrsToList (name: cfg: {
             "/${name}".return = "302 /${name}/";
             "/${name}/" = {
-              proxyPass = "http://localhost:${toString cfg.port}/";
+              proxyPass = "http://${toString cfg.narBridgeListenAddress}/";
             };
           }) config.services.tvix-binary-cache.caches
         );

pkgs/nar-bridge-go/default.nix

@@ -1,9 +0,0 @@
-{ buildGoModule, tvix-src }:
-buildGoModule rec {
-  pname = "nar-bridge-go";
-  version = "0.1.0";
-  src = tvix-src;
-  sourceRoot = "${src.name}/tvix/nar-bridge-go";
-  vendorHash = "sha256-7jugbC5sEGhppjiZgnoLP5A6kQSaHK9vE6cXVZBG22s=";
-  meta.mainProgram = "nar-bridge-http";
-}

pkgs/overlay.nix

@@ -1,5 +1,8 @@
 final: _: {
-  tvix-store = final.callPackage ./tvix { packages = [ "tvix-store" ]; };
-  nar-bridge-go = final.callPackage ./nar-bridge-go { };
-  tvix-src = final.callPackage ./tvix-src.nix { };
+  tvix = final.callPackage ./tvix {
+    packages = [
+      "nar-bridge"
+      "tvix-store"
+    ];
+  };
 }

pkgs/tvix-src.nix

@@ -1,7 +0,0 @@
-{ fetchgit }:
-fetchgit {
-  name = "tvix";
-  url = "https://code.tvl.fyi/depot.git";
-  rev = "f0b0a6572ff7908c890963f6462af27dcef80bd7";
-  hash = "sha256-7l/UHq19oyF4743olRgAk0ikc92VUqKrxcfj17GAgt0=";
-}

pkgs/tvix/default.nix

@@ -1,5 +1,5 @@
 {
-  tvix-src,
+  fetchgit,
   rustPlatform,
   protobuf,
   packages ? [ ],
@@ -10,23 +10,33 @@ rustPlatform.buildRustPackage rec {
   inherit pname;
   version = "0.1.0";
 
-  src = tvix-src;
+  src = fetchgit {
+    name = "tvix";
+    url = "https://cl.tvl.fyi/depot";
+    rev = "507a5c1b7376a2f9617b286f142ec8115cf19a6e";
+    hash = "sha256-QTAzM8b59NtQSjvlu5mcO13t4T58Mv00ax7/RnRENwk=";
+  };
 
-  sourceRoot = "${src.name}/tvix";
+  patches = [ ];
+  postPatch = "cd tvix";
 
   doCheck = false;
 
-  cargoBuildFlags = builtins.concatStringsSep " " (builtins.map (v: "-p ${v}") packages);
+  cargoBuildFlags = builtins.concatStringsSep " " (
+    builtins.map (v: "-p ${v}") packages
+    ++ [
+      "--features"
+      "tvix-store/xp-store-composition"
+    ]
+  );
 
   nativeBuildInputs = [ protobuf ];
 
   PROTO_ROOT = "/build/${src.name}";
 
   cargoLock = {
-    lockFile = ./Cargo.lock;
-    outputHashes = {
-      "wu-manber-0.1.0" = "sha256-7YIttaQLfFC/32utojh2DyOHVsZiw8ul/z0lvOhAE/4=";
-    };
+    lockFile = src + "/tvix/Cargo.lock";
+    allowBuiltinFetchGit = true;
   };
   meta = { };
 }

tests/default.nix

@@ -4,7 +4,11 @@
   pkgs ? import nixpkgs { overlays = [ (import ../pkgs/overlay.nix) ]; },
 }:
 {
+  # Disabled since nar-bridge doesn't expose gRPC
+  # ingest = pkgs.callPackage ./ingest.nix { };
   multi-cache = pkgs.callPackage ./multi-cache.nix { };
-  ingest = pkgs.callPackage ./ingest.nix { };
   substitution = pkgs.callPackage ./substitution.nix { };
+  upstream-cache-tvix = pkgs.callPackage ./upstream-cache-tvix.nix { };
+  upstream-cache-http-directory = pkgs.callPackage ./upstream-cache-http-directory.nix { };
+  upstream-cache-nix-serve = pkgs.callPackage ./upstream-cache-nix-serve.nix { };
 }

tests/ingest.nix

@@ -41,7 +41,7 @@ pkgs.testers.runNixOSTest (_: {
     import time
     start_all()
     cache.wait_for_unit("nginx.service")
-    cache.wait_for_unit("nar-bridge-cache.service")
+    cache.wait_for_unit("tvix-store-cache.service")
     time.sleep(1)
     socket_addr = "grpc+unix:///run/tvix-store-cache/socket"
     cache.succeed(f"BLOB_SERVICE_ADDR={socket_addr} DIRECTORY_SERVICE_ADDR={socket_addr} PATH_INFO_SERVICE_ADDR={socket_addr} tvix-store copy ${builtins.toString references}")

tests/multi-cache.nix

@@ -35,23 +35,19 @@ pkgs.testers.runNixOSTest (_: {
     };
   };
   testScript = ''
-    import sys
     import time
     start_all()
     machine.wait_for_unit("nginx.service")
-    machine.wait_for_unit("nar-bridge-one.service")
-    machine.wait_for_unit("nar-bridge-two.service")
+    machine.wait_for_unit("tvix-store-one.service")
+    machine.wait_for_unit("tvix-store-two.service")
     time.sleep(1)
     with subtest("Nar bridge home"):
-        out = machine.succeed("curl -L http://127.0.0.1/one")
-        if out != "nar-bridge":
-            sys.exit(1)
+        machine.succeed("curl -L http://127.0.0.1/one/nix-cache-info")
     with subtest("Nar upload"):
         machine.succeed("nix copy --to 'http://127.0.0.1/one/?compression=none' ${hello}")
     with subtest("narinfo retrieve"):
         narHash = "${hello}"[11:11+32]
         machine.succeed(f"curl -f 'http://127.0.0.1/one/{narHash}.narinfo'")
         machine.fail(f"curl -f 'http://127.0.0.1/two/{narHash}.narinfo'")
-
   '';
 })

tests/substitution.nix

@@ -22,7 +22,8 @@ pkgs.testers.runNixOSTest (_: {
           host = "cache";
         };
         caches = {
-          cache.port = 8000;
+          cache.grpcListenAddress = "[::1]:5000";
+          cache.narBridgeListenAddress = "[::1]:8000";
         };
       };
 
@@ -42,16 +43,13 @@ pkgs.testers.runNixOSTest (_: {
       };
   };
   testScript = ''
-    import sys
     import time
     start_all()
     cache.wait_for_unit("nginx.service")
-    cache.wait_for_unit("nar-bridge-cache.service")
+    cache.wait_for_unit("narbridge-cache.service")
     time.sleep(1)
     with subtest("Nar bridge home"):
-        out = cache.succeed("curl -L http://127.0.0.1/cache")
-        if out != "nar-bridge":
-            sys.exit(1)
+        cache.succeed("curl -f http://127.0.0.1/cache/nix-cache-info")
     with subtest("Path signature and copy"):
         # Sign
         cache.succeed("nix store sign -k ${./cache-keys/privkey} ${hello}")

tests/upstream-cache-http-directory.nix

@@ -0,0 +1,81 @@
+{ pkgs }:
+let
+  #hello = pkgs.hello.overrideAttrs { pname = "custom-hello"; };
+  inherit (pkgs) hello;
+in
+pkgs.testers.runNixOSTest (_: {
+  name = "caching of upstream nar-store";
+  nodes = {
+    cache = {
+      imports = [
+        ./common
+        ../modules
+      ];
+
+      system.extraDependencies = [ hello ];
+
+      services = {
+        tvix-binary-cache = {
+          enable = true;
+          enableNginx = true;
+          nginx = {
+            clientMaxBodySize = "50G";
+            host = "cache";
+          };
+          caches = {
+            cache = {
+              port = 8000;
+              remote-path-info-service-addr = "nix+http://localhost/upstream/"; # ?trusted-public-keys=${lib.escapeURL (builtins.readFile ./cache-keys/pubkey)}";
+            };
+          };
+        };
+
+        nginx = {
+          virtualHosts.cache = {
+            default = true;
+            locations = {
+              "/upstream".return = "302 /upstream/";
+              "/upstream/".alias = "/srv/";
+            };
+          };
+        };
+      };
+      networking.firewall.allowedTCPPorts = [ 80 ];
+    };
+    client =
+      { lib, ... }:
+      {
+        imports = [ ./common ];
+        nix.settings = {
+          substituters = lib.mkForce [ "http://cache/cache" ];
+          trusted-public-keys = lib.mkForce [ (builtins.readFile ./cache-keys/pubkey) ];
+        };
+      };
+  };
+  testScript = ''
+    import time
+    start_all()
+    cache.wait_for_unit("nginx.service")
+    cache.wait_for_unit("tvix-store-cache.service")
+    time.sleep(1)
+
+    def delete_and_substitute():
+        client.succeed("nix-store --delete ${hello}")
+        client.fail("stat ${hello}")
+        client.succeed("nix-store -r ${hello}")
+        client.succeed("stat ${hello}")
+
+    with subtest("Upload"):
+        cache.succeed("nix store sign -k ${./cache-keys/privkey} ${hello}")
+        cache.succeed("nix copy --to file:///srv ${hello}")
+        narHash = "${hello}"[11:11+32]
+        out = cache.succeed(f"curl -f 'http://localhost/upstream/{narHash}.narinfo'")
+        print(out)
+        cache.succeed(f"curl -f 'http://localhost/cache/{narHash}.narinfo'")
+    with subtest("Try to substitute from cache"):
+        delete_and_substitute()
+    with subtest("Check effective caching"):
+        cache.succeed("rm -rf /srv")
+        delete_and_substitute()
+  '';
+})

tests/upstream-cache-nix-serve.nix

@@ -0,0 +1,87 @@
+{ pkgs }:
+let
+  #hello = pkgs.hello.overrideAttrs { pname = "custom-hello"; };
+  inherit (pkgs) hello;
+in
+pkgs.testers.runNixOSTest (_: {
+  name = "caching of upstream nar-store";
+  nodes = {
+    cache = {
+      imports = [
+        ./common
+        ../modules
+      ];
+
+      system.extraDependencies = [ hello ];
+
+      services = {
+        tvix-binary-cache = {
+          enable = true;
+          enableNginx = true;
+          nginx = {
+            clientMaxBodySize = "50G";
+            host = "cache";
+          };
+          caches = {
+            cache = {
+              port = 8000;
+              remote-path-info-service-addr = "nix+http://localhost/upstream/"; # ?trusted-public-keys=${lib.escapeURL (builtins.readFile ./cache-keys/pubkey)}";
+            };
+          };
+        };
+
+        nix-serve = {
+          enable = true;
+          port = 8001;
+          secretKeyFile = "${./cache-keys/privkey}";
+        };
+
+        nginx = {
+          virtualHosts.cache = {
+            default = true;
+            locations = {
+              "/upstream".return = "302 /upstream/";
+              "/upstream/".proxyPass = "http://localhost:8001/";
+            };
+          };
+        };
+
+      };
+      networking.firewall.allowedTCPPorts = [ 80 ];
+    };
+    client =
+      { lib, ... }:
+      {
+        imports = [ ./common ];
+        nix.settings = {
+          substituters = lib.mkForce [ "http://cache/cache" ];
+          trusted-public-keys = lib.mkForce [ (builtins.readFile ./cache-keys/pubkey) ];
+        };
+      };
+  };
+  testScript = ''
+    import time
+    start_all()
+    cache.wait_for_unit("nginx.service")
+    cache.wait_for_unit("tvix-store-cache.service")
+    cache.wait_for_unit("nix-serve.service")
+    cache.wait_for_open_port(8001)
+    time.sleep(1)
+
+    def delete_and_substitute():
+        client.succeed("nix-store --delete ${hello}")
+        client.fail("stat ${hello}")
+        client.succeed("nix-store -r ${hello}")
+        client.succeed("stat ${hello}")
+
+    with subtest("Upload"):
+        narHash = "${hello}"[11:11+32]
+        cache.succeed(f"curl -f 'http://127.0.0.1/upstream/{narHash}.narinfo'")
+        cache.succeed(f"curl -f 'http://127.0.0.1/cache/{narHash}.narinfo'")
+    with subtest("Try to substitute from cache"):
+        delete_and_substitute()
+    with subtest("Check effective caching"):
+        cache.succeed("systemctl stop nix-serve.service")
+        delete_and_substitute()
+  '';
+})

tests/upstream-cache-tvix.nix

@@ -0,0 +1,79 @@
+{ pkgs }:
+let
+  inherit (pkgs) hello;
+in
+pkgs.testers.runNixOSTest (_: {
+  name = "caching of upstream nar-store";
+  nodes = {
+    cache = {
+      imports = [
+        ./common
+        ../modules
+      ];
+
+      system.extraDependencies = [ hello ];
+
+      services.tvix-binary-cache = {
+        enable = true;
+        enableNginx = true;
+        nginx = {
+          clientMaxBodySize = "50G";
+          host = "cache";
+        };
+        caches = {
+          cache = {
+            port = 8000;
+            path-info-service-addr = "nix+http://localhost/upstream/"; # ?trusted-public-keys=${lib.escapeURL (builtins.readFile ./cache-keys/pubkey)}";
+          };
+          upstream = {
+            port = 8001;
+            blob-service-addr = "objectstore+file://%S/tvix-store-upstream/blobs.object-store";
+            directory-service-addr = "sled://%S/tvix-store-upstream/directories.sled";
+          };
+        };
+      };
+
+      services.nginx = {
+        virtualHosts.cache = {
+          default = true;
+        };
+      };
+      networking.firewall.allowedTCPPorts = [ 80 ];
+    };
+    client =
+      { lib, ... }:
+      {
+        imports = [ ./common ];
+        nix.settings = {
+          substituters = lib.mkForce [ "http://cache/cache" ];
+          trusted-public-keys = lib.mkForce [ (builtins.readFile ./cache-keys/pubkey) ];
+        };
+      };
+  };
+  testScript = ''
+    import time
+    start_all()
+    cache.wait_for_unit("nginx.service")
+    cache.wait_for_unit("tvix-store-cache.service")
+    cache.wait_for_unit("tvix-store-upstream.service")
+    time.sleep(1)
+
+    def delete_and_substitute():
+        client.succeed("nix-store --delete ${hello}")
+        client.fail("stat ${hello}")
+        client.succeed("nix-store -r ${hello}")
+        client.succeed("stat ${hello}")
+
+    with subtest("Upload"):
+        cache.succeed("nix store sign -k ${./cache-keys/privkey} ${hello}")
+        cache.succeed("nix copy --to 'http://localhost/upstream/?compression=none' ${hello}")
+        narHash = "${hello}"[11:11+32]
+        cache.succeed(f"curl -f 'http://localhost/upstream/{narHash}.narinfo'")
+        cache.succeed(f"curl -f 'http://localhost/cache/{narHash}.narinfo'")
+    with subtest("Try to substitute from cache"):
+        delete_and_substitute()
+    with subtest("Check effective caching"):
+        cache.succeed("systemctl stop tvix-store-upstream.service")
+        delete_and_substitute()
+  '';
+})