Commit graph

3127 commits

Author SHA1 Message Date
Eelco Dolstra 9de6d10d11 Get rid of $NIX_BINARY_CACHES
You can use ‘--option binary-caches URLs’ instead.
2012-07-30 16:39:31 -04:00
Eelco Dolstra ab42bf1dab nix-install-package: Support binary caches
The .nixpkg file format is extended to optionally include the URL of a
binary cache, which will be used in preference to the manifest URL
(which can be set to a non-existent value).
2012-07-30 16:11:02 -04:00
Eelco Dolstra d059bf48e4 Pass configuration settings to the substituters
Previously substituters could read nix.conf themselves, but this
didn't take --option flags into account.
2012-07-30 16:09:54 -04:00
Eelco Dolstra f9613da180 Remove unused variables 2012-07-30 15:43:37 -04:00
Eelco Dolstra 6183cf2f19 Fix whitespace 2012-07-30 15:42:18 -04:00
Eelco Dolstra f15083c10a Document the --option flag
Pointed out by Daniel Santa Cruz on IRC.
2012-07-29 14:37:40 -04:00
Eelco Dolstra 66a3ac6a56 Allow a binary cache to declare that it doesn't support "nix-env -qas"
Querying all substitutable paths via "nix-env -qas" is potentially
hard on a server, since it involves sending thousands of HEAD
requests.  So a binary cache must now have a meta-info file named
"nix-cache-info" that specifies whether the server wants this.  It
also specifies the store prefix so that we don't send useless queries
to a binary cache for a different store prefix.
2012-07-27 18:16:05 -04:00
Eelco Dolstra 6ecf4f13f6 Use "set -x" in the tests to see where a test fails 2012-07-27 14:33:01 -04:00
Eelco Dolstra e6ab52cdd1 Test "nix-env -qas" with the binary cache substituter 2012-07-27 14:15:03 -04:00
Eelco Dolstra 73acb8b836 Let build.cc verify the expected hash of a substituter's output
Since SubstitutionGoal::finished() in build.cc computes the hash
anyway, we can prevent the inefficiency of computing the hash twice by
letting the substituter tell Nix about the expected hash, which can
then verify it.
2012-07-27 12:16:02 -04:00
Eelco Dolstra fbf59d95f6 Remove more tabs 2012-07-27 10:56:33 -04:00
Eelco Dolstra 3a8f841612 download-using-manifests: Don't use nix-prefetch-url
Instead call curl directly and pipe it into ‘nix-store --restore’.
This saves I/O and prevents creating garbage in the Nix store.
2012-07-27 10:47:36 -04:00
Eelco Dolstra b4ea83249b Remove trailing whitespace / tabs 2012-07-27 09:59:18 -04:00
Eelco Dolstra 7f8e805c8e download-from-binary-cache: Only use the default cache for /nix/store 2012-07-27 09:57:42 -04:00
Eelco Dolstra 67c6f3eded nix-push: Support generating a manifest again
This makes all the tests succeed.  Woohoo!
2012-07-26 18:28:12 -04:00
Eelco Dolstra 50395b71a9 Fix the substituter tests 2012-07-26 17:36:07 -04:00
Eelco Dolstra e3ce54ccee nix-push: Remove the upload feature 2012-07-26 17:13:38 -04:00
Eelco Dolstra 609586a16d Add a test for the binary cache substituter 2012-07-26 17:13:14 -04:00
Eelco Dolstra 7861260a5f Clear NIX_STORE when running the tests 2012-07-26 17:12:42 -04:00
Eelco Dolstra 7892ad15ab download-from-binary-cache: Support file://
The file:// URI schema requires checking for errors in a more general
way.  Also, don't cache file:// lookups.
2012-07-26 17:11:11 -04:00
Eelco Dolstra dbce685e91 Add some .gitignore entries 2012-07-26 17:10:28 -04:00
Eelco Dolstra 8c79100839 Merge branch 'master' into no-manifests 2012-07-26 15:14:33 -04:00
Eelco Dolstra 3a4623afbb Set permissions on temporary build directories to 0700
Fixes #39.
2012-07-26 15:04:40 -04:00
Eelco Dolstra 2605f4f4e6 nix-profile.sh: Don't set NIX_REMOTE on single user installations
Commit 6a214f3e06 reused the NixOS
environment initialisation for nix-profile.sh, but this is
inappropriate on systems that don't have multi-user support enabled.
2012-07-25 17:06:09 -04:00
Eelco Dolstra 477b0fbeca Subscribe to the Nixpkgs rather than NixOS channel 2012-07-25 16:56:56 -04:00
Shea Levy f5954e2d94 prim_import: When importing .drvs, allocate the intermediate attrset on the heap just in case it escapes the stack frame. 2012-07-25 09:30:17 -04:00
Shea Levy 1ef2d5765b Turn tests back on 2012-07-25 09:30:17 -04:00
Shea Levy b1112bbef1 import: If the path is a valid .drv file, parse it and generate a derivation attrset.
The generated attrset has drvPath and outPath with the right string context, type 'derivation', outputName with
the right name, all with a list of outputs, and an attribute for each output.

I see three uses for this (though certainly there may be more):

* Using derivations generated by something besides nix-instantiate (e.g. guix)

* Allowing packages provided by channels to be used in nix expressions. If a channel installed a valid deriver
  for each package it provides into the store, then those could be imported and used as dependencies or installed
  in environment.systemPackages, for example.

* Enable hydra to be consistent in how it treats inputs that are outputs of another build. Right now, if an
  input is passed as an argument to the job, it is passed as a derivation, but if it is accessed via NIX_PATH
  (i.e. through the <> syntax), then it is a path that can be imported. This is problematic because the build
  being depended upon may have been built with non-obvious arguments passed to its jobset file. With this
  feature, hydra can just set the name of that input to the path to its drv file in NIX_PATH
2012-07-25 09:30:17 -04:00
Shea Levy 566a30c007 Disable tests temporarily 2012-07-25 09:30:17 -04:00
Eelco Dolstra e98c029717 Handle platforms that don't support linking to a symlink
E.g. Darwin doesn't allow this.
2012-07-23 18:42:18 -04:00
Eelco Dolstra fd63c8bfcd Unlink the right file 2012-07-23 18:06:37 -04:00
Eelco Dolstra 0f65793f94 Add a test for Nix store optimisation 2012-07-23 17:40:23 -04:00
Eelco Dolstra 680ab6f83d Garbage collect unused links in /nix/store/.links
Incremental optimisation requires creating links in /nix/store/.links
to all files in the store.  However, this means that if we delete a
store path, no files are actually deleted because links in
/nix/store/.links still exists.  So we need to check /nix/store/.links
for files with a link count of 1 and delete them.
2012-07-23 17:14:16 -04:00
Eelco Dolstra 6193105710 Automatically optimise the Nix store when a new path is added
Auto-optimisation is enabled by default.  It can be turned off by
setting auto-optimise-store to false in nix.conf.
2012-07-23 17:14:15 -04:00
Eelco Dolstra 564fb7d9fa optimiseStore(): Use a content-addressed file store in /nix/store/.links
optimiseStore() now creates persistent, content-addressed hard links
in /nix/store/.links.  For instance, if it encounters a file P with
hash H, it will create a hard link

  P' = /nix/store/.link/<H>

to P if P' doesn't already exist; if P' exist, then P is replaced by a
hard link to P'.  This is better than the previous in-memory map,
because it had the tendency to unnecessarily replace hard links with a
hard link to whatever happened to be the first file with a given hash
it encountered.  It also allows on-the-fly, incremental optimisation.
2012-07-23 17:14:15 -04:00
Eelco Dolstra ed59bf7a18 nix-build: Support the ‘-’ argument to build an expression from stdin 2012-07-23 17:11:12 -04:00
Eelco Dolstra 6852289c46 Use lutimes() if available to canonicalise the timestamp of symlinks
Also use utimes() instead of utime() if lutimes() is not available.
2012-07-23 16:52:25 -04:00
Eelco Dolstra 1832ab71db Bump version 2012-07-18 17:17:23 -04:00
Eelco Dolstra 98193bb440 Remove RPM builds that don't evaluate 2012-07-18 16:50:56 -04:00
Eelco Dolstra b7fd2c2822 Use "#pragma once" to prevent repeated header file inclusion 2012-07-18 14:59:03 -04:00
Eelco Dolstra 58337e0e61 Set release date 2012-07-18 11:51:27 -04:00
Eelco Dolstra aa115e22df download-from-binary-cache: Print correct URL 2012-07-18 11:01:17 -04:00
Eelco Dolstra fe241ece29 Merge branch 'master' into no-manifests 2012-07-18 10:47:59 -04:00
Eelco Dolstra ccc52adfb2 Add function queryPathFromHashPart()
To implement binary caches efficiently, Hydra needs to be able to map
the hash part of a store path (e.g. "gbg...zr7") to the full store
path (e.g. "/nix/store/gbg...kzr7-subversion-1.7.5").  (The binary
cache mechanism uses hash parts as a key for looking up store paths to
ensure privacy.)  However, doing a search in the Nix store for
/nix/store/<hash>* is expensive since it requires reading the entire
directory.  queryPathFromHashPart() prevents this by doing a cheap
database lookup.
2012-07-17 18:55:39 -04:00
Eelco Dolstra 220818f758 queryPathInfo(): return hash in base-32 if desired
Cherry-picked from the no-manifests branch.
2012-07-17 16:55:45 -04:00
Eelco Dolstra a6f348599c Print some debug output 2012-07-17 16:19:40 -04:00
Eelco Dolstra 3a9fdf2747 Return an exit code of 100 for cached failed builds
Exit code 100 should be returned for all permanent failures.  This
includes cached failures.

Fixes #34.
2012-07-17 15:55:30 -04:00
Eelco Dolstra 1217204c81 Remove dead code 2012-07-17 14:07:52 -04:00
Eelco Dolstra 51d71ad3d7 Manual: Don't claim we support Cygwin 2012-07-17 11:49:47 -04:00
Eelco Dolstra 6c01fb4d68 Update Nix 1.1 release notes 2012-07-17 10:06:20 -04:00