Commit graph

158 commits

Author SHA1 Message Date
Eelco Dolstra 1aba0bf0fa nix-store -r: do substitutions in parallel
I.e. when multiple non-derivation arguments are passed to ‘nix-store
-r’ to be substituted, do them in parallel.
2012-06-27 16:58:15 -04:00
Eelco Dolstra 881beb170d "nix-store -l": support compressed logs 2012-05-30 00:00:02 -04:00
Eelco Dolstra 4bc4da331a Reserve some disk space for the garbage collector
We can't open a SQLite database if the disk is full.  Since this
prevents the garbage collector from running when it's most needed, we
reserve some dummy space that we can free just before doing a garbage
collection.  This actually revives some old code from the Berkeley DB
days.

Fixes #27.
2012-05-29 22:59:12 -04:00
Eelco Dolstra 8be1979f1a Remove the --max-links GC option
We don't need this anymore now that current filesystems support more
than 32,000 files in a directory.
2012-03-26 20:00:02 +02:00
Eelco Dolstra d5a5a83ad4 Use data() instead of c_str() where appropriate 2012-02-09 18:27:45 +01:00
Eelco Dolstra 4e624849b6 * Added a command ‘nix-store --print-env $drvpath’ that prints out the
environment of the given derivation in a format that can be sourced
  by the shell, e.g.

  $ eval "$(nix-store --print-env $(nix-instantiate /etc/nixos/nixpkgs -A pkg))"
  $ NIX_BUILD_TOP=/tmp
  $ source $stdenv/setup

  This is especially useful to reproduce the environment used to build
  a package outside of its builder for development purposes.

  TODO: add a nix-build option to do the above and fetch the
  dependencies of the derivation as well.
2012-01-17 23:07:22 +00:00
Eelco Dolstra 93b56acb2d * Support multiple outputs in nix-store (specifically the ‘--query’
and ‘--realise’ actions).
2011-12-30 17:25:19 +00:00
Eelco Dolstra 273b288a7e * importPath() -> importPaths(). Because of buffering of the input
stream it's now necessary for the daemon to process the entire
  sequence of exported paths, rather than letting the client do it.
2011-12-16 22:31:25 +00:00
Eelco Dolstra 49f59dceca * Move parseHash16or32 into libutil, and use in nix-hash. 2011-12-02 11:47:06 +00:00
Eelco Dolstra 5bbd693cae * Add an API function exportPaths() that provides the functionality of
‘nix-store --export’.
* Add a Perl module that provides the functionality of
  ‘nix-copy-closure --to’.  This is used by build-remote.pl so it no
  longer needs to start a separate nix-copy-closure process.  Also, it
  uses the Perl API to do the export, so it doesn't need to start a
  separate nix-store process either.  As a result, nix-copy-closure
  and build-remote.pl should no longer fail on very large closures due
  to an "Argument list too long" error.  (Note that having very many
  dependencies in a single derivation can still fail because the
  environment can become too large.  Can't be helped though.)
2011-11-23 15:13:37 +00:00
Eelco Dolstra e6cb3d0a0d * Added a command ‘nix-store --verify-paths PATHS’ to check whether
the contents of any of the given store paths have been modified.
  E.g.

    $ nix-store --verify-path $(nix-store -qR /var/run/current-system)
    path `/nix/store/m2smyiwbxidlprfxfz4rjlvz2c3mg58y-etc' was modified! expected hash `fc87e271c5fdf179b47939b08ad13440493805584b35e3014109d04d8436e7b8', got `20f1a47281b3c0cbe299ce47ad5ca7340b20ab34246426915fce0ee9116483aa'

  All paths are checked; the exit code is 1 if any path has been
  modified, 0 otherwise.
2011-09-06 12:06:30 +00:00
Eelco Dolstra 93227ff65c * Eliminate all uses of the global variable ‘store’ from libstore.
This should also fix:

    nix-instantiate: ./../boost/shared_ptr.hpp:254: T* boost::shared_ptr<T>::operator->() const [with T = nix::StoreAPI]: Assertion `px != 0' failed.

  which was caused by hashDerivationModulo() calling the ‘store’
  object (during store upgrades) before openStore() assigned it.
2011-08-31 21:11:50 +00:00
Eelco Dolstra e60c962fb8 * Add an operation `nix-store -q --size'. 2010-11-17 12:40:52 +00:00
Eelco Dolstra a3883cbd28 * Store the size of a store path in the database (to be precise, the
size of the NAR serialisation of the path, i.e., `nix-store --dump
  PATH').  This is useful for Hydra.
2010-11-16 17:11:46 +00:00
Eelco Dolstra fb9368b5a0 * Sync with the trunk. 2010-11-16 12:49:47 +00:00
Eelco Dolstra 705868a8a9 * Make sure that config.h is included before the system headers,
because it defines _FILE_OFFSET_BITS.  Without this, on 
  OpenSolaris the system headers define it to be 32, and then 
  the 32-bit stat() ends up being called with a 64-bit "struct 
  stat", or vice versa.

  This also ensures that we get 64-bit file sizes everywhere.

* Remove the redundant call to stat() in parseExprFromFile().
  The file cannot be a symlink because that's the exit condition
  of the loop before.
2010-10-04 17:55:38 +00:00
Eelco Dolstra bf87cc44b4 * Sync with the trunk. 2010-06-21 07:55:38 +00:00
Ludovic Courtès 8bcdd36f10 Add XML output to `nix-store'.
* src/nix-store/Makefile.am (nix_store_SOURCES): Add `xmlgraph.cc' and
  `xmlgraph.hh'.

* src/nix-store/help.txt (Operations): Document `--xml'.

* src/nix-store/nix-store.cc (opQuery): Handle `--xml'.

* src/nix-store/xmlgraph.cc, src/nix-store/xmlgraph.hh: New files.
2010-05-31 16:36:24 +00:00
Eelco Dolstra f92c9a0ac5 * Allow unprivileged users to do `nix-store --clear-failed-paths' and
`nix-store --query-failed-paths'.
2010-05-04 10:45:10 +00:00
Eelco Dolstra 6199f9b93e * Added a command `nix-store --clear-failed-paths <PATHS>' to clear
the "failed" status of the given store paths.  The special value `*'
  clears all failed paths.
2010-04-26 12:56:42 +00:00
Eelco Dolstra 2398af13c5 * Add an command `nix-store --query-failed-paths' to list the cached
failed paths (when using the `build-cache-failure' option).
2010-04-26 12:43:42 +00:00
Eelco Dolstra fefd467539 * `helpText' is now zero-terminated. 2010-02-24 13:24:27 +00:00
Eelco Dolstra fdcaf37361 * Made `nix-store -qR --include-outputs' much faster if there are
multiple paths specified on the command line (from O(n * m) to O(n +
  m), where n is the number of arguments and m is the size of the
  closure).
2010-01-25 17:18:44 +00:00
Eelco Dolstra 9b8fda796b * Templatise getIntArg / string2Int. 2009-11-24 12:26:25 +00:00
Eelco Dolstra 3d55f1eb57 * A command `nix-store --query --roots <paths>' to find the garbage
collector roots that point (directly or indirectly) to the given
  paths.
2009-11-23 18:16:25 +00:00
Eelco Dolstra ae6bf87273 * `nix-store --gc --print-roots': also print the path of the actual
root symlink, not just its target.  E.g.:

  /nix/var/nix/profiles/system-99-link -> /nix/store/76kwf88657nq7wgk1hx3l1z5q91zb9wd-system
2009-11-23 17:23:12 +00:00
Eelco Dolstra 8824d60fe5 * Remove the --use-atime / --max-atime garbage collector flags. Many
(Linux) machines no longer maintain the atime because it's too
  expensive, and on the machines where --use-atime is useful (like the
  buildfarm), reading the atimes on the entire Nix store takes way too
  much time to make it practical.
2009-11-20 17:12:38 +00:00
Eelco Dolstra 5dd8fb2069 * Don't use ULLONG_MAX in maxFreed - use 0 to mean "no limit".
18446744073709551615ULL breaks on GCC 3.3.6 (`integer constant is
  too large for "long" type').
2009-03-26 11:02:07 +00:00
Eelco Dolstra 7a57b2920b * Better error message when nix-store --import is applied to garbage
(previously it would likely say "implementation cannot deal with >
  32-bit integers").
2009-03-22 17:51:45 +00:00
Eelco Dolstra 77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
2009-03-22 17:36:43 +00:00
Eelco Dolstra 0008b0006d * Simplify deleting .lock files in /nix/store: just don't delete them
if they belong a path that's currently being built.  This gets rid
  of some Cygwin-specific code.
2008-12-12 17:03:18 +00:00
Eelco Dolstra ff762fb499 * Pass HashType values instead of strings. 2008-12-03 16:10:17 +00:00
Eelco Dolstra 64519cfd65 * Unify the treatment of sources copied to the store, and recursive
SHA-256 outputs of fixed-output derivations.  I.e. they now produce
  the same store path:

  $ nix-store --add x
  /nix/store/j2fq9qxvvxgqymvpszhs773ncci45xsj-x

  $ nix-store --add-fixed --recursive sha256 x
  /nix/store/j2fq9qxvvxgqymvpszhs773ncci45xsj-x

  the latter being the same as the path that a derivation

    derivation {
      name = "x";
      outputHashAlgo = "sha256";
      outputHashMode = "recursive";
      outputHash = "...";
      ...
    };

  produces.

  This does change the output path for such fixed-output derivations.
  Fortunately they are quite rare.  The most common use is fetchsvn
  calls with SHA-256 hashes.  (There are a handful of those is
  Nixpkgs, mostly unstable development packages.)
  
* Documented the computation of store paths (in store-api.cc).
2008-12-03 15:06:30 +00:00
Eelco Dolstra 5d250ad1ea * nix-store --dump-db: be more streamy. 2008-11-19 16:26:34 +00:00
Eelco Dolstra f32fef1b07 * GC option `--max-atime' that specifies an upper limit to the last
accessed time of paths that may be deleted.  Anything more recently
  used won't be deleted.  The time is specified in time_t,
  e.g. seconds since 1970-01-01 00:00:00 UTC; use `date +%s' to
  convert to time_t from the command line. 

  Example: to delete everything that hasn't been used in the last two
  months:

  $ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")
2008-09-17 14:52:35 +00:00
Eelco Dolstra 77afd97a99 * nix-store --gc / --delete: show how many store paths were deleted. 2008-09-17 12:53:33 +00:00
Eelco Dolstra 7ab68961e4 * Garbage collector: added an option `--use-atime' to delete paths in
order of ascending last access time.  This is useful in conjunction
  with --max-freed or --max-links to prefer deleting non-recently used
  garbage, which is good (especially in the build farm) since garbage
  may become live again.

  The code could easily be modified to accept other criteria for
  ordering garbage by changing the comparison operator used by the
  priority queue in collectGarbage().
2008-09-17 10:02:55 +00:00
Eelco Dolstra a1d310b6b5 * `nix-store --realise': print what paths will be built/downloaded,
just like nix-env.
* `nix-store --realise': --dry-run option.
2008-08-04 13:44:46 +00:00
Eelco Dolstra d3aa183beb * Garbage collector: option `--max-freed' to stop after at least N
bytes have been freed, `--max-links' to stop when the Nix store
  directory has fewer than N hard links (the latter being important
  for very large Nix stores on filesystems with a 32000 subdirectories
  limit).
2008-06-18 14:20:16 +00:00
Eelco Dolstra a72709afd8 * Some refactoring: put the GC options / results in separate structs.
* The garbage collector now also prints the number of blocks freed.
2008-06-18 09:34:17 +00:00
Eelco Dolstra b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00
Eelco Dolstra 66c51dc215 * nix-store --dump-db / --load-db to dump/load the Nix DB.
* nix-store --register-validity: option to supply the content hash of
  each path.
* Removed compatibility with Nix <= 0.7 stores.
2008-01-29 18:17:36 +00:00
Eelco Dolstra 633518628f * nix-env -e: support uninstalling by path, so that one can say
$ nix-env -e $(which firefox)

  or

    $ nix-env -e /nix/store/nywzlygrkfcgz7dfmhm5xixlx1l0m60v-pan-0.132

* nix-env -i: if an argument contains a slash anywhere, treat it as a
  path and follow it through symlinks into the Nix store.  This allows
  things like

    $ nix-build -A firefox
    $ nix-env -i ./result

* nix-env -q/-i/-e: don't complain when the `*' selector doesn't match
  anything.  In particular, `nix-env -q \*' doesn't fail anymore on an
  empty profile.
2007-11-29 16:18:24 +00:00
Eelco Dolstra c05783ad67 * nix-store --import: add a flag --require-signature. 2007-11-16 16:10:27 +00:00
Eelco Dolstra 14e47e9c2c * Doh. 2007-10-10 14:58:00 +00:00
Eelco Dolstra 315183f194 * nix-store --optimise: flag "--dry-run" to just query what the disk
savings would be.
2007-10-10 13:43:04 +00:00
Eelco Dolstra a8629de827 * New command `nix-store --optimise' to reduce Nix store disk space
usage by finding identical files in the store and hard-linking them
  to each other.  It typically reduces the size of the store by
  something like 25-35%.  This is what the optimise-store.pl script
  did, but the new command is faster and more correct (it's safe wrt
  garbage collection and concurrent builds).
2007-10-09 22:14:27 +00:00
Eelco Dolstra 9e975458b4 * Get rid of the substitutes database table (NIX-47). Instead, if we
need any info on substitutable paths, we just call the substituters
  (such as download-using-manifests.pl) directly.  This means that
  it's no longer necessary for nix-pull to register substitutes or for
  nix-channel to clear them, which makes those operations much faster
  (NIX-95).  Also, we don't have to worry about keeping nix-pull
  manifests (in /nix/var/nix/manifests) and the database in sync with
  each other.

  The downside is that there is some overhead in calling an external
  program to get the substitutes info.  For instance, "nix-env -qas"
  takes a bit longer.

  Abolishing the substitutes table also makes the logic in
  local-store.cc simpler, as we don't need to store info for invalid
  paths.  On the downside, you cannot do things like "nix-store -qR"
  on a substitutable but invalid path (but nobody did that anyway).

* Never catch interrupts (the Interrupted exception).
2007-08-12 00:29:28 +00:00
Eelco Dolstra 6d1a1191b0 * Support queryDeriver() in multi-user installations. 2007-06-12 16:53:44 +00:00
Eelco Dolstra 7f6161ab3a * Flush cout to show progress. 2007-02-21 23:08:55 +00:00
Eelco Dolstra 0db450024d * Export/import many paths in one go. 2007-02-21 23:00:31 +00:00
Eelco Dolstra 9da367b7d5 * `nix-store -qR' and friends: print the paths sorted topologically
under the references relation.  This is useful for commands that
  want to copy paths to another Nix store in the right order.
2007-02-21 22:45:10 +00:00
Eelco Dolstra 881feb9698 * Flag --print-invalid' in nix-store --check-validity' to print out
which paths specified on the command line are invalid (i.e., don't
  barf when encountering an invalid path, just print it).  This is
  useful for build-remote.pl to figure out which paths need to be
  copied to a remote machine.  (Currently we use rsync, but that's
  rather inefficient.)
2007-02-21 17:57:59 +00:00
Eelco Dolstra 43c4d18c6a * nix-store --import': import an archive created by nix-store
--export' into the Nix store, and optionally check the cryptographic
  signatures against /nix/etc/nix/signing-key.pub.  (TODO: verify
  against a set of public keys.)
2007-02-21 15:45:32 +00:00
Eelco Dolstra 46e0919ced * `nix-store --export --sign': sign the Nix archive using the RSA key
in /nix/etc/nix/signing-key.sec
2007-02-21 14:31:42 +00:00
Eelco Dolstra b824a1daee * Start of `nix-store --export' operation for serialising a store
path.  This is like `nix-store --dump', only it also dumps the
  meta-information of the store path (references, deriver).  Will add
  a `--sign' flag later to add a cryptographic signature, which we
  will use for exchanging store paths between build farm machines in a
  secure manner.
2007-02-20 23:17:20 +00:00
Eelco Dolstra afe23b5f38 * nix-pack-closure: store the top-level store paths in the closure.
* nix-unpack-closure: extract the top-level paths from the closure and
  print them on stdout.  This allows them to be installed, e.g.,
  "nix-env -i $(nix-unpack-closure)".  (NIX-64)
2007-01-13 19:50:42 +00:00
Eelco Dolstra 4ca01065c3 * Rename all those main.cc files. 2006-12-07 20:47:30 +00:00
Renamed from src/nix-store/main.cc (Browse further)