Commit graph

1486 commits

Author SHA1 Message Date
Shea Levy b4cee3f816 Revert previous commit
It doesn't detect indirect references
2011-12-21 17:31:34 +00:00
Shea Levy f3c88f297d Detect and reject mutually-referential outputs
There is probably a more efficient way to do this.
2011-12-21 17:14:28 +00:00
Eelco Dolstra b19a0f63db * Simplify the context handling logic. 2011-12-21 15:33:30 +00:00
Eelco Dolstra b5363810bb * Fix the build. 2011-12-20 16:37:01 +00:00
Eelco Dolstra 194d21f9f6 * Sync with the trunk. 2011-12-16 23:33:01 +00:00
Eelco Dolstra 273b288a7e * importPath() -> importPaths(). Because of buffering of the input
stream it's now necessary for the daemon to process the entire
  sequence of exported paths, rather than letting the client do it.
2011-12-16 22:31:25 +00:00
Eelco Dolstra 8d3dfa2c17 * Avoid expensive conversions from char arrays to STL strings. 2011-12-16 21:29:46 +00:00
Eelco Dolstra e0bd307802 * Make the import operation through the daemon much more efficient
(way fewer roundtrips) by allowing the client to send data in bigger
  chunks.
* Some refactoring.
2011-12-16 19:44:13 +00:00
Eelco Dolstra 78598d06f0 * Clean up exception handling. 2011-12-16 15:45:42 +00:00
Eelco Dolstra 5a1b9ed0aa * Refactoring: move sink/source buffering into separate classes.
* Buffer the HashSink.  This speeds up hashing a bit because it
  prevents lots of calls to the hash update functions (e.g. nix-hash
  went from 9.3s to 8.7s of user time on the closure of my
  /var/run/current-system).
2011-12-15 16:19:53 +00:00
Eelco Dolstra a3e0656cbb * Buffer reads in FdSource. Together with write buffering, this
significantly cuts down the number of syscalls (e.g., for "nix-store
  -qR /var/run/current-system" via the daemon, it reduced the number
  of syscalls in the client from 29134 to 4766 and in the daemon from
  44266 to 20666).
2011-12-15 12:32:08 +00:00
Eelco Dolstra 3a48282b06 * Buffer writes in FdSink. This significantly reduces the number of
system calls / context switches when dumping a NAR and in the worker
  protocol.
2011-12-14 23:30:06 +00:00
Eelco Dolstra 893cac1402 * Remove the terminate handler, which was only really needed because
of Berkeley DB (see r8632).
2011-12-14 22:41:10 +00:00
Eelco Dolstra c8c0380744 * Remove unnecessary quotes. showPaths() already adds quotes. 2011-12-05 21:04:20 +00:00
Eelco Dolstra 000160f5b9 * In ‘nix-store --verify --check-contents’, repair missing hashes
rather than complain about them.
2011-12-02 17:52:18 +00:00
Eelco Dolstra 49f59dceca * Move parseHash16or32 into libutil, and use in nix-hash. 2011-12-02 11:47:06 +00:00
Eelco Dolstra b12b21825c * Allow '<nixexpr>' syntax to be used in nix-instantiate, nix-build
and nix-env, e.g.,

  $ nix-env -f '<nixpkgs>' -i patchelf

  or

  $ nix-build '<nixos/tests>' -A login.test
2011-12-01 16:41:43 +00:00
Eelco Dolstra 23c38a04cc * Slight improvement. 2011-12-01 13:51:05 +00:00
Eelco Dolstra 24f863d86b * When doing "nix-store --add-fixed" without "--recursive" via the Nix
daemon (which is an error), print a nicer error message than
  "Connection reset by peer" or "broken pipe".
* In the daemon, log errors that occur during request parameter
  processing.
2011-12-01 13:48:48 +00:00
Eelco Dolstra b1eb8f4249 * Get rid of some superfluous error messages if a substituter fails.
* Say "fetch" instead of "substitute".
2011-11-29 13:00:41 +00:00
Eelco Dolstra 5bbd693cae * Add an API function exportPaths() that provides the functionality of
‘nix-store --export’.
* Add a Perl module that provides the functionality of
  ‘nix-copy-closure --to’.  This is used by build-remote.pl so it no
  longer needs to start a separate nix-copy-closure process.  Also, it
  uses the Perl API to do the export, so it doesn't need to start a
  separate nix-store process either.  As a result, nix-copy-closure
  and build-remote.pl should no longer fail on very large closures due
  to an "Argument list too long" error.  (Note that having very many
  dependencies in a single derivation can still fail because the
  environment can become too large.  Can't be helped though.)
2011-11-23 15:13:37 +00:00
Eelco Dolstra 993fa94fb4 * Move initialisation of variables like nixConfDir from libmain to
libstore so that the Perl bindings can use it as well.  It's vital
  that the Perl bindings use the configuration file, because otherwise
  nix-copy-closure will fail with a ‘database locked’ message if the
  value of ‘use-sqlite-wal’ is changed from the default.
2011-11-22 17:28:41 +00:00
Rob Vermaas 4e1ea17052 nix: add /etc/hosts with localhost entry to chroot builds. 2011-11-21 15:19:51 +00:00
Eelco Dolstra c0b706213d * Boldly make SQLite WAL mode the default again. Hopefully the
intermittent problems are gone by now.  WAL mode is preferrable
  because it does way fewer fsyncs.
2011-11-07 21:11:59 +00:00
Shea Levy 3c3107da86 There's no need to mess with drvPath at all 2011-11-06 07:18:19 +00:00
Shea Levy 2ab29be70c Fix faulty reversion of my changes to unsafeDiscardOutputDependency 2011-11-06 07:03:14 +00:00
Shea Levy ca0d47a70c Respect all outputs passed to the derivation, not just the last one 2011-11-06 06:54:05 +00:00
Shea Levy 24b65937e1 Remove the unused sCurrentOutput symbol 2011-11-06 06:28:34 +00:00
Shea Levy 3522730316 Embed output name into the context of the *OutPath attributes and extract it for input derivations
Multiple outputs test passes!
2011-11-06 06:28:30 +00:00
Shea Levy af2e53fd48 Include all outputs of derivations in the closure of explicitly-passed derivation paths
This required adding a queryOutputDerivationNames function in the store API
2011-11-06 06:28:20 +00:00
Shea Levy 981edeab7b The 'insert output between = signs' approach was not helpful 2011-11-06 06:28:14 +00:00
Shea Levy 2721e9f56f Merge from trunk 2011-11-06 00:13:09 +00:00
Eelco Dolstra a12095d3be * In printValueAsXML, handle the case where a "type" attribute is not
a string.  This happens in the NixOS option system.
* Remove a bogus comparison of a unsigned integer with -1.
2011-10-27 19:06:23 +00:00
Shea Levy e81c09edbf Remove the current output metadata from the string for unsaveDiscardOutputDependency 2011-09-16 11:31:00 +00:00
Shea Levy bf50d6ad32 Add information about which output is active to drvPath's context
This will break things that depend on being able to just strip away an equals sign, so those have to be updated next
2011-09-16 11:30:52 +00:00
Shea Levy ffa038f66d Add an sCurrentOutput member to EvalState 2011-09-16 11:30:44 +00:00
Shea Levy f3e410d4bf Add a currentOutput attribute to derivations keep track of which output is active 2011-09-16 11:30:03 +00:00
Shea Levy c172d16b00 First attempt at the output-as-derivation semantics
For each output, this adds a corresponding attribute to the derivation that is
the same as the derivation except for outPath, which is set to the path specific
to that output. Additionally, an "all" attribute is added that is a list of all
of the output derivations. This has to be done outside of derivationStrict as
each output is itself a derivation that contains itself (and all other outputs)
as an attribute. The derivation itself is equivalent to the first output in the
outputs list (or "out" if that list isn't set).
2011-09-14 05:59:17 +00:00
Eelco Dolstra 13114daa3e * Ouch. A store upgrade could cause a substituter to be triggered,
causing a deadlock.
2011-09-12 09:07:43 +00:00
Eelco Dolstra e6cb3d0a0d * Added a command ‘nix-store --verify-paths PATHS’ to check whether
the contents of any of the given store paths have been modified.
  E.g.

    $ nix-store --verify-path $(nix-store -qR /var/run/current-system)
    path `/nix/store/m2smyiwbxidlprfxfz4rjlvz2c3mg58y-etc' was modified! expected hash `fc87e271c5fdf179b47939b08ad13440493805584b35e3014109d04d8436e7b8', got `20f1a47281b3c0cbe299ce47ad5ca7340b20ab34246426915fce0ee9116483aa'

  All paths are checked; the exit code is 1 if any path has been
  modified, 0 otherwise.
2011-09-06 12:06:30 +00:00
Eelco Dolstra 93227ff65c * Eliminate all uses of the global variable ‘store’ from libstore.
This should also fix:

    nix-instantiate: ./../boost/shared_ptr.hpp:254: T* boost::shared_ptr<T>::operator->() const [with T = nix::StoreAPI]: Assertion `px != 0' failed.

  which was caused by hashDerivationModulo() calling the ‘store’
  object (during store upgrades) before openStore() assigned it.
2011-08-31 21:11:50 +00:00
Eelco Dolstra 2d663b502d * Cache the result of file evaluation (i.e, memoize evalFile()). This
prevents files from being evaluated and stored as values multiple
  times.  For instance, evaluation of the ‘system’ attribute in NixOS
  causes ‘nixpkgs/pkgs/lib/lists.nix’ to be evaluated 2019 times.

  Caching gives a modest speedup and a decent memory footprint
  reduction (e.g., from 1.44s to 1.28s, and from 81 MiB to 59 MiB with
  GC_INITIAL_HEAP_SIZE=100000 on my system).
2011-08-06 19:45:43 +00:00
Eelco Dolstra 9d091ee99a * Handle the case where the search path element is a regular file. 2011-08-06 18:45:28 +00:00
Eelco Dolstra 00a724ebc6 * Remove a debug statement. 2011-08-06 18:24:43 +00:00
Eelco Dolstra 07340b8be7 * Add the Nix corepkgs to the end of the search path. This makes it
possible for other Nix expressions to use corepkgs (mostly useful
  for the buildenv function).
2011-08-06 18:23:38 +00:00
Eelco Dolstra c7101dac0b * Allow redirections in search path entries. E.g. if you have a
directory

    /home/eelco/src/stdenv-updates

  that you want to use as the directory for import such as

    with (import <nixpkgs> { });

  then you can say

    $ nix-build -I nixpkgs=/home/eelco/src/stdenv-updates
2011-08-06 17:48:57 +00:00
Eelco Dolstra 1ecc97b6bd * Add a Nix expression search path feature. Paths between angle
brackets, e.g.

    import <nixpkgs/pkgs/lib>

  are resolved by looking them up relative to the elements listed in
  the search path.  This allows us to get rid of hacks like

    import "${builtins.getEnv "NIXPKGS_ALL"}/pkgs/lib"

  The search path can be specified through the ‘-I’ command-line flag
  and through the colon-separated ‘NIX_PATH’ environment variable,
  e.g.,

    $ nix-build -I /etc/nixos ...

  If a file is not found in the search path, an error message is
  lazily thrown.
2011-08-06 16:05:24 +00:00
Eelco Dolstra 54945a2950 * Refactoring: move parseExprFromFile() and parseExprFromString() into
the EvalState class.
2011-08-06 13:02:55 +00:00
Eelco Dolstra c8606664ab * Don't allow derivations with fixed and non-fixed outputs. 2011-07-20 18:26:00 +00:00
Eelco Dolstra b2027f70d9 * Fix a huuuuge security hole in the Nix daemon. It didn't check that
derivations added to the store by clients have "correct" output
  paths (meaning that the output paths are computed by hashing the
  derivation according to a certain algorithm).  This means that a
  malicious user could craft a special .drv file to build *any*
  desired path in the store with any desired contents (so long as the
  path doesn't already exist).  Then the attacker just needs to wait
  for a victim to come along and install the compromised path.

  For instance, if Alice (the attacker) knows that the latest Firefox
  derivation in Nixpkgs produces the path

    /nix/store/1a5nyfd4ajxbyy97r1fslhgrv70gj8a7-firefox-5.0.1

  then (provided this path doesn't already exist) she can craft a .drv
  file that creates that path (i.e., has it as one of its outputs),
  add it to the store using "nix-store --add", and build it with
  "nix-store -r".  So the fake .drv could write a Trojan to the
  Firefox path.  Then, if user Bob (the victim) comes along and does

    $ nix-env -i firefox
    $ firefox

  he executes the Trojan injected by Alice.

  The fix is to have the Nix daemon verify that derivation outputs are
  correct (in addValidPath()).  This required some refactoring to move
  the hash computation code to libstore.
2011-07-20 18:10:47 +00:00