Commit graph

1971 commits

Author SHA1 Message Date
Eelco Dolstra 65a6452240 nix-shell: Handle --option correctly
Fixes #181.
2013-12-20 13:10:14 +01:00
Shea Levy e36229d27f Bump language version for new storePath feature
This will allow e.g. channel expressions to use builtins.storePath IFF
it is safe to do so without knowing if the path is valid yet.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-10 15:07:04 +01:00
Shea Levy 22d665019a builtins.storePath: Try to substitute the path if it is not yet valid
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-12-10 15:07:04 +01:00
Eelco Dolstra a6add93d73 Garbage collector: Release locks on temporary root files
This allows processes waiting for such locks to proceed during the
trash deletion phase of the garbage collector.
2013-12-10 13:13:59 +01:00
Eelco Dolstra c5b8fe3151 Print a trace message if a build fails due to the platform being unknown 2013-12-05 14:31:57 -05:00
Eelco Dolstra f3cf0436b5 Install bsdiff and bspatch in $(libexecdir)/nix 2013-12-02 20:21:31 +00:00
Eelco Dolstra 0202ce6b94 Add support for ‘make installcheck’ 2013-11-25 18:47:03 +01:00
Eelco Dolstra 7ce0e05ad8 Rename Makefile.new -> Makefile 2013-11-25 15:25:13 +00:00
Eelco Dolstra 1a1d8b073a Add a Makefile for bsdiff 2013-11-25 11:39:21 +00:00
Eelco Dolstra 2bd0fcc966 Use libnix as a prefix for all Nix libraries
In particular "libutil" was always a problem because it collides with
Glibc's libutil.  Even if we install into $(libdir)/nix, the linker
sometimes got confused (e.g. if a program links against libstore but
not libutil, then ld would report undefined symbols in libstore
because it was looking at Glibc's libutil).
2013-11-23 23:53:41 +00:00
Eelco Dolstra abb5bd66de Add Makefile for nix-log2xml 2013-11-23 23:42:50 +00:00
Eelco Dolstra c1f3a1a89b Disallow undefined symbols in dynamic libraries by default
This encourages that each library declares its own dependencies
properly.
2013-11-24 00:10:00 +01:00
Eelco Dolstra 5a1114ecdb Drop the dependency on libgc in libmain
Instead, libexpr now depends on libgc.  This means commands like
nix-store that don't do any evaluation no longer require libgc.
2013-11-23 20:19:36 +00:00
Eelco Dolstra 06a8ac96e7 Initialise Boehm GC only once 2013-11-23 20:15:57 +00:00
Eelco Dolstra 90dfb37f14 Allow (dynamic) libraries to depend on other libraries 2013-11-23 20:11:02 +00:00
Eelco Dolstra 611868a909 Implement basic ‘make install’ 2013-11-23 17:04:27 +00:00
Eelco Dolstra d1b3ca0b4a Improve building dynamic libraries
They now get a correct RPATH.
2013-11-23 17:26:43 +01:00
Eelco Dolstra 6dd74b48f8 Support building dynamic libraries 2013-11-22 20:56:58 +00:00
Eelco Dolstra 9b11a8bfbc Fix building without Boehm GC 2013-11-22 20:05:00 +00:00
Eelco Dolstra 6b5f89f2cf Drop the dependency on Automake 2013-11-22 19:30:24 +00:00
Eelco Dolstra 754c05ed6c Rename $(here) to $(d) for brevity, and remove trailing slash 2013-11-22 16:45:52 +00:00
Eelco Dolstra 62e35cc3a8 Add ‘make dist’ support 2013-11-22 16:42:25 +01:00
Eelco Dolstra eff6c4b791 Add missing #include 2013-11-22 15:41:48 +00:00
Eelco Dolstra 1474ecfe42 Generate the parser and the lexer 2013-11-22 16:22:31 +01:00
Eelco Dolstra b8e9efc476 New non-recursive, plain Make-based build system 2013-11-22 15:54:18 +01:00
Eelco Dolstra 709cbe4e76 Include <cstring> for memset
This should fix building on Illumos.
2013-11-22 10:00:43 +00:00
Eelco Dolstra 30b986908e Check meta values and warn about bad ones 2013-11-19 14:29:39 +01:00
Eelco Dolstra 0f24400d90 Generalise meta attributes 2013-11-19 14:09:14 +01:00
Eelco Dolstra 990126cde0 Shorter error message 2013-11-19 14:09:14 +01:00
Eelco Dolstra af94a70ba6 Drop support for user environment manifests in ATerm format 2013-11-19 14:09:14 +01:00
Eelco Dolstra 245e26408f nix-env -q: Add a --json flag 2013-11-19 00:41:45 +01:00
Eelco Dolstra 5fea98111b Refactor JSON output 2013-11-19 00:33:06 +01:00
Eelco Dolstra 77c13cdf56 Add a toJSON primop 2013-11-19 00:04:11 +01:00
Eelco Dolstra 285df765b9 Add a primop unsafeGetAttrPos to return the position of an attribute 2013-11-18 22:22:35 +01:00
Eelco Dolstra fc33fd86b7 Add a symbol __curPos that expands to the current source location
I.e. an attribute set { file = <string>; line = <int>; column = <int>; }.
2013-11-18 20:16:02 +01:00
Eelco Dolstra 90b5e69284 Support quoted attribute names in -A
This is requires if you have attribute names with dots in them.  So
you can now say:

  $ nix-instantiate '<nixos>' -A 'config.systemd.units."postgresql.service".text' --eval-only

Fixes #151.
2013-11-18 10:21:12 +00:00
Eelco Dolstra a478e8a7bb Remove nix-setuid-helper
AFAIK, nobody uses it, it's not maintained, and it has no tests.
2013-11-14 11:57:37 +01:00
Eelco Dolstra 89e6781cc5 Make function calls show up in stack traces again
Note that adding --show-trace prevents functions calls from being
tail-recursive, so an expression that evaluates without --show-trace
may fail with a stack overflow if --show-trace is given.
2013-11-12 12:51:59 +01:00
Eelco Dolstra c897bac549 Make function calls tail-recursive 2013-11-12 11:32:23 +00:00
Eelco Dolstra 273322c773 Make ifs and asserts tail-recursive
The local Value object prevented g++ from making a tail call.  Not
clear why.  In any case, not using a temporary makes g++ do the tail
call.
2013-11-12 11:32:23 +00:00
Eelco Dolstra 4badd7ed17 Get rid of an intermediary on the stack 2013-11-12 11:32:23 +00:00
Eelco Dolstra 8d6418d46e Fix building without a garbage collector
http://hydra.nixos.org/build/6695350
2013-10-28 22:51:12 +01:00
Eelco Dolstra dec2f19502 Fix a segfault in genericClosure
It kept temporary data in STL containers that were not scanned by
Boehm GC, so Nix programs using genericClosure could randomly crash if
the garbage collector kicked in at a bad time.

Also make it a bit more efficient by copying points to values rather
than values.
2013-10-28 18:52:26 +01:00
Eelco Dolstra ea6bf0c21f Slightly optimize listToAttrs 2013-10-28 07:34:44 +01:00
Eelco Dolstra 5bc41d78ff Rename "attribute sets" to "sets"
We don't have any other kind of sets so calling them attribute sets is
unnecessarily verbose.
2013-10-24 16:41:04 +02:00
Eelco Dolstra 69befd33a9 Remove unnecessary call to forceStringNoCtx 2013-10-24 03:08:34 +02:00
Eelco Dolstra 05d02f798f Add a typeOf primop
We already have some primops for determining the type of a value, such
as isString, but they're incomplete: for instance, there is no isPath.
Rather than adding more isBla functions, the generic typeOf function
returns a string representing the type of the argument (e.g. "int").
2013-10-24 02:49:13 +02:00
Eelco Dolstra 543d8a5942 Don't require NIX_SHOW_STATS for NIX_COUNT_CALLS 2013-10-24 02:20:54 +02:00
Eelco Dolstra fe95650487 Memoize evalFile() lookups under both the original and resolved name
Previously we only used the resolved name, causing repeated resolution
(e.g. /dir to /dir/default.nix).
2013-10-23 11:19:01 +00:00
Eelco Dolstra c086183843 For auto roots, show the intermediate link
I.e. "nix-store -q --roots" will now show (for example)

  /home/eelco/Dev/nixpkgs/result

rather than

  /nix/var/nix/gcroots/auto/53222qsppi12s2hkap8dm2lg8xhhyk6v
2013-10-22 11:39:10 +02:00
Eelco Dolstra 792fd51f41 Fold two stack trace messages in derivations
Combined with the previous changes, stack traces involving derivations
are now much less verbose, since something like

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  while instantiating the derivation named `gtk+-2.24.20' at `/home/eelco/Dev/nixpkgs/pkgs/development/libraries/gtk+/2.x.nix:11:3':
  while evaluating the derivation attribute `propagatedNativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:78:17':
  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':
  ...

now reads

  while evaluating the attribute `propagatedNativeBuildInputs' of the derivation `gtk+-2.24.20' at `/home/eelco/Dev/nixpkgs/pkgs/development/libraries/gtk+/2.x.nix:11:3':
  ...
2013-10-17 11:57:25 +02:00
Eelco Dolstra f440558acc Don't show <nix/derivation.nix> in stack traces
Messages like

  while evaluating the attribute `outPath' at `/nix/store/212ngf4ph63mp6p1np2bapkfikpakfv7-nix-1.6/share/nix/corepkgs/derivation.nix:18:9':

are redundant, because Nix already shows that it's evaluating a derivation:

  while instantiating the derivation named `firefox-24.0' at `/home/eelco/Dev/nixpkgs/pkgs/applications/networking/browsers/firefox/default.nix:131:5':
  while evaluating the derivation attribute `nativeBuildInputs' at `/home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/default.nix:76:17':
2013-10-17 11:47:38 +02:00
Eelco Dolstra b8034e5581 Ensure proper type checking/coercion of "${expr}"
Now we only rewrite "${expr}" to expr if expr is a string literal.
2013-10-17 00:57:24 +02:00
Eelco Dolstra d6a7aa8f48 Revert the behaviour of antiquoted paths to pre-Nix 1.6
Commit 159e621d1a accidentally changed
the behaviour of antiquoted paths, e.g.

  "${/foo}/bar"

used to evaluate to "/nix/store/<hash>-foo/bar" (where /foo gets
copied to the store), but in Nix 1.6 it evaluates to "/foo/bar".  This
is inconsistent, since

  " ${/foo}/bar"

evaluates to " /nix/store/<hash>-foo/bar".  So revert to the old
behaviour.
2013-10-17 00:39:59 +02:00
Eelco Dolstra a737f51fd9 Retry all SQLite operations
To deal with SQLITE_PROTOCOL, we also need to retry read-only
operations.
2013-10-16 15:58:20 +02:00
Eelco Dolstra ff02f5336c Fix a race in registerFailedPath()
Registering the path as failed can fail if another process does the
same thing after the call to hasPathFailed().  This is extremely
unlikely though.
2013-10-16 14:55:53 +02:00
Eelco Dolstra 4bd5282573 Convenience macros for retrying a SQLite transaction 2013-10-16 14:46:35 +02:00
Eelco Dolstra bce14d0f61 Don't wrap read-only queries in a transaction
There is no risk of getting an inconsistent result here: if the ID
returned by queryValidPathId() is deleted from the database
concurrently, subsequent queries involving that ID will simply fail
(since IDs are never reused).
2013-10-16 14:36:53 +02:00
Eelco Dolstra 7cdefdbe73 Print a distinct warning for SQLITE_PROTOCOL 2013-10-16 14:27:36 +02:00
Eelco Dolstra d05bf04444 Treat SQLITE_PROTOCOL as SQLITE_BUSY
In the Hydra build farm we fairly regularly get SQLITE_PROTOCOL errors
(e.g., "querying path in database: locking protocol").  The docs for
this error code say that it "is returned if some other process is
messing with file locks and has violated the file locking protocol
that SQLite uses on its rollback journal files."  However, the SQLite
source code reveals that this error can also occur under high load:

  if( cnt>5 ){
    int nDelay = 1;                      /* Pause time in microseconds */
    if( cnt>100 ){
      VVA_ONLY( pWal->lockError = 1; )
      return SQLITE_PROTOCOL;
    }
    if( cnt>=10 ) nDelay = (cnt-9)*238;  /* Max delay 21ms. Total delay 996ms */
    sqlite3OsSleep(pWal->pVfs, nDelay);
  }

i.e. if certain locks cannot be not acquired, SQLite will retry a
number of times before giving up and returing SQLITE_PROTOCOL.  The
comments say:

  Circumstances that cause a RETRY should only last for the briefest
  instances of time.  No I/O or other system calls are done while the
  locks are held, so the locks should not be held for very long. But
  if we are unlucky, another process that is holding a lock might get
  paged out or take a page-fault that is time-consuming to resolve,
  during the few nanoseconds that it is holding the lock.  In that case,
  it might take longer than normal for the lock to free.
  ...
  The total delay time before giving up is less than 1 second.

On a heavily loaded machine like lucifer (the main Hydra server),
which often has dozens of processes waiting for I/O, it seems to me
that a page fault could easily take more than a second to resolve.
So, let's treat SQLITE_PROTOCOL as SQLITE_BUSY and retry the
transaction.

Issue NixOS/hydra#14.
2013-10-16 14:19:59 +02:00
Eelco Dolstra 7bdb85453d printStats(): Print the size of the symbol table in bytes 2013-10-08 15:37:08 +02:00
Eelco Dolstra 9deb822180 Deduplicate filenames in Pos
This saves ~4 MiB of RAM for NixOS system instantiation, and ~18 MiB
for "nix-env -qa".
2013-10-08 15:36:10 +02:00
Eelco Dolstra b1e3b1a4ac Treat undefined variable errors consistently
Previously, a undefined variable inside a "with" caused an EvalError
(which can be caught), while outside, it caused a ParseError (which
cannot be caught).  Now both cause an UndefinedVarError (which cannot
be caught).
2013-10-08 14:45:36 +02:00
Eelco Dolstra 6b47de580f Show the exact position of undefined variables
In particular, undefined variable errors in a "with" previously didn't
show *any* position information, so this should help a lot in those
cases.
2013-10-08 14:40:51 +02:00
Eelco Dolstra a5e0f64db3 Remove some unused functions 2013-10-08 12:30:23 +00:00
Eelco Dolstra 221a2daf34 Merge VarRef into ExprVar 2013-10-08 14:24:53 +02:00
Eelco Dolstra 176c666f36 Don't show calls to primops in stack traces
Since they don't have location information, they just give you crap
like:

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  ...
2013-10-07 18:02:32 +02:00
Eelco Dolstra c945f015de Fix segfault in nix-repl / hydra-eval-jobs
If a "with" attribute set fails to evaluate, we have to make sure its
Env record remains unchanged.  Otherwise, repeated evaluation gives a
segfault:

  nix-repl> :a with 0; { a = x; b = x; }
  Added 2 variables.

  nix-repl> a
  error: value is an integer while an attribute set was expected

  nix-repl> b
  Segmentation fault
2013-10-02 15:24:45 +02:00
Eelco Dolstra 28e0742966 Report OOM errors better 2013-10-02 14:34:36 +02:00
Eelco Dolstra a5fb4b5b7c Fix typo 2013-10-02 14:22:49 +02:00
Eelco Dolstra f53574ebd6 RestoreSink: Slightly reduce the number of concurrent FDs 2013-09-17 12:06:59 +00:00
Eelco Dolstra 0220da3e10 Remove stray debug line 2013-09-06 17:20:19 +02:00
Eelco Dolstra 936f9d45ba Don't apply the CPU affinity hack to nix-shell (and other Perl programs)
As discovered by Todd Veldhuizen, the shell started by nix-shell has
its affinity set to a single CPU.  This is because nix-shell connects
to the Nix daemon, which causes the affinity hack to be applied.  So
we turn this off for Perl programs.
2013-09-06 16:36:56 +02:00
Eelco Dolstra 2c1ecf8e81 nix-env -i: Add a flag ‘--remove-all’ / ‘-r’
This is equivalent to running ‘nix-env -e '*'’ first, except that it
happens in a single transaction.  Thus, ‘nix-env -i pkgs...’ replaces
the profile with the specified set of packages.

The main motivation is to support declarative package management
(similar to environment.systemPackages in NixOS).  That is, if you
have a specification ‘profile.nix’ like this:

  with import <nixpkgs> {};
  [ thunderbird
    geeqie
    ...
  ]

then after any change to ‘profile.nix’, you can run:

  $ nix-env -f profile.nix -ir

to update the profile to match the specification.  (Without the ‘-r’
flag, if you remove a package from ‘profile.nix’, it won't be removed
from the actual profile.)

Suggested by @zefhemel.
2013-09-03 21:21:14 +02:00
Eelco Dolstra 88c07341a6 nix-env: Use wildcard match by default
That is, you don't need to pass '*' anymore, so

  nix-env -qa

is equivalent to

  nix-env -qa '*'
2013-09-03 16:35:49 +02:00
Eelco Dolstra 07a08bddf0 nix-env: Load files in ~/.nix-defexpr on demand
So if you do "nix-env -qa -A nixos", then other channels won't be
parsed/evaluated at all.
2013-09-03 15:45:32 +02:00
Eelco Dolstra c57ed84e28 Check for name collisions in the input Nix expressions 2013-09-03 15:25:51 +02:00
Eelco Dolstra ef4f5ba85e Work on Values instead of Exprs
This prevents some duplicate evaluation in nix-env and
nix-instantiate.

Also, when traversing ~/.nix-defexpr, only read regular files with the
extension .nix.  Previously it was reading files like
.../channels/binary-caches/<name>.  The only reason this didn't cause
problems is pure luck (namely, <name> shadows an actual Nix
expression, the binary-caches files happen to be syntactically valid
Nix expressions, and we iterate over the directory contents in just
the right order).
2013-09-03 13:17:51 +00:00
Eelco Dolstra 06bb2d95b4 Reformat 2013-09-03 11:04:21 +00:00
Eelco Dolstra 6f809194d7 Get rid of the parse tree cache
Since we already cache files in normal form (fileEvalCache), caching
parse trees is redundant.

Note that getting rid of this cache doesn't actually save much memory
at the moment, because parse trees are currently not freed / GC'ed.
2013-09-03 13:01:42 +02:00
Eelco Dolstra 57d18df7d0 Add some support code for nix-repl 2013-09-02 18:34:04 +02:00
Eelco Dolstra 92077b4547 Get rid of a signedness warning 2013-09-02 16:39:17 +02:00
Eelco Dolstra 33972629d7 Fix whitespace 2013-09-02 16:29:15 +02:00
Eelco Dolstra b29d3f4aee Only show trace messages when tracing is enabled 2013-09-02 12:01:04 +02:00
Eelco Dolstra efe4289464 Add an option to limit the log output of builders
This is mostly useful for Hydra to deal with builders that get stuck
in an infinite loop writing data to stdout/stderr.
2013-09-02 11:58:18 +02:00
Shea Levy afc6c1bad6 Simplify inherited attribute handling
This reduces the difference between inherited and non-inherited
attribute handling to the choice of which env to use (in recs and lets)
by setting the AttrDef::e to a new ExprVar in the parser rather than
carrying a separate AttrDef::v VarRef member.

As an added bonus, this allows inherited attributes that inherit from a
with to delay forcing evaluation of the with's attributes.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-08-26 11:31:56 +02:00
Ivan Kozik 34bb806f74 Fix typos, especially those that end up in the Nix manual 2013-08-26 11:15:22 +02:00
Gergely Risko c6c024ca6f Fix personality switching from x86_64 to i686
On Linux, Nix can build i686 packages even on x86_64 systems.  It's not
enough to recognize this situation by settings.thisSystem, we also have
to consult uname().  E.g. we can be running on a i686 Debian with an
amd64 kernel.  In that situation settings.thisSystem is i686-linux, but
we still need to change personality to i686 to make builds consistent.
2013-08-26 11:12:35 +02:00
Eelco Dolstra d308aeaf53 Store Nix integers as longs
So on 64-bit systems, integers are now 64-bit.

Fixes #158.
2013-08-19 12:35:03 +02:00
Eelco Dolstra 46222bbc43 Typo 2013-08-14 22:32:49 +02:00
Eelco Dolstra 3fb7ae0586 Respect MINSIGSTKSZ when allocating an alternative stack
http://hydra.nixos.org/build/5663577
2013-08-07 15:44:19 +00:00
Eelco Dolstra 161a2ccf7a Fix build on non-Linux
http://hydra.nixos.org/build/5662914
2013-08-07 17:26:41 +02:00
Eelco Dolstra a583a2bc59 Run the daemon worker on the same CPU as the client
On a system with multiple CPUs, running Nix operations through the
daemon is significantly slower than "direct" mode:

$ NIX_REMOTE= nix-instantiate '<nixos>' -A system
real    0m0.974s
user    0m0.875s
sys     0m0.088s

$ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system
real    0m2.118s
user    0m1.463s
sys     0m0.218s

The main reason seems to be that the client and the worker get moved
to a different CPU after every call to the worker.  This patch adds a
hack to lock them to the same CPU.  With this, the overhead of going
through the daemon is very small:

$ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system
real    0m1.074s
user    0m0.809s
sys     0m0.098s
2013-08-07 14:02:04 +02:00
Eelco Dolstra fd7d979c79 Remove obsolete reference to ATerms 2013-08-06 14:15:11 +02:00
Eelco Dolstra 46ffcce0c3 In the profiler output, show function names (if available) 2013-08-02 18:29:23 +00:00
Eelco Dolstra 8e74c0bfd1 Let the ordering operators also work on strings
E.g. ‘"foo" < "bar"’ now works.
2013-08-02 18:53:02 +02:00
Eelco Dolstra 3d77b28eac Add comparison operators ‘<’, ‘<=’, ‘>’ and ‘>=’ 2013-08-02 18:39:40 +02:00
Eelco Dolstra 47701677e8 Add integer ‘-’, ‘*’ and ‘/’ operators 2013-08-02 16:03:02 +00:00
Eelco Dolstra 5d147e125c Add a unary integer negation operator
This allows saying "-1" instead of "builtins.sub 0 1".
2013-08-02 15:43:31 +00:00
Eelco Dolstra 159e621d1a Overload the ‘+’ operator to support integer addition 2013-08-02 15:21:17 +00:00
Eelco Dolstra 0a470fc345 Make Env smaller
Commit 20866a7031 added a ‘withAttrs’
field to Env, which is annoying because it makes every Env structure
bigger and we allocate millions of them.  E.g. NixOS evaluation took
18 MiB more.  So this commit squeezes ‘withAttrs’ into values[0].
Probably should use a union...
2013-07-31 12:44:21 +02:00
Eelco Dolstra 8ae6d55db1 Don't use NULL 2013-07-31 12:11:14 +02:00
Shea Levy e068f49f7d Avoid thunks when a fromWith var can be looked up without evaluation
Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Shea Levy 20866a7031 Delay evaulation of with attrs until a variable lookup needs them
Evaluation of attribute sets is strict in the attribute names, which
means immediate evaluation of `with` attribute sets rules out some
potentially interesting use cases (e.g. where the attribute names of one
set depend in some way on another but we want to bring those names into
scope for some values in the second set).

The major example of this is overridable self-referential package sets
(e.g. all-packages.nix). With immediate `with` evaluation, the only
options for such sets are to either make them non-recursive and
explicitly use the name of the overridden set in non-overridden one
every time you want to reference another package, or make the set
recursive and use the `__overrides` hack. As shown in the test case that
comes with this commit, though, delayed `with` evaluation allows a nicer
third alternative.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-31 11:48:39 +02:00
Eelco Dolstra 70e68e0ec6 Detect stack overflows
Previously, if the Nix evaluator gets a stack overflow due to a deep
or infinite recursion in the Nix expression, the user gets an
unhelpful message ("Segmentation fault") that doesn't indicate that
the problem is in the user's code rather than Nix itself.  Now it
prints:

  error: stack overflow (possible infinite recursion)

This only works on x86_64-linux and i686-linux.

Fixes #35.
2013-07-30 23:37:10 +02:00
Shea Levy e87d1a63bd killUser: Don't let the child kill itself on Apple
The kill(2) in Apple's libc follows POSIX semantics, which means that
kill(-1, SIGKILL) will kill the calling process too. Since nix has no
way to distinguish between the process successfully killing everything
and the process being killed by a rogue builder in that case, it can't
safely conclude that killUser was successful.

Luckily, the actual kill syscall takes a parameter that determines
whether POSIX semantics are followed, so we can call that syscall
directly and avoid the issue on Apple.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-30 21:25:16 +02:00
Eelco Dolstra a4921b8ceb Revert "build-remote.pl: Enforce timeouts locally"
This reverts commit 69b8f9980f.

The timeout should be enforced remotely.  Otherwise, if the garbage
collector is running either locally or remotely, if will block the
build or closure copying for some time.  If the garbage collector
takes too long, the build may time out, which is not what we want.
Also, on heavily loaded systems, copying large paths to and from the
remote machine can take a long time, also potentially resulting in a
timeout.
2013-07-18 12:52:29 +02:00
Shea Levy 16591eb3cc Allow bind-mounting regular files into the chroot
mount(2) with MS_BIND allows mounting a regular file on top of a regular
file, so there's no reason to only bind directories. This allows finer
control over just which files are and aren't included in the chroot
without having to build symlink trees or the like.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-15 16:01:33 +02:00
Eelco Dolstra aeb810b01e Garbage collector: Don't follow symlinks arbitrarily
Only indirect roots (symlinks to symlinks to the Nix store) are now
supported.
2013-07-12 14:03:36 +02:00
Ludovic Courtès 212e96f39c Leave `HAVE_HUP_NOTIFICATION' undefined on GNU/Hurd. 2013-07-07 13:21:06 +02:00
Eelco Dolstra 7ccd946407 Don't set $preferLocalBuild and $requiredSystemFeatures in builders
With C++ std::map, doing a comparison like ‘map["foo"] == ...’ has the
side-effect of adding a mapping from "foo" to the empty string if
"foo" doesn't exist in the map.  So we ended up setting some
environment variables by accident.
2013-06-20 18:07:27 +00:00
Eelco Dolstra 5558652709 Don't substitute derivations that have preferLocalBuild set
In particular this means that "trivial" derivations such as writeText
are not substituted, reducing the number of GET requests to the binary
cache by about 200 on a typical NixOS configuration.
2013-06-20 19:26:31 +02:00
Eelco Dolstra 1906cce6fc Increase SQLite's auto-checkpoint interval
Common operations like instantiating a NixOS system config no longer
fitted in 8192 pages, leading to more fsyncs.  So increase this limit.
2013-06-20 14:01:33 +00:00
Eelco Dolstra 9b11165aec Disable the copy-from-other-stores substituter
This substituter basically cannot work reliably since we switched to
SQLite, since SQLite databases may need write access to open them even
just for reading (and in WAL mode they always do).
2013-06-20 12:01:33 +02:00
Eelco Dolstra 22144afa8d Don't keep "disabled" substituters running
For instance, it's pointless to keep copy-from-other-stores running if
there are no other stores, or download-using-manifests if there are no
manifests.  This also speeds things up because we don't send queries
to those substituters.
2013-06-20 11:55:15 +02:00
Eelco Dolstra 1b6ee8f4c7 Allow hard links between the outputs of a derivation 2013-06-13 17:29:56 +02:00
Eelco Dolstra cd49ee0897 Fix a security bug in hash rewriting
Before calling dumpPath(), we have to make sure the files are owned by
the build user.  Otherwise, the build could contain a hard link to
(say) /etc/shadow, which would then be read by the daemon and
rewritten as a world-readable file.

This only affects systems that don't have hard link restrictions
enabled.
2013-06-13 17:12:24 +02:00
Eelco Dolstra 1e2c7c04b1 Fix assertion failure in canonicalisePathMetaData() after hash rewriting
The assertion in canonicalisePathMetaData() failed because the
ownership of the path already changed due to the hash rewriting.  The
solution is not to check the ownership of rewritten paths.

Issue #122.
2013-06-13 17:12:06 +02:00
Eelco Dolstra 6cc2a8f8ed computeFSClosure: Only process the missing/corrupt paths
Issue #122.
2013-06-13 16:43:20 +02:00
Eelco Dolstra f9ff67e948 In repair mode, update the hash of rebuilt paths
Otherwise subsequent invocations of "--repair" will keep rebuilding
the path.  This only happens if the path content differs between
builds (e.g. due to timestamps).
2013-06-13 14:46:07 +02:00
Eelco Dolstra 6b05f688ee nix-daemon: Trust options like binary-caches when the client is root
Fixes #127.
2013-06-12 12:10:26 +02:00
Eelco Dolstra ca70fba0bf Remove obsolete EOF checks 2013-06-07 15:10:23 +02:00
Eelco Dolstra 5959c591a0 Process stderr from substituters while doing have/info queries 2013-06-07 15:02:14 +02:00
Eelco Dolstra c5f9d0d080 Buffer reads from the substituter
This greatly reduces the number of system calls.
2013-06-07 14:00:23 +02:00
Eelco Dolstra b09b87321c nix-store --export: Export paths in topologically sorted order
Fixes #118.
2013-05-23 14:55:36 -04:00
Eelco Dolstra 18a48d80a0 Show function names in error messages
Functions in Nix are anonymous, but if they're assigned to a
variable/attribute, we can use the variable/attribute name in error
messages, e.g.

while evaluating `concatMapStrings' at `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/strings.nix:18:25':
...
2013-05-16 19:08:02 +02:00
Eelco Dolstra 1b3a03f161 Show which function argument was unexpected
Fixes #116.
2013-05-16 17:56:14 +02:00
Eelco Dolstra 229567293c Shut up a compiler warning 2013-05-16 17:48:19 +02:00
Eelco Dolstra 2ee9da9e22 In trace messages, don't print the output path
This doesn't work if there is no output named "out".  Hydra didn't use
it anyway.
2013-05-10 00:24:33 +02:00
Eelco Dolstra 6eba05613a Communicate build timeouts to Hydra 2013-05-09 18:39:04 +02:00
Eelco Dolstra 69b8f9980f build-remote.pl: Enforce timeouts locally
Don't pass --timeout / --max-silent-time to the remote builder.
Instead, let the local Nix process terminate the build if it exceeds a
timeout.  The remote builder will be killed as a side-effect.  This
gives better error reporting (since the timeout message from the
remote side wasn't properly propagated) and handles non-Nix problems
like SSH hangs.
2013-05-09 17:17:17 +02:00
Eelco Dolstra 470553bd05 Don't let stderr writes in substituters cause a deadlock 2013-05-01 13:21:39 +02:00
Eelco Dolstra 0374d94437 addAdditionalRoots(): Check each path only once 2013-04-26 12:07:25 +02:00
Eelco Dolstra 772b70952f Fix --timeout
I'm not sure if it has ever worked correctly.  The line "lastWait =
after;" seems to mean that the timer was reset every time a build
produced log output.

Note that the timeout is now per build, as documented ("the maximum
number of seconds that a builder can run").
2013-04-23 18:04:59 +02:00
Eelco Dolstra 934cf2d1f4 Nix daemon: respect build timeout from the client 2013-04-23 16:59:06 +02:00
Eelco Dolstra 258897c265 Complain if /homeless-shelter exists 2013-04-04 11:16:26 +02:00
Shea Levy cc63db1dd5 makeStoreWritable: Ask forgiveness, not permission
It is surprisingly impossible to check if a mountpoint is a bind mount
on Linux, and in my previous commit I forgot to check if /nix/store was
even a mountpoint at all. statvfs.f_flag is not populated with MS_BIND
(and even if it were, my check was wrong in the previous commit).

Luckily, the semantics of mount with MS_REMOUNT | MS_BIND make both
checks unnecessary: if /nix/store is not a mountpoint, then mount will
fail with EINVAL, and if /nix/store is not a bind-mount, then it will
not be made writable. Thus, if /nix/store is not a mountpoint, we fail
immediately (since we don't know how to make it writable), and if
/nix/store IS a mountpoint but not a bind-mount, we fail at first write
(see below for why we can't check and fail immediately).

Note that, due to what is IMO buggy behavior in Linux, calling mount
with MS_REMOUNT | MS_BIND on a non-bind readonly mount makes the
mountpoint appear writable in two places: In the sixth (but not the
10th!) column of mountinfo, and in the f_flags member of struct statfs.
All other syscalls behave as if the mount point were still readonly (at
least for Linux 3.9-rc1, but I don't think this has changed recently or
is expected to soon). My preferred semantics would be for MS_REMOUNT |
MS_BIND to fail on a non-bind mount, as it doesn't make sense to remount
a non bind-mount as a bind mount.
2013-03-25 19:00:16 +01:00
Shea Levy 2c9cf50746 makeStoreWritable: Use statvfs instead of /proc/self/mountinfo to find out if /nix/store is a read-only bind mount
/nix/store could be a read-only bind mount even if it is / in its own filesystem, so checking the 4th field in mountinfo is insufficient.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-03-25 19:00:16 +01:00
Eelco Dolstra 804709706c Fix building against Bison 2.6 2013-03-14 18:31:08 +01:00
Eelco Dolstra c56bc3d81c Make sure that thunks are restored properly if an exception occurs
Fixes Hydra bug #67.
2013-03-14 17:21:13 +01:00
Eelco Dolstra bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra 28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra 8057a192e3 Handle systems without lutimes() or lchown() 2013-02-28 19:55:09 +01:00
Eelco Dolstra f45c731cd7 Handle symlinks properly
Now it's really brown paper bag time...
2013-02-28 14:51:08 +01:00
Eelco Dolstra 0111ba98ea Handle hard links to other files in the output 2013-02-27 17:18:41 +01:00
Eelco Dolstra b008674e46 Refactoring: Split off the non-recursive canonicalisePathMetaData()
Also, change the file mode before changing the owner.  This prevents a
slight time window in which a setuid binary would be setuid root.
2013-02-27 16:42:19 +01:00
Eelco Dolstra 5526a282b5 Security: Don't allow builders to change permissions on files they don't own
It turns out that in multi-user Nix, a builder may be able to do

  ln /etc/shadow $out/foo

Afterwards, canonicalisePathMetaData() will be applied to $out/foo,
causing /etc/shadow's mode to be set to 444 (readable by everybody but
writable by nobody).  That's obviously Very Bad.

Fortunately, this fails in NixOS's default configuration because
/nix/store is a bind mount, so "ln" will fail with "Invalid
cross-device link".  It also fails if hard-link restrictions are
enabled, so a workaround is:

  echo 1 > /proc/sys/fs/protected_hardlinks

The solution is to check that all files in $out are owned by the build
user.  This means that innocuous operations like "ln
${pkgs.foo}/some-file $out/" are now rejected, but that already failed
in chroot builds anyway.
2013-02-26 02:30:19 +01:00
Eelco Dolstra dadf7a5b46 build-remote: Use the --quiet flag
‘--option verbosity 0’ doesn't actually do anything.
2013-02-19 16:56:31 +01:00
Ludovic Courtès 3e067ac11c Add `Settings::nixDaemonSocketFile'. 2013-02-19 10:19:18 +01:00
Ludovic Courtès 5ea138dc4b Enable chroot support on old glibc versions. 2013-02-19 10:19:11 +01:00