Commit graph

2241 commits

Author SHA1 Message Date
aszlig 8b88d25cda libutil: Limit readLink() error to only overflows.
Let's not just improve the error message itself, but also the behaviour
to actually work around the ntfs-3g symlink bug. If the readlink() call
returns a smaller size than the stat() call, this really isn't a problem
even if the symlink target really has changed between the calls.

So if stat() reports the size for the absolute path, it's most likely
that the relative path is smaller and thus it should also work for file
system bugs as mentioned in 93002d69fc58c2b71e2dfad202139230c630c53a.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Tested-by: John Ericson <Ericson2314@Yahoo.com>
2015-01-02 12:53:42 +01:00
aszlig bbd45ac80f libutil: Improve errmsg on readLink size mismatch.
A message like "error: reading symbolic link `...' : Success" really is
quite confusing, so let's not indicate "success" but rather point out
the real issue.

We could also limit the check of this to just check for non-negative
values, but this would introduce a race condition between stat() and
readlink() if the link target changes between those two calls, thus
leading to a buffer overflow vulnerability.

Reported by @Ericson2314 on IRC. Happened due to a possible ntfs-3g bug
where a relative symlink returned the absolute path (st_)size in stat()
while readlink() returned the relative size.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Tested-by: John Ericson <Ericson2314@Yahoo.com>
2015-01-02 12:53:42 +01:00
Shea Levy 3d97b8d1e7 LocalStore initialization: Don't die if build-users-group doesn't exist
See NixOS/nixpkgs@9245516
2014-12-29 14:40:13 +01:00
Eelco Dolstra bd0f362d2f Revive running builds in a PID namespace 2014-12-23 17:25:06 +01:00
Eelco Dolstra 4e0607369e Pedantry 2014-12-14 01:51:14 +01:00
Eelco Dolstra 8bdff8c100 Merge branch 'cygwin-master' of https://github.com/ternaris/nix 2014-12-14 01:49:14 +01:00
Eelco Dolstra fa2063ca35 Better error message 2014-12-13 16:54:40 +01:00
Eelco Dolstra b77037b8fd Silence some warnings on GCC 4.9 2014-12-12 17:14:28 +01:00
Eelco Dolstra 46f3eb6fdd Shut up a Valgrind warning 2014-12-12 15:10:02 +01:00
Eelco Dolstra f52b6c944e Fix some memory leaks 2014-12-12 15:01:16 +01:00
Eelco Dolstra 28f22b4653 Ensure we're writing to stderr in the builder
http://hydra.nixos.org/build/17862041
2014-12-12 14:35:44 +01:00
Eelco Dolstra 5a2d451648 Don't abort if we get a signal while waiting for the pager 2014-12-12 14:05:23 +01:00
Eelco Dolstra 54616be64f Get rid of unnecessary "interrupted by the user" message with -vvv 2014-12-12 14:01:14 +01:00
Eelco Dolstra 4acca1a5b9 Remove chatty message
This broke building with "-vv", because the builder is not allowed to
write to stderr at this point.
2014-12-12 13:43:59 +01:00
Eelco Dolstra ad790022fd Doh 2014-12-12 13:41:00 +01:00
Eelco Dolstra dcdb254281 Remove tabs 2014-12-12 12:39:50 +01:00
Eelco Dolstra df319047f4 Remove dead code 2014-12-12 12:30:38 +01:00
Eelco Dolstra 8aedaf111e Remove canary stuff 2014-12-12 10:59:50 +01:00
Eelco Dolstra c56bfbe863 Provide default pagers
Borrowed from systemd.
2014-12-10 18:16:05 +01:00
Eelco Dolstra 851b47bd7d Don't do vfork in conjunction with setuid 2014-12-10 18:01:01 +01:00
Eelco Dolstra 0e8fc118b3 Use vfork 2014-12-10 17:25:39 +01:00
Eelco Dolstra b5ed5b6e66 Rename function 2014-12-10 17:25:38 +01:00
Eelco Dolstra e529823635 Don't wait for PID -1
The pid field can be -1 if forking the substituter process failed.
2014-12-10 13:53:04 +01:00
Eelco Dolstra ad332e1718 Revert "Use posix_spawn to run the pager"
This reverts commit d34d2b2bbf.
2014-12-10 13:48:50 +01:00
Shea Levy 50c3352811 builtins.readFile: realise context associated with the path 2014-12-10 12:26:53 +01:00
Marko Durkovic 629e8da3aa Explicitly include required C headers 2014-12-09 13:00:59 +01:00
Eelco Dolstra 8d169b2b75 Define ‘environ’
http://hydra.nixos.org/build/17690555
2014-12-05 21:05:24 +01:00
Eelco Dolstra d34d2b2bbf Use posix_spawn to run the pager
In low memory environments, "nix-env -qa" failed because the fork to
run the pager hit the kernel's overcommit limits. Using posix_spawn
gets around this. (Actually, you have to use posix_spawn with the
undocumented POSIX_SPAWN_USEVFORK flag, otherwise it just uses
fork/exec...)
2014-12-05 20:34:41 +01:00
Shea Levy 608110804c Make all ExternalValueBase functions const 2014-12-02 10:27:10 -05:00
Shea Levy 320659b0cd Allow external code using libnixexpr to add types
Code that links to libnixexpr (e.g. plugins loaded with importNative, or
nix-exec) may want to provide custom value types and operations on
values of those types. For example, nix-exec is currently using sets
where a custom IO value type would be more appropriate. This commit
provides a generic hook for such types in the form of tExternal and the
ExternalBase virtual class, which contains all functions necessary for
libnixexpr's type-polymorphic functions (e.g. `showType`) to be
implemented.
2014-12-02 10:27:04 -05:00
Eelco Dolstra 976df480c9 Add a primop for regular expression pattern matching
The function ‘builtins.match’ takes a POSIX extended regular
expression and an arbitrary string. It returns ‘null’ if the string
does not match the regular expression. Otherwise, it returns a list
containing substring matches corresponding to parenthesis groups in
the regex. The regex must match the entire string (i.e. there is an
implied "^<pat>$" around the regex).  For example:

  match "foo" "foobar" => null
  match "foo" "foo" => []
  match "f(o+)(.*)" "foooobar" => ["oooo" "bar"]
  match "(.*/)?([^/]*)" "/dir/file.nix" => ["/dir/" "file.nix"]
  match "(.*/)?([^/]*)" "file.nix" => [null "file.nix"]

The following example finds all regular files with extension .nix or
.patch underneath the current directory:

  let

    findFiles = pat: dir: concatLists (mapAttrsToList (name: type:
      if type == "directory" then
        findFiles pat (dir + "/" + name)
      else if type == "regular" && match pat name != null then
        [(dir + "/" + name)]
      else []) (readDir dir));

  in findFiles ".*\\.(nix|patch)" (toString ./.)
2014-11-25 11:47:06 +01:00
Eelco Dolstra 4e340a983f forceString(): Accept pos argument 2014-11-25 10:23:36 +01:00
Eelco Dolstra b7b6e3ddec Build derivations in a more predictable order
Derivations are now built in order of derivation name, so a package
named "aardvark" is built before "baboon".

Fixes #399.
2014-11-24 16:50:46 +01:00
Eelco Dolstra 9e3389c337 Don't create unnecessary substitution goals for derivations 2014-11-24 16:50:46 +01:00
Shea Levy b0c5c2ac34 import derivation: cleanup
Before this there was a bug where a `find` was being called on a
not-yet-sorted set. The code was just a mess before anyway, so I cleaned
it up while fixing it.
2014-11-20 22:48:12 -05:00
Eelco Dolstra 8299aaf079 Disable vacuuming the DB after garbage collection
Especially in WAL mode on a highly loaded machine, this is not a good
idea because it results in a WAL file of approximately the same size
ad the database, which apparently cannot be deleted while anybody is
accessing it.
2014-11-19 18:14:24 +01:00
Eelco Dolstra a3e5c99d66 nix-daemon: Call exit(), not _exit()
This was preventing destructors from running. In particular, it was
preventing the deletion of the temproot file for each worker
process. It may also have been responsible for the excessive WAL
growth on Hydra (due to the SQLite database not being closed
properly).

Apparently broken by accident in
8e9140cfde.
2014-11-19 17:09:27 +01:00
Eelco Dolstra 1256ab3b44 Clean up temp roots in a more C++ way 2014-11-19 17:07:29 +01:00
Eelco Dolstra 35aad73bb6 Fix message 2014-11-17 01:00:39 +01:00
Shea Levy 2719627bbe realiseContext: Handle all context types
Avoids an assertion
2014-11-15 21:43:51 -05:00
Shea Levy 997defa166 Add functors (callable attribute sets).
With this, attribute sets with a `__functor` attribute can be applied
just like normal functions. This can be used to attach arbitrary
metadata to a function without callers needing to treat it specially.
2014-11-15 16:12:05 -05:00
Eelco Dolstra 8cfe939b0f Don't use ADDR_LIMIT_3GB
This gives 32-bit builds on x86_64-linux more memory.
2014-11-14 14:16:20 +01:00
Eelco Dolstra bab8d9b52a Make ~DerivationGoal more reliable 2014-11-12 11:35:53 +01:00
Eelco Dolstra 06a86aee15 nix-store --gc: Don't warn about missing manifests directory 2014-11-04 10:41:29 +01:00
Eelco Dolstra 526811c87a nix-daemon: Get peer credentials on Mac OS X
This makes allowed-users and trusted-users work on Mac OS X.
2014-10-31 10:08:59 +01:00
Eelco Dolstra e389f4ea55 Improve error message if the daemon worker fails to start 2014-10-31 09:36:09 +01:00
Eelco Dolstra 3a9b4a1467 Fix more warnings 2014-10-31 08:49:15 +01:00
Eelco Dolstra 80893a13a7 Shut up a clang warning 2014-10-31 08:49:15 +01:00
Eelco Dolstra 11aef17a77 Remove comments claiming we use a private PID namespace
This is no longer the case since
524f89f139.
2014-10-29 15:49:34 +01:00
Shea Levy 6062b12160 Fix build on gcc < 4.7 2014-10-20 12:15:50 -04:00
Shea Levy f040159f77 Revert "Drop support for pre-c++11 compilers."
The breakage this fixed can be worked around without removing support.

This reverts commit 84a13dc576.
2014-10-20 11:33:48 -04:00
Eelco Dolstra ecc2c8f464 Improve printing of ASTs 2014-10-20 09:13:21 +02:00
Shea Levy 84a13dc576 Drop support for pre-c++11 compilers.
In particular, gcc 4.6's std::exception::~exception has an exception
specification in c++0x mode, which requires us to use that deprecated
feature in nix (and led to breakage after some recent changes that were
valid c++11).

nix already uses several c++11 features and gcc 4.7 has been around for
over 2 years.
2014-10-18 22:44:59 -04:00
Shea Levy 0ee1ca628a baseNameOf: Don't copy paths to the store first 2014-10-18 20:28:28 -04:00
Shea Levy d16e3c7f09 Export realiseContext in libnixexpr
Useful for importNative plugins
2014-10-17 22:15:09 -04:00
Eelco Dolstra d04e1ff12a nix-store -q: Check for conflicting flags
Fixes #364.
2014-10-14 12:15:39 +02:00
Eelco Dolstra c6849e2dee Remove redundant space in usage errors 2014-10-14 11:41:04 +02:00
Eelco Dolstra 2a4dbcff74 Improved error message when encountering unsupported file types
Fixes #269.
2014-10-14 10:51:19 +02:00
Eelco Dolstra 6bb4c0b712 mkList: Scrub better
Clearing v.app.right was not enough, because the length field of a
list only takes 32 bits, so the most significant 32 bits of v.app.left
(a.k.a. v.thunk.env) would remain. This could cause Boehm GC to
interpret it as a valid pointer.

This change reduces maximum RSS for evaluating the ‘tested’ job in
nixos/release-small.nix from 1.33 GiB to 0.80 GiB, and runtime by
about 8%.
2014-10-09 13:08:53 +02:00
Eelco Dolstra 986fbd6fab Typo 2014-10-09 11:34:48 +02:00
Eelco Dolstra b6809608cc Get rid of some unnecessary ExprConcatStrings nodes in dynamic attrs
This gives a ~18% speedup in NixOS evaluation (after converting
most calls to hasAttr/getAttr to dynamic attrs).
2014-10-05 01:04:58 +02:00
Eelco Dolstra 1418806969 Show total allocations 2014-10-05 00:39:28 +02:00
Eelco Dolstra c3f0a489f9 Add primop ‘catAttrs’ 2014-10-04 18:15:03 +02:00
Eelco Dolstra d4fcbe1687 Add primop ‘attrValues’ 2014-10-04 16:41:24 +02:00
Eelco Dolstra 58d8a213b0 Tweak 2014-10-04 11:27:23 +02:00
Eelco Dolstra 3f8576a6ab Remove some duplicate code 2014-10-03 22:37:51 +02:00
Shea Levy c08c802bf3 Add readDir primop 2014-10-03 22:32:11 +02:00
Eelco Dolstra 3b5fa8d50c Don't recompile the same regex over and over 2014-10-03 21:29:40 +02:00
Eelco Dolstra 104e55bb7f nix-env: Add regular expression support in selectors
So you can now do things like:

  $ nix-env -qa '.*zip.*'
  $ nix-env -qa '.*(firefox|chromium).*'
2014-10-03 21:29:40 +02:00
Eelco Dolstra 3800f441e4 createDirs(): Handle ‘path’ being a symlink
In particular, this fixes "nix-build -o /tmp/result" on Mac OS X
(where /tmp is a symlink).
2014-10-03 16:53:28 +02:00
Eelco Dolstra a17c23426e printValue(): Don't print <CYCLE> for repeated values 2014-10-01 15:54:40 +02:00
Eelco Dolstra d61853430a Support control characters in JSON output 2014-09-30 00:41:18 +02:00
Eelco Dolstra f77be20c16 printMissing(): Print derivations in approximate build order 2014-09-26 14:09:20 +02:00
Eelco Dolstra 9b146a52f1 nix-daemon: Close unnecessary fd 2014-09-25 18:45:43 +02:00
Eelco Dolstra 0ed1b924be Bindings: Remove copy constructor 2014-09-24 15:29:05 +02:00
Shell Turner cf72a61af2 Add --force-name support for --set in nix-env, to support nix-install-package --set 2014-09-23 15:11:24 +02:00
Eelco Dolstra ebb1dbb3e1 Add missing static 2014-09-23 15:08:27 +02:00
Eelco Dolstra 53b044c2f6 Don't evaluate inside a "throw"
Workaround for
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41174. This caused
hydra-eval-jobs to ignore SIGINT.
2014-09-22 19:18:05 +02:00
Eelco Dolstra 0cd6596b0e Add ‘deepSeq’ primop
Note that unlike ‘lib.deepSeq’ in Nixpkgs, this handles cycles.
2014-09-22 16:05:00 +02:00
Eelco Dolstra 831fc8ea21 Make forceValueDeep work on values with cycles 2014-09-22 16:05:00 +02:00
Eelco Dolstra 1e0a799bef Rename strictForceValue -> forceValueDeep 2014-09-22 16:05:00 +02:00
Eelco Dolstra 022618c794 Handle cycles when printing a value
So this no longer crashes with a stack overflow:

  nix-instantiate -E --eval 'let as = { x = as; }; in as'

Instead it prints:

  { x = { x = <CYCLE>; }; }
2014-09-22 16:05:00 +02:00
Eelco Dolstra a54c263402 Add ‘seq’ primop 2014-09-22 16:05:00 +02:00
Eelco Dolstra eff120d1b9 Add a function ‘valueSize’
It returns the size of value, including all other values and
environments reachable from it. It is intended for debugging memory
consumption issues.
2014-09-22 16:05:00 +02:00
Eelco Dolstra 2d6cd8aafd attrNames: Don't allocate duplicates of the symbols 2014-09-19 18:11:46 +02:00
Eelco Dolstra ea525a261f Fix off-by-one 2014-09-19 18:08:14 +02:00
Eelco Dolstra 93e4f01ee3 Inline Bindings::find() 2014-09-19 16:56:13 +02:00
Eelco Dolstra 5b58991a71 Store Attrs inside Bindings
This prevents a double allocation per attribute set.
2014-09-19 16:49:41 +02:00
Eelco Dolstra 0342eb1705 Remove bogus comment 2014-09-19 15:07:22 +02:00
Eelco Dolstra d4a71ec3bf Update spec file
http://hydra.nixos.org/build/14344391
2014-09-18 15:42:01 +02:00
Eelco Dolstra 8be9990cdb Install some pkgconfig files 2014-09-18 12:00:40 +02:00
Eelco Dolstra d98bfcbf81 On Linux, disable address space randomization 2014-09-17 17:21:13 +02:00
Eelco Dolstra 6e5b02bee4 Add some instrumentation for debugging GC leaks 2014-09-17 15:19:07 +02:00
Eelco Dolstra d37d012774 Settings: Add bool get() 2014-09-17 15:18:13 +02:00
Eelco Dolstra 9d65287b91 Fix dependency ordering 2014-09-04 20:02:08 +02:00
Eelco Dolstra 9472b4157d Fix boost::too_many_args error
Fixes #333.
2014-09-02 22:53:01 +02:00
Ludovic Courtès 8fb8c26b6d Add an 'optimiseStore' remote procedure call. 2014-09-01 23:53:26 +02:00
Eelco Dolstra b72e93bca8 Add disallowedReferences / disallowedRequisites
For the "stdenv accidentally referring to bootstrap-tools", it seems
easier to specify the path that we don't want to depend on, e.g.

  disallowedRequisites = [ bootstrapTools ];
2014-08-28 18:57:13 +02:00
Gergely Risko fd61069a42 Introduce allowedRequisites feature 2014-08-28 18:23:55 +02:00
Joel Taylor b224ac1520 fix disappearing bash arguments 2014-08-21 23:17:50 +02:00