philiptaron/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
4465 commits 26 branches 4 tags 76 MiB
Commit graph

22 commits

Author SHA1 Message Date
Eelco Dolstra 5f862658c3 Remove bad daemon connections from the pool 
This is necessary for long-running processes like hydra-queue-runner:
if a nix-daemon worker is killed, we need to stop reusing that
connection.
 2016-02-24 11:39:56 +01:00
Eelco Dolstra 71a5161365 Implement buildDerivation() via the daemon 2015-09-03 12:56:59 +02:00
Eelco Dolstra b3491c781c More cleanup 2015-07-20 01:16:16 +02:00
Eelco Dolstra 6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra 1511aa9f48 Allow remote builds without sending the derivation closure 
Previously, to build a derivation remotely, we had to copy the entire
closure of the .drv file to the remote machine, even though we only
need the top-level derivation. This is very wasteful: the closure can
contain thousands of store paths, and in some Hydra use cases, include
source paths that are very large (e.g. Git/Mercurial checkouts).

So now there is a new operation, StoreAPI::buildDerivation(), that
performs a build from an in-memory representation of a derivation
(BasicDerivation) rather than from a on-disk .drv file. The only files
that need to be in the Nix store are the sources of the derivation
(drv.inputSrcs), and the needed output paths of the dependencies (as
described by drv.inputDrvs). "nix-store --serve" exposes this
interface.

Note that this is a privileged operation, because you can construct a
derivation that builds any store path whatsoever. Fixing this will
require changing the hashing scheme (i.e., the output paths should be
computed from the other fields in BasicDerivation, allowing them to be
verified without access to other derivations). However, this would be
quite nice because it would allow .drv-free building (e.g. "nix-env
-i" wouldn't have to write any .drv files to disk).

Fixes #173.
 2015-07-17 17:57:40 +02:00
Eelco Dolstra 829af22759 Print a warning when loading a large path into memory 
I.e. if you have a derivation with

  src = ./huge-directory;

you'll get a warning that this is not a good idea.
 2014-06-10 13:30:09 +02:00
Eelco Dolstra fb8d8f5428 Remove unnecessary null pointer checks 
Fixes #225.
 2014-03-12 14:42:25 +01:00
Eelco Dolstra 5959c591a0 Process stderr from substituters while doing have/info queries 2013-06-07 15:02:14 +02:00
Eelco Dolstra d5a5a83ad4 Use data() instead of c_str() where appropriate 2012-02-09 18:27:45 +01:00
Eelco Dolstra 273b288a7e * importPath() -> importPaths(). Because of buffering of the input 
stream it's now necessary for the daemon to process the entire
  sequence of exported paths, rather than letting the client do it.
 2011-12-16 22:31:25 +00:00
Eelco Dolstra 8d3dfa2c17 * Avoid expensive conversions from char arrays to STL strings. 2011-12-16 21:29:46 +00:00
Eelco Dolstra e0bd307802 * Make the import operation through the daemon much more efficient 
(way fewer roundtrips) by allowing the client to send data in bigger
  chunks.
* Some refactoring.
 2011-12-16 19:44:13 +00:00
Eelco Dolstra 78598d06f0 * Clean up exception handling. 2011-12-16 15:45:42 +00:00
Eelco Dolstra 5a1b9ed0aa * Refactoring: move sink/source buffering into separate classes. 
* Buffer the HashSink.  This speeds up hashing a bit because it
  prevents lots of calls to the hash update functions (e.g. nix-hash
  went from 9.3s to 8.7s of user time on the closure of my
  /var/run/current-system).
 2011-12-15 16:19:53 +00:00
Eelco Dolstra a3e0656cbb * Buffer reads in FdSource. Together with write buffering, this 
significantly cuts down the number of syscalls (e.g., for "nix-store
  -qR /var/run/current-system" via the daemon, it reduced the number
  of syscalls in the client from 29134 to 4766 and in the daemon from
  44266 to 20666).
 2011-12-15 12:32:08 +00:00
Eelco Dolstra 3a48282b06 * Buffer writes in FdSink. This significantly reduces the number of 
system calls / context switches when dumping a NAR and in the worker
  protocol.
 2011-12-14 23:30:06 +00:00
Eelco Dolstra 77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it 
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
 2009-03-22 17:36:43 +00:00
Eelco Dolstra a72709afd8 * Some refactoring: put the GC options / results in separate structs. 
* The garbage collector now also prints the number of blocks freed.
 2008-06-18 09:34:17 +00:00
Eelco Dolstra bd955e15e1 * GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour and 
Armijn Hemel.
 2008-05-21 11:17:31 +00:00
Eelco Dolstra 0130ef88ea * Daemon mode (`nix-worker --daemon'). Clients connect to the server 
via the Unix domain socket in /nix/var/nix/daemon.socket.  The
  server forks a worker process per connection.
* readString(): use the heap, not the stack.
* Some protocol fixes.
 2006-12-04 17:17:13 +00:00
Eelco Dolstra 0565b5f2b3 * More remote operations. 
* Added new operation hasSubstitutes(), which is more efficient than
  querySubstitutes().size() > 0.
 2006-11-30 22:43:55 +00:00
Eelco Dolstra 40b3f64b55 * Skeleton of the privileged worker program. 
* Some refactoring: put the NAR archive integer/string serialisation
  code in a separate file so it can be reused by the worker protocol
  implementation.
 2006-11-30 19:19:59 +00:00