lbodor/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
2349 commits 21 branches 2 tags 66 MiB
Commit graph

374 commits

Author SHA1 Message Date
Eelco Dolstra
bd48fd97f6 * Impose a configurable time limit on patch generation. This is 
necessary because bsdiff has some pathological worst-case behaviour,
  e.g. O(n^2) on files consisting only of 0s.
 2010-11-29 09:15:45 +00:00
Eelco Dolstra
1a211d812f * Oops. 2010-11-17 17:54:49 +00:00
Eelco Dolstra
a4f0365b2d * When checking whether a patch is applicable, for efficiency, use 
`nix-store -q --hash' to get the hash of the base path rather than
  `nix-hash'.  However, only do this for estimating the size of a
  download, not for the actual substitution, because sometimes the
  contents of store paths are modified (which they shouldn't, of
  course).
 2010-11-17 17:41:59 +00:00
Eelco Dolstra
3d38a49840 * In the download size indication, take binary patches into account. 
Hopefully this doesn't slow things down too much.
 2010-11-17 17:32:25 +00:00
Eelco Dolstra
bdf089f463 * Before a build, show the disk space that the downloaded store paths 
will approximately require.
 2010-11-17 14:31:42 +00:00
Eelco Dolstra
06699d4219 * Store the NAR size in the manifest. 2010-11-17 12:57:52 +00:00
Eelco Dolstra
5693b8a7e2 * nix-push: no need to compute the NAR hash, since the Nix database 
already has it (`nix-store -q --hash').
 2010-11-17 12:51:54 +00:00
Eelco Dolstra
bfa6ee7d91 * Don't use SSH's `-tt' flag because it doesn't seem to work 
on OpenSolaris when using connection sharing.  Instead have
  the remote side check for disconnection and kill the process 
  group when that happens.
 2010-10-04 12:30:46 +00:00
Eelco Dolstra
bf0dde9597 * Always print hook output on stderr, even if --no-build-output is 
set.
* In the build hook, print a trace message to allow Hydra to pick up
  the name of the remote machine used for the build.
 2010-08-31 12:36:24 +00:00
Eelco Dolstra
80e722278c * When using the build hook, distinguish between the stderr of the 
hook script proper, and the stdout/stderr of the builder.  Only the
  latter should be saved in /nix/var/log/nix/drvs.
* Allow the verbosity to be set through an option.
* Added a flag --quiet to lower the verbosity level.
 2010-08-30 14:53:03 +00:00
Eelco Dolstra
766f708418 * Experimental feature: allow a derivation to tell the build hook that 
it requires a certain feature on the build machine, e.g.

    requiredSystemFeatures = [ "kvm" ];

  We need this in Hydra to make sure that builds that require KVM
  support are forwarded to machines that have KVM support.  Probably
  this should also be enforced for local builds.
 2010-08-27 13:18:13 +00:00
Eelco Dolstra
e437b08250 * Made the build hook mechanism more efficient. Rather than starting 
the hook every time we want to ask whether we can run a remote build
  (which can be very often), we now reuse a hook process for answering
  those queries until it accepts a build.  So if there are N
  derivations to be built, at most N hooks will be started.
 2010-08-25 20:44:28 +00:00
Eelco Dolstra
1a396f3789 * Don't call "cat". 2010-08-25 12:19:30 +00:00
Eelco Dolstra
95deba581d * In the build hook, temporarily register the derivation and its 
output as GC roots.  This prevents a race if the garbage collector
  is running during the build.
 2010-08-25 11:54:11 +00:00
Eelco Dolstra
1e5f5ea2e9 * Correctly handle SSH failing to establish a connection. Because 
this didn't work right, the build hook wouldn't fall back to using
  other machines of the desired type.
 2010-08-24 14:27:07 +00:00
Eelco Dolstra
5f9aad44ca * nix-build: recognise --cores. 2010-08-11 15:28:02 +00:00
Eelco Dolstra
587dc8aa00 * Sync with the trunk. 2010-08-04 17:48:29 +00:00
Eelco Dolstra
532d766c27 * Don't barf if the source NAR for a patch has disappeared. 2010-07-21 11:30:23 +00:00
Eelco Dolstra
560ab22f7d * Ignore packages that don't have a version. 
* Work on a manifest instead of a channel directory.
 2010-06-23 21:11:33 +00:00
Eelco Dolstra
87ef5907e9 * Sync. 2010-06-22 14:42:34 +00:00
Eelco Dolstra
819548d92f * Pass `--fallback' to the remote build to ignore failing 
substituters.
 2010-06-22 14:41:22 +00:00
Eelco Dolstra
c778ed1768 * Fix the copy-from-other-stores substituter. 2010-04-26 13:39:55 +00:00
Eelco Dolstra
05fbf61f0e * nix-prefetch-url depends on sed (see 
http://hydra.nixos.org/build/311170).
 2010-03-05 18:26:47 +00:00
Eelco Dolstra
68e55cd9da * Since Hydra now escapes +', =' and `?' to %NN in URIs, 
nix-prefetch-url should unescape them, because `%' is not a valid
  character in store path names.
 2010-02-16 00:10:39 +00:00
Eelco Dolstra
719cebcac2 * "Fix" incorrect help message. 2010-02-04 09:38:09 +00:00
Eelco Dolstra
443673620d * Don't use ssh's -f flag since it leads to lots of lingering ssh 
processes.
 2010-02-04 02:38:40 +00:00
Eelco Dolstra
7ec5a65925 * Doh! The scope of $slotLock should extend to the end of the script, 
because otherwise the lock will be released at the end of the while
  loop.
 2010-02-04 02:18:29 +00:00
Eelco Dolstra
c9b2d80bcd * Typo. 2010-02-04 02:05:34 +00:00
Eelco Dolstra
3cfe65e516 * Doh! Calling `system' in an END block causes the exit status in $? 
to be changed to 0.
 2010-02-04 02:05:22 +00:00
Eelco Dolstra
e51a276907 * Remove the `cat' calls when not using --gzip. 2010-02-04 01:39:23 +00:00
Eelco Dolstra
408913bbaf * Revert r19796 for now. 2010-02-04 00:12:57 +00:00
Eelco Dolstra
c45de33c67 * Respect @sysconfdir@. 2010-02-03 21:13:37 +00:00
Eelco Dolstra
d0c32dc135 * In the build hook, if connecting to a machine fails, try the other 
machines of the right type (if available).  This makes the build
  farm more robust to failures.
 2010-02-03 20:35:37 +00:00
Eelco Dolstra
f56a039775 * Use SSH connection sharing in the remote build script. 
* Removed the Cygwin password hack since the problem is apparently
  fixed in Visual Studio.
 2010-02-03 20:12:18 +00:00
Eelco Dolstra
bc1e478db1 * nix-copy-closure: start only one SSH connection to the server, or 
recycle an already existing connection (using  OpenSSH's connection
  sharing feature).
 2010-02-03 15:34:52 +00:00
Eelco Dolstra
e839802720 * Don't do a chdir to $tmpDir. It's not necessary, and Windows doesn't 
support deleting the current directory.
 2010-02-02 15:29:18 +00:00
Eelco Dolstra
50e34891f0 * Disable gzip compression in build-remote.pl because it puts too much 
load on the Hydra build farm (where it's unnecessary anyway because
  it has a fast connection to the build machines).  In any case,
  compression can be enabled by using the `-C' option to ssh.
 2010-01-25 16:14:45 +00:00
Eelco Dolstra
7ca9972636 * When doing a nix-pull, remove old manifests downloaded from the same 
URL.  This prevents lots of old cruft accumulating in
  /nix/var/nix/manifests.
 2009-12-09 19:36:54 +00:00
Eelco Dolstra
c4c84d1edb * nix-build: be less verbose. 2009-12-09 18:08:28 +00:00
Eelco Dolstra
997db91e07 * Don't pass -K. It should really inherit the setting of the calling 
Nix though.
 2009-11-17 16:22:39 +00:00
Eelco Dolstra
3392d32e8b * In nix-pull/nix-channel, create the manifests directory if it 
doesn't exist.  The Debian packages don't include the manifests
  directory, so nix-channel would silently skip doing a nix-pull,
  resulting in everything being built from source.  Thanks to Juan
  Pedro Bolívar Puente.
 2009-11-13 10:08:31 +00:00
Eelco Dolstra
e1df4ef73c 2009-09-17 17:02:14 +00:00
Eelco Dolstra
86408b3f47 * build-remote.pl: Pick machines in a round-robin order, rather than 
giving jobs to the first machine until it hits its job limit, then
  the second machine and so on.  This should improve utilisation of
  the Hydra build farm a lot.  Also take an optional speed factor
  into account to cause fast machines to be preferred over slower
  machines with a similar load.
 2009-09-17 15:48:17 +00:00
Eelco Dolstra
57e0d73c77 * build-remote.pl: allow the system type to be a comma-separated list 
of system types.  Don't treat the x86_64-linux system type
  specially.
 2009-09-17 13:51:04 +00:00
Eelco Dolstra
20b6f94b65 * nix-build: pass the --show-trace flag. 2009-07-15 09:10:38 +00:00
Eelco Dolstra
f24cf5d303 * nix-build: if nix-instantiate or nix-store fails due to a signal 
(e.g. out of memory or a segfault), print an error message.
  Otherwise the user doesn't see anything.
 2009-06-10 11:30:34 +00:00
Eelco Dolstra
c710fe540e * Machines of type x86_64-linux can do i686-linux builds. 2009-04-21 12:42:47 +00:00
Eelco Dolstra
0460ea4c39 * Cleanup. 2009-04-16 09:27:33 +00:00
Eelco Dolstra
096affb55b * Update the mtime on the lock file to make it easy to see when a slot 
was last used.
 2009-03-29 18:40:44 +00:00
Eelco Dolstra
737423a89c * Use polling to wait for a remote build slot when using a build hook 
(that is, call the build hook with a certain interval until it
  accepts the build).
* build-remote.pl was totally broken: for all system types other than
  the local system type, it would send all builds to the *first*
  machine of the appropriate type.
 2009-03-29 18:00:45 +00:00