Commit graph

80 commits

Author SHA1 Message Date
Wout Mertens e974f20c98 Preload linked hashes to speed up lookups
By preloading all inodes in the /nix/store/.links directory, we can
quickly determine of a hardlinked file was already linked to the hashed
links.
This is tolerant of removing the .links directory, it will simply
recalculate all hashes in the store.
2014-05-13 23:10:06 +02:00
wmertens a84f503d86 Shortcut already-hardlinked files
If an inode in the Nix store has more than 1 link, it probably means that it was linked into .links/ by us. If so, skip.

There's a possibility that something else hardlinked the file, so it would be nice to be able to override this.

Also, by looking at the number of hardlinks for each of the files in .links/, you can get deduplication numbers and space savings.
2014-05-10 15:53:01 +02:00
Eelco Dolstra bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra 28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra b008674e46 Refactoring: Split off the non-recursive canonicalisePathMetaData()
Also, change the file mode before changing the owner.  This prevents a
slight time window in which a setuid binary would be setuid root.
2013-02-27 16:42:19 +01:00
Eelco Dolstra 5526a282b5 Security: Don't allow builders to change permissions on files they don't own
It turns out that in multi-user Nix, a builder may be able to do

  ln /etc/shadow $out/foo

Afterwards, canonicalisePathMetaData() will be applied to $out/foo,
causing /etc/shadow's mode to be set to 444 (readable by everybody but
writable by nobody).  That's obviously Very Bad.

Fortunately, this fails in NixOS's default configuration because
/nix/store is a bind mount, so "ln" will fail with "Invalid
cross-device link".  It also fails if hard-link restrictions are
enabled, so a workaround is:

  echo 1 > /proc/sys/fs/protected_hardlinks

The solution is to check that all files in $out are owned by the build
user.  This means that innocuous operations like "ln
${pkgs.foo}/some-file $out/" are now rejected, but that already failed
in chroot builds anyway.
2013-02-26 02:30:19 +01:00
Eelco Dolstra def5160b61 Clear any immutable bits in the Nix store
Doing this once makes subsequent operations like garbage collecting
more efficient since we don't have to call makeMutable() first.
2013-01-03 12:59:23 +01:00
Eelco Dolstra b9c2b4d5b4 Remove setting of the immutable bit
Using the immutable bit is problematic, especially in conjunction with
store optimisation.  For instance, if the garbage collector deletes a
file, it has to clear its immutable bit, but if the file has
additional hard links, we can't set the bit afterwards because we
don't know the remaining paths.

So now that we support having the entire Nix store as a read-only
mount, we may as well drop the immutable bit.  Unfortunately, we have
to keep the code to clear the immutable bit for backwards
compatibility.
2012-09-19 16:17:54 -04:00
Eelco Dolstra e94806d030 Merge branch 'master' into no-manifests 2012-08-27 11:09:07 -04:00
Eelco Dolstra 325d1cfebf Don't warn about maximum link count exceeded on 0-byte files 2012-08-07 16:22:54 -04:00
Eelco Dolstra b6c989b801 Fix race condition when two processes create a hard link to a file in .links
This is a problem because one process may set the immutable bit before
the second process has created its link.

Addressed random Hydra failures such as:

error: cannot rename `/nix/store/.tmp-link-17397-1804289383' to
`/nix/store/rsvzm574rlfip3830ac7kmaa028bzl6h-nixos-0.1pre-git/upstart-interface-version':
Operation not permitted
2012-08-05 21:41:44 -04:00
Eelco Dolstra 108e14bb18 Fix race condition when two processes create the same link in /nix/store/.links 2012-08-05 18:17:55 -04:00
Eelco Dolstra 9c2decaa19 Merge branch 'master' into no-manifests 2012-08-01 18:03:49 -04:00
Eelco Dolstra 234ce610e0 Doh 2012-08-01 16:09:47 -04:00
Eelco Dolstra 7b10562370 Make ‘nix-store --optimise’ interruptible 2012-08-01 16:06:49 -04:00
Eelco Dolstra 97421eb5ec Refactor settings processing
Put all Nix configuration flags in a Settings object.
2012-07-30 19:55:41 -04:00
Eelco Dolstra 8c79100839 Merge branch 'master' into no-manifests 2012-07-26 15:14:33 -04:00
Eelco Dolstra e98c029717 Handle platforms that don't support linking to a symlink
E.g. Darwin doesn't allow this.
2012-07-23 18:42:18 -04:00
Eelco Dolstra fd63c8bfcd Unlink the right file 2012-07-23 18:06:37 -04:00
Eelco Dolstra 6193105710 Automatically optimise the Nix store when a new path is added
Auto-optimisation is enabled by default.  It can be turned off by
setting auto-optimise-store to false in nix.conf.
2012-07-23 17:14:15 -04:00
Eelco Dolstra 564fb7d9fa optimiseStore(): Use a content-addressed file store in /nix/store/.links
optimiseStore() now creates persistent, content-addressed hard links
in /nix/store/.links.  For instance, if it encounters a file P with
hash H, it will create a hard link

  P' = /nix/store/.link/<H>

to P if P' doesn't already exist; if P' exist, then P is replaced by a
hard link to P'.  This is better than the previous in-memory map,
because it had the tendency to unnecessarily replace hard links with a
hard link to whatever happened to be the first file with a given hash
it encountered.  It also allows on-the-fly, incremental optimisation.
2012-07-23 17:14:15 -04:00
Eelco Dolstra 667d5f1936 Rename queryValidPaths() to queryAllValidPaths() 2012-07-11 10:49:04 -04:00
Eelco Dolstra bd013b6f98 On Linux, make the Nix store really read-only by using the immutable bit
I was bitten one time too many by Python modifying the Nix store by
creating *.pyc files when run as root.  On Linux, we can prevent this
by setting the immutable bit on files and directories (as in ‘chattr
+i’).  This isn't supported by all filesystems, so it's not an error
if setting the bit fails.  The immutable bit is cleared by the garbage
collector before deleting a path.  The only tricky aspect is in
optimiseStore(), since it's forbidden to create hard links to an
immutable file.  Thus optimiseStore() temporarily clears the immutable
bit before creating the link.
2012-02-15 01:31:56 +01:00
Eelco Dolstra a3883cbd28 * Store the size of a store path in the database (to be precise, the
size of the NAR serialisation of the path, i.e., `nix-store --dump
  PATH').  This is useful for Hydra.
2010-11-16 17:11:46 +00:00
Eelco Dolstra 07ca66cf24 * Applied a patch from David Brown to prevent `nix-store --optimise'
from failing on rename() on BtrFS.
2010-06-04 13:56:11 +00:00
Eelco Dolstra c7057fc1f2 * And some more. 2009-09-24 07:39:55 +00:00
Eelco Dolstra d5eab2fc82 * Use foreach in a lot of places. 2009-04-21 11:52:16 +00:00
Eelco Dolstra a8f3b02092 * `nix-store --optimise': handle files with >= 32000 hard links.
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18 14:13:00 +00:00
Eelco Dolstra a72709afd8 * Some refactoring: put the GC options / results in separate structs.
* The garbage collector now also prints the number of blocks freed.
2008-06-18 09:34:17 +00:00
Eelco Dolstra b0e92f6d47 * Merged the no-bdb branch (-r10900:HEAD
https://svn.nixos.org/repos/nix/nix/branches/no-bdb).
2008-06-09 13:52:45 +00:00