qyriad/lix
1
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
8990 commits 57 branches 0 tags 73 MiB
Commit graph

5579 commits

Author SHA1 Message Date
Maximilian Bosch e4940e90f3
Restore backwards-compat for current builtins.fetchGit 
If a repo is dirty, it used to return a `rev` object with an "empty"
sha1 (0000000000000000000000000000000000000000). Please note that this
only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{
type = "git"; }`.
 2020-07-28 22:46:39 +02:00
Carlo Nucera 2980b244b7 Use assert(false) instead of abort() 2020-07-28 15:39:45 -04:00
Carlo Nucera cfe96bd7ce Merge branch 'master' of github.com:NixOS/nix into ca-drv 2020-07-28 15:36:45 -04:00
Eelco Dolstra b8eea7e81a Remove putBytes 
istream->tellg() returns -1 so we can't get the number of bytes
written.

Fixes 'uploaded 's3://nix-cache/nar/00819r9lp5kajr6baxfw5dhhc0cx8ndxaz43qmd2f0gn1hk1ynlp.nar.xz' (-1 bytes) in 11620 ms' messages.
 2020-07-28 13:27:56 +02:00
Maximilian Bosch f742438465
Merge legacy fetchGit-builtin with the generic fetchTree-function 
The original idea was to implement a git-fetcher in Nix's core that
supports content hashes[1]. In #3549[2] it has been suggested to
actually use `fetchTree` for this since it's a fairly generic wrapper
over the new fetcher-API[3] and already supports content-hashes.

This patch implements a new git-fetcher based on `fetchTree` by
incorporating the following changes:

* Removed the original `fetchGit`-implementation and replaced it with an
  alias on the `fetchTree` implementation.

* Ensured that the `git`-fetcher from `libfetchers` always computes a
  content-hash and returns an "empty" revision on dirty trees (the
  latter one is needed to retain backwards-compatibility).

* The hash-mismatch error in the fetcher-API exits with code 102 as it
  usually happens whenever a hash-mismatch is detected by Nix.

* Removed the `flakes`-feature-flag: I didn't see a reason why this API
  is so tightly coupled to the flakes-API and at least `fetchGit` should
  remain usable without any feature-flags.

* It's only possible to specify a `narHash` for a `git`-tree if either a
  `ref` or a `rev` is given[4].

* It's now possible to specify an URL without a protocol. If it's missing,
  `file://` is automatically added as it was the case in the original
  `fetchGit`-implementation.

[1] https://github.com/NixOS/nix/pull/3216
[2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383
[3] https://github.com/NixOS/nix/pull/3459
[4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
 2020-07-28 00:44:39 +02:00
John Ericson 8065c6d160 Abstract out topo sorting logic 2020-07-27 20:45:34 +00:00
John Ericson 7cf978440c Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca 2020-07-27 17:47:40 +00:00
John Ericson e6115ca004 Merge remote-tracking branch 'upstream/master' into ca-derivation-data-types 2020-07-27 17:41:43 +00:00
John Ericson 1d7d94ceea Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-27 16:17:50 +00:00
John Ericson 43f2bd8dc5 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 16:13:57 +00:00
John Ericson 387699fe7f Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca 2020-07-27 15:00:28 +00:00
John Ericson 78466bcb2f Merge branch 'optional-derivation-output-storepath' into ca-derivation-data-types 2020-07-27 14:40:08 +00:00
John Ericson d5bb67cfa4 Merge remote-tracking branch 'upstream/master' into optional-derivation-output-storepath 2020-07-27 14:29:32 +00:00
John Ericson 699fc89b39 Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-27 14:22:45 +00:00
Eelco Dolstra 2292814049
createUnixDomainSocket(): Fix off-by-one error in copying the socket path 
Reported by Kane York.
 2020-07-24 11:19:17 +02:00
John Ericson a2af25c55a Merge remote-tracking branch 'upstream/master' into misc-ca 2020-07-23 19:14:17 +00:00
John Ericson 6fcd9966bf Merge remote-tracking branch 'upstream/master' into misc-ca 2020-07-23 15:02:29 +00:00
John Ericson 66a2067288 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-23 14:56:35 +00:00
John Ericson 2274f63453 Get rid of basicDerivation::findOutput 
It's a tiny function which is:

 - hardly worth abstrating over, and also only used once.

 - doesn't work once we get CA drvs

I rewrote the one callsite to be forwards compatable with CA
derivations, and also potentially more performant: instead of reading in
the derivation it can ust consult the SQLite DB in the common case.
 2020-07-23 14:39:02 +00:00
Fabian Möller b9ead08ca8
Save changes made by "nix registry pin" to user registry 2020-07-23 14:21:27 +02:00
John Ericson c56356bacc Separate concerns in scanForReferences with TeeSink 
This also will make it easier to use a `HashModuloSink` instead for CA
derivations.
 2020-07-22 22:13:44 +00:00
Eelco Dolstra ff314f186e
Merge pull request #3736 from obsidiansystems/allow-relative-paths-in-store-option 
Allow relative paths in --store option
 2020-07-21 19:43:07 +02:00
Eelco Dolstra 0951330680
Merge pull request #3714 from obsidiansystems/add-body-to-network-errors 
Add response body to network errors
 2020-07-21 18:09:42 +02:00
John Ericson 6cce32c8e8 Change logic for deciding what is a relative path for the local store 
The was Eelco's prefered logic, and it looks good to me!
 2020-07-21 15:39:47 +00:00
Carlo Nucera 922a845ffc Update chunkSize to the suggested value 
This was a suggested course of action in a review in one of our earlier
commits, https://github.com/NixOS/nix/pull/3801#discussion_r457557079
 2020-07-21 10:24:19 -04:00
John Ericson 54e507a7aa Merge remote-tracking branch 'upstream/master' into add-body-to-network-errors 2020-07-21 14:17:59 +00:00
John Ericson 5ce95b9529
Update src/libstore/build.cc 2020-07-21 09:47:40 -04:00
John Ericson cdb3f39b64 Merge remote-tracking branch 'upstream/master' into better-ca-parse-errors 2020-07-21 13:43:20 +00:00
John Ericson 5055c595bd Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into ca-derivation-data-types 2020-07-21 01:20:53 +00:00
John Ericson 5a37413796 Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into from-dump-stream 2020-07-21 01:20:14 +00:00
John Ericson bf0dca7790 Merge branch 'fix-and-document-addToStoreSlow' of github.com:obsidiansystems/nix into misc-ca 2020-07-21 01:18:33 +00:00
Carlo Nucera 9aae179f34 Correct bug, thoroughly document addToStoreSlow 2020-07-20 20:18:12 -04:00
Carlo Nucera 0ca9744694 Use heuristics to decide when to show the response 
Due to https://github.com/NixOS/nix/issues/3841 we don't know how print
different messages for different verbosity levels.
 2020-07-20 15:57:58 -04:00
Carlo Nucera 6357b1b0fb Add another Unimplemented case 2020-07-20 14:17:25 -04:00
Carlo Nucera 007bf6d17a Merge branch 'ca-derivation-data-types' of github.com:obsidiansystems/nix into misc-ca 2020-07-20 14:14:39 -04:00
Carlo Nucera 362ae93851 Add UnimplementedError to ease grepping for these 2020-07-20 14:13:37 -04:00
John Ericson c58c6165c5 Remove period at the end of the exception message 2020-07-20 17:43:19 +00:00
John Ericson bf61871271 parser.hh -> split.hh 2020-07-20 17:42:34 +00:00
John Ericson eab945f591 Merge remote-tracking branch 'upstream/master' into from-dump-stream 2020-07-20 17:29:23 +00:00
Eelco Dolstra 1c5f8bbfb5
Merge pull request #3822 from obsidiansystems/dump-thrice-fixme 
Optimize `addToStoreSlow` and remove `TeeParseSink`
 2020-07-20 18:55:05 +02:00
John Ericson ac2fc7ba1f
Apply suggestions from code review 
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-07-20 11:29:46 -04:00
John Ericson 6756cecfcf Add DerivationType::CAFloating 2020-07-17 19:55:41 +00:00
Carlo Nucera fefd6c9e5f Merge branch 'master' of github.com:NixOS/nix into allow-relative-paths-in-store-option 2020-07-17 14:34:20 -04:00
Carlo Nucera bbc633c98c Revert "Don't anticipate CA but not fixed outputs for now" 
This reverts commit 3a9e4c3262.
 2020-07-17 13:10:32 -04:00
Carlo Nucera 205dcd140d Revert "Don't anticipate multiple CA outputs for now" 
This reverts commit 74b251b2f3.
 2020-07-17 12:43:46 -04:00
Carlo Nucera 1feb8981df Revert "Don't anticipate hash algo without hash in derivation for now" 
This reverts commit 3804e3df9b.
 2020-07-17 12:05:54 -04:00
Lars Mühmel bc73590151 nix edit: call restoreSignals() before execvp-ing the $EDITOR 
Currently resizing of the terminal doesn't play nicely with
nix edit when using kakoune as the editor, as it relies on the
SIGWINCH signal which is trapped by nix. How this is not a problem
with e.g. vim is beyond me.

Virtually all other exec* calls are following a call to
restoreSignals(). This commit adds this behavior to nix edit
as well.
 2020-07-17 17:35:59 +02:00
Carlo Nucera 487c5751c6 Merge branch 'ca-drv' of github.com:Ericson2314/nix into misc-ca 2020-07-17 11:32:59 -04:00
Carlo Nucera a395c12830 Merge branch 'master' of github.com:NixOS/nix into misc-ca 2020-07-17 11:06:54 -04:00
Carlo Nucera 5cb840541b Merge branch 'multi-output-hashDerivationModulo' of github.com:Ericson2314/nix into misc-ca 2020-07-17 10:28:33 -04:00
Eelco Dolstra 17f75f9cc4 parseFlakeRef(): Only search for the top-level directory for CLI flakerefs 2020-07-17 14:54:21 +02:00
Eelco Dolstra 52c8be38e0 nix profile diff-closures: Don't inherit EvalCommand 2020-07-17 12:36:12 +02:00
Carlo Nucera 745a03cef5 Merge branch 'optional-derivation-output-storepath' of github.com:obsidiansystems/nix into ca-derivation-data-types 2020-07-16 13:36:01 -04:00
Carlo Nucera 048e916f64 Merge branch 'master' of github.com:NixOS/nix into optional-derivation-output-storepath 2020-07-16 13:32:28 -04:00
John Ericson cc0d77f8c9 Merge branch 'hash-always-has-type' of github.com:obsidiansystems/nix into better-ca-parse-errors 2020-07-16 17:28:52 +00:00
Carlo Nucera e3a3406db8 Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors 2020-07-16 12:04:10 -04:00
Eelco Dolstra 16c9f6762d Add command 'nix profile diff-closure' 
This shows all changes between generations of a profile. E.g.

  $ nix profile diff-closures --profile /nix/var/nix/profiles/system
  Generation 654 -> 655:
    nix: 2.4pre20200617_5d69bbf → 2.4pre20200701_6ff9aa8, +42.2 KiB

  Generation 655 -> 656:
    blender-bin: 2.83.0 → 2.83.1, -294.2 KiB

  Generation 656 -> 657:
    curl: 7.68.0 → 7.70.0, +19.1 KiB
    firmware-linux-nonfree: 2020-01-22 → 2020-05-19, +30827.7 KiB
    ibus: -21.8 KiB
    initrd-linux: 5.4.46 → 5.4.49
    ...
 2020-07-16 17:00:42 +02:00
John Ericson 5ea817dace Merge remote-tracking branch 'upstream/master' into hash-always-has-type 2020-07-16 14:58:53 +00:00
John Ericson 3dcca18c30 Fix bug in TeeSource 
We use this to simplify `LocalStore::addToStoreFromDump`.

Also, hope I fixed build error with old clang (used in Darwin CI).
 2020-07-16 13:39:27 +00:00
Eelco Dolstra 5517eee17e Generations API cleanup 2020-07-16 15:18:21 +02:00
Eelco Dolstra 8807ff902e nix diff-closures: Fix build 2020-07-16 14:25:51 +02:00
Eelco Dolstra 716eafaca4 Merge branch 'diff-closures' 2020-07-16 14:19:35 +02:00
Eelco Dolstra 3f264916db Merge remote-tracking branch 'origin/flakes' 2020-07-16 14:07:32 +02:00
John Ericson 68dfb8c6ae Optimize addToStoreSlow and remove TeeParseSink 2020-07-16 05:09:41 +00:00
John Ericson 5602637d9e Revert "LocalStore::addToStoreFromDump copy in chunks" 
This reverts commit 592851fb67. We don't
need this extra feature anymore
 2020-07-15 23:37:49 +00:00
John Ericson bc109648c4 Get rid of LocalStore::addToStoreCommon 
I got it to just become `LocalStore::addToStoreFromDump`, cleanly taking
a store and then doing nothing too fancy with it.

`LocalStore::addToStore(...Path...)` is now just a simple wrapper with a
bare-bones sinkToSource of the right dump command.
 2020-07-15 23:14:30 +00:00
John Ericson 64b7421741 Merge branch 'handle-flat-case' into from-dump-stream 2020-07-15 21:22:55 +00:00
Carlo Nucera d090562348 Merge branch 'master' of github.com:NixOS/nix into hash-always-has-type 2020-07-15 17:21:01 -04:00
John Ericson d087cf4855 Revert "Revert "LocalStore::addToStore(srcPath): Handle the flat case"" 
This reverts commit cff2157185.
 2020-07-15 21:10:33 +00:00
John Ericson aff69e229c Merge remote-tracking branch 'obsidian/fix-add-to-star-narhash' into from-dump-stream 2020-07-15 21:05:29 +00:00
Eelco Dolstra 36a1242603 nix why-depends: Fix shortest path calculation 
This was completely broken since d8972317fc.
 2020-07-15 21:11:07 +02:00
Eelco Dolstra 2d6d53bc87 nix: Fix examples 2020-07-15 20:28:16 +02:00
Eelco Dolstra 3624c042ac nix: Add --derivation flag to operate on .drv paths 
For instance, 'nix why-depends --use-derivation nixpkgs#hello
nixpkgs#glibc' shows why hello's .drv depends on glibc's .drv.
 2020-07-15 20:25:10 +02:00
Eelco Dolstra dfe8f3ebc6 nix why-depends: Fix misleading message 2020-07-15 20:09:50 +02:00
Eelco Dolstra 94eb5fad76 Clean up RealiseMode 2020-07-15 20:05:42 +02:00
Eelco Dolstra e3c2b00237 Make InstallableStorePath behave consistently with InstallableValue 
That is, the commands 'nix path-info nixpkgs#hello' and 'nix path-info
/nix/store/00ls0qi49qkqpqblmvz5s1ajl3gc63lr-hello-2.10.drv' now do the
same thing (i.e. build the derivation and operate on the output store
path, rather than the .drv path).
 2020-07-15 19:50:32 +02:00
Carlo Nucera 2d2a10e79a Merge branch 'master' of github.com:NixOS/nix into add-body-to-network-errors 2020-07-15 13:06:08 -04:00
Eelco Dolstra 0356f14459 Add 'nix diff-closures' command 
This command makes it easier to see what changed between two closures,
i.e. what packages/versions got added or removed, and whether there
were any notable changes in path size.

For example:

  $ nix diff-closures /nix/var/nix/profiles/system-655-link /nix/var/nix/profiles/system-658-link
  blender-bin: 2.83.0 → 2.83.2, -294.2 KiB
  curl: 7.68.0 → 7.70.0, +19.1 KiB
  firmware-linux-nonfree: 2020-01-22 → 2020-05-19, +30827.7 KiB
  ibus: -21.8 KiB
  initrd-linux: 5.4.46 → 5.4.51, +16.9 KiB
  libexif: 0.6.21 → 0.6.22, +497.6 KiB
  linux: 5.4.46 → 5.4.51, +13.2 KiB
  mesa: 19.3.3 → 19.3.5, -183.9 KiB
  nix: 2.4pre20200701_6ff9aa8 → 2.4pre20200708_9223603, +9.7 KiB
  nix-bash-completions: 0.6.8 → ∅, -57.6 KiB
  nixos-system-hagbard: 20.03.20200615.a84b797 → 20.03.20200713.add5529
  nvidia-persistenced: 440.82 → 440.100
  nvidia-settings: 440.82 → 440.100
  nvidia-x11: 440.82-5.4.46 → 440.100-5.4.51, +664.7 KiB
  pcre: 8.43 → 8.44
  php: 7.3.16 → 7.3.20, -26.2 KiB
  python3.7-youtube-dl: 2020.06.06 → 2020.06.16.1, +8.4 KiB
  samba: 4.11.5 → 4.11.9, +30.1 KiB
  sane-backends: 1.0.28 → 1.0.30, +680.5 KiB
  source: -182.0 KiB
  zfs-kernel: 0.8.3-5.4.46 → 0.8.4-5.4.51, +9.9 KiB
  zfs-user: 0.8.3 → 0.8.4, +20.1 KiB
 2020-07-15 16:24:25 +02:00
Eelco Dolstra cff2157185 Revert "LocalStore::addToStore(srcPath): Handle the flat case" 
This reverts commit a2c27022e9. See
addToStoreSlow(), we don't need to handle this case efficiently
anymore. In fact, we can almost remove the method/hashAlgo arguments
since the non-recursive and/or non-SHA256 are almost not used anymore.
 2020-07-15 12:49:03 +02:00
John Ericson 650c2c6558 Rename variable nar -> dump according to TODO 2020-07-14 21:28:50 +00:00
John Ericson 8173e7bfef Fix localhost::addToStore(...Path...) 
We were calculating the nar hash wrong when the file ingestion method
was flat. I don't think there's anything we can do in that case but dump
the file again, so that's what I do.

As an optomization, we again could reuse the original dump for just the
recursive and non-sha256 case, but I rather do that after this fix, and
after my other PRs which deduplicate this code.
 2020-07-14 21:12:11 +00:00
John Ericson 592851fb67 LocalStore::addToStoreFromDump copy in chunks 
Rather than copying byte-by-byte, we let the coroutine know how much
data we would like it to send back to us.
 2020-07-14 13:56:36 +00:00
John Ericson 9de96ef7d4 Dedup LocalStore::addToStore* 
The downsides is that the coroutine has byte-by-byte loop transfer. Will
fix that next.
 2020-07-14 13:56:36 +00:00
John Ericson c86fc3a965 Crudely make addToStoreFromDump take Source not string 
I just as little beyond the type as possible, so the implementation
changes this enables can be reviewed separately.
 2020-07-14 13:56:36 +00:00
John Ericson 9ec10046e0 Narrow scope of temporary value 2020-07-14 13:54:29 +00:00
Eelco Dolstra 298ff6af8f
Merge pull request #3809 from Ma27/gitlab-refs 
Fix gitlab-fetcher to obtain tags and branches
 2020-07-14 15:22:52 +02:00
Eelco Dolstra da3aea291d EvalCache: Ignore SQLite errors 
Fixes #3794.
 2020-07-14 15:17:38 +02:00
Eelco Dolstra 832e111494 Merge remote-tracking branch 'origin/master' into flakes 2020-07-14 13:56:18 +02:00
Eelco Dolstra 926c3a6664 Doh 2020-07-14 11:55:54 +02:00
Eelco Dolstra 43b8e96d30 Fix 'nix verify --all' on a binary cache (cached case) 2020-07-13 20:17:00 +02:00
Eelco Dolstra 9502c0e2eb nix verify: Show correct path when using --all on a binary cache 2020-07-13 20:12:44 +02:00
Eelco Dolstra 7c2fef0a81 Make 'nix copy' to s3:// binary caches run in constant memory 2020-07-13 20:07:19 +02:00
Maximilian Bosch cf9f33995b
Fix gitlab-fetcher to obtain tags and branches 
Until now, the `gitlab`-fetcher determined the source's rev by checking
the latest commit of the given `ref` using the
`/repository/branches`-API.

This breaks however when trying to fetch a gitlab-repo by its tag:

```
$ nix repl
nix-repl> builtins.fetchTree gitlab:Ma27/nvim.nix/0.2.0
error: --- Error ------------------------------------------------------------------------------------- nix
unable to download 'https://gitlab.com/api/v4/projects/Ma27%2Fnvim.nix/repository/branches/0.2.0': HTTP error 404 ('')
```

When using the `/commits?ref_name`-endpoint[1] you can pass any kind of
valid ref to the `gitlab`-fetcher.

Please note that this fetches the only first 20 commits on a ref,
unfortunately there's currently no endpoint which only retrieves the
latest commit of any kind of `ref`.

[1] https://docs.gitlab.com/ee/api/commits.html#list-repository-commits
 2020-07-13 19:22:59 +02:00
Eelco Dolstra 493961b689 Remove istringstream_nocopy 2020-07-13 18:31:19 +02:00
Eelco Dolstra 545bb2ed03 Remove 'accessor' from addToStore() 
This is only used by hydra-queue-runner and it's better to implement
it there.
 2020-07-13 18:31:19 +02:00
Eelco Dolstra 0a9da00a10 NarAccessor: Run in constant memory 2020-07-13 17:30:42 +02:00
Eelco Dolstra fc84c358d9 Make 'nix copy' to file:// binary caches run in constant memory 2020-07-13 16:28:45 +02:00
Eelco Dolstra 400f1a9b59 Store::pathInfoToJSON(): Use consistent format for downloadHash 2020-07-13 16:26:09 +02:00
Eelco Dolstra c0dd05131e toStorePath(): Return a StorePath and the suffix 2020-07-13 16:25:48 +02:00
Eelco Dolstra 1d01ae816b Fix 'nix verify --all' on a binary cache and add a test 2020-07-13 14:35:01 +02:00