moraxyc/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity
9321 commits 1 branch 2 tags 68 MiB
Commit graph

5824 commits

Author SHA1 Message Date
regnat d72927aa7a Fix the s3 store 
Add some necessary casts in the initialisation of the store's config
 2020-09-16 13:53:09 +02:00
regnat e0817cbcdc Don't include nlohmann/json.hpp in config.hh 
Instead make a separate header with the template implementation of
`BaseSetting<T>::toJSONObj` that can be included where needed
 2020-09-16 13:53:09 +02:00
regnat 93c0e14a30 Include the full nlohmann/json header in config.hh 
It is apparently required for using `toJSONObject()`, which we do inside
the header file (because it's in a template).

This was accidentally working when building Nix itself (presumably because
`config.hh` was always included after `nlohman/json.hpp`) but caused a
(pretty dirty) build failure in the perl bindings package.
 2020-09-16 13:53:09 +02:00
regnat fc2d31c423 Add (StoreConfig*) casts to work around a GCC bug 
Work around https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80431 that was
already there in the code but was accidentally removed in the last
commits
 2020-09-16 13:53:09 +02:00
regnat a1e82ba450 fixup! Add a default value for the settings 2020-09-16 13:53:09 +02:00
regnat 888f7afe9f Fix build issues with gcc 2020-09-16 13:53:09 +02:00
regnat 634cb2a5ae Add a markdown output to nix describe-stores 2020-09-16 13:53:09 +02:00
regnat b73adacc1e Add a name to the stores 
So that it can be printed by `nix describe-stores`
 2020-09-16 13:53:09 +02:00
regnat f24f0888f9 Document the new store hierarchy 2020-09-16 13:53:09 +02:00
regnat d65962db4d Make uri schemes grammar more RFC-compliant 
Allow `-` and `.` in the RFC schemes as stated by
[RFC3986](https://tools.ietf.org/html/rfc3986#section-3.1).

Practically, this is needed so that `ssh-ng` is a valid URI scheme
 2020-09-16 13:53:09 +02:00
regnat 29a632386e fixup! Make the store plugins more introspectable 2020-09-16 13:53:09 +02:00
regnat 1129913c4e fixup! Correctly call all the parent contructors of the stores 2020-09-16 13:53:09 +02:00
regnat 7f103dcddd Properly filter the stores according to their declared uriSchemes 
When opening a store, only try the stores whose `uriSchemes()` include
the current one
 2020-09-16 13:53:09 +02:00
regnat 5895184df4 Correctly call all the parent contructors of the stores 
Using virtual inheritance means that only the default constructors of
the parent classes will be called, which isn't what we want
 2020-09-16 13:53:09 +02:00
regnat dae39f0a7a Make nix describe-stores functional 
Using the `*Config` class hierarchy
 2020-09-16 13:53:08 +02:00
regnat 22afa8fb4d Separate store configs from the implems 
Rework the `Store` hierarchy so that there's now one hierarchy for the
store configs and one for the implementations (where each implementation
extends the corresponding config). So a class hierarchy like

```
StoreConfig-------->Store
    |                 |
    v                 v
SubStoreConfig----->SubStore
    |                 |
    v                 v
SubSubStoreConfig-->SubSubStore
```

(with virtual inheritance to prevent DDD).

The advantage of this architecture is that we can now introspect the configuration of a store without having to instantiate the store itself
 2020-09-16 13:53:08 +02:00
regnat aa4eac3788 fixup! Separate the instantiation and initialisation of the stores 2020-09-16 13:53:08 +02:00
regnat 35042c9623 Add a default value for the settings 
The default value is initialized when creating the setting and unchanged
after that
 2020-09-16 13:53:08 +02:00
regnat 3c525d1590 Complete the toJSON instance for Setting<T> 
Don't let it just contain the value, but also the other fields of the
setting (description, aliases, etc..)
 2020-09-16 13:53:08 +02:00
regnat 3b57181f8e Separate the instantiation and initialisation of the stores 
Add a new `init()` method to the `Store` class that is supposed to
handle all the effectful initialisation needed to set-up the store.
The constructor should remain side-effect free and just initialize the
c++ data structure.

The goal behind that is that we can create “dummy” instances of each
store to query static properties about it (the parameters it accepts for
example)
 2020-09-16 13:53:08 +02:00
regnat fa32560169 Fix the registration of stores 2020-09-16 13:53:08 +02:00
regnat 7d5bdf8b56 Make the store plugins more introspectable 
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
 2020-09-16 13:53:08 +02:00
John Ericson 3a5cdd737c Rename Derivation::pathOpt to Derivation::path 
We no longer need the `*Opt` to disambiguate.
 2020-09-15 15:21:39 +00:00
John Ericson 6387550d58 Get rid of confusing std::optional<bool> for validity 2020-09-15 15:19:45 +00:00
John Ericson c4bf219b55 Don't link deriver until after any delayed exception is thrown 
Otherwise, we will associate fixed-output derivations with outputs that
they did indeed produce, but which had the wrong hash. That's no good.
 2020-09-15 14:28:06 +00:00
John Ericson 3ba552b245 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-15 14:17:06 +00:00
John Ericson c08c9f08c7 Merge remote-tracking branch 'upstream/master' into remove-storetype-delegate-regStore 2020-09-15 14:08:35 +00:00
Eelco Dolstra 2a80170920
Merge pull request #4014 from tweag/fix-queryDrvOutputMaps-for-old-daemons 
gracefully handle old daemon versions
 2020-09-15 11:25:19 +02:00
regnat 057c6203b5 gracefully handle old daemon versions 
Add a fallback path in `queryPartialDerivationOutputMap` for daemons
that don't support it.

Also upstreams a couple methods from `SSHStore` to `RemoteStore` as this
is needed to handle the fallback path.
 2020-09-15 09:58:01 +02:00
regnat 250f8a4bba Escape ${ in strings when printing Nix expressions 
Otherwise the result of the printing can't be parsed back correctly by
Nix (because the unescaped `${` will be parsed as the begining of an
anti-quotation).

Fix #3989
 2020-09-14 17:19:25 +02:00
Eelco Dolstra a59e77d9e5 nix-daemon: Lower verbosity of restricted setting warning 
Fixes #3992.
 2020-09-14 13:48:51 +02:00
Eelco Dolstra 35ba092019
Merge pull request #4005 from Infinisil/fix-autoArgs 
Fix auto argument passing for more auto arguments than formals
 2020-09-14 13:31:23 +02:00
aszlig 525b38eee8
Fix unspecified behaviour in readStorePathCAMap 
When deploying a Hydra instance with current Nix master, most builds
would not run because of errors like this:

  queue monitor: error: --- Error --- hydra-queue-runner
  error: --- UsageError --- nix-daemon
  not a content address because it is not in the form '<prefix>:<rest>': /nix/store/...-somedrv

The last error message is from parseContentAddress, which expects a
colon-separated string, however what we got here is a store path.

Looking at the worker protocol, the following message sent to the Nix
daemon caused the error above:

  0x1E -> wopQuerySubstitutablePathInfos
  0x01 -> Number of paths
  0x16 -> Length of string
  "/nix/store/...-somedrv"
  0x00 -> Length of string
  ""

Looking at writeStorePathCAMap, the store path is indeed the first field
that's transmitted. However, readStorePathCAMap expects it to be the
*second* field *on my machine*, since expression evaluation order is a
classic form of unspecified behaviour[1] in C++.

This has been introduced in https://github.com/NixOS/nix/pull/3689,
specifically in commit 66a62b3189.

[1]: https://en.wikipedia.org/wiki/Unspecified_behavior#Order_of_evaluation_of_subexpressions

Signed-off-by: aszlig <aszlig@nix.build>
 2020-09-13 02:40:51 +02:00
Silvan Mosberger 8dbd57a6a5
Fix auto argument passing for more auto arguments than formals 
The change in 626200713b didn't account
for when the number of auto arguments is bigger than the number of
formal arguments. This causes the following:

  $ nix-instantiate --eval -E '{ ... }@args: args.foo' --argstr foo foo
  nix-instantiate: src/libexpr/attr-set.hh:55: void nix::Bindings::push_back(const nix::Attr&): Assertion `size_ < capacity_' failed.
  Aborted (core dumped)
 2020-09-11 20:41:51 +02:00
Gabriel Gonzalez ee5906243a Add nix-shell support for preserving PS1 
Fixes https://github.com/NixOS/nix/issues/1268

`nix-shell` will now preserve `PS1` if the `NIX_SHELL_PRESERVE_PROMPT`
environment variable is set.
 2020-09-04 20:05:43 -07:00
John Ericson 5aed6f9b25 Document mkOutputString 2020-09-04 15:58:42 +00:00
John Ericson e9fad3006b Fix some of the issues raised by @edolstra 
- More and better comments

 - The easier renames
 2020-09-04 15:18:25 +00:00
John Ericson e86dd59dcc
Apply suggestions from code review 
Thanks!

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
 2020-09-04 10:48:50 -04:00
John Ericson c9f1ed912c Don't chmod symlink before moving outputs around 
Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
 2020-09-04 14:41:53 +00:00
John Ericson e7d93e7ece Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-03 15:43:17 +00:00
Eelco Dolstra 8a945d6ddb
Remove gc-check-reachability 2020-09-03 11:30:15 +02:00
Eelco Dolstra 82b77a7726
addPermRoot(): Remove indirect flag 2020-09-03 11:29:34 +02:00
Eelco Dolstra 00d25e8457
Remove the --indirect flag 
All GC roots are now indirect.
 2020-09-03 11:22:00 +02:00
Eelco Dolstra b07167be5a
createGeneration(): Always create an indirect root 
This means profiles outside of /nix/var/nix/profiles don't get
garbage-collected. It also means we don't need to scan
/nix/var/nix/profiles for GC roots anymore, except for compatibility
with previously existing generations.
 2020-09-03 11:13:53 +02:00
Eelco Dolstra b74f5cdd23
createGeneration(): Take a StorePath 2020-09-03 11:06:56 +02:00
Eelco Dolstra 94a043ff3b
EvalCache: Fix caching of strings 
This was broken in 50f13b06fb. Once
again it turns out that putting a bool in a std::variant is a bad
idea, since pointers get silently cast to them...
 2020-09-02 14:16:44 +02:00
Domen Kožar dd4b56c87f
Allow HTTP binary cache to request absolute uris 2020-09-01 21:35:48 +02:00
John Ericson ef278d00f9 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-09-01 18:01:48 +00:00
Eelco Dolstra 6d7f7efb89
github: Use access token when calling .../commits API 2020-09-01 15:29:04 +02:00
Eelco Dolstra 84f5cabbea Merge remote-tracking branch 'origin/master' into markdown 2020-08-31 14:24:26 +02:00
John Ericson 8017fe7487 Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-28 19:59:14 +00:00
Eelco Dolstra f15651303f nix develop: Add convenience flags for running specific phases 
For example, for building the Nix flake, you would do:

  $ nix develop --configure
  $ nix develop --install
  $ nix develop --installcheck
 2020-08-28 19:24:29 +02:00
Eelco Dolstra 3156560d41 nix develop: Set output paths to writable locations 
Currently, they're set to $(pwd)/outputs/$outputName. This allows
commands like 'make install' to work.
 2020-08-28 18:16:03 +02:00
Eelco Dolstra 691a1bd717 Merge branch 'minimal-logger' of https://github.com/Ma27/nix 2020-08-28 10:50:52 +02:00
Griffin Smith 626200713b Pass all args when auto-calling a function with an ellipsis 
The command line options --arg and --argstr that are used by a bunch of
CLI commands to pass arguments to top-level functions in files go
through the same code-path as auto-calling top-level functions with
their default arguments - this, however, was only passing the arguments
that were *explicitly* mentioned in the formals of the function - in the
case of an as-pattern with an ellipsis (eg args @ { ... }) extra passed
arguments would get omitted. This fixes that to instead pass *all*
specified auto args in the case that our function has an ellipsis.

Fixes #598
 2020-08-27 12:38:25 -04:00
Eelco Dolstra eb75282b8d
Merge pull request #3434 from Ericson2314/derivation-header-include-order 
Revise division of labor in deserialization of derivations
 2020-08-27 16:39:28 +02:00
Eelco Dolstra e915fd6d2a Typo 2020-08-27 14:51:50 +02:00
Eelco Dolstra a0f19d9f3a RemoteStore::addToStore(): Fix race between stderrThread and NAR writer 
As pointed out by @B4dM4n, the call to to.flush() on stderrThread is
unsafe because the NAR writer thread is also writing to 'to'.

Fixes #3943.
 2020-08-27 14:50:51 +02:00
Eelco Dolstra c3efef9275 Remove obsolete comment 2020-08-26 09:28:10 +02:00
Eelco Dolstra 4bf5faf416 Merge remote-tracking branch 'origin/master' into markdown 2020-08-25 19:47:34 +02:00
Eelco Dolstra 7a02865b94
Move import docs 2020-08-25 14:06:01 +02:00
Eelco Dolstra f53b5f1058
Add getDoc() function 2020-08-25 13:31:11 +02:00
Eelco Dolstra 2a2121d264
Use RegisterPrimOp for some undocumented primops 2020-08-25 11:25:01 +02:00
Eelco Dolstra b8416779e3
Document some primops 2020-08-25 11:16:45 +02:00
Eelco Dolstra 24b1c2c66b
Fix tests 2020-08-25 10:51:14 +02:00
Eelco Dolstra b42789f013
Fix clang build 2020-08-24 21:13:39 +02:00
Eelco Dolstra d9a8619762
Don't barf if corepkgs is in the store but not a valid path 
This can happen when using a dummy store (or indeed any non-local store).
 2020-08-24 19:15:17 +02:00
Eelco Dolstra 6a67e57019
Add DummyStore (dummy://) 
DummyStore does not allow building or adding paths. This is useful for
evaluation tests when you don't want to initialize a "proper" store.
 2020-08-24 18:54:16 +02:00
Eelco Dolstra d0690bc311
nix repl ':doc': Render using lowdown 2020-08-24 18:10:33 +02:00
Eelco Dolstra 0f314f3c25
Generate builtins section of the manual 2020-08-24 14:49:30 +02:00
Eelco Dolstra a990f063ff
Move primop docs inline 
This makes them available to 'nix repl'.
 2020-08-24 14:31:10 +02:00
Eelco Dolstra 33b1679d75
Allow primops to have Markdown documentation 2020-08-24 13:16:02 +02:00
Eelco Dolstra 88d5c9ec58
Fix tests 2020-08-24 10:37:10 +02:00
Eelco Dolstra 2530cc6827
Merge pull request #3950 from obsidiansystems/tabs-spaces 
tabs -> spaces
 2020-08-24 10:20:28 +02:00
John Ericson 35e6288be1 writeDerivation just needs a plain store reference 2020-08-23 15:01:11 +00:00
John Ericson 3a7b330b64 "Downstream placeholders" should not be store paths 
Insead they should be opaque `/<hash>` like the placeholders we already
have.
 2020-08-21 19:35:35 +00:00
John Ericson 422affe102 tabs -> spaces 
Sorry I let the tab sneak in there in the first place.
 2020-08-21 19:19:24 +00:00
John Ericson 27a3f82c0b Merge remote-tracking branch 'upstream/master' into single-ca-drv-build 2020-08-20 18:28:17 +00:00
John Ericson 45a2f1baab Rename drv output querying functions, like master 
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap
 2020-08-20 18:14:12 +00:00
Eelco Dolstra 4d77513d97
Merge pull request #3859 from obsidiansystems/drv-outputs-map-allow-missing 
`queryDerivationOutputMap` no longer assumes all outputs have a mapping
 2020-08-20 16:49:23 +02:00
John Ericson 9a9d834dc7 Rename drv output querying functions 
- `queryDerivationOutputMapAssumeTotal` -> `queryPartialDerivationOutputMap`
 - `queryDerivationOutputMapAssumeTotal` -> `queryDerivationOutputMap`
 2020-08-20 14:12:51 +00:00
Eelco Dolstra 25ecfffdc3
Remove PrimOp constructor 2020-08-20 12:34:04 +02:00
Eelco Dolstra dc2f278c95
Allow 'nix' subcommands to provide docs in Markdown format 2020-08-20 12:21:46 +02:00
Eelco Dolstra 3c4f8c9175
List deprecated option aliases in the docs 2020-08-20 11:13:17 +02:00
Eelco Dolstra acb99f03f9
Config: Use nlohmann/json 2020-08-20 11:02:16 +02:00
John Ericson 3df78858f2 Fix max fd calc and add test 2020-08-20 05:08:50 +00:00
Ryan Mulligan f36793c7b9
fix spelling 2020-08-19 20:31:01 -07:00
John Ericson a83694c7a1 Use RemoteStore to open connection for proxying daemon 
Removes duplicate websocket opening code, and also means we should be
able to to ssh-ssh-... daemon relays, not just uds-uds-... ones.
 2020-08-19 19:34:47 +00:00
Eelco Dolstra c8fa39324a
Generate the nix.conf docs from the source code 
This means we don't have two (divergent) sets of option descriptions
anymore.
 2020-08-19 18:28:04 +02:00
John Ericson d5af5763cf Merge branch 'master' of github.com:NixOS/nix into remove-storetype-delegate-regStore 2020-08-19 15:54:50 +00:00
Eelco Dolstra 34b22e0123
Change option descriptions to Markdown 2020-08-19 14:21:27 +02:00
John Ericson 950ddfdb82 Merge remote-tracking branch 'upstream/master' into derivation-header-include-order 2020-08-18 14:36:44 +00:00
Eelco Dolstra 1c8b550e34
Merge pull request #3917 from obsidiansystems/output-env-var-unconditional 
Simplify code as output env vars are unconditional
 2020-08-18 16:21:17 +02:00
Eelco Dolstra dfeb76dbf9
Merge pull request #3930 from obsidiansystems/legacy-ssh-build-paths 
Define `LegacySSHStore::buildPaths` using `cmdBuildPaths`
 2020-08-18 16:07:40 +02:00
Eelco Dolstra 069340179e
Improve nix.1 manpage generator 2020-08-18 15:15:35 +02:00
Eelco Dolstra 6f19c776db
Start generation of the nix.1 manpage 2020-08-17 19:33:18 +02:00
Eelco Dolstra a72a20d68f
Add 'nix dump-args' to dump all commands/flags for manpage generation 2020-08-17 17:44:52 +02:00
Eelco Dolstra 7cdc739ece
Merge remote-tracking branch 'origin/master' into markdown 2020-08-17 13:43:39 +02:00
Eelco Dolstra e849b19872
Merge pull request #3932 from chkno/no-show-signature 
Don't try to parse signature check as commit timestamp
 2020-08-17 11:00:24 +02:00
Eelco Dolstra 847a5392f4 Merge branch 'ca-no-need-trust' of https://github.com/obsidiansystems/nix 2020-08-17 10:57:02 +02:00