puck/lix
0
0
Fork 0
forked from lix-project/lix
Code Pull requests Activity

Document "Import From Derivation" (#7332)

Browse source 
* document "Import From Derivation"

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
This commit is contained in:
Valentin Gagarin 2023-09-26 03:49:03 +02:00 committed by GitHub
parent 9a78d87bc0
commit b17f200b11
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 150 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
doc/manual/src/SUMMARY.md.in
View file

@ -33,6 +33,7 @@
- [Operators](language/operators.md) - [Operators](language/operators.md)
- [Derivations](language/derivations.md) - [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md) - [Advanced Attributes](language/advanced-attributes.md)
- [Import From Derivation](language/import-from-derivation.md)
- [Built-in Constants](language/builtin-constants.md) - [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md) - [Built-in Functions](language/builtins.md)
- [Advanced Topics](advanced-topics/advanced-topics.md) - [Advanced Topics](advanced-topics/advanced-topics.md)

5
doc/manual/src/glossary.md
View file

@ -105,12 +105,15 @@
- [store object]{#gloss-store-object} - [store object]{#gloss-store-object}
A store object consists of a [file system object], [reference]s to other store objects, and other metadata. A store object consists of a [file system object], [reference]s to other store objects, and other metadata.
It can be referred to by a [store path]. It can be referred to by a [store path].
[store object]: #gloss-store-object [store object]: #gloss-store-object
- [IFD]{#gloss-ifd}
[Import From Derivation](./language/import-from-derivation.md)
- [input-addressed store object]{#gloss-input-addressed-store-object} - [input-addressed store object]{#gloss-input-addressed-store-object}
A store object produced by building a A store object produced by building a

139
doc/manual/src/language/import-from-derivation.md Normal file
View file

@ -0,0 +1,139 @@
# Import From Derivation
The value of a Nix expression can depend on the contents of a [store object](@docroot@/glossary.md#gloss-store-object).
Passing an expression `expr` that evaluates to a [store path](@docroot@/glossary.md#gloss-store-path) to any built-in function which reads from the filesystem constitutes Import From Derivation (IFD):
- [`import`](./builtins.md#builtins-import)` expr`
- [`builtins.readFile`](./builtins.md#builtins-readFile)` expr`
- [`builtins.readFileType`](./builtins.md#builtins-readFileType)` expr`
- [`builtins.readDir`](./builtins.md#builtins-readDir)` expr`
- [`builtins.pathExists`](./builtins.md#builtins-pathExists)` expr`
- [`builtins.filterSource`](./builtins.md#builtins-filterSource)` f expr`
- [`builtins.path`](./builtins.md#builtins-path)` { path = expr; }`
- [`builtins.hashFile`](./builtins.md#builtins-hashFile)` t expr`
- `builtins.scopedImport x drv`
When the store path needs to be accessed, evaluation will be paused, the corresponding store object [realised], and then evaluation resumed.
[realised]: @docroot@/glossary.md#gloss-realise
This has performance implications:
Evaluation can only finish when all required store objects are realised.
Since the Nix language evaluator is sequential, it only finds store paths to read from one at a time.
While realisation is always parallel, in this case it cannot be done for all required store paths at once, and is therefore much slower than otherwise.
Realising store objects during evaluation can be disabled by setting [`allow-import-from-derivation`](../command-ref/conf-file.md#conf-allow-import-from-derivation) to `false`.
Without IFD it is ensured that evaluation is complete and Nix can produce a build plan before starting any realisation.
## Example
In the following Nix expression, the inner derivation `drv` produces a file with contents `hello`.
```nix
# IFD.nix
let
drv = derivation {
name = "hello";
builder = "/bin/sh";
args = [ "-c" "echo -n hello > $out" ];
system = builtins.currentSystem;
};
in "${builtins.readFile drv} world"
```
```shellSession
nix-instantiate IFD.nix --eval --read-write-mode
```
```
building '/nix/store/348q1cal6sdgfxs8zqi9v8llrsn4kqkq-hello.drv'...
"hello world"
```
The contents of the derivation's output have to be [realised] before they can be read with [`readFile`](./builtins.md#builtins-readFile).
Only then evaluation can continue to produce the final result.
## Illustration
As a first approximation, the following data flow graph shows how evaluation and building are interleaved, if the value of a Nix expression depends on realising a [store object].
Boxes are data structures, arrow labels are transformations.
```
+----------------------+ +------------------------+
| Nix evaluator | | Nix store |
| .----------------. | | |
| | Nix expression | | | |
| '----------------' | | |
| | | | |
| evaluate | | |
| | | | |
| V | | |
| .------------. | | .------------------. |
| | derivation |----|-instantiate-|->| store derivation | |
| '------------' | | '------------------' |
| | | | |
| | | realise |
| | | | |
| | | V |
| .----------------. | | .--------------. |
| | Nix expression |<-|----read-----|----| store object | |
| '----------------' | | '--------------' |
| | | | |
| evaluate | | |
| | | | |
| V | | |
| .------------. | | |
| | value | | | |
| '------------' | | |
+----------------------+ +------------------------+
```
In more detail, the following sequence diagram shows how the expression is evaluated step by step, and where evaluation is blocked to wait for the build output to appear.
```
.-------. .-------------. .---------.
|Nix CLI| |Nix evaluator| |Nix store|
'-------' '-------------' '---------'
| | |
|evaluate IFD.nix| |
|--------------->| |
| | |
| evaluate `"${readFile drv} world"` |
| | |
| evaluate `readFile drv` |
| | |
| evaluate `drv` as string |
| | |
| |instantiate /nix/store/...-hello.drv|
| |----------------------------------->|
| : |
| : realise /nix/store/...-hello.drv |
| :----------------------------------->|
| : |
| |--------.
| : | |
| (evaluation blocked) | echo hello > $out
| : | |
| |<-------'
| : /nix/store/...-hello |
| |<-----------------------------------|
| | |
| resume `readFile /nix/store/...-hello` |
| | |
| | readFile /nix/store/...-hello |
| |----------------------------------->|
| | |
| | hello |
| |<-----------------------------------|
| | |
| resume `"${"hello"} world"` |
| | |
| resume `"hello world"` |
| | |
| "hello world" | |
|<---------------| |
.-------. .-------------. .---------.
|Nix CLI| |Nix evaluator| |Nix store|
'-------' '-------------' '---------'
```

11
src/libexpr/eval-settings.hh
View file

@ -47,11 +47,12 @@ struct EvalSettings : Config
Setting<bool> enableImportFromDerivation{ Setting<bool> enableImportFromDerivation{
this, true, "allow-import-from-derivation", this, true, "allow-import-from-derivation",
R"( R"(
By default, Nix allows you to `import` from a derivation, allowing By default, Nix allows [Import from Derivation](@docroot@/language/import-from-derivation.md).
building at evaluation time. With this option set to false, Nix will
throw an error when evaluating an expression that uses this feature, With this option set to `false`, Nix will throw an error when evaluating an expression that uses this feature,
allowing users to ensure their evaluation will not require any even when the required store object is readily available.
builds to take place. This ensures that evaluation will not require any builds to take place,
regardless of the state of the store.
)"}; )"};
Setting<Strings> allowedUris{this, {}, "allowed-uris", Setting<Strings> allowedUris{this, {}, "allowed-uris",