From be8744f937150c65ffa5d98ffdc74c502dbcdd5e Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Mon, 19 Sep 2022 22:21:40 +0200 Subject: [PATCH 1/7] manual: architecture overview these changes were not merged properly and had to be reverted. see merge commit d8e54d19f71f78540dd967b2e42be6a5d8a0b1bb for full history leading up to here. --- doc/manual/src/SUMMARY.md.in | 1 + doc/manual/src/architecture/architecture.md | 77 +++++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 doc/manual/src/architecture/architecture.md diff --git a/doc/manual/src/SUMMARY.md.in b/doc/manual/src/SUMMARY.md.in index 908e7e3d9..b386aa14b 100644 --- a/doc/manual/src/SUMMARY.md.in +++ b/doc/manual/src/SUMMARY.md.in @@ -59,6 +59,7 @@ @manpages@ - [Files](command-ref/files.md) - [nix.conf](command-ref/conf-file.md) +- [Architecture](architecture/architecture.md) - [Glossary](glossary.md) - [Contributing](contributing/contributing.md) - [Hacking](contributing/hacking.md) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md new file mode 100644 index 000000000..24150fc2b --- /dev/null +++ b/doc/manual/src/architecture/architecture.md @@ -0,0 +1,77 @@ +# Architecture + +This chapter describes how Nix works. +It should help users understand why Nix behaves as it does, and it should help developers understand how to modify Nix and how to write similar tools. + +## Overview + +Nix consists of [hierarchical layers]. + +[hierarchical layers]: https://en.m.wikipedia.org/wiki/Multitier_architecture#Layers + +``` ++---------------------------------------------------------------+ +| Nix | +| [ commmand line interface ]------, | +| | | | +| evaluates | | +| | manages | +| V | | +| [ configuration language ] | | +| | | | +| +----------------------------|-------------------V----------+ | +| | store evaluates to | | +| | | | | +| | referenced by V builds | | +| | [ build input ] ---> [ build plan ] ---> [ build result ] | | +| | | | +| +-----------------------------------------------------------+ | ++---------------------------------------------------------------+ +``` + +At the top is the [command line interface](../command-ref/command-ref.md), translating from invocations of Nix executables to interactions with the underlying layers. + +Below that is the [Nix language](../language/index.md), a [purely functional] configuration language. +It is used to compose expressions which ultimately evaluate to self-contained *build plans*, used to derive *build results* from referenced *build inputs*. + +[purely functional]: https://en.m.wikipedia.org/wiki/Purely_functional_programming + +Command line interface and Nix language are what users interact with most. + +> **Note** +> The Nix language itself does not have a notion of *packages* or *configurations*. +> As far as we are concerned here, the inputs and results of a build plan are just data. + +Underlying these is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them. +It can also execute build plans to produce new data. + +A build plan is a series of *build tasks*. +Each build task has a special build input, which is used as *build instructions*. +The result of a build task can be input to another build task. + +> **Important** +> A build task in Nix is called [derivation](../glossary#gloss-derivation). + +``` ++----------------------------------------------------------------------------------+ +| store .............................................. | +| : build plan : | +| : : | +| [ build input ]---instructions-, : | +| : | : | +| : v : | +| [ build input ]--------->[ build task ]-instructions-, : | +| : | : | +| : v : | +| [ build input ]---instructions-, [ build task ]--->[ build result ] | +| : | ^ : | +| : v | : | +| [ build input ]--------->[ build task ]--------------' : | +| : ^ : | +| : | : | +| [ build input ]----------------' : | +| : : | +| :............................................: | ++----------------------------------------------------------------------------------+ +``` + From b5728ace5d5250a6d52df05802c9fc325153dc2c Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Fri, 23 Sep 2022 14:48:54 +0200 Subject: [PATCH 2/7] add articles --- doc/manual/src/architecture/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md index 24150fc2b..972ff2d58 100644 --- a/doc/manual/src/architecture/architecture.md +++ b/doc/manual/src/architecture/architecture.md @@ -36,7 +36,7 @@ It is used to compose expressions which ultimately evaluate to self-contained *b [purely functional]: https://en.m.wikipedia.org/wiki/Purely_functional_programming -Command line interface and Nix language are what users interact with most. +The command line interface and the Nix language are what users interact with most. > **Note** > The Nix language itself does not have a notion of *packages* or *configurations*. From 98447c1a7f1538f4d3132c0b79ad5b67e136af85 Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Fri, 23 Sep 2022 14:49:47 +0200 Subject: [PATCH 3/7] clarify subject of sentence Co-authored-by: Bryan Honof --- doc/manual/src/architecture/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md index 972ff2d58..f113cbf4d 100644 --- a/doc/manual/src/architecture/architecture.md +++ b/doc/manual/src/architecture/architecture.md @@ -42,7 +42,7 @@ The command line interface and the Nix language are what users interact with mos > The Nix language itself does not have a notion of *packages* or *configurations*. > As far as we are concerned here, the inputs and results of a build plan are just data. -Underlying these is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them. +Underlying the command line interface and the Nix language is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them. It can also execute build plans to produce new data. A build plan is a series of *build tasks*. From 3d716df7ce0639e61170ea1e18a370138dce8e5b Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Sat, 15 Oct 2022 23:46:32 +0200 Subject: [PATCH 4/7] make diagrams compatible with svgbob this will at some point enable rendering them nicely for the web --- doc/manual/src/architecture/architecture.md | 53 ++++++++++++--------- 1 file changed, 31 insertions(+), 22 deletions(-) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md index f113cbf4d..0df4fbc68 100644 --- a/doc/manual/src/architecture/architecture.md +++ b/doc/manual/src/architecture/architecture.md @@ -11,20 +11,25 @@ Nix consists of [hierarchical layers]. ``` +---------------------------------------------------------------+ -| Nix | -| [ commmand line interface ]------, | +| Nix .-------------------------. | +| | commmand line interface |------. | +| '-------------------------' | | | | | | | evaluates | | | | manages | | V | | -| [ configuration language ] | | +| .-------------------------. | | +| | configuration language | | | +| '-------------------------' | | | | | | -| +----------------------------|-------------------V----------+ | -| | store evaluates to | | -| | | | | +| evaluates to | | +| | V | +| +----------------------------|------------------------------+ | +| | store | | | | | referenced by V builds | | -| | [ build input ] ---> [ build plan ] ---> [ build result ] | | -| | | | +| | .-------------. .------------. .--------------. | | +| | | build input |----->| build plan | ---->| build result | | | +| | '-------------' '------------' '--------------' | | | +-----------------------------------------------------------+ | +---------------------------------------------------------------+ ``` @@ -54,24 +59,28 @@ The result of a build task can be input to another build task. ``` +----------------------------------------------------------------------------------+ -| store .............................................. | +| store - - - - - - - - - - - - - - - - - - - - - - | | : build plan : | -| : : | -| [ build input ]---instructions-, : | -| : | : | +| .-------------. : : | +| | build input |---instructions-. : | +| '-------------' : | : | | : v : | -| [ build input ]--------->[ build task ]-instructions-, : | -| : | : | +| .-------------. : .------------. : | +| | build input |--------->| build task |-instructions-. : | +| '-------------' : '------------' | : | | : v : | -| [ build input ]---instructions-, [ build task ]--->[ build result ] | -| : | ^ : | -| : v | : | -| [ build input ]--------->[ build task ]--------------' : | +| .-------------. : .------------. : .--------------. | +| | build input |---instructions-. | build task |--->| build result | | +| '-------------' : | '------------' : '--------------' | +| : v ^ : | +| .-------------. : .------------. | : | +| | build input |--------->| build task |--------------' : | +| '-------------' : '------------' : | | : ^ : | -| : | : | -| [ build input ]----------------' : | -| : : | -| :............................................: | +| .-------------. : | : | +| | build input |----------------' : | +| '-------------' : : | +| :_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _: | +----------------------------------------------------------------------------------+ ``` From 9d20a056c82b129ab2584169e692c7f3e324fa29 Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Wed, 9 Nov 2022 01:36:17 +0100 Subject: [PATCH 5/7] remove external link the language has its own overview page where its properties are described in sufficient detail. --- doc/manual/src/architecture/architecture.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md index 0df4fbc68..4b0704894 100644 --- a/doc/manual/src/architecture/architecture.md +++ b/doc/manual/src/architecture/architecture.md @@ -36,10 +36,8 @@ Nix consists of [hierarchical layers]. At the top is the [command line interface](../command-ref/command-ref.md), translating from invocations of Nix executables to interactions with the underlying layers. -Below that is the [Nix language](../language/index.md), a [purely functional] configuration language. -It is used to compose expressions which ultimately evaluate to self-contained *build plans*, used to derive *build results* from referenced *build inputs*. - -[purely functional]: https://en.m.wikipedia.org/wiki/Purely_functional_programming +Below that is the [Nix language](../language/index.md), the configuration language for Nix. +Its expressions ultimately evaluate to self-contained *build plans*, used to derive *build results* from referenced *build inputs*. The command line interface and the Nix language are what users interact with most. From 4ed8bb1cb132a6671a51e96abbd8241fbe01d6ef Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Mon, 12 Dec 2022 14:29:02 +0100 Subject: [PATCH 6/7] suggestions from review --- doc/manual/src/architecture/architecture.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md index 4b0704894..3a7d1d6f5 100644 --- a/doc/manual/src/architecture/architecture.md +++ b/doc/manual/src/architecture/architecture.md @@ -15,14 +15,14 @@ Nix consists of [hierarchical layers]. | | commmand line interface |------. | | '-------------------------' | | | | | | -| evaluates | | +| calls | | | | manages | | V | | | .-------------------------. | | -| | configuration language | | | +| | language evaluator | | | | '-------------------------' | | | | | | -| evaluates to | | +| produces | | | | V | | +----------------------------|------------------------------+ | | | store | | | @@ -36,7 +36,7 @@ Nix consists of [hierarchical layers]. At the top is the [command line interface](../command-ref/command-ref.md), translating from invocations of Nix executables to interactions with the underlying layers. -Below that is the [Nix language](../language/index.md), the configuration language for Nix. +Below that is the evaluator for the [Nix language](../language/index.md), the configuration language for Nix. Its expressions ultimately evaluate to self-contained *build plans*, used to derive *build results* from referenced *build inputs*. The command line interface and the Nix language are what users interact with most. From c162c90b432590e1dbdeb5f4361bf4b48de82bac Mon Sep 17 00:00:00 2001 From: Valentin Gagarin Date: Tue, 20 Dec 2022 09:59:59 +0100 Subject: [PATCH 7/7] add more explanation to diagrams this is to help reading the diagrams, otherwise arrows and labels were reported as being ambiguous. --- doc/manual/src/architecture/architecture.md | 125 ++++++++++++-------- 1 file changed, 78 insertions(+), 47 deletions(-) diff --git a/doc/manual/src/architecture/architecture.md b/doc/manual/src/architecture/architecture.md index 3a7d1d6f5..2d1b26558 100644 --- a/doc/manual/src/architecture/architecture.md +++ b/doc/manual/src/architecture/architecture.md @@ -9,18 +9,29 @@ Nix consists of [hierarchical layers]. [hierarchical layers]: https://en.m.wikipedia.org/wiki/Multitier_architecture#Layers +The following [concept map] shows its main components (rectangles), the objects they operate on (rounded rectangles), and their interactions (connecting phrases): + +[concept map]: https://en.m.wikipedia.org/wiki/Concept_map + ``` -+---------------------------------------------------------------+ -| Nix .-------------------------. | -| | commmand line interface |------. | -| '-------------------------' | | -| | | | -| calls | | -| | manages | -| V | | -| .-------------------------. | | -| | language evaluator | | | -| '-------------------------' | | + + .----------------. + | Nix expression |----------. + '----------------' | + | passed to + | | ++----------|-------------------|--------------------------------+ +| Nix | V | +| | +-------------------------+ | +| | | commmand line interface |------. | +| | +-------------------------+ | | +| | | | | +| evaluated by calls manages | +| | | | | +| | V | | +| | +--------------------+ | | +| '-------->| language evaluator | | | +| +--------------------+ | | | | | | | produces | | | | V | @@ -28,57 +39,77 @@ Nix consists of [hierarchical layers]. | | store | | | | | referenced by V builds | | | | .-------------. .------------. .--------------. | | -| | | build input |----->| build plan | ---->| build result | | | +| | | build input |----->| build plan |----->| build result | | | | | '-------------' '------------' '--------------' | | -| +-----------------------------------------------------------+ | -+---------------------------------------------------------------+ +| +-------------------------------------------------|---------+ | ++---------------------------------------------------|-----------+ + | + represented as + | + V + .---------------. + | file | + '---------------' ``` -At the top is the [command line interface](../command-ref/command-ref.md), translating from invocations of Nix executables to interactions with the underlying layers. +At the top is the [command line interface](../command-ref/command-ref.md) that drives the underlying layers. -Below that is the evaluator for the [Nix language](../language/index.md), the configuration language for Nix. -Its expressions ultimately evaluate to self-contained *build plans*, used to derive *build results* from referenced *build inputs*. +The [Nix language](../language/index.md) evaluator transforms Nix expressions into self-contained *build plans*, which are used to derive *build results* from referenced *build inputs*. -The command line interface and the Nix language are what users interact with most. +The command line interface and Nix expressions are what users deal with most. > **Note** > The Nix language itself does not have a notion of *packages* or *configurations*. > As far as we are concerned here, the inputs and results of a build plan are just data. -Underlying the command line interface and the Nix language is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them. -It can also execute build plans to produce new data. +Underlying the command line interface and the Nix language evaluator is the [Nix store](../glossary.md#gloss-store), a mechanism to keep track of build plans, data, and references between them. +It can also execute build plans to produce new data, which are made available to the operating system as files. -A build plan is a series of *build tasks*. -Each build task has a special build input, which is used as *build instructions*. -The result of a build task can be input to another build task. +A build plan itself is a series of *build tasks*, together with their build inputs. > **Important** > A build task in Nix is called [derivation](../glossary#gloss-derivation). +Each build task has a special build input executed as *build instructions* in order to perform the build. +The result of a build task can be input to another build task. + +The following [data flow diagram] shows a build plan for illustration. +Build inputs used as instructions to a build task are marked accordingly: + +[data flow diagram]: https://en.m.wikipedia.org/wiki/Data-flow_diagram + ``` -+----------------------------------------------------------------------------------+ -| store - - - - - - - - - - - - - - - - - - - - - - | -| : build plan : | -| .-------------. : : | -| | build input |---instructions-. : | -| '-------------' : | : | -| : v : | -| .-------------. : .------------. : | -| | build input |--------->| build task |-instructions-. : | -| '-------------' : '------------' | : | -| : v : | -| .-------------. : .------------. : .--------------. | -| | build input |---instructions-. | build task |--->| build result | | -| '-------------' : | '------------' : '--------------' | -| : v ^ : | -| .-------------. : .------------. | : | -| | build input |--------->| build task |--------------' : | -| '-------------' : '------------' : | -| : ^ : | -| .-------------. : | : | -| | build input |----------------' : | -| '-------------' : : | -| :_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _: | -+----------------------------------------------------------------------------------+ ++--------------------------------------------------------------------+ +| build plan | +| | +| .-------------. | +| | build input |---------. | +| '-------------' | | +| instructions | +| | | +| v | +| .-------------. .----------. | +| | build input |-->( build task )-------. | +| '-------------' '----------' | | +| instructions | +| | | +| v | +| .-------------. .----------. .--------------. | +| | build input |---------. ( build task )--->| build result | | +| '-------------' | '----------' '--------------' | +| instructions ^ | +| | | | +| v | | +| .-------------. .----------. | | +| | build input |-->( build task )-------' | +| '-------------' '----------' | +| ^ | +| | | +| | | +| .-------------. | | +| | build input |---------' | +| '-------------' | +| | ++--------------------------------------------------------------------+ ```