Commit graph

15242 commits

Author SHA1 Message Date
Eelco Dolstra 0c5eac9c45 Git fetcher: Handle submodules for workdirs 2023-10-31 15:59:25 +01:00
Eelco Dolstra 669b074f51 Cleanup 2023-10-31 14:52:21 +01:00
Eelco Dolstra d88106df24 Git fetcher: Improve submodule handling
Instead of making a complete copy of the repo, fetching the
submodules, and writing the result to the store (which is all
superexpensive), we now fetch the submodules recursively using the Git
fetcher, and return a union accessor that "mounts" the accessors for
the submodules on top of the root accessor.
2023-10-31 14:52:21 +01:00
Eelco Dolstra ee36a44bf2 GitInputScheme: Use libgit2
This replaces most calls to the "git" binary with libgit2.
2023-10-31 14:52:21 +01:00
Eelco Dolstra 1d0e3d84b6 Provide a InputScheme::fetch() built on top of InputScheme::getAccessor()
This is for graceful migration to lazy-trees fetchers (which are all
accessor-based). Eventually fetch() will be removed.
2023-10-31 14:52:17 +01:00
Eelco Dolstra e1b8442fa1 Fetcher cache: Add support for caching facts not related to store paths 2023-10-31 14:51:39 +01:00
Eelco Dolstra fa6bc33604
Merge pull request #9239 from edolstra/putFile
Input: Replace markFileChanged() by putFile()
2023-10-31 14:49:42 +01:00
Eelco Dolstra 05316d401f Cleanup 2023-10-30 17:03:06 +01:00
John Ericson a6e587923c
Merge pull request #9247 from obsidiansystems/derivation-test-with-files
Turn derivation unit tests into unit characterization tests
2023-10-26 18:39:00 -04:00
John Ericson a419b61497 Turn derivation unit tests into unit characterization tests
The brings a number of advantages, including:

- Easier to update test data if design changes (and I do think our
  derivation JSON is not yet complaint with the guidelines).

- Easier to reuse test data in other implementations, inching closer to
  compliance tests for Nix *the concept* rather than any one
  implementation.
2023-10-26 18:09:01 -04:00
John Ericson 232e236672
Merge pull request #9246 from trofi/clarify-builder-uid-exhaustion
local-derivation-goal.cc: slightly clarify waiting message
2023-10-26 16:31:49 -04:00
Sergei Trofimovich e69c764708 local-derivation-goal.cc: slightly clarify waiting message
Before the change builder ID exhaustion printed the following message:

    [0/1 built] waiting for UID to build '/nix/store/hiy9136x0iyib4ssh3w3r5m8pxjnad50-python3.11-breathe-4.35.0.drv'

After the change it should be:

    [0/1 built] waiting for a free build user ID for '/nix/store/hiy9136x0iyib4ssh3w3r5m8pxjnad50-python3.11-breathe-4.35.0.drv'
2023-10-26 20:54:21 +01:00
John Ericson 1dc6a65d36
Merge pull request #9238 from tfc/small-improvements2
Small improvements 2
2023-10-26 09:35:03 -04:00
Jacek Galowicz 28c39c370c Provide default value for id to fix warning 2023-10-26 09:47:40 +02:00
Jacek Galowicz b66381e8d8 Use using instead of typedef 2023-10-26 09:47:40 +02:00
Silvan Mosberger 46028ff764
doc: Fix fetchGit default name (#9241) 2023-10-26 07:05:48 +02:00
John Ericson bfd51a4137
Merge pull request #9235 from fricklerhandwerk/doc-style-guide
add notes on comments in code samples
2023-10-25 15:32:07 -04:00
Eelco Dolstra 622191c2b5
Merge pull request #8965 from Artturin/bindfilesinchroot
Bindmount files instead of hardlinking or copying to chroot
2023-10-25 19:10:03 +02:00
Eelco Dolstra 15c430f389 Remove unused LockFile::write() 2023-10-25 18:44:09 +02:00
Eelco Dolstra 95d657c8b3 Input: Replace markFileChanged() by putFile()
Committing a lock file using markFileChanged() required the input to
be writable by the caller in the local filesystem (using the path
returned by getSourcePath()). putFile() abstracts over this.
2023-10-25 18:30:29 +02:00
John Ericson 5ac87a75dd
Merge pull request #9229 from tfc/small-improvements
Remove warnings, small improvements
2023-10-25 10:50:46 -04:00
Théophane Hufschmitt d571f17ff1
Merge pull request #9237 from iFreilicht/clarify-flake-implicit-types
docs: clarify flake types and implied defaults
2023-10-25 16:13:47 +02:00
Jacek Galowicz f555c98a34 Improve loop over gid container 2023-10-25 16:10:35 +02:00
Felix Uhl 7bc45c6136 docs: clarify flake types and implied defaults 2023-10-25 15:10:35 +02:00
Valentin Gagarin 78278f2b3f add notes on comments in code samples 2023-10-25 12:00:56 +02:00
Eelco Dolstra 3ca0dade5b
Merge pull request #9212 from trofkm/args-refactoring
Clean up `args.hh`
2023-10-25 10:33:53 +02:00
Valentin Gagarin 8d9e0b7aed
document the store concept (#9206)
* document the store concept and its purpose

reword the glossary to link to more existing information instead of
repeating it.

move the store documentation to the top of the table of contents, in
front of the Nix language. this will provide a natural place to
document other aspects of the store as well as the various store types.

move the package management section after the Nix language and before
Advanced Topics to follow the pattern to layer more complex concepts on
top of each other.

this structure of the manual will also nudge beginners to learn Nix
bottom-up and hopefully make more likely that they understand underlying
concepts first before delving into complex use cases that may or may not
be easy to implement with what's currently there.

[John adds this note] The sort of beginner who likes to dive straight into reference documentation should prefer this approach. Conversely, the sort of beginner who would prefer the opposite top-down approach of trying to solve problems before they understand everything that is going on is better off reading other tutorial/guide material anyways, and will just "random-access" the reference manual as a last resort. For such random-access the order doesn't matter, so this restructure doesn't make them any worse off.

Co-authored-by: John Ericson <git@JohnEricson.me>
2023-10-25 02:28:35 +00:00
Jacek Galowicz b113d925de Fix warning 2023-10-24 19:57:38 +02:00
Jacek Galowicz eaced12c94 Fix signed vs. unsigned comparison warning and improve code 2023-10-24 19:57:38 +02:00
Silvan Mosberger f269911641
Document builtins.substring negative length behavior (#9226) 2023-10-24 11:22:02 +02:00
Vignesh abb1c829c8
Release notes updated for #9150 reverted (#9227) 2023-10-24 11:18:00 +02:00
Robert Hensing c7c5d370e0
Merge pull request #9218 from NixOS/system-features-darwin-typo
system-features: Typo
2023-10-23 20:22:59 +02:00
Robert Hensing cde3c63617
system-features: Typo
There I was, thinking all of Apple's OSes started with lower case.
2023-10-23 19:30:00 +02:00
John Ericson a58d7f143e
Merge pull request #9216 from obsidiansystems/addDrvOutputDependencies-pre
Add `builtins.addDrvOutputDependencies`
2023-10-23 13:26:51 -04:00
Valentin Gagarin cd680bd53d
Merge how-to section on S3 buckets into S3 store docs (#7972)
Rather than having a misc tutorial page in the grab-bag "package management" section, this information should just be part of the S3 store docs.

---------

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2023-10-23 13:22:33 -04:00
Кирилл Трофимов c9528d2081
fix: Remove extra to from README.md (#9213) 2023-10-23 19:20:23 +02:00
John Ericson 765436e300 Add builtins.addDrvOutputDependencies
End goal: make `(mkDerivation x).drvPath` behave like a non-DrvDeep
context.

Problem: users won't be able to recover the DrvDeep behavior when
nixpkgs makes this change.

Solution: add this primop.

The new primop is fairly simple, and is supposed to complement other
existing ones (`builtins.storePath`, `builtins.outputOf`) so there are
simple ways to construct strings with every type of string context
element.

(It allows nothing we couldn't already do with `builtins.getContext` and `builtins.appendContext`, which is also true of those other two primops.)

This was originally in #8595, but then it was proposed to land some doc
changes separately. So now the code changes proper is just moved to
this, and the doc will be done in that.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.nore
github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io
2023-10-23 12:49:14 -04:00
Robert Hensing 3b99c62913
Merge pull request #8470 from ncfavier/shebang-single-quotes
nix-shell: support single quotes in shebangs, fix whitespace parsing
2023-10-23 18:23:48 +02:00
Naïm Favier e053eeb272 tests: test nix-shell shebang quoting 2023-10-23 17:32:50 +02:00
Kirill Trofimov 90e3ed06f8 fix: Use default destructor. 2023-10-23 18:07:57 +03:00
Kirill Trofimov a31fc5cc86 fix: Use using instead of typedef for type aliasing.
Since C++ 11 we shouldn't use c-style `typedefs`. In addition, `using` can be templated.
2023-10-23 18:07:17 +03:00
Kirill Trofimov b205da16ef fix: Explicitly pass lambda scope variables.
Default capture implicitly also capture *this, which would automatically be used if for example you referenced a method from the enclosing scope.
2023-10-23 18:06:15 +03:00
Kirill Trofimov c82066cf73 fix: Declare constructor as default 2023-10-23 16:59:19 +03:00
Naïm Favier 595010b631 nix-shell: fix shebang whitespace parsing
Leading whitespace after `nix-shell` used to produce an empty argument,
while an empty argument at the end of the line was ignored.

Fix the first issue by consuming the initial whitespace before calling
shellwords; fix the second issue by returning immediately if whitespace
is found at the end of the string instead of checking for an empty
string.

Also throw if quotes aren't terminated.
2023-10-23 15:56:07 +02:00
Naïm Favier fa9642ec45 nix-shell: support single quotes in shebangs
Single quotes are a basic feature of shell syntax that people expect to
work. They are also more convenient for writing literal code expressions
with less escaping.
2023-10-23 15:56:05 +02:00
John Ericson 8b68bbb777
Merge pull request #6223 from obsidiansystems/worker-proto-with-version
Give `nix daemon` and `nix-store --serve` protocols separate serializers with version info
2023-10-23 09:16:23 -04:00
John Ericson b461cac21a
Overhaul completions, redo #6693 (#8131)
As I complained in
https://github.com/NixOS/nix/pull/6784#issuecomment-1421777030 (a
comment on the wrong PR, sorry again!), #6693 introduced a second
completions mechanism to fix a bug. Having two completion mechanisms
isn't so nice.

As @thufschmitt also pointed out, it was a bummer to go from `FlakeRef`
to `std::string` when collecting flake refs. Now it is `FlakeRefs`
again.

The underlying issue that sought to work around was that completion of
arguments not at the end can still benefit from the information from
latter arguments.

To fix this better, we rip out that change and simply defer all
completion processing until after all the (regular, already-complete)
arguments have been passed.

In addition, I noticed the original completion logic used some global
variables. I do not like global variables, because even if they save
lines of code, they also obfuscate the architecture of the code.

I got rid of them  moved them to a new `RootArgs` class, which now has
`parseCmdline` instead of `Args`. The idea is that we have many argument
parsers from subcommands and what-not, but only one root args that owns
the other per actual parsing invocation. The state that was global is
now part of the root args instead.

This did, admittedly, add a bunch of new code. And I do feel bad about
that. So I went and added a lot of API docs to try to at least make the
current state of things clear to the next person.

--

This is needed for RFC 134 (tracking issue #7868). It was very hard to
modularize `Installable` parsing when there were two completion
arguments. I wouldn't go as far as to say it is *easy* now, but at least
it is less hard (and the completions test finally passed).

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-10-23 15:03:11 +02:00
Eelco Dolstra 955bbe53c5
Merge pull request #9177 from edolstra/input-accessors
Backport FSInputAccessor and MemoryInputAccessor from lazy-trees
2023-10-23 11:42:04 +02:00
Eelco Dolstra 34a42f0d0a Move PosixSourceAccessor into its own file 2023-10-23 11:05:50 +02:00
Valentin Gagarin 256dfb98e8
remove Basic Package Management section (#7974)
this is the first thing most beginners see, and it misleads them into
assuming `nix-env` is appropriate for doing anything but setting and
reverting profile generations.

this chapter is the root of most evil around the ecosystem, and today we
finally close it for good.
2023-10-23 04:05:02 +02:00