Cannot compile lix in lix docker container #452
Labels
No labels
Area/build-packaging
Area/cli
Area/evaluator
Area/fetching
Area/flakes
Area/language
Area/profiles
Area/protocol
Area/releng
Area/remote-builds
Area/repl
Area/store
bug
crash 💥
Cross Compilation
devx
docs
Downstream Dependents
E/easy
E/hard
E/help wanted
E/reproducible
E/requires rearchitecture
imported
Needs Langver
OS/Linux
OS/macOS
performance
regression
release-blocker
RFD
stability
Status
blocked
Status
invalid
Status
postponed
Status
wontfix
testing
testing/flakey
ux
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: lix-project/lix#452
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Lix fails compilation in docker container
While building a raspberry pi image with lix, this happened:
This is on an m2 Mac:
Steps To Reproduce
docker compose run --rm lix bashnix --extra-experimental-features 'nix-command flakes' build --print-build-logs ".#nixosConfigurations.pi.config.system.build.sdImage"Expected behavior
This should work.
nix --versionoutputAdditional context
Do you need anything else to reproduce this? I have attached the flake.nix, configuration.nix and docker-compose.yaml files to make this reproducible.
Cannot compile lix in docker containerto Cannot compile lix in lix docker containerIs this possibly a duplicate of #310?
Hard to say, as I'm not deep into the other bug. If it should be the same, this is probably easier to reproduce?
It does look similar, that's for sure.
please try running your lix container as privileged. so for compose something like:
The tests are most likely failing, even though lix was successfully built, because they use sandboxing/namespacing features, and the required rights are dropped by default for containers by docker/podman.
In that case instead of a bug with lix or the lix docker image, this might be considered a problem with documentation, as most users probably do no require these privileges for their containers
Nix 2.24 surprisingly fixes this issue compared to previous releases, disabling a number of tests including the ones failing, for the default build, which does not seem optimal
i would suggest to set the seccomp policy to unconfined rather than running as privileged; lix does not need privileges, it just needs to not have its syscalls blocked.
@71rd I think this output suggests that adding
privileged: trueto the container seems to fix the tests:Partial Test Output from Image Build
That being said, the image build went through fine, and I hope to be able to test it on Thursday.
@jade Can you give me a hint on how to test disabling the sec comp policies? I'll gladly test that next.
here you go: https://docs.docker.com/engine/security/seccomp/#run-without-the-default-seccomp-profile
This leads to failures again:
Extract from the build log
This is how I've configured this
Did I misunderstand your example?
Configuration looks right, but unfortunately the provided log is too truncated to have any specific details about why anything is actually failing. I am mildly surprised at it failing for some kind of unclear container related reasons that aren't seccomp. For now, a privileged container is a workaround, I guess.
@jade Sure, here's the full output.
not running the container as privileged in production is clearly the better suggestion, it was only meant as a quick way the find the problem and get it to work not knowing the exact issue.
Unmasking
/proc/*allows building lix in a unprivileged container with podman, or alternatively only unmasking the specific subpaths that are accessed there.Trying to debug this the list of required paths appears to be:
Not clear why lix would require access to to some of those paths, but removing any of them results in the build tests failing again.
Uncertain how this should be documented, because documenting this as a list might be brittle and still not optimal(offering the least privileges). Maybe documenting 'unmask=/proc/*' with a warning that it is better to only unmask the required paths, without offering an authoritative list that might be outdated/wrong.
This is honestly incomprehensible why those are required. Thank you for debugging it, and now I have many more questions.