Compare commits

..

3 commits

Author SHA1 Message Date
alois31 bbfeed2f00
libstore/build: copy ca-certificates too
In b469c6509b, the ca-certificates file was
missed. It should be copied too so that we don't end up bind-mounting a broken
symlink.

Change-Id: Ic9b292d602eb94b0e78f77f2a27a19d24665783c
2024-05-29 18:24:30 +02:00
alois31 ac41104988
libstore/build: use an allowlist approach to syscall filtering
Previously, system call filtering (to prevent builders from storing files with
setuid/setgid permission bits or extended attributes) was performed using a
blocklist. While this looks simple at first, it actually carries significant
security and maintainability risks: after all, the kernel may add new syscalls
to achieve the same functionality one is trying to block, and it can even be
hard to actually add the syscall to the blocklist when building against a C
library that doesn't know about it yet. For a recent demonstration of this
happening in practice to Nix, see the introduction of fchmodat2 [0] [1].

The allowlist approach does not share the same drawback. While it does require
a rather large list of harmless syscalls to be maintained in the codebase,
failing to update this list (and roll out the update to all users) in time has
rather benign effects; at worst, very recent programs that already rely on new
syscalls will fail with an error the same way they would on a slightly older
kernel that doesn't support them yet. Most importantly, no unintended new ways
of performing dangerous operations will be silently allowed.

Another possible drawback is reduced system call performance due to the larger
filter created by the allowlist requiring more computation [2]. However, this
issue has not convincingly been demonstrated yet in practice, for example in
systemd or various browsers.

This commit tries to keep the behavior as close to unchanged as possible. Only
newer syscalls that are not supported by glibc 2.38 (as found in NixOS 23.11)
are blocked. Since this includes fchmodat2, the compatibility code added for
handling this syscall can be removed too.

[0] https://github.com/NixOS/nixpkgs/issues/300635
[1] https://github.com/NixOS/nix/issues/10424
[2] https://github.com/flatpak/flatpak/pull/4462#issuecomment-1061690607

Change-Id: I541be3ea9b249bcceddfed6a5a13ac10b11e16ad
2024-05-29 18:24:30 +02:00
alois31 e07d281e01
libstore/build: always treat seccomp setup failures as fatal
In f047e4357b, I missed the behavior that if
building without a dedicated build user (i.e. in single-user setups), seccomp
setup failures are silently ignored. This was introduced without explanation 7
years ago (ff6becafa8). Hopefully the only
use-case nowadays is causing spurious test suite successes when messing up the
seccomp filter during development. Let's try removing it.

Change-Id: Ibe51416d9c7a6dd635c2282990224861adf1ceab
2024-05-29 18:24:30 +02:00
165 changed files with 3315 additions and 3540 deletions

View file

@ -11,10 +11,6 @@ additional-js = ["redirects.js"]
# to just submit a Gerrit CL by the web for trivial stuff.
edit-url-template = "https://github.com/lix-project/lix/tree/main/doc/manual/{path}"
git-repository-url = "https://git.lix.systems/lix-project/lix"
# Folding by default would prevent things like "Ctrl+F for nix-env" from working
# trivially, but the user should be able to fold if they want to.
fold.enable = true
fold.level = 30
# Handles replacing @docroot@ with a path to ./src relative to that markdown file,
# {{#include handlebars}}, and the @generated@ syntax used within these. it mostly

View file

@ -85,7 +85,6 @@
let
inherit (__forDefaults) canRunInstalled;
inherit (lib) fileset;
inherit (stdenv) hostPlatform buildPlatform;
version = lib.fileContents ./.version + versionSuffix;
@ -188,23 +187,23 @@ stdenv.mkDerivation (finalAttrs: {
dontBuild = false;
mesonFlags =
lib.optionals hostPlatform.isLinux [
lib.optionals stdenv.hostPlatform.isLinux [
# You'd think meson could just find this in PATH, but busybox is in buildInputs,
# which don't actually get added to PATH. And buildInputs is correct over
# nativeBuildInputs since this should be a busybox executable on the host.
"-Dsandbox-shell=${lib.getExe' busybox-sandbox-shell "busybox"}"
]
++ lib.optional hostPlatform.isStatic "-Denable-embedded-sandbox-shell=true"
++ lib.optional stdenv.hostPlatform.isStatic "-Denable-embedded-sandbox-shell=true"
++ lib.optional (finalAttrs.dontBuild) "-Denable-build=false"
++ [
# mesonConfigurePhase automatically passes -Dauto_features=enabled,
# so we must explicitly enable or disable features that we are not passing
# dependencies for.
(lib.mesonEnable "internal-api-docs" internalApiDocs)
(lib.mesonBool "enable-tests" finalAttrs.finalPackage.doCheck)
(lib.mesonBool "enable-tests" finalAttrs.doCheck)
(lib.mesonBool "enable-docs" canRunInstalled)
]
++ lib.optional (hostPlatform != buildPlatform) "--cross-file=${mesonCrossFile}";
++ lib.optional (stdenv.hostPlatform != stdenv.buildPlatform) "--cross-file=${mesonCrossFile}";
# We only include CMake so that Meson can locate toml11, which only ships CMake dependency metadata.
dontUseCmakeConfigure = true;
@ -232,7 +231,7 @@ stdenv.mkDerivation (finalAttrs: {
jq
lsof
]
++ lib.optional hostPlatform.isLinux util-linuxMinimal
++ lib.optional stdenv.hostPlatform.isLinux util-linuxMinimal
++ lib.optional (!officialRelease && buildUnreleasedNotes) build-release-notes
++ lib.optional internalApiDocs doxygen;
@ -252,14 +251,14 @@ stdenv.mkDerivation (finalAttrs: {
toml11
lix-doc
]
++ lib.optionals hostPlatform.isLinux [
++ lib.optionals stdenv.hostPlatform.isLinux [
libseccomp
busybox-sandbox-shell
]
++ lib.optional internalApiDocs rapidcheck
++ lib.optional hostPlatform.isx86_64 libcpuid
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (hostPlatform == buildPlatform) aws-sdk-cpp-nix
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform) aws-sdk-cpp-nix
++ lib.optionals (finalAttrs.dontBuild) maybePropagatedInputs;
checkInputs = [
@ -279,18 +278,18 @@ stdenv.mkDerivation (finalAttrs: {
};
preConfigure =
lib.optionalString (!finalAttrs.dontBuild && !hostPlatform.isStatic) ''
lib.optionalString (!finalAttrs.dontBuild && !stdenv.hostPlatform.isStatic) ''
# Copy libboost_context so we don't get all of Boost in our closure.
# https://github.com/NixOS/nixpkgs/issues/45462
mkdir -p $out/lib
cp -pd ${boost}/lib/{libboost_context*,libboost_thread*,libboost_system*} $out/lib
rm -f $out/lib/*.a
''
+ lib.optionalString (!finalAttrs.dontBuild && hostPlatform.isLinux && !hostPlatform.isStatic) ''
+ lib.optionalString (!finalAttrs.dontBuild && stdenv.hostPlatform.isLinux) ''
chmod u+w $out/lib/*.so.*
patchelf --set-rpath $out/lib:${stdenv.cc.cc.lib}/lib $out/lib/libboost_thread.so.*
''
+ lib.optionalString (!finalAttrs.dontBuild && hostPlatform.isDarwin) ''
+ lib.optionalString (!finalAttrs.dontBuild && stdenv.hostPlatform.isDarwin) ''
for LIB in $out/lib/*.dylib; do
chmod u+w $LIB
install_name_tool -id $LIB $LIB
@ -334,7 +333,7 @@ stdenv.mkDerivation (finalAttrs: {
mkdir -p $doc/nix-support
echo "doc manual $doc/share/doc/nix/manual" >> $doc/nix-support/hydra-build-products
''
+ lib.optionalString hostPlatform.isStatic ''
+ lib.optionalString stdenv.hostPlatform.isStatic ''
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
''
@ -365,12 +364,12 @@ stdenv.mkDerivation (finalAttrs: {
runHook postInstallCheck
'';
separateDebugInfo = !hostPlatform.isStatic && !finalAttrs.dontBuild;
separateDebugInfo = !stdenv.hostPlatform.isStatic && !finalAttrs.dontBuild;
strictDeps = true;
# strictoverflow is disabled because we trap on signed overflow instead
hardeningDisable = [ "strictoverflow" ] ++ lib.optional hostPlatform.isStatic "pie";
hardeningDisable = [ "strictoverflow" ] ++ lib.optional stdenv.hostPlatform.isStatic "pie";
meta = {
mainProgram = "nix";
@ -400,7 +399,7 @@ stdenv.mkDerivation (finalAttrs: {
check-syscalls,
}:
let
glibcFix = lib.optionalAttrs (buildPlatform.isLinux && glibcLocales != null) {
glibcFix = lib.optionalAttrs (stdenv.buildPlatform.isLinux && glibcLocales != null) {
# Required to make non-NixOS Linux not complain about missing locale files during configure in a dev shell
LOCALE_ARCHIVE = "${lib.getLib pkgs.glibcLocales}/lib/locale/locale-archive";
};
@ -418,19 +417,13 @@ stdenv.mkDerivation (finalAttrs: {
name = "lix-shell-env";
# finalPackage is necessary to propagate stuff that is set by mkDerivation itself,
# like doCheck.
inputsFrom = [ finalAttrs.finalPackage ];
inputsFrom = [ finalAttrs ];
# For Meson to find Boost.
env = finalAttrs.env;
# I guess this is necessary because mesonFlags to mkDerivation doesn't propagate in inputsFrom,
# which only propagates stuff set in hooks? idk.
inherit (finalAttrs) mesonFlags;
packages =
lib.optional (stdenv.cc.isClang && hostPlatform == buildPlatform) clang-tools_llvm
lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) clang-tools_llvm
++ [
check-syscalls
just
@ -444,41 +437,37 @@ stdenv.mkDerivation (finalAttrs: {
llvmPackages.clang-unwrapped.dev
]
++ lib.optional (pre-commit-checks ? enabledPackages) pre-commit-checks.enabledPackages
++ lib.optional (lib.meta.availableOn buildPlatform clangbuildanalyzer) clangbuildanalyzer
++ lib.optional (lib.meta.availableOn stdenv.buildPlatform clangbuildanalyzer) clangbuildanalyzer
++ finalAttrs.checkInputs;
shellHook = ''
# don't re-run the hook in (other) nested nix-shells
function lixShellHook() {
if [[ $name != lix-shell-env ]]; then
return;
fi
if [[ $name != lix-shell-env ]]; then
return;
fi
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
${lib.optionalString (pre-commit-checks ? shellHook) pre-commit-checks.shellHook}
# Allow `touch .nocontribmsg` to turn this notice off.
if ! [[ -f .nocontribmsg ]]; then
cat ${contribNotice}
fi
${lib.optionalString (pre-commit-checks ? shellHook) pre-commit-checks.shellHook}
# Allow `touch .nocontribmsg` to turn this notice off.
if ! [[ -f .nocontribmsg ]]; then
cat ${contribNotice}
fi
# Install the Gerrit commit-msg hook.
# (git common dir is the main .git, including for worktrees)
if gitcommondir=$(git rev-parse --git-common-dir 2>/dev/null) && [[ ! -f "$gitcommondir/hooks/commit-msg" ]]; then
echo 'Installing Gerrit commit-msg hook (adds Change-Id to commit messages)' >&2
mkdir -p "$gitcommondir/hooks"
curl -s -Lo "$gitcommondir/hooks/commit-msg" https://gerrit.lix.systems/tools/hooks/commit-msg
chmod u+x "$gitcommondir/hooks/commit-msg"
fi
unset gitcommondir
}
lixShellHook
# Install the Gerrit commit-msg hook.
# (git common dir is the main .git, including for worktrees)
if gitcommondir=$(git rev-parse --git-common-dir 2>/dev/null) && [[ ! -f "$gitcommondir/hooks/commit-msg" ]]; then
echo 'Installing Gerrit commit-msg hook (adds Change-Id to commit messages)' >&2
mkdir -p "$gitcommondir/hooks"
curl -s -Lo "$gitcommondir/hooks/commit-msg" https://gerrit.lix.systems/tools/hooks/commit-msg
chmod u+x "$gitcommondir/hooks/commit-msg"
fi
unset gitcommondir
'';
}
);

View file

@ -11,6 +11,7 @@
#include "derivations.hh"
#include "globals.hh"
#include "store-api.hh"
#include "util.hh"
#include "crypto.hh"
#include <sodium.h>

View file

@ -2,6 +2,7 @@
#include "common-eval-args.hh"
#include "shared.hh"
#include "filetransfer.hh"
#include "util.hh"
#include "eval.hh"
#include "fetchers.hh"
#include "registry.hh"

View file

@ -1,5 +1,5 @@
#include "util.hh"
#include "editor-for.hh"
#include "environment-variables.hh"
#include "source-path.hh"
namespace nix {

View file

@ -1,6 +1,7 @@
#include "globals.hh"
#include "installable-attr-path.hh"
#include "outputs-spec.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"

View file

@ -4,6 +4,7 @@
#include "globals.hh"
#include "installable-value.hh"
#include "outputs-spec.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"

View file

@ -2,6 +2,7 @@
#include "installable-flake.hh"
#include "installable-derived-path.hh"
#include "outputs-spec.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"

View file

@ -3,21 +3,26 @@
#include "installable-derived-path.hh"
#include "installable-attr-path.hh"
#include "installable-flake.hh"
#include "logging.hh"
#include "outputs-spec.hh"
#include "util.hh"
#include "command.hh"
#include "attr-path.hh"
#include "common-eval-args.hh"
#include "derivations.hh"
#include "eval-inline.hh"
#include "eval.hh"
#include "eval-settings.hh"
#include "get-drvs.hh"
#include "store-api.hh"
#include "shared.hh"
#include "flake/flake.hh"
#include "eval-cache.hh"
#include "url.hh"
#include "registry.hh"
#include "build-result.hh"
#include <regex>
#include <queue>
#include <nlohmann/json.hpp>

View file

@ -1,6 +1,7 @@
#pragma once
///@file
#include "util.hh"
#include "path.hh"
#include "outputs-spec.hh"
#include "derived-path.hh"

View file

@ -1,7 +1,6 @@
#include "markdown.hh"
#include "error.hh"
#include "util.hh"
#include "finally.hh"
#include "terminal.hh"
#include <sys/queue.h>
#include <lowdown.h>

View file

@ -1,7 +1,3 @@
#include "error.hh"
#include "file-system.hh"
#include "logging.hh"
#include <csignal>
#include <cstdio>
#include <iostream>
#include <string>
@ -22,8 +18,11 @@ extern "C" {
}
#endif
#include "signals.hh"
#include "finally.hh"
#include "repl-interacter.hh"
#include "util.hh"
#include "repl.hh"
namespace nix {

View file

@ -31,7 +31,6 @@
#include "print.hh"
#include "progress-bar.hh"
#include "gc-small-vector.hh"
#include "users.hh"
#if HAVE_BOEHMGC
#define GC_INCLUDE_NEW

View file

@ -1,5 +1,6 @@
#include "attr-path.hh"
#include "eval-inline.hh"
#include "util.hh"
namespace nix {

View file

@ -1,8 +1,8 @@
#include "eval-cache.hh"
#include "sqlite.hh"
#include "eval.hh"
#include "eval-inline.hh"
#include "store-api.hh"
#include "users.hh"
namespace nix::eval_cache {

View file

@ -1,7 +1,6 @@
#include "file-system.hh"
#include "globals.hh"
#include "profiles.hh"
#include "users.hh"
#include "eval.hh"
#include "eval-settings.hh"
namespace nix {

View file

@ -5,6 +5,7 @@
#include "print-options.hh"
#include "shared.hh"
#include "types.hh"
#include "util.hh"
#include "store-api.hh"
#include "derivations.hh"
#include "downstream-placeholder.hh"

View file

@ -1,6 +1,5 @@
#include "flake.hh"
#include "logging.hh"
#include "users.hh"
#include "globals.hh"
#include "fetch-settings.hh"
#include <nlohmann/json.hpp>

View file

@ -8,7 +8,6 @@
#include "fetchers.hh"
#include "finally.hh"
#include "fetch-settings.hh"
#include "terminal.hh"
namespace nix {

View file

@ -1,4 +1,5 @@
#include "get-drvs.hh"
#include "util.hh"
#include "eval-inline.hh"
#include "derivations.hh"
#include "store-api.hh"

View file

@ -2,6 +2,7 @@
#include "derivations.hh"
#include "eval.hh"
#include "symbol-table.hh"
#include "util.hh"
#include "print.hh"
#include "escape-string.hh"

View file

@ -19,7 +19,7 @@
#include <variant>
#include "finally.hh"
#include "users.hh"
#include "util.hh"
#include "nixexpr.hh"
#include "eval.hh"

View file

@ -9,8 +9,8 @@
#include "json-to-value.hh"
#include "names.hh"
#include "path-references.hh"
#include "processes.hh"
#include "store-api.hh"
#include "util.hh"
#include "value-to-json.hh"
#include "value-to-xml.hh"
#include "primops.hh"

View file

@ -1,7 +1,6 @@
#include "print-ambiguous.hh"
#include "attr-set.hh"
#include "logging.hh"
#include "print.hh"
#include "eval.hh"
#include "signals.hh"
#include "escape-string.hh"

View file

@ -9,7 +9,6 @@
#include "english.hh"
#include "signals.hh"
#include "eval.hh"
#include "terminal.hh"
namespace nix {

View file

@ -1,4 +1,5 @@
#include "search-path.hh"
#include "util.hh"
namespace nix {

View file

@ -1,5 +1,6 @@
#include "value-to-json.hh"
#include "eval-inline.hh"
#include "util.hh"
#include "signals.hh"
#include "store-api.hh"

View file

@ -1,7 +1,11 @@
#include "value-to-xml.hh"
#include "xml-writer.hh"
#include "eval-inline.hh"
#include "util.hh"
#include "signals.hh"
#include <cstdlib>
namespace nix {

View file

@ -1,6 +1,7 @@
#pragma once
///@file
#include "util.hh"
#include "comparator.hh"
#include "derived-path.hh"
#include "variant-wrapper.hh"

View file

@ -2,7 +2,6 @@
#include "sqlite.hh"
#include "sync.hh"
#include "store-api.hh"
#include "users.hh"
#include <nlohmann/json.hpp>

View file

@ -3,6 +3,7 @@
#include "types.hh"
#include "config.hh"
#include "util.hh"
#include <map>
#include <limits>

View file

@ -3,6 +3,7 @@
#include "source-path.hh"
#include "store-api.hh"
#include "util.hh"
#include "repair-flag.hh"
#include "content-address.hh"

View file

@ -1,12 +1,11 @@
#include "fetchers.hh"
#include "cache.hh"
#include "globals.hh"
#include "processes.hh"
#include "tarfile.hh"
#include "store-api.hh"
#include "url-parts.hh"
#include "pathlocks.hh"
#include "users.hh"
#include "util.hh"
#include "git.hh"
#include "logging.hh"
#include "finally.hh"

View file

@ -1,9 +1,9 @@
#include "fetchers.hh"
#include "cache.hh"
#include "processes.hh"
#include "globals.hh"
#include "tarfile.hh"
#include "store-api.hh"
#include "url-parts.hh"
#include "users.hh"
#include "fetch-settings.hh"

View file

@ -1,6 +1,6 @@
#include "registry.hh"
#include "fetchers.hh"
#include "users.hh"
#include "util.hh"
#include "globals.hh"
#include "store-api.hh"
#include "local-fs-store.hh"

View file

@ -2,7 +2,6 @@
#include "args/root.hh"
#include "globals.hh"
#include "loggers.hh"
#include "logging.hh"
namespace nix {

View file

@ -1,6 +1,6 @@
#include "environment-variables.hh"
#include "loggers.hh"
#include "progress-bar.hh"
#include "util.hh"
namespace nix {

View file

@ -1,8 +1,8 @@
#include "progress-bar.hh"
#include "util.hh"
#include "sync.hh"
#include "store-api.hh"
#include "names.hh"
#include "terminal.hh"
#include <atomic>
#include <map>

View file

@ -2,10 +2,10 @@
#include "shared.hh"
#include "store-api.hh"
#include "gc-store.hh"
#include "util.hh"
#include "signals.hh"
#include "loggers.hh"
#include "progress-bar.hh"
#include "current-process.hh"
#include <algorithm>
#include <cctype>

View file

@ -1,12 +1,12 @@
#pragma once
///@file
#include "util.hh"
#include "args.hh"
#include "args/root.hh"
#include "common-args.hh"
#include "path.hh"
#include "derived-path.hh"
#include "processes.hh"
#include "exit.hh"
#include <signal.h>

View file

@ -1,33 +0,0 @@
#include "current-process.hh"
#include "logging.hh"
namespace nix {
void commonChildInit()
{
logger = makeSimpleLogger();
const static std::string pathNullDevice = "/dev/null";
restoreProcessContext(false);
/* Put the child in a separate session (and thus a separate
process group) so that it has no controlling terminal (meaning
that e.g. ssh cannot open /dev/tty) and it doesn't receive
terminal signals. */
if (setsid() == -1)
throw SysError("creating a new session");
/* Dup stderr to stdout. */
if (dup2(STDERR_FILENO, STDOUT_FILENO) == -1)
throw SysError("cannot dup stderr into stdout");
/* Reroute stdin to /dev/null. */
int fdDevNull = open(pathNullDevice.c_str(), O_RDWR);
if (fdDevNull == -1)
throw SysError("cannot open '%1%'", pathNullDevice);
if (dup2(fdDevNull, STDIN_FILENO) == -1)
throw SysError("cannot dup null device into stdin");
close(fdDevNull);
}
}

View file

@ -1,11 +0,0 @@
#pragma once
///@file
namespace nix {
/**
* Common initialisation performed in child processes.
*/
void commonChildInit();
}

View file

@ -5,6 +5,7 @@
#include "builtins/buildenv.hh"
#include "references.hh"
#include "finally.hh"
#include "util.hh"
#include "archive.hh"
#include "compression.hh"
#include "common-protocol.hh"

View file

@ -1,5 +1,3 @@
#include "child.hh"
#include "file-system.hh"
#include "globals.hh"
#include "hook-instance.hh"

View file

@ -2,7 +2,6 @@
///@file
#include "logging.hh"
#include "processes.hh"
#include "serialise.hh"
namespace nix {

View file

@ -6,6 +6,7 @@
#include "builtins/buildenv.hh"
#include "path-references.hh"
#include "finally.hh"
#include "util.hh"
#include "archive.hh"
#include "compression.hh"
#include "daemon.hh"
@ -14,8 +15,6 @@
#include "cgroup.hh"
#include "personality.hh"
#include "namespaces.hh"
#include "child.hh"
#include "unix-domain-socket.hh"
#include <regex>
#include <queue>
@ -63,11 +62,6 @@ extern "C" int sandbox_init_with_parameters(const char *profile, uint64_t flags,
namespace nix {
/**
* The system for which Nix is compiled.
*/
constexpr std::string_view nativeSystem = SYSTEM;
void handleDiffHook(
uid_t uid, uid_t gid,
const Path & tryA, const Path & tryB,

View file

@ -3,7 +3,6 @@
#include "derivation-goal.hh"
#include "local-store.hh"
#include "processes.hh"
namespace nix {

View file

@ -1,8 +1,11 @@
#include "serialise.hh"
#include "util.hh"
#include "path-with-outputs.hh"
#include "store-api.hh"
#include "build-result.hh"
#include "common-protocol.hh"
#include "common-protocol-impl.hh"
#include "archive.hh"
#include "derivations.hh"
#include <nlohmann/json.hpp>

View file

@ -1,7 +1,6 @@
#include "crypto.hh"
#include "file-system.hh"
#include "util.hh"
#include "globals.hh"
#include "strings.hh"
#include <sodium.h>

View file

@ -1,9 +1,9 @@
#pragma once
///@file
#include "types.hh"
#include <map>
#include <string>
namespace nix {

View file

@ -3,6 +3,7 @@
#include "store-api.hh"
#include "globals.hh"
#include "types.hh"
#include "util.hh"
#include "split.hh"
#include "common-protocol.hh"
#include "common-protocol-impl.hh"

View file

@ -1,7 +1,7 @@
#pragma once
///@file
#include "config.hh"
#include "util.hh"
#include "path.hh"
#include "outputs-spec.hh"
#include "comparator.hh"

View file

@ -1,5 +1,5 @@
#include "filetransfer.hh"
#include "namespaces.hh"
#include "util.hh"
#include "globals.hh"
#include "store-api.hh"
#include "s3.hh"
@ -19,6 +19,7 @@
#include <algorithm>
#include <cmath>
#include <cstring>
#include <iostream>
#include <queue>
#include <random>
#include <thread>
@ -50,6 +51,7 @@ struct curlFileTransfer : public FileTransfer
std::function<void(TransferItem &, std::string_view data)> dataCallback;
CURL * req = 0;
bool active = false; // whether the handle has been added to the multi object
bool headersProcessed = false;
std::string statusMsg;
unsigned int attempt = 0;
@ -134,11 +136,35 @@ struct curlFileTransfer : public FileTransfer
std::exception_ptr writeException;
std::optional<std::string> getHeader(const char * name)
{
curl_header * result;
auto e = curl_easy_header(req, name, 0, CURLH_HEADER, -1, &result);
if (e == CURLHE_OK) {
return result->value;
} else if (e == CURLHE_MISSING || e == CURLHE_NOHEADERS) {
return std::nullopt;
} else {
throw nix::Error("unexpected error from curl_easy_header(): %i", e);
}
}
size_t writeCallback(void * contents, size_t size, size_t nmemb)
{
const size_t realSize = size * nmemb;
try {
if (!headersProcessed) {
if (auto h = getHeader("content-encoding")) {
encoding = std::move(*h);
}
if (auto h = getHeader("accept-ranges"); h && *h == "bytes") {
acceptRanges = true;
}
headersProcessed = true;
}
result.bodySize += realSize;
if (successfulStatuses.count(getHTTPStatus()) && this->dataCallback) {
@ -174,31 +200,7 @@ struct curlFileTransfer : public FileTransfer
statusMsg = trim(match.str(1));
acceptRanges = false;
encoding = "";
} else {
auto i = line.find(':');
if (i != std::string::npos) {
std::string name = toLower(trim(line.substr(0, i)));
if (name == "etag") {
result.etag = trim(line.substr(i + 1));
}
else if (name == "content-encoding")
encoding = trim(line.substr(i + 1));
else if (name == "accept-ranges" && toLower(trim(line.substr(i + 1))) == "bytes")
acceptRanges = true;
else if (name == "link" || name == "x-amz-meta-link") {
auto value = trim(line.substr(i + 1));
static std::regex linkRegex("<([^>]*)>; rel=\"immutable\"", std::regex::extended | std::regex::icase);
if (std::smatch match; std::regex_match(value, match, linkRegex))
result.immutableUrl = match.str(1);
else
debug("got invalid link header '%s'", value);
warn("foo %s", value);
}
}
headersProcessed = false;
}
return realSize;
}
@ -334,6 +336,25 @@ struct curlFileTransfer : public FileTransfer
debug("finished %s of '%s'; curl status = %d, HTTP status = %d, body = %d bytes",
request.verb(), request.uri, code, httpStatus, result.bodySize);
auto link = getHeader("link");
if (!link) {
link = getHeader("x-amz-meta-link");
}
if (link) {
static std::regex linkRegex(
"<([^>]*)>; rel=\"immutable\"", std::regex::extended | std::regex::icase
);
if (std::smatch match; std::regex_match(*link, match, linkRegex)) {
result.immutableUrl = match.str(1);
} else {
debug("got invalid link header '%s'", *link);
}
}
if (auto etag = getHeader("etag")) {
result.etag = std::move(*etag);
}
// this has to happen here until we can return an actual future.
// wrapping user `callback`s instead is not possible because the
// Callback api expects std::functions, and copying Callbacks is

View file

@ -1,9 +1,8 @@
#pragma once
///@file
#include "logging.hh"
#include "serialise.hh"
#include "types.hh"
#include "hash.hh"
#include "config.hh"
#include <string>

View file

@ -1,14 +1,16 @@
#include "derivations.hh"
#include "globals.hh"
#include "local-store.hh"
#include "pathlocks.hh"
#include "processes.hh"
#include "signals.hh"
#include "finally.hh"
#include "unix-domain-socket.hh"
#include <functional>
#include <queue>
#include <algorithm>
#include <regex>
#include <random>
#include <climits>
#include <errno.h>
#include <fcntl.h>
#include <poll.h>

View file

@ -1,15 +1,12 @@
#include "environment-variables.hh"
#include "globals.hh"
#include "file-system.hh"
#include "logging.hh"
#include "strings.hh"
#include "users.hh"
#include "util.hh"
#include "archive.hh"
#include "args.hh"
#include "abstract-setting-to-json.hh"
#include "compute-levels.hh"
#include "current-process.hh"
#include <algorithm>
#include <map>
#include <mutex>
#include <thread>
#include <dlfcn.h>
@ -28,7 +25,6 @@
#include "config-impl.hh"
#ifdef __APPLE__
#include "processes.hh"
#include <curl/curl.h>
#include <sys/sysctl.h>
#endif

View file

@ -1,9 +1,9 @@
#pragma once
///@file
#include "environment-variables.hh"
#include "types.hh"
#include "config.hh"
#include "util.hh"
#include <map>
#include <limits>

View file

@ -3,9 +3,11 @@
#include "sqlite.hh"
#include "pathlocks.hh"
#include "store-api.hh"
#include "indirect-root-store.hh"
#include "sync.hh"
#include "util.hh"
#include <chrono>
#include <future>

View file

@ -1,6 +1,4 @@
#include "lock.hh"
#include "logging.hh"
#include "file-system.hh"
#include "globals.hh"
#include "pathlocks.hh"

View file

@ -1,7 +1,7 @@
#include "machines.hh"
#include "util.hh"
#include "globals.hh"
#include "store-api.hh"
#include "strings.hh"
#include <algorithm>

View file

@ -79,7 +79,6 @@ libstore_sources = files(
'store-api.cc',
'uds-remote-store.cc',
'worker-protocol.cc',
'build/child.cc',
'build/derivation-goal.cc',
'build/drv-output-substitution-goal.cc',
'build/entry-points.cc',
@ -97,7 +96,6 @@ libstore_sources = files(
libstore_headers = files(
'binary-cache-store.hh',
'build/child.hh',
'build/derivation-goal.hh',
'build/drv-output-substitution-goal.hh',
'build/goal.hh',

View file

@ -1,5 +1,5 @@
#include "names.hh"
#include "strings.hh"
#include "util.hh"
#include <regex>

View file

@ -1,9 +1,7 @@
#include "nar-info-disk-cache.hh"
#include "logging.hh"
#include "sync.hh"
#include "sqlite.hh"
#include "globals.hh"
#include "users.hh"
#include <sqlite3.h>
#include <nlohmann/json.hpp>

View file

@ -1,3 +1,4 @@
#include "util.hh"
#include "local-store.hh"
#include "globals.hh"
#include "signals.hh"

View file

@ -1,10 +1,10 @@
#include <regex>
#include <nlohmann/json.hpp>
#include "util.hh"
#include "regex-combinators.hh"
#include "outputs-spec.hh"
#include "path-regex.hh"
#include "strings.hh"
namespace nix {

View file

@ -1,8 +1,12 @@
#include "path-references.hh"
#include "hash.hh"
#include "util.hh"
#include "archive.hh"
#include <map>
#include <cstdlib>
#include <mutex>
#include <algorithm>
namespace nix {

View file

@ -1,8 +1,10 @@
#include "pathlocks.hh"
#include "logging.hh"
#include "util.hh"
#include "signals.hh"
#include "sync.hh"
#include <cerrno>
#include <cstdlib>
#include <fcntl.h>
#include <sys/types.h>

View file

@ -1,7 +1,7 @@
#pragma once
///@file
#include "file-descriptor.hh"
#include "util.hh"
namespace nix {

View file

@ -1,7 +1,7 @@
#include "profiles.hh"
#include "store-api.hh"
#include "local-fs-store.hh"
#include "users.hh"
#include "util.hh"
#include <sys/types.h>
#include <sys/stat.h>

View file

@ -1,4 +1,5 @@
#include "serialise.hh"
#include "util.hh"
#include "signals.hh"
#include "path-with-outputs.hh"
#include "gc-store.hh"

View file

@ -1,9 +1,11 @@
#include "serialise.hh"
#include "util.hh"
#include "path-with-outputs.hh"
#include "store-api.hh"
#include "build-result.hh"
#include "serve-protocol.hh"
#include "serve-protocol-impl.hh"
#include "archive.hh"
#include "path-info.hh"
#include <nlohmann/json.hpp>

View file

@ -1,11 +1,12 @@
#include "sqlite.hh"
#include "globals.hh"
#include "logging.hh"
#include "util.hh"
#include "signals.hh"
#include "url.hh"
#include <sqlite3.h>
#include <atomic>
namespace nix {

View file

@ -1,9 +1,5 @@
#include "current-process.hh"
#include "environment-variables.hh"
#include "ssh.hh"
#include "finally.hh"
#include "logging.hh"
#include "strings.hh"
namespace nix {

View file

@ -1,8 +1,7 @@
#pragma once
///@file
#include "file-system.hh"
#include "processes.hh"
#include "util.hh"
#include "sync.hh"
namespace nix {

View file

@ -1,7 +1,9 @@
#include "crypto.hh"
#include "fs-accessor.hh"
#include "globals.hh"
#include "derivations.hh"
#include "store-api.hh"
#include "util.hh"
#include "nar-info-disk-cache.hh"
#include "thread-pool.hh"
#include "url.hh"
@ -12,7 +14,6 @@
// FIXME this should not be here, see TODO below on
// `addMultipleToStore`.
#include "worker-protocol.hh"
#include "users.hh"
#include <nlohmann/json.hpp>
#include <regex>

View file

@ -1,7 +1,6 @@
#pragma once
///@file
#include "logging.hh"
#include "nar-info.hh"
#include "realisation.hh"
#include "path.hh"

View file

@ -1,5 +1,4 @@
#include "uds-remote-store.hh"
#include "unix-domain-socket.hh"
#include "worker-protocol.hh"
#include <sys/types.h>

View file

@ -1,4 +1,5 @@
#include "serialise.hh"
#include "util.hh"
#include "path-with-outputs.hh"
#include "store-api.hh"
#include "build-result.hh"

View file

@ -12,9 +12,8 @@
#include <fcntl.h>
#include "archive.hh"
#include "file-system.hh"
#include "util.hh"
#include "config.hh"
#include "logging.hh"
#include "signals.hh"
namespace nix {

View file

@ -3,7 +3,6 @@
#include "types.hh"
#include "serialise.hh"
#include "file-system.hh"
namespace nix {

View file

@ -2,10 +2,7 @@
#include "args/root.hh"
#include "hash.hh"
#include "json-utils.hh"
#include "environment-variables.hh"
#include "experimental-features-json.hh"
#include "logging.hh"
#include <glob.h>

View file

@ -1,17 +1,13 @@
#pragma once
///@file
#include "experimental-features.hh"
#include "types.hh"
#include <functional>
#include <iostream>
#include <map>
#include <memory>
#include <limits>
#include <nlohmann/json_fwd.hpp>
#include <optional>
#include <set>
#include "util.hh"
namespace nix {

View file

@ -1,5 +1,5 @@
#include "canon-path.hh"
#include "file-system.hh"
#include "util.hh"
namespace nix {

View file

@ -1,17 +1,14 @@
#include "logging.hh"
#if __linux__
#include "cgroup.hh"
#include "file-system.hh"
#include "util.hh"
#include "finally.hh"
#include "strings.hh"
#include <chrono>
#include <cmath>
#include <regex>
#include <unordered_set>
#include <thread>
#include <signal.h>
#include <dirent.h>
#include <mntent.h>

View file

@ -1,8 +1,6 @@
#pragma once
///@file
#include <tuple>
#define DECLARE_ONE_CMP(PRE, QUAL, COMPARATOR, MY_TYPE) \
PRE bool QUAL operator COMPARATOR(const MY_TYPE & other) const;
#define DECLARE_EQUAL(prefix, qualification, my_type) \

View file

@ -1,5 +1,7 @@
#include "compression.hh"
#include "tarfile.hh"
#include "util.hh"
#include "finally.hh"
#include "signals.hh"
#include "logging.hh"
@ -11,6 +13,7 @@
#include <brotli/decode.h>
#include <brotli/encode.h>
#include <iostream>
namespace nix {

View file

@ -12,9 +12,7 @@
* instantiation.
*/
#include "args.hh"
#include "config.hh"
#include "logging.hh"
namespace nix {

View file

@ -2,9 +2,6 @@
#include "args.hh"
#include "abstract-setting-to-json.hh"
#include "experimental-features.hh"
#include "file-system.hh"
#include "logging.hh"
#include "strings.hh"
#include "config-impl.hh"

View file

@ -1,111 +0,0 @@
#include "current-process.hh"
#include "error.hh"
#include "file-system.hh"
#include "logging.hh"
#include "namespaces.hh"
#include "signals.hh"
#include "strings.hh"
#ifdef __APPLE__
# include <mach-o/dyld.h>
#endif
#if __linux__
# include <sys/resource.h>
#endif
#include <sys/mount.h>
#include <cgroup.hh>
namespace nix {
unsigned int getMaxCPU()
{
#if __linux__
try {
auto cgroupFS = getCgroupFS();
if (!cgroupFS) return 0;
auto cgroups = getCgroups("/proc/self/cgroup");
auto cgroup = cgroups[""];
if (cgroup == "") return 0;
auto cpuFile = *cgroupFS + "/" + cgroup + "/cpu.max";
auto cpuMax = readFile(cpuFile);
auto cpuMaxParts = tokenizeString<std::vector<std::string>>(cpuMax, " \n");
if (cpuMaxParts.size() != 2) {
return 0;
}
auto quota = cpuMaxParts[0];
auto period = cpuMaxParts[1];
if (quota != "max")
return std::ceil(std::stoi(quota) / std::stof(period));
} catch (Error &) { ignoreException(lvlDebug); }
#endif
return 0;
}
rlim_t savedStackSize = 0;
void setStackSize(rlim_t stackSize)
{
struct rlimit limit;
if (getrlimit(RLIMIT_STACK, &limit) == 0 && limit.rlim_cur < stackSize) {
savedStackSize = limit.rlim_cur;
limit.rlim_cur = std::min(stackSize, limit.rlim_max);
if (setrlimit(RLIMIT_STACK, &limit) != 0) {
logger->log(
lvlError,
HintFmt(
"Failed to increase stack size from %1% to %2% (maximum allowed stack size: %3%): %4%",
savedStackSize,
stackSize,
limit.rlim_max,
std::strerror(errno)
).str()
);
}
}
}
void restoreProcessContext(bool restoreMounts)
{
restoreSignals();
if (restoreMounts) {
restoreMountNamespace();
}
if (savedStackSize) {
struct rlimit limit;
if (getrlimit(RLIMIT_STACK, &limit) == 0) {
limit.rlim_cur = savedStackSize;
setrlimit(RLIMIT_STACK, &limit);
}
}
}
std::optional<Path> getSelfExe()
{
static auto cached = []() -> std::optional<Path>
{
#if __linux__
return readLink("/proc/self/exe");
#elif __APPLE__
char buf[1024];
uint32_t size = sizeof(buf);
if (_NSGetExecutablePath(buf, &size) == 0)
return buf;
else
return std::nullopt;
#else
return std::nullopt;
#endif
}();
return cached;
}
}

View file

@ -1,37 +0,0 @@
#pragma once
///@file
#include <optional>
#include <sys/resource.h>
#include "types.hh"
namespace nix {
/**
* If cgroups are active, attempt to calculate the number of CPUs available.
* If cgroups are unavailable or if cpu.max is set to "max", return 0.
*/
unsigned int getMaxCPU();
/**
* Change the stack size.
*/
void setStackSize(rlim_t stackSize);
/**
* Restore the original inherited Unix process context (such as signal
* masks, stack size).
* See startSignalHandlerThread(), saveSignalMask().
*/
void restoreProcessContext(bool restoreMounts = true);
/**
* @return the path of the current executable.
*/
std::optional<Path> getSelfExe();
}

View file

@ -1,51 +0,0 @@
#include <cstring>
#include <map>
#include <optional>
#include <string>
extern char * * environ __attribute__((weak));
namespace nix {
std::optional<std::string> getEnv(const std::string & key)
{
char * value = getenv(key.c_str());
if (!value) return {};
return std::string(value);
}
std::optional<std::string> getEnvNonEmpty(const std::string & key) {
auto value = getEnv(key);
if (value == "") return {};
return value;
}
std::map<std::string, std::string> getEnv()
{
std::map<std::string, std::string> env;
for (size_t i = 0; environ[i]; ++i) {
auto s = environ[i];
auto eq = strchr(s, '=');
if (!eq)
// invalid env, just keep going
continue;
env.emplace(std::string(s, eq), std::string(eq + 1));
}
return env;
}
void clearEnv()
{
for (auto & name : getEnv())
unsetenv(name.first.c_str());
}
void replaceEnv(const std::map<std::string, std::string> & newEnv)
{
clearEnv();
for (auto & newEnvVar : newEnv)
setenv(newEnvVar.first.c_str(), newEnvVar.second.c_str(), 1);
}
}

View file

@ -1,42 +0,0 @@
#pragma once
/**
* @file
*
* Utilities for working with the current process's environment
* variables.
*/
#include <map>
#include <optional>
#include <string>
namespace nix {
/**
* @return an environment variable.
*/
std::optional<std::string> getEnv(const std::string & key);
/**
* @return a non empty environment variable. Returns nullopt if the env
* variable is set to ""
*/
std::optional<std::string> getEnvNonEmpty(const std::string & key);
/**
* Get the entire environment.
*/
std::map<std::string, std::string> getEnv();
/**
* Clear the environment.
*/
void clearEnv();
/**
* Replace the entire environment with the given one.
*/
void replaceEnv(const std::map<std::string, std::string> & newEnv);
}

View file

@ -1,15 +1,15 @@
#include "environment-variables.hh"
#include "error.hh"
#include "logging.hh"
#include "position.hh"
#include "terminal.hh"
#include <iostream>
#include <optional>
#include "serialise.hh"
#include <sstream>
namespace nix {
const std::string nativeSystem = SYSTEM;
void BaseError::addTrace(std::shared_ptr<Pos> && e, HintFmt hint)
{
err.traces.push_front(Trace { .pos = std::move(e), .hint = hint });
@ -415,17 +415,4 @@ std::ostream & showErrorInfo(std::ostream & out, const ErrorInfo & einfo, bool s
return out;
}
void ignoreException(Verbosity lvl)
{
/* Make sure no exceptions leave this function.
printError() also throws when remote is closed. */
try {
try {
throw;
} catch (std::exception & e) {
printMsg(lvl, "error (ignored): %1%", e.what());
}
} catch (...) { }
}
}

View file

@ -202,10 +202,4 @@ public:
}
};
/**
* Exception handling in destructors: print an error message, then
* ignore the exception.
*/
void ignoreException(Verbosity lvl = lvlError);
}

View file

@ -1,7 +1,7 @@
#include "experimental-features.hh"
// Required for instances of to_json and from_json for ExperimentalFeature
#include "experimental-features-json.hh"
#include "strings.hh"
#include "util.hh"
#include "nlohmann/json.hpp"

View file

@ -1,252 +0,0 @@
#include "file-system.hh"
#include "finally.hh"
#include "logging.hh"
#include "serialise.hh"
#include "signals.hh"
#include <fcntl.h>
#include <unistd.h>
namespace nix {
std::string readFile(int fd)
{
struct stat st;
if (fstat(fd, &st) == -1)
throw SysError("statting file");
return drainFD(fd, true, st.st_size);
}
std::string readLine(int fd)
{
std::string s;
while (1) {
checkInterrupt();
char ch;
// FIXME: inefficient
ssize_t rd = read(fd, &ch, 1);
if (rd == -1) {
if (errno != EINTR)
throw SysError("reading a line");
} else if (rd == 0)
throw EndOfFile("unexpected EOF reading a line");
else {
if (ch == '\n') return s;
s += ch;
}
}
}
void writeLine(int fd, std::string s)
{
s += '\n';
writeFull(fd, s);
}
void readFull(int fd, char * buf, size_t count)
{
while (count) {
checkInterrupt();
ssize_t res = read(fd, buf, count);
if (res == -1) {
if (errno == EINTR) continue;
throw SysError("reading from file");
}
if (res == 0) throw EndOfFile("unexpected end-of-file");
count -= res;
buf += res;
}
}
void writeFull(int fd, std::string_view s, bool allowInterrupts)
{
while (!s.empty()) {
if (allowInterrupts) checkInterrupt();
ssize_t res = write(fd, s.data(), s.size());
if (res == -1 && errno != EINTR)
throw SysError("writing to file");
if (res > 0)
s.remove_prefix(res);
}
}
std::string drainFD(int fd, bool block, const size_t reserveSize)
{
// the parser needs two extra bytes to append terminating characters, other users will
// not care very much about the extra memory.
StringSink sink(reserveSize + 2);
drainFD(fd, sink, block);
return std::move(sink.s);
}
void drainFD(int fd, Sink & sink, bool block)
{
// silence GCC maybe-uninitialized warning in finally
int saved = 0;
if (!block) {
saved = fcntl(fd, F_GETFL);
if (fcntl(fd, F_SETFL, saved | O_NONBLOCK) == -1)
throw SysError("making file descriptor non-blocking");
}
Finally finally([&]() {
if (!block) {
if (fcntl(fd, F_SETFL, saved) == -1)
throw SysError("making file descriptor blocking");
}
});
std::array<unsigned char, 64 * 1024> buf;
while (1) {
checkInterrupt();
ssize_t rd = read(fd, buf.data(), buf.size());
if (rd == -1) {
if (!block && (errno == EAGAIN || errno == EWOULDBLOCK))
break;
if (errno != EINTR)
throw SysError("reading from file");
}
else if (rd == 0) break;
else sink({(char *) buf.data(), (size_t) rd});
}
}
AutoCloseFD::AutoCloseFD() : fd{-1} {}
AutoCloseFD::AutoCloseFD(int fd) : fd{fd} {}
AutoCloseFD::AutoCloseFD(AutoCloseFD && that) : fd{that.fd}
{
that.fd = -1;
}
AutoCloseFD & AutoCloseFD::operator =(AutoCloseFD && that)
{
close();
fd = that.fd;
that.fd = -1;
return *this;
}
AutoCloseFD::~AutoCloseFD()
{
try {
close();
} catch (...) {
ignoreException();
}
}
int AutoCloseFD::get() const
{
return fd;
}
void AutoCloseFD::close()
{
if (fd != -1) {
if (::close(fd) == -1)
/* This should never happen. */
throw SysError("closing file descriptor %1%", fd);
fd = -1;
}
}
void AutoCloseFD::fsync()
{
if (fd != -1) {
int result;
#if __APPLE__
result = ::fcntl(fd, F_FULLFSYNC);
#else
result = ::fsync(fd);
#endif
if (result == -1)
throw SysError("fsync file descriptor %1%", fd);
}
}
AutoCloseFD::operator bool() const
{
return fd != -1;
}
int AutoCloseFD::release()
{
int oldFD = fd;
fd = -1;
return oldFD;
}
void Pipe::create()
{
int fds[2];
#if HAVE_PIPE2
if (pipe2(fds, O_CLOEXEC) != 0) throw SysError("creating pipe");
#else
if (pipe(fds) != 0) throw SysError("creating pipe");
closeOnExec(fds[0]);
closeOnExec(fds[1]);
#endif
readSide = AutoCloseFD{fds[0]};
writeSide = AutoCloseFD{fds[1]};
}
void Pipe::close()
{
readSide.close();
writeSide.close();
}
void closeMostFDs(const std::set<int> & exceptions)
{
#if __linux__
try {
for (auto & s : readDirectory("/proc/self/fd")) {
auto fd = std::stoi(s.name);
if (!exceptions.count(fd)) {
debug("closing leaked FD %d", fd);
close(fd);
}
}
return;
} catch (SysError &) {
}
#endif
int maxFD = 0;
maxFD = sysconf(_SC_OPEN_MAX);
for (int fd = 0; fd < maxFD; ++fd)
if (!exceptions.count(fd))
close(fd); /* ignore result */
}
void closeOnExec(int fd)
{
int prev;
if ((prev = fcntl(fd, F_GETFD, 0)) == -1 ||
fcntl(fd, F_SETFD, prev | FD_CLOEXEC) == -1)
throw SysError("setting close-on-exec flag");
}
}

View file

@ -1,80 +0,0 @@
#pragma once
///@file
#include "error.hh"
namespace nix {
struct Sink;
struct Source;
/**
* Read a line from a file descriptor.
*/
std::string readLine(int fd);
/**
* Write a line to a file descriptor.
*/
void writeLine(int fd, std::string s);
/**
* Read the contents of a file into a string.
*/
std::string readFile(int fd);
/**
* Wrappers arount read()/write() that read/write exactly the
* requested number of bytes.
*/
void readFull(int fd, char * buf, size_t count);
void writeFull(int fd, std::string_view s, bool allowInterrupts = true);
/**
* Read a file descriptor until EOF occurs.
*/
std::string drainFD(int fd, bool block = true, const size_t reserveSize=0);
void drainFD(int fd, Sink & sink, bool block = true);
class AutoCloseFD
{
int fd;
public:
AutoCloseFD();
explicit AutoCloseFD(int fd);
AutoCloseFD(const AutoCloseFD & fd) = delete;
AutoCloseFD(AutoCloseFD&& fd);
~AutoCloseFD();
AutoCloseFD& operator =(const AutoCloseFD & fd) = delete;
AutoCloseFD& operator =(AutoCloseFD&& fd) noexcept(false);
int get() const;
explicit operator bool() const;
int release();
void close();
void fsync();
void reset() { *this = {}; }
};
class Pipe
{
public:
AutoCloseFD readSide, writeSide;
void create();
void close();
};
/**
* Close all file descriptors except those listed in the given set.
* Good practice in child processes.
*/
void closeMostFDs(const std::set<int> & exceptions);
/**
* Set the close-on-exec flag for the given file descriptor.
*/
void closeOnExec(int fd);
MakeError(EndOfFile, Error);
}

View file

@ -1,670 +0,0 @@
#include <sys/time.h>
#include <filesystem>
#include <atomic>
#include "environment-variables.hh"
#include "file-descriptor.hh"
#include "file-system.hh"
#include "finally.hh"
#include "logging.hh"
#include "serialise.hh"
#include "signals.hh"
#include "types.hh"
#include "users.hh"
namespace fs = std::filesystem;
namespace nix {
Path absPath(Path path, std::optional<PathView> dir, bool resolveSymlinks)
{
if (path.empty() || path[0] != '/') {
if (!dir) {
#ifdef __GNU__
/* GNU (aka. GNU/Hurd) doesn't have any limitation on path
lengths and doesn't define `PATH_MAX'. */
char *buf = getcwd(NULL, 0);
if (buf == NULL)
#else
char buf[PATH_MAX];
if (!getcwd(buf, sizeof(buf)))
#endif
throw SysError("cannot get cwd");
path = concatStrings(buf, "/", path);
#ifdef __GNU__
free(buf);
#endif
} else
path = concatStrings(*dir, "/", path);
}
return canonPath(path, resolveSymlinks);
}
Path canonPath(PathView path, bool resolveSymlinks)
{
assert(path != "");
std::string s;
s.reserve(256);
if (path[0] != '/')
throw Error("not an absolute path: '%1%'", path);
std::string temp;
/* Count the number of times we follow a symlink and stop at some
arbitrary (but high) limit to prevent infinite loops. */
unsigned int followCount = 0, maxFollow = 1024;
while (1) {
/* Skip slashes. */
while (!path.empty() && path[0] == '/') path.remove_prefix(1);
if (path.empty()) break;
/* Ignore `.'. */
if (path == "." || path.substr(0, 2) == "./")
path.remove_prefix(1);
/* If `..', delete the last component. */
else if (path == ".." || path.substr(0, 3) == "../")
{
if (!s.empty()) s.erase(s.rfind('/'));
path.remove_prefix(2);
}
/* Normal component; copy it. */
else {
s += '/';
if (const auto slash = path.find('/'); slash == std::string::npos) {
s += path;
path = {};
} else {
s += path.substr(0, slash);
path = path.substr(slash);
}
/* If s points to a symlink, resolve it and continue from there */
if (resolveSymlinks && isLink(s)) {
if (++followCount >= maxFollow)
throw Error("infinite symlink recursion in path '%1%'", path);
temp = concatStrings(readLink(s), path);
path = temp;
if (!temp.empty() && temp[0] == '/') {
s.clear(); /* restart for symlinks pointing to absolute path */
} else {
s = dirOf(s);
if (s == "/") { // we dont want trailing slashes here, which dirOf only produces if s = /
s.clear();
}
}
}
}
}
return s.empty() ? "/" : std::move(s);
}
void chmodPath(const Path & path, mode_t mode)
{
if (chmod(path.c_str(), mode) == -1)
throw SysError("setting permissions on '%s'", path);
}
Path dirOf(const PathView path)
{
Path::size_type pos = path.rfind('/');
if (pos == std::string::npos)
return ".";
return pos == 0 ? "/" : Path(path, 0, pos);
}
std::string_view baseNameOf(std::string_view path)
{
if (path.empty())
return "";
auto last = path.size() - 1;
if (path[last] == '/' && last > 0)
last -= 1;
auto pos = path.rfind('/', last);
if (pos == std::string::npos)
pos = 0;
else
pos += 1;
return path.substr(pos, last - pos + 1);
}
std::string expandTilde(std::string_view path)
{
// TODO: expand ~user ?
auto tilde = path.substr(0, 2);
if (tilde == "~/" || tilde == "~")
return getHome() + std::string(path.substr(1));
else
return std::string(path);
}
bool isInDir(std::string_view path, std::string_view dir)
{
return path.substr(0, 1) == "/"
&& path.substr(0, dir.size()) == dir
&& path.size() >= dir.size() + 2
&& path[dir.size()] == '/';
}
bool isDirOrInDir(std::string_view path, std::string_view dir)
{
return path == dir || isInDir(path, dir);
}
struct stat stat(const Path & path)
{
struct stat st;
if (stat(path.c_str(), &st))
throw SysError("getting status of '%1%'", path);
return st;
}
struct stat lstat(const Path & path)
{
struct stat st;
if (lstat(path.c_str(), &st))
throw SysError("getting status of '%1%'", path);
return st;
}
std::optional<struct stat> maybeLstat(const Path & path)
{
std::optional<struct stat> st{std::in_place};
if (lstat(path.c_str(), &*st))
{
if (errno == ENOENT || errno == ENOTDIR)
st.reset();
else
throw SysError("getting status of '%s'", path);
}
return st;
}
bool pathExists(const Path & path)
{
return maybeLstat(path).has_value();
}
bool pathAccessible(const Path & path)
{
try {
return pathExists(path);
} catch (SysError & e) {
// swallow EPERM
if (e.errNo == EPERM) return false;
throw;
}
}
Path readLink(const Path & path)
{
checkInterrupt();
std::vector<char> buf;
for (ssize_t bufSize = PATH_MAX/4; true; bufSize += bufSize/2) {
buf.resize(bufSize);
ssize_t rlSize = readlink(path.c_str(), buf.data(), bufSize);
if (rlSize == -1)
if (errno == EINVAL)
throw Error("'%1%' is not a symlink", path);
else
throw SysError("reading symbolic link '%1%'", path);
else if (rlSize < bufSize)
return std::string(buf.data(), rlSize);
}
}
bool isLink(const Path & path)
{
struct stat st = lstat(path);
return S_ISLNK(st.st_mode);
}
DirEntries readDirectory(DIR *dir, const Path & path)
{
DirEntries entries;
entries.reserve(64);
struct dirent * dirent;
while (errno = 0, dirent = readdir(dir)) { /* sic */
checkInterrupt();
std::string name = dirent->d_name;
if (name == "." || name == "..") continue;
entries.emplace_back(name, dirent->d_ino,
#ifdef HAVE_STRUCT_DIRENT_D_TYPE
dirent->d_type
#else
DT_UNKNOWN
#endif
);
}
if (errno) throw SysError("reading directory '%1%'", path);
return entries;
}
DirEntries readDirectory(const Path & path)
{
AutoCloseDir dir(opendir(path.c_str()));
if (!dir) throw SysError("opening directory '%1%'", path);
return readDirectory(dir.get(), path);
}
unsigned char getFileType(const Path & path)
{
struct stat st = lstat(path);
if (S_ISDIR(st.st_mode)) return DT_DIR;
if (S_ISLNK(st.st_mode)) return DT_LNK;
if (S_ISREG(st.st_mode)) return DT_REG;
return DT_UNKNOWN;
}
std::string readFile(const Path & path)
{
AutoCloseFD fd{open(path.c_str(), O_RDONLY | O_CLOEXEC)};
if (!fd)
throw SysError("opening file '%1%'", path);
return readFile(fd.get());
}
void readFile(const Path & path, Sink & sink)
{
AutoCloseFD fd{open(path.c_str(), O_RDONLY | O_CLOEXEC)};
if (!fd)
throw SysError("opening file '%s'", path);
drainFD(fd.get(), sink);
}
void writeFile(const Path & path, std::string_view s, mode_t mode, bool sync)
{
AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
if (!fd)
throw SysError("opening file '%1%'", path);
try {
writeFull(fd.get(), s);
} catch (Error & e) {
e.addTrace({}, "writing file '%1%'", path);
throw;
}
if (sync)
fd.fsync();
// Explicitly close to make sure exceptions are propagated.
fd.close();
if (sync)
syncParent(path);
}
void writeFile(const Path & path, Source & source, mode_t mode, bool sync)
{
AutoCloseFD fd{open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC, mode)};
if (!fd)
throw SysError("opening file '%1%'", path);
std::vector<char> buf(64 * 1024);
try {
while (true) {
try {
auto n = source.read(buf.data(), buf.size());
writeFull(fd.get(), {buf.data(), n});
} catch (EndOfFile &) { break; }
}
} catch (Error & e) {
e.addTrace({}, "writing file '%1%'", path);
throw;
}
if (sync)
fd.fsync();
// Explicitly close to make sure exceptions are propagated.
fd.close();
if (sync)
syncParent(path);
}
void syncParent(const Path & path)
{
AutoCloseFD fd{open(dirOf(path).c_str(), O_RDONLY, 0)};
if (!fd)
throw SysError("opening file '%1%'", path);
fd.fsync();
}
static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed)
{
checkInterrupt();
std::string name(baseNameOf(path));
struct stat st;
if (fstatat(parentfd, name.c_str(), &st, AT_SYMLINK_NOFOLLOW) == -1) {
if (errno == ENOENT) return;
throw SysError("getting status of '%1%'", path);
}
if (!S_ISDIR(st.st_mode)) {
/* We are about to delete a file. Will it likely free space? */
switch (st.st_nlink) {
/* Yes: last link. */
case 1:
bytesFreed += st.st_size;
break;
/* Maybe: yes, if 'auto-optimise-store' or manual optimisation
was performed. Instead of checking for real let's assume
it's an optimised file and space will be freed.
In worst case we will double count on freed space for files
with exactly two hardlinks for unoptimised packages.
*/
case 2:
bytesFreed += st.st_size;
break;
/* No: 3+ links. */
default:
break;
}
}
if (S_ISDIR(st.st_mode)) {
/* Make the directory accessible. */
const auto PERM_MASK = S_IRUSR | S_IWUSR | S_IXUSR;
if ((st.st_mode & PERM_MASK) != PERM_MASK) {
if (fchmodat(parentfd, name.c_str(), st.st_mode | PERM_MASK, 0) == -1)
throw SysError("chmod '%1%'", path);
}
int fd = openat(parentfd, path.c_str(), O_RDONLY);
if (fd == -1)
throw SysError("opening directory '%1%'", path);
AutoCloseDir dir(fdopendir(fd));
if (!dir)
throw SysError("opening directory '%1%'", path);
for (auto & i : readDirectory(dir.get(), path))
_deletePath(dirfd(dir.get()), path + "/" + i.name, bytesFreed);
}
int flags = S_ISDIR(st.st_mode) ? AT_REMOVEDIR : 0;
if (unlinkat(parentfd, name.c_str(), flags) == -1) {
if (errno == ENOENT) return;
throw SysError("cannot unlink '%1%'", path);
}
}
static void _deletePath(const Path & path, uint64_t & bytesFreed)
{
Path dir = dirOf(path);
if (dir == "")
dir = "/";
AutoCloseFD dirfd{open(dir.c_str(), O_RDONLY)};
if (!dirfd) {
if (errno == ENOENT) return;
throw SysError("opening directory '%1%'", path);
}
_deletePath(dirfd.get(), path, bytesFreed);
}
void deletePath(const Path & path)
{
uint64_t dummy;
deletePath(path, dummy);
}
void deletePath(const Path & path, uint64_t & bytesFreed)
{
//Activity act(*logger, lvlDebug, "recursively deleting path '%1%'", path);
bytesFreed = 0;
_deletePath(path, bytesFreed);
}
Paths createDirs(const Path & path)
{
Paths created;
if (path == "/") return created;
struct stat st;
if (lstat(path.c_str(), &st) == -1) {
created = createDirs(dirOf(path));
if (mkdir(path.c_str(), 0777) == -1 && errno != EEXIST)
throw SysError("creating directory '%1%'", path);
st = lstat(path);
created.push_back(path);
}
if (S_ISLNK(st.st_mode) && stat(path.c_str(), &st) == -1)
throw SysError("statting symlink '%1%'", path);
if (!S_ISDIR(st.st_mode)) throw Error("'%1%' is not a directory", path);
return created;
}
//////////////////////////////////////////////////////////////////////
AutoDelete::AutoDelete() : del{false} {}
AutoDelete::AutoDelete(const std::string & p, bool recursive) : path(p)
{
del = true;
this->recursive = recursive;
}
AutoDelete::~AutoDelete()
{
try {
if (del) {
if (recursive)
deletePath(path);
else {
if (remove(path.c_str()) == -1)
throw SysError("cannot unlink '%1%'", path);
}
}
} catch (...) {
ignoreException();
}
}
void AutoDelete::cancel()
{
del = false;
}
void AutoDelete::reset(const Path & p, bool recursive) {
path = p;
this->recursive = recursive;
del = true;
}
//////////////////////////////////////////////////////////////////////
static Path tempName(Path tmpRoot, const Path & prefix, bool includePid,
std::atomic<unsigned int> & counter)
{
tmpRoot = canonPath(tmpRoot.empty() ? getEnv("TMPDIR").value_or("/tmp") : tmpRoot, true);
if (includePid)
return fmt("%1%/%2%-%3%-%4%", tmpRoot, prefix, getpid(), counter++);
else
return fmt("%1%/%2%-%3%", tmpRoot, prefix, counter++);
}
Path createTempDir(const Path & tmpRoot, const Path & prefix,
bool includePid, bool useGlobalCounter, mode_t mode)
{
static std::atomic<unsigned int> globalCounter = 0;
std::atomic<unsigned int> localCounter = 0;
auto & counter(useGlobalCounter ? globalCounter : localCounter);
while (1) {
checkInterrupt();
Path tmpDir = tempName(tmpRoot, prefix, includePid, counter);
if (mkdir(tmpDir.c_str(), mode) == 0) {
#if __FreeBSD__
/* Explicitly set the group of the directory. This is to
work around around problems caused by BSD's group
ownership semantics (directories inherit the group of
the parent). For instance, the group of /tmp on
FreeBSD is "wheel", so all directories created in /tmp
will be owned by "wheel"; but if the user is not in
"wheel", then "tar" will fail to unpack archives that
have the setgid bit set on directories. */
if (chown(tmpDir.c_str(), (uid_t) -1, getegid()) != 0)
throw SysError("setting group of directory '%1%'", tmpDir);
#endif
return tmpDir;
}
if (errno != EEXIST)
throw SysError("creating directory '%1%'", tmpDir);
}
}
std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix)
{
Path tmpl(getEnv("TMPDIR").value_or("/tmp") + "/" + prefix + ".XXXXXX");
// Strictly speaking, this is UB, but who cares...
// FIXME: use O_TMPFILE.
AutoCloseFD fd(mkstemp((char *) tmpl.c_str()));
if (!fd)
throw SysError("creating temporary file '%s'", tmpl);
closeOnExec(fd.get());
return {std::move(fd), tmpl};
}
void createSymlink(const Path & target, const Path & link)
{
if (symlink(target.c_str(), link.c_str()))
throw SysError("creating symlink from '%1%' to '%2%'", link, target);
}
void replaceSymlink(const Path & target, const Path & link)
{
for (unsigned int n = 0; true; n++) {
Path tmp = canonPath(fmt("%s/.%d_%s", dirOf(link), n, baseNameOf(link)));
try {
createSymlink(target, tmp);
} catch (SysError & e) {
if (e.errNo == EEXIST) continue;
throw;
}
renameFile(tmp, link);
break;
}
}
void setWriteTime(const fs::path & p, const struct stat & st)
{
struct timeval times[2];
times[0] = {
.tv_sec = st.st_atime,
.tv_usec = 0,
};
times[1] = {
.tv_sec = st.st_mtime,
.tv_usec = 0,
};
if (lutimes(p.c_str(), times) != 0)
throw SysError("changing modification time of '%s'", p);
}
void copy(const fs::directory_entry & from, const fs::path & to, CopyFileFlags flags)
{
// TODO: Rewrite the `is_*` to use `symlink_status()`
auto statOfFrom = lstat(from.path().c_str());
auto fromStatus = from.symlink_status();
// Mark the directory as writable so that we can delete its children
if (flags.deleteAfter && fs::is_directory(fromStatus)) {
fs::permissions(from.path(), fs::perms::owner_write, fs::perm_options::add | fs::perm_options::nofollow);
}
if (fs::is_symlink(fromStatus) || fs::is_regular_file(fromStatus)) {
auto opts = fs::copy_options::overwrite_existing;
if (!flags.followSymlinks) {
opts |= fs::copy_options::copy_symlinks;
}
fs::copy(from.path(), to, opts);
} else if (fs::is_directory(fromStatus)) {
fs::create_directory(to);
for (auto & entry : fs::directory_iterator(from.path())) {
copy(entry, to / entry.path().filename(), flags);
}
} else {
throw Error("file '%s' has an unsupported type", from.path());
}
setWriteTime(to, statOfFrom);
if (flags.deleteAfter) {
if (!fs::is_symlink(fromStatus))
fs::permissions(from.path(), fs::perms::owner_write, fs::perm_options::add | fs::perm_options::nofollow);
fs::remove(from.path());
}
}
void copyFile(const Path & oldPath, const Path & newPath, CopyFileFlags flags)
{
return copy(fs::directory_entry(fs::path(oldPath)), fs::path(newPath), flags);
}
void renameFile(const Path & oldName, const Path & newName)
{
fs::rename(oldName, newName);
}
void moveFile(const Path & oldName, const Path & newName)
{
try {
renameFile(oldName, newName);
} catch (fs::filesystem_error & e) {
auto oldPath = fs::path(oldName);
auto newPath = fs::path(newName);
// For the move to be as atomic as possible, copy to a temporary
// directory
fs::path temp = createTempDir(newPath.parent_path(), "rename-tmp");
Finally removeTemp = [&]() { fs::remove(temp); };
auto tempCopyTarget = temp / "copy-target";
if (e.code().value() == EXDEV) {
fs::remove(newPath);
warn("Cant rename %s as %s, copying instead", oldName, newName);
copy(fs::directory_entry(oldPath), tempCopyTarget, { .deleteAfter = true });
renameFile(tempCopyTarget, newPath);
}
}
}
}

View file

@ -1,269 +0,0 @@
#pragma once
/**
* @file
*
* Utiltities for working with the file sytem and file paths.
*/
#include "types.hh"
#include "file-descriptor.hh"
#include <sys/types.h>
#include <sys/stat.h>
#include <dirent.h>
#include <unistd.h>
#include <boost/lexical_cast.hpp>
#include <functional>
#include <optional>
#ifndef HAVE_STRUCT_DIRENT_D_TYPE
#define DT_UNKNOWN 0
#define DT_REG 1
#define DT_LNK 2
#define DT_DIR 3
#endif
namespace nix {
struct Sink;
struct Source;
/**
* @return An absolutized path, resolving paths relative to the
* specified directory, or the current directory otherwise. The path
* is also canonicalised.
*/
Path absPath(Path path,
std::optional<PathView> dir = {},
bool resolveSymlinks = false);
/**
* Canonicalise a path by removing all `.` or `..` components and
* double or trailing slashes. Optionally resolves all symlink
* components such that each component of the resulting path is *not*
* a symbolic link.
*/
Path canonPath(PathView path, bool resolveSymlinks = false);
/**
* Change the permissions of a path
* Not called `chmod` as it shadows and could be confused with
* `int chmod(char *, mode_t)`, which does not handle errors
*/
void chmodPath(const Path & path, mode_t mode);
/**
* @return The directory part of the given canonical path, i.e.,
* everything before the final `/`. If the path is the root or an
* immediate child thereof (e.g., `/foo`), this means `/`
* is returned.
*/
Path dirOf(const PathView path);
/**
* @return the base name of the given canonical path, i.e., everything
* following the final `/` (trailing slashes are removed).
*/
std::string_view baseNameOf(std::string_view path);
/**
* Perform tilde expansion on a path.
*/
std::string expandTilde(std::string_view path);
/**
* Check whether 'path' is a descendant of 'dir'. Both paths must be
* canonicalized.
*/
bool isInDir(std::string_view path, std::string_view dir);
/**
* Check whether 'path' is equal to 'dir' or a descendant of
* 'dir'. Both paths must be canonicalized.
*/
bool isDirOrInDir(std::string_view path, std::string_view dir);
/**
* Get status of `path`.
*/
struct stat stat(const Path & path);
struct stat lstat(const Path & path);
/**
* `lstat` the given path if it exists.
* @return std::nullopt if the path doesn't exist, or an optional containing the result of `lstat` otherwise
*/
std::optional<struct stat> maybeLstat(const Path & path);
/**
* @return true iff the given path exists.
*/
bool pathExists(const Path & path);
/**
* A version of pathExists that returns false on a permission error.
* Useful for inferring default paths across directories that might not
* be readable.
* @return true iff the given path can be accessed and exists
*/
bool pathAccessible(const Path & path);
/**
* Read the contents (target) of a symbolic link. The result is not
* in any way canonicalised.
*/
Path readLink(const Path & path);
bool isLink(const Path & path);
/**
* Read the contents of a directory. The entries `.` and `..` are
* removed.
*/
struct DirEntry
{
std::string name;
ino_t ino;
/**
* one of DT_*
*/
unsigned char type;
DirEntry(std::string name, ino_t ino, unsigned char type)
: name(std::move(name)), ino(ino), type(type) { }
};
typedef std::vector<DirEntry> DirEntries;
DirEntries readDirectory(const Path & path);
unsigned char getFileType(const Path & path);
/**
* Read the contents of a file into a string.
*/
std::string readFile(const Path & path);
void readFile(const Path & path, Sink & sink);
/**
* Write a string to a file.
*/
void writeFile(const Path & path, std::string_view s, mode_t mode = 0666, bool sync = false);
void writeFile(const Path & path, Source & source, mode_t mode = 0666, bool sync = false);
/**
* Flush a file's parent directory to disk
*/
void syncParent(const Path & path);
/**
* Delete a path; i.e., in the case of a directory, it is deleted
* recursively. It's not an error if the path does not exist. The
* second variant returns the number of bytes and blocks freed.
*/
void deletePath(const Path & path);
void deletePath(const Path & path, uint64_t & bytesFreed);
/**
* Create a directory and all its parents, if necessary. Returns the
* list of created directories, in order of creation.
*/
Paths createDirs(const Path & path);
inline Paths createDirs(PathView path)
{
return createDirs(Path(path));
}
/**
* Create a symlink.
*/
void createSymlink(const Path & target, const Path & link);
/**
* Atomically create or replace a symlink.
*/
void replaceSymlink(const Path & target, const Path & link);
void renameFile(const Path & src, const Path & dst);
/**
* Similar to 'renameFile', but fallback to a copy+remove if `src` and `dst`
* are on a different filesystem.
*
* Beware that this might not be atomic because of the copy that happens behind
* the scenes
*/
void moveFile(const Path & src, const Path & dst);
struct CopyFileFlags
{
/**
* Delete the file after copying.
*/
bool deleteAfter = false;
/**
* Follow symlinks and copy the eventual target.
*/
bool followSymlinks = false;
};
/**
* Recursively copy the content of `oldPath` to `newPath`. If `andDelete` is
* `true`, then also remove `oldPath` (making this equivalent to `moveFile`, but
* with the guaranty that the destination will be fresh, with no stale inode
* or file descriptor pointing to it).
*/
void copyFile(const Path & oldPath, const Path & newPath, CopyFileFlags flags);
/**
* Automatic cleanup of resources.
*/
class AutoDelete
{
Path path;
bool del;
bool recursive;
public:
AutoDelete();
AutoDelete(const Path & p, bool recursive = true);
~AutoDelete();
void cancel();
void reset(const Path & p, bool recursive = true);
operator Path() const { return path; }
operator PathView() const { return path; }
};
struct DIRDeleter
{
void operator()(DIR * dir) const {
closedir(dir);
}
};
typedef std::unique_ptr<DIR, DIRDeleter> AutoCloseDir;
/**
* Create a temporary directory.
*/
Path createTempDir(const Path & tmpRoot = "", const Path & prefix = "nix",
bool includePid = true, bool useGlobalCounter = true, mode_t mode = 0755);
/**
* Create a temporary file, returning a file handle and its path.
*/
std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix = "nix");
/**
* Used in various places.
*/
typedef std::function<bool(const Path & path)> PathFilter;
extern PathFilter defaultPathFilter;
}

175
src/libutil/filesystem.cc Normal file
View file

@ -0,0 +1,175 @@
#include <sys/time.h>
#include <filesystem>
#include <atomic>
#include "finally.hh"
#include "util.hh"
#include "signals.hh"
#include "types.hh"
namespace fs = std::filesystem;
namespace nix {
static Path tempName(Path tmpRoot, const Path & prefix, bool includePid,
std::atomic<unsigned int> & counter)
{
tmpRoot = canonPath(tmpRoot.empty() ? getEnv("TMPDIR").value_or("/tmp") : tmpRoot, true);
if (includePid)
return fmt("%1%/%2%-%3%-%4%", tmpRoot, prefix, getpid(), counter++);
else
return fmt("%1%/%2%-%3%", tmpRoot, prefix, counter++);
}
Path createTempDir(const Path & tmpRoot, const Path & prefix,
bool includePid, bool useGlobalCounter, mode_t mode)
{
static std::atomic<unsigned int> globalCounter = 0;
std::atomic<unsigned int> localCounter = 0;
auto & counter(useGlobalCounter ? globalCounter : localCounter);
while (1) {
checkInterrupt();
Path tmpDir = tempName(tmpRoot, prefix, includePid, counter);
if (mkdir(tmpDir.c_str(), mode) == 0) {
#if __FreeBSD__
/* Explicitly set the group of the directory. This is to
work around around problems caused by BSD's group
ownership semantics (directories inherit the group of
the parent). For instance, the group of /tmp on
FreeBSD is "wheel", so all directories created in /tmp
will be owned by "wheel"; but if the user is not in
"wheel", then "tar" will fail to unpack archives that
have the setgid bit set on directories. */
if (chown(tmpDir.c_str(), (uid_t) -1, getegid()) != 0)
throw SysError("setting group of directory '%1%'", tmpDir);
#endif
return tmpDir;
}
if (errno != EEXIST)
throw SysError("creating directory '%1%'", tmpDir);
}
}
std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix)
{
Path tmpl(getEnv("TMPDIR").value_or("/tmp") + "/" + prefix + ".XXXXXX");
// Strictly speaking, this is UB, but who cares...
// FIXME: use O_TMPFILE.
AutoCloseFD fd(mkstemp((char *) tmpl.c_str()));
if (!fd)
throw SysError("creating temporary file '%s'", tmpl);
closeOnExec(fd.get());
return {std::move(fd), tmpl};
}
void createSymlink(const Path & target, const Path & link)
{
if (symlink(target.c_str(), link.c_str()))
throw SysError("creating symlink from '%1%' to '%2%'", link, target);
}
void replaceSymlink(const Path & target, const Path & link)
{
for (unsigned int n = 0; true; n++) {
Path tmp = canonPath(fmt("%s/.%d_%s", dirOf(link), n, baseNameOf(link)));
try {
createSymlink(target, tmp);
} catch (SysError & e) {
if (e.errNo == EEXIST) continue;
throw;
}
renameFile(tmp, link);
break;
}
}
void setWriteTime(const fs::path & p, const struct stat & st)
{
struct timeval times[2];
times[0] = {
.tv_sec = st.st_atime,
.tv_usec = 0,
};
times[1] = {
.tv_sec = st.st_mtime,
.tv_usec = 0,
};
if (lutimes(p.c_str(), times) != 0)
throw SysError("changing modification time of '%s'", p);
}
void copy(const fs::directory_entry & from, const fs::path & to, CopyFileFlags flags)
{
// TODO: Rewrite the `is_*` to use `symlink_status()`
auto statOfFrom = lstat(from.path().c_str());
auto fromStatus = from.symlink_status();
// Mark the directory as writable so that we can delete its children
if (flags.deleteAfter && fs::is_directory(fromStatus)) {
fs::permissions(from.path(), fs::perms::owner_write, fs::perm_options::add | fs::perm_options::nofollow);
}
if (fs::is_symlink(fromStatus) || fs::is_regular_file(fromStatus)) {
auto opts = fs::copy_options::overwrite_existing;
if (!flags.followSymlinks) {
opts |= fs::copy_options::copy_symlinks;
}
fs::copy(from.path(), to, opts);
} else if (fs::is_directory(fromStatus)) {
fs::create_directory(to);
for (auto & entry : fs::directory_iterator(from.path())) {
copy(entry, to / entry.path().filename(), flags);
}
} else {
throw Error("file '%s' has an unsupported type", from.path());
}
setWriteTime(to, statOfFrom);
if (flags.deleteAfter) {
if (!fs::is_symlink(fromStatus))
fs::permissions(from.path(), fs::perms::owner_write, fs::perm_options::add | fs::perm_options::nofollow);
fs::remove(from.path());
}
}
void copyFile(const Path & oldPath, const Path & newPath, CopyFileFlags flags)
{
return copy(fs::directory_entry(fs::path(oldPath)), fs::path(newPath), flags);
}
void renameFile(const Path & oldName, const Path & newName)
{
fs::rename(oldName, newName);
}
void moveFile(const Path & oldName, const Path & newName)
{
try {
renameFile(oldName, newName);
} catch (fs::filesystem_error & e) {
auto oldPath = fs::path(oldName);
auto newPath = fs::path(newName);
// For the move to be as atomic as possible, copy to a temporary
// directory
fs::path temp = createTempDir(newPath.parent_path(), "rename-tmp");
Finally removeTemp = [&]() { fs::remove(temp); };
auto tempCopyTarget = temp / "copy-target";
if (e.code().value() == EXDEV) {
fs::remove(newPath);
warn("Cant rename %s as %s, copying instead", oldName, newName);
copy(fs::directory_entry(oldPath), tempCopyTarget, { .deleteAfter = true });
renameFile(tempCopyTarget, newPath);
}
}
}
}

Some files were not shown because too many files have changed in this diff Show more