summaryrefslogtreecommitdiff
path: root/nix/libstore/build.cc
AgeCommit message (Expand)Author
2020-12-19daemon: Delegate deduplication to 'guix substitute'....This removes the main source of latency between subsequent downloads. * nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a "deduplicate" key to ENV. (SubstitutionGoal::finished): Remove call to 'optimisePath'. * guix/scripts/substitute.scm (process-substitution)[destination-in-store?] [dump-file/deduplicate*]: New variables. Pass #:dump-file to 'restore-file'. * guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New variable. Pass #:deduplicate? to 'process-substitution'. * guix/serialization.scm (dump-file): Export and augment 'dump-file'. Ludovic Courtès
2020-12-19daemon: Do not reset timestamps and permissions on substituted items....'guix substitute' now takes care of it via 'restore-file'. * nix/libstore/build.cc (SubstitutionGoal::finished): Remove call to 'canonicalisePathMetaData'. Ludovic Courtès
2020-12-19daemon: Let 'guix substitute' perform hash checks....This way, the hash of the store item can be computed as it is restored, thereby avoiding an additional file tree traversal ('hashPath' call) later on in the daemon. Consequently, it should reduce latency between subsequent substitute downloads. This is a followup to 5ff521452b9ec2aae9ed8e4bb7bdc250a581f203. * guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New procedure. (process-substitution): Wrap INPUT into a hash input port, 'hashed', and read from it. Compare the actual and expected hashes, and print a "hash-mismatch" status line when they differ. When they match, print not just "success" but also the nar hash and size. * nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]: Remove. (SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle "success" and "hash-mismatch" accordingly. Call 'hashPath' only when the returned hash is not SHA256. (SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr' handling. * tests/substitute.scm ("substitute, invalid hash"): Rename to... ("substitute, invalid narinfo hash"): ... this. ("substitute, invalid hash"): New test. Ludovic Courtès
2020-12-08daemon: Raise an error if substituter doesn't send the expected hash....It was already impossible in practice for 'expectedHashStr' to be empty if 'status' == "success". * nix/libstore/build.cc (SubstitutionGoal::finished): Throw 'SubstError' when 'expectedHashStr' is empty. Ludovic Courtès
2020-12-08substitute: Cache and reuse connections while substituting....That way, when fetching a series of substitutes from the same server(s), the connection is reused instead of being closed/opened for each substitutes, which saves on network round trips and TLS handshakes. * guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it. * guix/progress.scm (progress-report-port): Add #:close? parameter and honor it. * guix/scripts/substitute.scm (at-most): Return the tail as a second value. (fetch): Add #:port and #:keep-alive? and honor them. (%max-cached-connections): New variable. (open-connection-for-uri/cached, call-with-cached-connection): New procedures. (with-cached-connection): New macro. (process-substitution): Wrap 'fetch' call in 'with-cached-connection'. Pass #:close? to 'progress-report-port'. Ludovic Courtès
2020-12-08daemon: Run 'guix substitute --substitute' as an agent....This avoids spawning one substitute process per substitution. * nix/libstore/build.cc (class Worker)[substituter]: New field. [outPipe, logPipe, pid]: Remove. (class SubstitutionGoal)[expectedHashStr, status, substituter]: New fields. (SubstitutionGoal::timedOut): Adjust to check 'substituter'. (SubstitutionGoal::tryToRun): Remove references to 'outPipe' and 'logPipe'. Run "guix substitute --substitute" as an 'Agent'. Send the request with 'writeLine'. (SubstitutionGoal::finished): Likewise. (SubstitutionGoal::handleChildOutput): Change to fill in 'expectedHashStr' and 'status'. (SubstitutionGoal::handleEOF): Call 'wakeUp' unconditionally. (SubstitutionGoal::~SubstitutionGoal): Adjust to check 'substituter'. * guix/scripts/substitute.scm (process-substitution): Write "success\n" to stdout upon success. (%error-to-file-descriptor-4?): New variable. (guix-substitute): Set 'current-error-port' to file descriptor 4 unless (%error-to-file-descriptor-4?) is false. Remove "--substitute" arguments. Loop reading line from stdin. * tests/substitute.scm <top level>: Call '%error-to-file-descriptor-4?'. (request-substitution): New procedure. ("substitute, no signature") ("substitute, invalid hash") ("substitute, unauthorized key") ("substitute, authorized key") ("substitute, unauthorized narinfo comes first") ("substitute, unsigned narinfo comes first") ("substitute, first narinfo is unsigned and has wrong hash") ("substitute, first narinfo is unsigned and has wrong refs") ("substitute, two invalid narinfos") ("substitute, narinfo with several URLs"): Adjust to new "guix substitute --substitute" calling convention. Ludovic Courtès
2020-12-08daemon: Use 'Agent' to spawn 'guix substitute --query'....* nix/libstore/local-store.hh (RunningSubstituter): Remove. (LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>. [setSubstituterEnv, didSetSubstituterEnv]: Remove. [getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'. * nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove reference to 'runningSubstituter'. (LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove. (LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'. (LocalStore::querySubstitutablePaths): Spawn substituter agent if needed. Adjust to 'Agent' interface. (LocalStore::querySubstitutablePathInfos): Likewise. * nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to 'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead. (SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'. * guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?): Remove second argument to 'make-parameter'. (process-query): Call 'warn-about-missing-authentication' when (%allow-unauthenticated-substitutes?) is #t. (guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port' too. No longer exit when 'substitute-urls' returns the empty list. No longer print newline initially. * tests/substitute.scm (test-quit): Parameterize 'current-error-port' to account for the port changes in 'guix-substitute'. Ludovic Courtès
2020-12-01daemon: Remove unneeded forward declaration....This is a followup to ee9dff34f9317509cb2b833d07a0d5e01a36a4ae. * nix/libstore/build.cc: Remove 'struct Agent' forward declaration. Ludovic Courtès
2020-11-29daemon: Remove pre-Guix hack....* nix/libstore/build.cc (DerivationGoal::startBuilder): Remove "NIX_OUTPUT_CHECKED" hack. Ludovic Courtès
2020-10-09nix: Honor '--rounds' when also using '--check'....Fixes <https://issues.guix.gnu.org/40144>. Until now, the '--rounds' option, when also using '--check', was ignored. This change makes it possible to use both, so that an item that has already been built once can be rebuilt as many times as desired. * nix/libstore/build.cc: Remove the conditionals causing the daemon to complete a build task early when 'buildMode' is equal to 'nix::bmCheck'. Reported-by: Brice Waegeneire <brice@waegenei.re> Maxim Cournoyer
2020-10-01daemon: Try to execute derivation builders only for matching OS kernels....Fixes <https://bugs.gnu.org/43668>. Previously, guix-daemon would try to run GNU/Hurd executables on GNU/Linux. execve(2) would succeed, but the executable would immediately crash. This change prevents it from attempting to execute "i586-gnu" code on "*-linux", while preserving the binfmt_misc-friendly behavior implemented in commit 7bf2a70a4ffd976d50638d3b9f2ec409763157df. * nix/libstore/build.cc (sameOperatingSystemKernel): New function. (DerivationGoal::runChild): Call 'execve' only when 'sameOperatingSystemKernel' returns true. Ludovic Courtès
2020-09-14daemon: Move 'Agent' to libutil....* nix/libstore/build.cc (DerivationGoal::tryBuildHook): Add "offload" to 'args' and pass settings.guixProgram as the first argument to Agent::Agent. (pathNullDevice, commonChildInit, Agent, Agent::Agent) (Agent::~Agent): Move to... * nix/libutil/util.cc: ... here. * nix/libutil/util.hh (struct Agent, commonChildInit): New declarations. Ludovic Courtès
2020-09-14daemon: Generalize 'HookInstance' to 'Agent'....* nix/libstore/build.cc (HookInstance): Rename to... (Agent): ... this. Rename 'toHook' and 'fromHook' similarly and update users. Change constructor to require a command and an argument list. (DerivationGoal::tryBuildHook): Pass arguments to the 'Agent' constructor. Ludovic Courtès
2020-03-26daemon: Do not use clone on the Hurd....Checking for CLONE_NEWNS is only needed for using tha Linux specific clone(2), otherwise we can use fork(2). Using clone on the Hurd needs some work, only support LINUX for now. See https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00190.html * nix/libstore/build.cc (CHROOT_ENABLED): Break into CHROOT_ENABLED and CLONE_ENABLED. (DerivationGoal::startBuilder): Replace CHROOT_ENABLED with __linux__. (DerivationGoal::runChild): Only define pivot_root() if SYS_pivot_root is defined. Co-authored-by: Jan Nieuwenhuizen <janneke@gnu.org> Manolis Ragkousis
2019-11-13daemon: Don't include <linux/fs.h>....As of GNU libc 2.29, <sys/mount.h> declares all the constants and functions we need, so there's no use in including <linux/fs.h> anymore. This silences annoying warnings like this one: In file included from nix/libstore/local-store.cc:32:0: /gnu/store/…-linux-libre-headers-4.19.56/include/linux/fs.h:108:0: warning: "MS_RDONLY" redefined #define MS_RDONLY 1 /* Mount read-only */ In file included from nix/libstore/local-store.cc:28:0: /gnu/store/…-glibc-2.29/include/sys/mount.h:36:0: note: this is the location of the previous definition #define MS_RDONLY MS_RDONLY * config-daemon.ac: Remove check for <linux/fs.h>. * nix/libstore/build.cc: Remove conditional inclusion of <linux/fs.h>. * nix/libstore/local-store.cc: Remove "#if HAVE_LINUX_FS_H" and inclusion of <linux/fs.h>. Ludovic Courtès
2019-11-04daemon: Unregister build hook from the worker's children upon build failure....Fixes <https://bugs.gnu.org/38062>. This is a followup to ada9a19a2dca74feafcf24df1152abd685d4142f. * nix/libstore/build.cc (DerivationGoal::killChild): Add conditional call to 'worker.childTerminated' for 'hook->pid'. Ludovic Courtès
2019-09-28daemon: Strictly respect timeouts for 'guix offload'....Until now it was up to 'guix offload' to honor timeouts. Unfortunately it would sometimes fail to do that, for example due to the libssh bug at <https://bugs.libssh.org/T33>. With this change, 'guix offload' is automatically killed by the daemon when one of the timeouts expires. Thus, data transfers performed by 'guix offload' now count as part of the timeouts, rather than just actual build time. * nix/libstore/build.cc (DerivationGoal::tryBuildHook): Pass true as the 'respectTimeouts' argument to 'childStarted'. Ludovic Courtès
2019-09-08daemon: Run 'guix substitute' directly and assume a single substituter....The daemon had a mechanism that allows it to handle a list of substituters and try them sequentially; this removes it. * nix/scripts/substitute.in: Remove. * nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove. * config-daemon.ac: Don't output 'nix/scripts/substitute'. * nix/libstore/build.cc (SubstitutionGoal)[subs, sub, hasSubstitute]: Remove. [tryNext]: Make private. (SubstitutionGoal::SubstitutionGoal, SubstitutionGoal::init): Remove now unneeded initializers. (SubstitutionGoal::tryNext): Adjust to assume a single substituter: call 'amDone' upfront when we couldn't find substitutes. (SubstitutionGoal::tryToRun): Adjust to run 'guix substitute' via 'settings.guixProgram'. (SubstitutionGoal::finished): Call 'amDone(ecFailed)' upon failure instead of setting 'state' to 'tryNext'. * nix/libstore/globals.hh (Settings)[substituters]: Remove. * nix/libstore/local-store.cc (LocalStore::~LocalStore): Adjust to handle a single substituter. (LocalStore::startSubstituter): Remove 'path' parameter. Adjust to invoke 'settings.guixProgram'. Don't refer to 'run.program', which no longer exists. (LocalStore::querySubstitutablePaths): Adjust for 'runningSubstituters' being a singleton instead of a list. (LocalStore::querySubstitutablePathInfos): Likewise, and remove 'substituter' parameter. * nix/libstore/local-store.hh (RunningSubstituter)[program]: Remove. (LocalStore)[runningSubstituters]: Remove. [runningSubstituter]: New field. [querySubstitutablePathInfos]: Remove 'substituter' parameter. [startSubstituter]: Remove 'substituter' parameter. * nix/nix-daemon/guix-daemon.cc (main): Remove references to 'settings.substituters'. * nix/nix-daemon/nix-daemon.cc (performOp): Ignore the user's "build-use-substitutes" value when 'settings.useSubstitutes' is false. Ludovic Courtès
2019-09-08daemon: Run 'guix offload' directly....* nix/scripts/offload.in: Remove. * nix/local.mk (nodist_pkglibexec_SCRIPTS) [BUILD_DAEMON_OFFLOAD]: Remove 'scripts/offload'. * config-daemon.ac: Don't output 'nix/scripts/offload'. * build-aux/pre-inst-env.in: Don't set 'NIX_BUILD_HOOK'. * nix/libstore/build.cc (HookInstance::HookInstance): Run 'guix offload'. (DerivationGoal::tryBuildHook): Remove reference to 'NIX_BUILD_HOOK'. * nix/nix-daemon/guix-daemon.cc (main) [HAVE_DAEMON_OFFLOAD_HOOK]: Don't set 'NIX_BUILD_HOOK'. * nix/nix-daemon/nix-daemon.cc (performOp) [!HAVE_DAEMON_OFFLOAD_HOOK]: Leave 'settings.useBuildHook' unchanged. Ludovic Courtès
2019-06-13daemon: Replace "illegal" by "invalid" in error messages....* nix/libstore/build.cc (parseReferenceSpecifiers): Replace "illegal" by "invalid". * nix/libstore/globals.cc (Settings::pack): Likewise. * nix/libstore/store-api.cc (checkStoreName): Likewise. Ludovic Courtès
2019-02-06daemon: Emit a 'build-succeeded' event in check mode....Until now, something like "guix build sed -v1 --check" would not get a 'build-succeeded' event, which in turn meant that the spinner would not be erased upon build completion. * nix/libstore/build.cc (DerivationGoal::registerOutputs): When 'buildMode' is bmCheck and 'settings.printBuildTrace' emit a "@ build-succeeded" trace upon success. * tests/store.scm ("build-succeeded trace in check mode"): New test. Ludovic Courtès
2018-12-16daemon: Use unbranded phrases in comments and messages....* nix/libstore/build.cc, nix/libstore/globals.cc, nix/libstore/gc.cc, nix/libstore/local-store.cc, nix/libstore/optimise-store.cc, nix/libstore/store-api.cc, nix/libutil/archive.cc, nix/nix-daemon/nix-daemon.cc: Replace "Nix store" by "store", and "Nix daemon" by "build daemon". Ludovic Courtès
2018-10-15daemon: Support multiplexed build output....This allows clients to tell whether output comes from the daemon or, if it comes from a builder, from which builder it comes. The latter is particularly useful when MAX-BUILD-JOBS > 1. * nix/libstore/build.cc (DerivationGoal::tryBuildHook) (DerivationGoal::startBuilder): Print the child's PID in "@ build-started" traces. (DerivationGoal::handleChildOutput): Define 'prefix', pass it to 'writeToStderr'. * nix/libstore/globals.cc (Settings:Settings): Initialize 'multiplexedBuildOutput'. (Settings::update): Likewise. * nix/libstore/globals.hh (Settings)[multiplexedBuildOutput]: New field. Update 'printBuildTrace' documentation. * nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0.163. * nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: Special-case "multiplexed-build-output" and remove "use-ssh-substituter". * guix/store.scm (set-build-options): Add #:multiplexed-build-output? and honor it. (%protocol-version): Bump to #x163. * tests/store.scm ("multiplexed-build-output"): New test. fixlet Ludovic Courtès
2018-09-27Add (guix status) and use it for pretty colored output....* guix/progress.scm (progress-reporter/trace): New procedure. (%progress-interval): New variable. (progress-reporter/file): Use it. * guix/scripts/build.scm (set-build-options-from-command-line): Pass #:print-extended-build-trace?. (%default-options): Add 'print-extended-build-trace?'. (guix-build): Parameterize CURRENT-TERMINAL-COLUMNS. Use 'build-status-updater'. * guix/scripts/environment.scm (%default-options): Add 'print-extended-build-trace?'. (guix-environment): Wrap body in 'with-status-report'. * guix/scripts/pack.scm (%default-options): Add 'print-build-trace?' and 'print-extended-build-trace?'. (guix-pack): Wrap body in 'with-status-report'. * guix/scripts/package.scm (%default-options, guix-package): Likewise. * guix/scripts/system.scm (%default-options, guix-system): Likewise. * guix/scripts/pull.scm (%default-options, guix-pull): Likewise. * guix/scripts/substitute.scm (progress-report-port): Don't call STOP when TOTAL is zero. (process-substitution): Add #:print-build-trace? and honor it. (guix-substitute)[print-build-trace?]: New variable. Pass #:print-build-trace? to 'process-substitution'. * guix/status.scm: New file. * guix/store.scm (set-build-options): Add #:print-extended-build-trace?; pass it into PAIRS. (%protocol-version): Bump. (protocol-version, nix-server-version): New procedures. (current-store-protocol-version): New variable. (with-store, build-things): Parameterize it. * guix/ui.scm (build-output-port): Remove. (colorize-string): Export. * po/guix/POTFILES.in: Add guix/status.scm. * tests/status.scm: New file. * Makefile.am (SCM_TESTS): Add it. * nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x162. * nix/libstore/build.cc (DerivationGoal::registerOutputs) (SubstitutionGoal::finished): Print a "@ hash-mismatch" trace before throwing. Ludovic Courtès
2018-05-17daemon: Allow building for armhf-linux on aarch64-linux....* nix/libstore/build.cc (canBuildLocally): Allow building armhf-linux builds on aarch64-linux. (DerivationGoal::runChild) Throw error if attempting to build for armhf-linux on an unsupported platform. * doc/guix.texi (Invoking guix build): Document how to build natively for armhf-linux on aarch64-linux. Add note that on some aarch64 machines this is unsupported. Efraim Flashner
2018-01-11daemon: Always try to execute the builder regardless of the platform....* nix/libstore/build.cc (runChild): Move platform check after 'execve' call. Check specifically for ENOEXEC. Ludovic Courtès
2018-01-07daemon: Make libbz2 an optional dependency....* config-daemon.ac: Don't bail out when libbz2 is missing. Define 'HAVE_LIBBZ2' Automake conditional. * nix/libstore/build.cc: Wrap relevant bits in '#if HAVE_BZLIB_H'. * nix/libstore/globals.cc (Settings::Settings): 'logCompression' defaults to COMPRESSION_GZIP when HAVE_BZLIB_H is false. * nix/libstore/globals.hh (CompressionType): Make 'COMPRESSION_BZIP2' conditional on HAVE_BZLIB_H. * nix/local.mk (guix_register_LDADD, guix_daemon_LDADD): Add -lbz2 only when HAVE_LIBBZ2. * nix/nix-daemon/guix-daemon.cc (parse_opt): Ignore "bzip2" when not HAVE_BZLIB_H. Ludovic Courtès
2018-01-07daemon: Add gzip log compression....* nix/nix-daemon/guix-daemon.cc (GUIX_OPT_LOG_COMPRESSION): New macro. (options): Mark "disable-log-compression" as hidden and add "log-compression". (parse_opt): Handle GUIX_OPT_LOG_COMPRESSION. * nix/libstore/build.cc (DerivationGoal): Add 'gzLogFile'. (openLogFile): Initialize it when 'logCompression' is COMPRESSION_GZIP. (closeLogFile, handleChildOutput): Honor 'gzLogFile'. * nix/libstore/globals.hh (Settings)[compressLog]: Remove. [logCompression]: New field. (CompressionType): New enum. * nix/libstore/globals.cc (Settings::Settings): Initialize it. (update): Remove '_get' call for 'compressLog'. * nix/local.mk (guix_daemon_LDADD, guix_register_LDADD): Add -lz. * guix/store.scm (log-file): Handle '.gz' log files. * tests/guix-daemon.sh: Add test with '--log-compression=gzip'. * doc/guix.texi (Invoking guix-daemon): Adjust accordingly. * config-daemon.ac: Check for libz and zlib.h. Ludovic Courtès
2017-08-18Simplify remouting with MS_PRIVATE in sandbox build...also fix race condition if mounts are added after mountinfo is read. Jörg Thalheim
2017-08-07daemon: Ensure the child stack is aligned on a 16-byte boundary....* nix/libstore/build.cc (DerivationGoal::startBuilder): When calling 'clone', ensure that the stack is aligned on a 16-byte boundary. Mark H Weaver
2017-08-05daemon: On aarch64, use increments of 16 on the stack....* nix/libstore/build.cc (DerivationGoal::startBuilder): When on aarch64, when calling clone(), increment the stack by 16. Efraim Flashner
2017-08-05Revert "daemon: Ensure proper alignment on the stack."...This reverts commit ab4ccc8fcfaf2215d4b33b1376147e4c2c70426a. Mark H Weaver
2017-05-30daemon: Report hash mismatches in a cleaner way....Suggested by Hartmut Goebel. * nix/libstore/build.cc (DerivationGoal::registerOutputs): Report hash mismatches on 3 lines for clarity. (SubstitutionGoal::finished): Likewise. Ludovic Courtès
2017-02-20daemon: Ensure proper alignment on the stack....* nix/libstore/build.cc (startBuilder): When calling 'clone', increase the step to 16 and ensure it aligns properly on the stack. Efraim Flashner
2017-01-11daemon: Allow check builds of 'builtin:download' derivations....Fixes <http://bugs.gnu.org/25089>. Reported by Leo Famulari <leo@famulari.name>. * nix/libstore/build.cc (DerivationGoal::runChild): In the 'isBuiltin' case, check whether DRV's output is in 'redirectedOutputs', and pass an 'output' argument to the built-in builder. (DerivationGoal::addHashRewrite): Add 'printMsg' call. * nix/libstore/builtins.hh (derivationBuilder): Add 'output' parameter. * nix/libstore/builtins.cc (builtinDownload): Likewise. Add OUTPUT to ARGV. * guix/scripts/perform-download.scm (perform-download): Add 'output' parameter. (guix-perform-download): Adjust 'match' clauses accordingly. * tests/derivations.scm ("'download' built-in builder, check mode"): New test. Ludovic Courtès
2016-12-29daemon: Allow fixed-output derivation builds with TMPDIR set....Fixes <http://bugs.gnu.org/25242>. Reported by Leo Famulari <leo@famulari.name>. The regression was introduced in 94d92c7796a3dd50c27d532315f7d497ac99f08e. * nix/libstore/build.cc (DerivationGoal::startBuilder): Set 'useChroot' as a function 'of isBuiltin(drv)'. (DerivationGoal::runChild): Use 'useChroot' instead of 'useChroot && !isBuiltin(drv)'. Ludovic Courtès
2016-12-09daemon: Set ownership of kept build directories to the calling user....Fixes <http://bugs.gnu.org/15890>. * nix/libstore/globals.hh (Settings) Add clientUid and clientGid. * nix/nix-daemon/nix-daemon.cc (daemonLoop] Store UID and GID of the caller in settings. * nix/libstore/build.cc (_chown): New function. (DerivationGoal::deleteTmpDir): Use it, change ownership of build directory if it is kept and the new owner is not root. Hartmut Goebel
2016-11-16daemon: Add "builtin:download" derivation builder....This ensures that 1) the derivation doesn't change when Guix changes; 2) the derivation closure doesn't contain Guix and its dependencies; 3) we don't have to rely on ugly chroot hacks. Adapted from Nix commit 0a2bee307b20411f5b0dda0c662b1f9bb9e0e131. * nix/libstore/build.cc (DerivationGoal::runChild): Add special case for 'isBuiltin(drv)'. Disable chroot when 'isBuiltin(drv)'. * nix/libstore/builtins.cc, nix/libstore/builtins.hh, nix/scripts/download.in, guix/scripts/perform-download.scm: New files. * guix/ui.scm (show-guix-help)[internal?]: Add 'perform-download'. * nix/local.mk (libstore_a_SOURCES): Add builtins.cc. (libstore_headers): Add builtins.hh. (nodist_pkglibexec_SCRIPTS): Add 'scripts/download'. * config-daemon.ac: Emit 'scripts/download'. * Makefile.am (MODULES): Add 'guix/scripts/perform-download.scm'. * tests/derivations.scm ("unknown built-in builder") ("'download' built-in builder") ("'download' built-in builder, invalid hash") ("'download' built-in builder, not found") ("'download' built-in builder, not fixed-output"): New tests. Co-authored-by: Eelco Dolstra <eelco.dolstra@logicblox.com> Ludovic Courtès
2016-05-31daemon: Fix typo....Fixes a regression/typo introduced in e08380fb6cefd3fd67c3c220a3ddaf385e6413cf. * nix/libstore/build.cc (DerivationGoal::startBuilder): Canonicalize "/tmp", not "/tmp/guix-build". Ludovic Courtès
2016-05-31daemon: rounds: Keep the differing output if -K is given....Regardless of -K, we now also print which output differs. Eelco Dolstra
2016-05-31daemon: check: Keep the differing output if -K is given....This makes it easier to investigate the non-determinism, e.g. $ nix-build pkgs/stdenv/linux -A stage1.pkgs.zlib --check -K error: derivation ‘/nix/store/l54i8wlw22656i4pk05c52ngv9rpl39q-zlib-1.2.8.drv’ may not be deterministic: output ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8’ differs from ‘/nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check’ $ diffoscope /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8 /nix/store/11a27shh6n2ivi4a7s964i65ql80cf27-zlib-1.2.8-check ... ├── lib/libz.a │ ├── metadata │ │ @@ -1,15 +1,15 @@ │ │ -rw-r--r-- 30001/30000 3096 Jan 12 15:20 2016 adler32.o ... │ │ +rw-r--r-- 30001/30000 3096 Jan 12 15:28 2016 adler32.o ... Co-authored-by: Ludovic Courtès <ludo@gnu.org> Eelco Dolstra
2016-05-31daemon: check: Fix "failed to produce output path"....This occured when sandbox building is disabled, at least one output exists, and at least one other output does not. Eelco Dolstra
2016-05-31daemon: check: Fix assertion failure when some outputs are missing....E.g. $ nix-build pkgs/stdenv/linux/ -A stage1.pkgs.perl --check nix-store: src/libstore/build.cc:1323: void nix::DerivationGoal::tryToBuild(): Assertion `buildMode != bmCheck || validPaths.size() == drv->outputs.size()' failed. when perl.out exists but perl.man doesn't. The fix is to only check the outputs that exist. Note that "nix-build -A stage1.pkgs.all --check" will still give a (proper) error in this case. Eelco Dolstra
2016-05-31daemon: When repairing, rebuild if there is no substituter.Eelco Dolstra
2016-05-31daemon: Fix --repair failure on multiple-output derivations....If repair found a corrupted/missing path that depended on a multiple-output derivation, and some of the outputs of the latter were not present, it failed with a message like error: path ‘/nix/store/cnfn9d5fjys1y93cz9shld2xwaibd7nn-bash-4.3-p42-doc’ is not valid Co-authored-by: Ludovic Courtès <ludo@gnu.org> Eelco Dolstra
2016-05-31daemon: Handle /tmp being a symlink....* nix/libstore/build.cc (DerivationGoal::startBuilder): Call 'canonPath' on "/tmp". Eelco Dolstra
2015-12-30daemon: Build in /tmp/guix-build-*....* nix/libstore/build.cc (DerivationGoal::startBuilder): Use "guix-build" instead of "nix-build" for TMPDIR. * doc/guix.texi (Build Environment Setup): Adjust accordingly. Ludovic Courtès
2015-12-13daemon: Better distinguish build statuses....In Nix itself, the new 'BuildResult' type is returned by the new 'buildDerivation' method, which we don't have and need. * nix/libstore/build.cc (Goal)[cancel]: Remove. [timeOut]: New pure virtual method. (DerivationGoal)[result]: New field. [cancel]: Remove. [timedOut, getResult, done]: New methods. (DerivationGoal::cancel): Remove. (DerivationGoal::timedOut): New method. (DerivationGoal::haveDerivation): Call 'done' instead of 'amDone'. (DerivationGoal::outputsSubstituted): Ditto. (DerivationGoal::inputsRealised): Ditto. (DerivationGoal::buildDone): Ditto. (DerivationGoal::handleChildOutput): Call 'timedOut' instead of 'cancel'. (DerivationGoal::done): New method. (SubstitutionGoal)[cancel]: Remove. [timedOut]: New method. (SubstitutionGoal::cancel): Remove. (SubstitutionGoal::timedOut): New method. (Worker::waitForInput): Use it. * nix/libstore/store-api.hh (BuildResult): New struct. Co-authored-by: Ludovic Courtès <ludo@gnu.org> Eelco Dolstra
2015-12-09daemon: Use deterministic $TMPDIR in chroot....Rather than using $<host-TMPDIR>/nix-build-<drvname>-<number>, the temporary directory is now always /tmp/nix-build-<drvname>-0. This improves bitwise-exact reproducibility for builds that store $TMPDIR in their build output. (Of course, those should still be fixed...) * nix/libstore/build.cc (DerivationGoal)[tmpDirInSandbox]: New field. (DerivationGoal::startBuilder): Initialize 'useChroot' earlier. Compute 'tmpDirInSandbox', and use it when populating 'dirsInChroot'. * doc/guix.texi (Build Environment Setup): Document it. Co-authored-by: Ludovic Courtès <ludo@gnu.org> Eelco Dolstra
2015-12-08daemon: Allow builds to be repeated....This makes it easy to detect non-deterministic builds. * nix/libstore/build.cc (DerivationGoal): Remove 'InodesSeen'; add 'curRound', 'nrRound', and 'prevInfos'. (DerivationGoal::inputsRealised): Initialize 'nrRound'. (NotDeterministic): New error type. (DerivationGoal::buildDone): Check whether we need to repeat. (DerivationGoal::startBuilder): Adjust message. (DerivationGoal::registerOutputs): Check whether we get the same result. * nix/libstore/globals.cc (Settings::get(const string & name, int def)): New method. * nix/libstore/globals.hh (Settings): Add it. * nix/libstore/store-api.hh (ValidPathInfo): Add operator ==. * nix/nix-daemon/nix-daemon.cc (performOp): Allow "build-repeat" for "untrusted" users. Co-authored-by: Ludovic Courtès <ludo@gnu.org> Eelco Dolstra