| Age | Commit message (Expand) | Author |
|---|
| 2020-12-19 | daemon: Delegate deduplication to 'guix substitute'....This removes the main source of latency between subsequent downloads.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a
"deduplicate" key to ENV.
(SubstitutionGoal::finished): Remove call to 'optimisePath'.
* guix/scripts/substitute.scm (process-substitution)[destination-in-store?]
[dump-file/deduplicate*]: New variables.
Pass #:dump-file to 'restore-file'.
* guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New
variable.
Pass #:deduplicate? to 'process-substitution'.
* guix/serialization.scm (dump-file): Export and augment 'dump-file'.
| Ludovic Courtès |
| 2020-12-19 | daemon: Let 'guix substitute' perform hash checks....This way, the hash of the store item can be computed as it is restored,
thereby avoiding an additional file tree traversal ('hashPath' call)
later on in the daemon. Consequently, it should reduce latency between
subsequent substitute downloads.
This is a followup to 5ff521452b9ec2aae9ed8e4bb7bdc250a581f203.
* guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New
procedure.
(process-substitution): Wrap INPUT into a hash input port, 'hashed', and
read from it. Compare the actual and expected hashes, and print a
"hash-mismatch" status line when they differ. When they match, print
not just "success" but also the nar hash and size.
* nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]:
Remove.
(SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle
"success" and "hash-mismatch" accordingly. Call 'hashPath' only when
the returned hash is not SHA256.
(SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr'
handling.
* tests/substitute.scm ("substitute, invalid hash"): Rename to...
("substitute, invalid narinfo hash"): ... this.
("substitute, invalid hash"): New test.
| Ludovic Courtès |
| 2020-12-18 | status: Do not emit ANSI escapes when stderr is not a tty....Fixes <https://bugs.gnu.org/44985>.
Reported by Simon Josefsson <simon@josefsson.org>.
* guix/progress.scm (display-download-progress): Add #:tty? and honor it.
* guix/status.scm (print-build-event): Pass #:tty? to
'display-download-progress'.
| Ludovic Courtès |
| 2020-12-18 | import: elpa: Support working with MELPA....* guix/import/elpa.scm (default-files-spec): New variable.
(download-git-repository, package-name->melpa-recipe, file-hash, vcs-file?,
git-repository->origin, melpa-recipe->origin, melpa-recipe->maybe-arguments):
New procedures.
(elpa-package->sexp): Add optional repo argument, and use it to determine
whether to attempt to construct a source using the MELPA recipe.
(elpa->guix-package): Pass repo to elpa-package->sexp.
Signed-off-by: Christopher Baines <mail@cbaines.net>
| Carlo Zancanaro |
| 2020-12-15 | import: cran: Use CRAN’s canonical URL as home-page....* guix/import/cran.scm (%cran-canonical-url): New variable.
(description->package): Construct home-page using canonical URL.
Signed-off-by: Christopher Baines <mail@cbaines.net>
| Lars-Dominik Braun |
| 2020-12-15 | deduplicate: Create the '.links' directory lazily....This avoids repeated (mkdir-p "/gnu/store/.links") calls when
deduplicating lots of files.
* guix/store/deduplication.scm (deduplicate): Remove initial call to
'mkdir-p'. Add ENOENT case in 'link' exception handler. Reindent.
* tests/store-deduplication.scm ("deduplicate, ENOSPC"): Check
for (<= links 4) to account for the initial 'link' call.
| Ludovic Courtès |
| 2020-12-15 | database: Honor 'SOURCE_DATE_EPOCH'....* guix/store/database.scm (timestamp): New procedure.
(sqlite-register): Use it as the default for #:time.
(register-items): Likewise for #:registeration-time.
| Ludovic Courtès |
| 2020-12-15 | database: Remove 'register-path'....* guix/store/database.scm (register-path): Remove.
* tests/store-database.scm ("register-path"): Rename to...
("register-items"): ... this, and use 'register-items' instead of
'register-path'.
("register-path, directory"): Rename to...
("register-items, directory"): ... this, and use 'register-items'
instead of 'register-path'.
("register-path with unregistered references"): Rename to...
("sqlite-register with unregistered references"): ... this.
| Ludovic Courtès |
| 2020-12-15 | system: 'init' does not recompute the hash of each store item....Fixes <https://bugs.gnu.org/44760>.
Previously, the 'register-path' call would re-traverse ITEM to compute
its nar hash, even though that hash is already known in the initial
store. This patch also avoids repeated opening/closing of the
database.
* guix/store/database.scm (call-with-database): Export.
* guix/scripts/system.scm (copy-item): Add 'db' parameter. Call
'sqlite-register' instead of 'register-path'.
(copy-closure): Remove redundant call to 'references*'. Call
'call-with-database' and pass the database to 'copy-item'.
| Ludovic Courtès |
| 2020-12-15 | database: Remove #:deduplicate? and #:reset-timestamps? from 'register-path'....* guix/store/database.scm (register-path): Remove #:deduplicate?
and #:reset-timestamps?.
* guix/scripts/system.scm (copy-item): Adjust accordingly.
* tests/store-database.scm ("register-path")
("register-path, directory"): Call 'reset-timestamps'.
| Ludovic Courtès |
| 2020-12-15 | guix system: 'init' copies, resets timestamps, and deduplicates at once....Partly fixes <https://bugs.gnu.org/44760>.
* guix/build/store-copy.scm (copy-store-item): New procedure.
(populate-store): Use it instead of the inline 'copy-recursively' call.
* guix/scripts/system.scm (copy-item): Likewise.
Pass #:reset-timestamps? and #:deduplicate? to 'register-path'.
| Ludovic Courtès |
| 2020-12-15 | database: Remove #:deduplicate? from 'register-items'....It is now up to the caller to deduplicate store contents.
* guix/store/database.scm (register-items): Remove #:deduplicate?
parameter and call to 'deduplicate'.
(register-path): Call 'deduplicate' when #:deduplicate? is true.
* gnu/build/image.scm (register-closure): Adjust call accordingly.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Likewise.
* guix/scripts/pack.scm (store-database): Likewise.
| Ludovic Courtès |
| 2020-12-15 | store-copy: 'populate-store' can optionally deduplicate files....Until now deduplication was performed as an additional pass after
copying files, which involve re-traversing all the files that had just
been copied.
* guix/store/deduplication.scm (copy-file/deduplicate): New procedure.
* tests/store-deduplication.scm ("copy-file/deduplicate"): New test.
* guix/build/store-copy.scm (populate-store): Add #:deduplicate?
parameter and honor it.
* tests/gexp.scm ("gexp->derivation, store copy"): Pass #:deduplicate? #f
to 'populate-store'.
* gnu/build/image.scm (initialize-root-partition): Pass #:deduplicate?
to 'populate-store'. Pass #:deduplicate? #f to 'register-closure'.
* gnu/build/vm.scm (root-partition-initializer): Likewise.
* gnu/build/install.scm (populate-single-profile-directory): Pass
#:deduplicate? #f to 'populate-store'.
* gnu/build/linux-initrd.scm (build-initrd): Likewise.
* guix/scripts/pack.scm (self-contained-tarball)[import-module?]: New
procedure.
[build]: Pass it as an argument to 'source-module-closure'.
* guix/scripts/pack.scm (squashfs-image)[build]: Wrap in
'with-extensions'.
* gnu/system/linux-initrd.scm (expression->initrd)[import-module?]: New
procedure.
[builder]: Pass it to 'source-module-closure'.
* gnu/system/install.scm (cow-store-service-type)[import-module?]: New
procedure. Pass it to 'source-module-closure'.
| Ludovic Courtès |
| 2020-12-15 | database: Remove #:reset-timestamps? from 'register-items'....The assumption now is that the caller took care of resetting timestamps
and permissions.
* guix/store/database.scm (register-items): Remove #:reset-timestamps?
parameter and the call to 'reset-timestamps'.
(register-path): Adjust accordingly and add call to 'reset-timestamps'.
* gnu/build/image.scm (register-closure): Remove #:reset-timestamps?
parameter to 'register-items'.
* gnu/build/vm.scm (register-closure): Likewise.
* guix/nar.scm (finalize-store-file): Adjust accordingly.
* guix/scripts/pack.scm (store-database)[build]: Likewise.
| Ludovic Courtès |
| 2020-12-15 | store-copy: 'populate-store' resets timestamps....Until now, 'populate-store' would reset permissions but not timestamps,
so callers would resort to going through an extra directory traversal to
reset timestamps.
* guix/build/store-copy.scm (reset-permissions): Remove.
(copy-recursively): New procedure.
(populate-store): Pass #:keep-permissions? to 'copy-recursively'.
Remove call to 'reset-permissions'.
* tests/gexp.scm ("gexp->derivation, store copy"): In BUILD-DRV, check
whether 'populate-store' canonicalizes permissions and timestamps.
* gnu/build/image.scm (initialize-root-partition): Pass #:reset-timestamps? #f
to 'register-closure'.
* gnu/build/vm.scm (root-partition-initializer): Likewise.
| Ludovic Courtès |
| 2020-12-15 | nar: Deduplicate files right as they are restored....This avoids having to traverse and re-read the files that we have just
restored, thereby reducing I/O.
* guix/serialization.scm (dump-file): New procedure.
(restore-file): Add #:dump-file parameter and honor it.
* guix/store/deduplication.scm (tee, dump-file/deduplicate): New
procedures.
* guix/nar.scm (restore-one-item): Pass #:dump-file to 'restore-file'.
(finalize-store-file): Pass #:deduplicate? #f to 'register-items'.
* tests/nar.scm <top level>: Call 'setenv' to set "NIX_STORE".
| Ludovic Courtès |
| 2020-12-15 | serialization: 'restore-file' sets canonical timestamp and permissions....* guix/serialization.scm (restore-file): Set the permissions and mtime
of FILE.
* guix/nar.scm (finalize-store-file): Pass #:reset-timestamps? #f to
'register-items'.
* tests/nar.scm (rm-rf): Add 'chmod' calls to ensure files are writable.
("write-file + restore-file with symlinks"): Ensure every file in OUTPUT
passes 'canonical-file?'.
* tests/guix-archive.sh: Run "chmod -R +w" before "rm -rf".
| Ludovic Courtès |
| 2020-12-15 | serialization: 'fold-archive' notifies about directory processing completion....* guix/serialization.scm (fold-archive): Call PROC with a
'directory-complete tag when done with a directory.
(restore-file): Handle it.
* guix/scripts/archive.scm (list-contents): Likewise.
* guix/scripts/challenge.scm (archive-contents): Likewise.
* tests/nar.scm ("write-file-tree + fold-archive"): Adjust accordingly.
| Ludovic Courtès |
| 2020-12-15 | offload: Print warning in context....* guix/scripts/offload.scm (build-machine): Add `location` field.
(build-machine-system): Print location with warning.
| Julien Lepiller |
| 2020-12-13 | scripts: discover: Remove file locks....* guix/scripts/discover.scm (call-with-read-file-lock, with-read-file-lock):
Remove them.
(write-publish-file): Use "with-atomic-file-output" instead of
"with-file-lock".
(read-substitute-urls): Remove file lock.
| Mathieu Othacehe |
| 2020-12-13 | download: Update ImageMagick mirror list....* guix/download.scm (%mirrors): Remove defunct ImageMagick mirrors, update
URLs for directory structure changes (/releases instead of /legacy). Move
official site last.
| Marius Bakke |
| 2020-12-12 | import/cran: Abort with error message when recursive import fails....Previously, after a failed recursive import "guix import" would signal
success.
* guix/import/cran.scm (cran->guix-package): Raise a condition when all
repositories have been exhausted.
* guix/scripts/import/cran.scm (guix-import-cran): Handle errors.
| Ricardo Wurmus |
| 2020-12-11 | self: Remove the empty string from '%load-extensions'....* guix/self.scm (guix-command): Set '%load-extensions'.
| Ludovic Courtès |
| 2020-12-11 | self: Move Guile early in the module search path....Until now, Guile modules would first be searched for in
MODULE-DIRECTORY, then in each $GUILE_LOAD_PATH entry, and finally in
Guile itself.
* guix/self.scm (guix-command): Make GUILE the second entry in the
%LOAD-PATH and %LOAD-COMPILED-PATH.
| Ludovic Courtès |
| 2020-12-10 | avahi: Ignore local services....* guix/avahi.scm (avahi-browse-service-thread): Add "ignore-local?" argument
and honor it.
| Mathieu Othacehe |
| 2020-12-09 | guix: Add globstar support....* guix/glob.scm (string->sglob)
(glob-match?): Add globstar support.
* tests/glob.scm: Update accordingly.
Signed-off-by: Jelle Licht <jlicht@fsfe.org>
| Giacomo Leidi |
| 2020-12-09 | import/utils: Fix conditional....This is a follow-up to commit 45584061a9ebe961e2be08ee94c3fe8ad74e2713.
* guix/import/utils.scm (package->definition): Use EQ? instead of = to compare
truthiness; use ELSE instead of (#T).
| Ricardo Wurmus |
| 2020-12-09 | import: cran: Update the Bioconductor version to 3.12....* guix/import/cran.scm (%bioconductor-version): Set to 3.12.
* guix/build-system/r.scm (bioconductor-uri): Update to 3.12.
| Roel Janssen |
| 2020-12-08 | import: opam: Adjust test to latest 'opam->guix-package' changes....This is a followup to a8dccd4bdc1e58219d4ba08fe1649bf0b8325f44, which
broke the test.
* guix/import/opam.scm (get-opam-repository): Prevent inlining.
* tests/opam.scm ("opam->guix-package"): Mock 'get-opam-repository'.
| Ludovic Courtès |
| 2020-12-08 | substitute: Cache and reuse connections while substituting....That way, when fetching a series of substitutes from the same server(s),
the connection is reused instead of being closed/opened for each
substitutes, which saves on network round trips and TLS handshakes.
* guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it.
* guix/progress.scm (progress-report-port): Add #:close? parameter and
honor it.
* guix/scripts/substitute.scm (at-most): Return the tail as a second
value.
(fetch): Add #:port and #:keep-alive? and honor them.
(%max-cached-connections): New variable.
(open-connection-for-uri/cached, call-with-cached-connection): New
procedures.
(with-cached-connection): New macro.
(process-substitution): Wrap 'fetch' call in 'with-cached-connection'.
Pass #:close? to 'progress-report-port'.
| Ludovic Courtès |
| 2020-12-08 | daemon: Run 'guix substitute --substitute' as an agent....This avoids spawning one substitute process per substitution.
* nix/libstore/build.cc (class Worker)[substituter]: New field.
[outPipe, logPipe, pid]: Remove.
(class SubstitutionGoal)[expectedHashStr, status, substituter]: New fields.
(SubstitutionGoal::timedOut): Adjust to check 'substituter'.
(SubstitutionGoal::tryToRun): Remove references to 'outPipe' and
'logPipe'. Run "guix substitute --substitute" as an 'Agent'. Send the
request with 'writeLine'.
(SubstitutionGoal::finished): Likewise.
(SubstitutionGoal::handleChildOutput): Change to fill in
'expectedHashStr' and 'status'.
(SubstitutionGoal::handleEOF): Call 'wakeUp' unconditionally.
(SubstitutionGoal::~SubstitutionGoal): Adjust to check 'substituter'.
* guix/scripts/substitute.scm (process-substitution): Write "success\n"
to stdout upon success.
(%error-to-file-descriptor-4?): New variable.
(guix-substitute): Set 'current-error-port' to file descriptor 4
unless (%error-to-file-descriptor-4?) is false.
Remove "--substitute" arguments. Loop reading line from stdin.
* tests/substitute.scm <top level>: Call '%error-to-file-descriptor-4?'.
(request-substitution): New procedure.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key")
("substitute, authorized key")
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, two invalid narinfos")
("substitute, narinfo with several URLs"): Adjust to new "guix
substitute --substitute" calling convention.
| Ludovic Courtès |
| 2020-12-08 | daemon: Use 'Agent' to spawn 'guix substitute --query'....* nix/libstore/local-store.hh (RunningSubstituter): Remove.
(LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>.
[setSubstituterEnv, didSetSubstituterEnv]: Remove.
[getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove
reference to 'runningSubstituter'.
(LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove.
(LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'.
(LocalStore::querySubstitutablePaths): Spawn substituter agent if
needed. Adjust to 'Agent' interface.
(LocalStore::querySubstitutablePathInfos): Likewise.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to
'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead.
(SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'.
* guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?):
Remove second argument to 'make-parameter'.
(process-query): Call 'warn-about-missing-authentication'
when (%allow-unauthenticated-substitutes?) is #t.
(guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port'
too. No longer exit when 'substitute-urls' returns the empty list. No
longer print newline initially.
* tests/substitute.scm (test-quit): Parameterize 'current-error-port' to
account for the port changes in 'guix-substitute'.
| Ludovic Courtès |
| 2020-12-08 | database: Remove unnecessary module imports....* guix/store/database.scm: Remove unnecessary imports added in
4b9eecd322e566783369795ebea63a479b51f486.
| Ludovic Courtès |
| 2020-12-08 | guix: opam: Add coq support in the importer....* guix/import/opam.scm (get-opam-repository): Add support for coq
repositories.
(ocaml-name->guix-name): Properly name coq package.
* doc/guix.texi (Invoking guix import): Document it.
| Julien Lepiller |
| 2020-12-08 | guix: opam: Add --repo argument to importer....* guix/scripts/import/opam.scm (guix-import-opam): Pass --repo argument
to recursive and non-recursive importers.
* guix/import/opam.scm (get-opam-repository): Select proper repository
location depending on a new repo argument.
(opam->guix-package): Use get-opam-repository in the procedure body.
| Julien Lepiller |
| 2020-12-08 | guix: opam: Pass default repository to recursive importer....* guix/import/opam.scm (opam->guix-package): Rename #:repository key
to #:repo.
(opam-recursive-import): Pass #:repo keyword.
* tests/opam.scm (opam->guix-package): Rename #:repository to #:repo.
| Julien Lepiller |
| 2020-12-08 | guix: opam: Filter implicit inputs....* guix/import/opam.scm (opam->guix-package, depends->inputs): Filter out
implicit inputs.
| Julien Lepiller |
| 2020-12-08 | guix: opam: Remove stray `pk'....* guix/import/opam.scm (opam->guix-package): Remove stray `pk'.
| Julien Lepiller |
| 2020-12-07 | ci: Add "evaluation" method....* guix/ci.scm (evaluation): New procedure.
| Mathieu Othacehe |
| 2020-12-07 | ci: Add evaluation field to build record....* guix/ci.scm (<build>): Add "evaluation" field.
| Mathieu Othacehe |
| 2020-12-07 | profiles: Delete duplicate manifest entries in packages->manifest....* gnu/profiles.scm (packages->manifest): Delete duplicate entries.
* tests/profiles.scm ("packages->manifest, no duplicates"): New test.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
| Leo Prikler |
| 2020-12-07 | profiles: Remove duplicates in manifest transactions....Fixes <https://bugs.gnu.org/23874>.
Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>.
* guix/profiles.scm (manifest-transaction-effects): Delete duplicates in
install and remove. Let multiple upgrades and downgrades shadow previous
transactions of the same kind.
* tests/profiles.scm
("manifest-transaction-effects no double install or upgrades")
("manifest-transaction-effects no double downgrade")
("manifest-transaction-effects no double removal"): New tests.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
| Leo Prikler |
| 2020-12-05 | ui: Handle multiword and empty $PAGER values....* guix/ui.scm (call-with-paginated-output-port): Empty PAGER values
disable paging. Non-empty ones are split into command arguments.
Reported by Daniel Brooks <db48x@db48x.net>.
| Tobias Geerinckx-Rice |
| 2020-12-05 | pull: Accept no non-option arguments....* guix/scripts/pull.scm (guix-pull): LEAVE if any non-option arguments
were passed.
| Tobias Geerinckx-Rice |
| 2020-12-05 | scripts: discover: Check for cache file existence....Fixes: https://issues.guix.gnu.org/45045.
* guix/scripts/discover.scm (read-substitute-urls): Check for file existence.
| Mathieu Othacehe |
| 2020-12-04 | git: 'reference-available?' handles short commit IDs....Reported by Simon Tournier on #guix.
Until now 'reference-available?' would always return #f when passed a
short commit ID.
* guix/git.scm (reference-available?): Call 'object-lookup-prefix' when
COMMIT is shorter than 40 characters.
| Ludovic Courtès |
| 2020-12-04 | import: crate: Skip tests when Guile-Semver is missing....* guix/import/crate.scm: Add comment for the 'module-autoload!' calls.
* tests/crate.scm (have-guile-semver?): New variable.
("crate->guix-package", "cargo-recursive-import")
("cargo-recursive-import-hoors-existing-packages"): Skip when
HAVE-GUILE-SEMVER? is false.
| Ludovic Courtès |
| 2020-12-04 | scripts: substitute: Fix discover option....The discover option can be unset or set to "yes/no". Handle both cases.
* guix/scripts/substitute.scm (%local-substitute-urls): Fix discover option.
| Mathieu Othacehe |
| 2020-12-04 | import/cran: Do not attempt to generate definitions for failed imports....* guix/scripts/import/cran.scm (guix-import-cran): Remove failed imports from
list of packages to define.
| Ricardo Wurmus |
| 2020-12-04 | import/cran: Fix calls to CRAN->GUIX-PACKAGE....* guix/import/cran.scm (cran->guix-package, latest-bioconductor-release):
Invoke CRAN->GUIX-PACKAGE with #:REPO keyword argument.
| Ricardo Wurmus |
