summaryrefslogtreecommitdiff
path: root/guix/scripts/substitute.scm
AgeCommit message (Expand)Author
2021-07-16substitute: Fix handling of short option "-h"....The short option was listed in the help-text, but not recognized. Hartmut Goebel
2021-06-25guix substitute: Adjust comment about GnuTLS bug....* guix/scripts/substitute.scm (call-with-cached-connection): Adjust comment. Ludovic Courtès
2021-06-18Start enabling substitutes from bordeaux.guix.gnu.org....In addition to substitutes from ci.guix.gnu.org. There are more changes that can be made in the future, but these changes seem like a good start. * config-daemon.ac (guix_substitute_urls): Add https://bordeaux.guix.gnu.org. * guix/scripts/substitute.scm (%default-substitute-urls): Add http://bordeaux.guix.gnu.org. * guix/store.scm (%default-substitute-urls): Add bordeaux.guix.gnu.org. * doc/guix.texi: Adjust accordingly. * doc/contributing.texi: Adjust accordingly. Christopher Baines
2021-06-06scripts: substitute: Cache connection when looking for narinfos....The process-substitution procedure is opening two distinct connections. The first one when looking for narinfo by calling lookup-narinfo and the other one when fetching nar files. Cache the connection when looking for narinfos so that process-substitution only opens one connection. * guix/scripts/substitute.scm (lookup-narinfo): Cache connection by using open-connection-for-uri/cached. Mathieu Othacehe
2021-06-01publish: Add keep-alive support when sending nar....The default Guile web server implementation supports the keep alive mechanism. However, in our custom http-write implementation, the connection is unconditionally close after sending nar files. To prevent that, when supported, add the client port to the server poll set so that further requests can be handled without closing the connection. * guix/scripts/publish.scm (nar-response-port): Rename it into ... (nar-compressed-port): ... this procedure. Operate directly on a given PORT. (http-write): Add keep-alive support when sending nar files. * guix/scripts/substitute.scm (process-substitution): Pass the download size to the progress-report-port procedure so that it doesn't block reading from the input port when keep-alive is supported. Mathieu Othacehe
2021-04-24http-client, substitute: Gracefully handle GnuTLS EAGAIN/EINTR....Partly fixes <https://bugs.gnu.org/47867>. Reported by Florian Pelz <pelzflorian@pelzflorian.de>. In GnuTLS up to 3.7.1 included, GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED are not handled by 'write_to_session_record_port' and could be thrown at the caller. This patch works around that by dropping connections altogether and restarting when this happens. * guix/http-client.scm (false-if-networking-error): Swallow ERROR/AGAIN and ERROR/INTERRUPTED. * guix/scripts/substitute.scm (call-with-cached-connection): Likewise. Ludovic Courtès
2021-04-09daemon: 'guix substitute' replies on FD 4....This avoids the situation where error messages would unintentionally go to stderr and be wrongfully interpreted as a reply by the daemon. Fixes <https://bugs.gnu.org/46362>. This is a followup to ee3226e9d54891c7e696912245e4904435be191c. * guix/scripts/substitute.scm (display-narinfo-data): Add 'port' parameter and honor it. (process-query): Likewise. (process-substitution): Likewise. (%error-to-file-descriptor-4?, with-redirected-error-port): Remove. (%reply-file-descriptor): New variable. (guix-substitute): Remove use of 'with-redirected-error-port'. Define 'reply-port' and pass it to 'process-query' and 'process-substitution'. * nix/libstore/build.cc (SubstitutionGoal::handleChildOutput): Swap 'builderOut' and 'fromAgent'. * nix/libstore/local-store.cc (LocalStore::getLineFromSubstituter): Likewise. * tests/substitute.scm <top level>: Set '%reply-file-descriptor' rather than '%error-to-file-descriptor-4?'. Ludovic Courtès
2021-03-31substitute: Emit a single newline upon completion....The immediate effect is that, with '--verbosity=1', only two lines are displayed for each substitute, instead of two lines followed by an empty line. * guix/scripts/substitute.scm (process-substitution): Emit a single newline upon completion when PRINT-BUILD-TRACE? is true. Ludovic Courtès
2021-03-29Fix substitutes server discovery....This is a follow-up of bc3896db25c788c181c7bcd65754e7cd378e9d9f. * guix/scripts/substitute.scm (%local-substitute-urls): Test for "true" instead of "yes". Mathieu Othacehe
2021-03-21substitute: Choose compression method based on past CPU usage....This stems from the observation that substitute download can be CPU-bound when high-speed networks are in use: https://lists.gnu.org/archive/html/guix-devel/2020-12/msg00177.html * guix/narinfo.scm (decompresses-faster?): New procedure. (narinfo-best-uri): Add #:fast-decompression?. * guix/scripts/substitute.scm (%prefer-fast-decompression?): New variable. (call-with-cpu-usage-monitoring): New procedure. (with-cpu-usage-monitoring): New macro. (display-narinfo-data, process-substitution): Pass #:fast-decompression? to 'narinfo-best-uri'. (process-substitution): Wrap 'restore-file' call in 'with-cpu-usage-monitoring'. Set '%prefer-fast-decompression?'. Ludovic Courtès
2021-03-17scripts: substitute: Tweak error reporting in process-substitution....The call-with-connection-error-handling was added in 20c08a8a45d0f137ead7c05e720456b2aea44402, but that error handling was previously inside of open-connection-for-uri/maybe, which is related to (call-)with-cached-connection which was used in process-substitution, but only actually used with call-with-cached-connection when used in fetch-narinfos. There's some handling for similar errors within with-networking, which is used within process-substitution. * guix/scripts/substitute.scm (process-substitution): Remove call-with-connection-error-handling call. Christopher Baines
2021-03-17scripts: substitute: Add back some error handling....In f50f5751fff4cfc6d5abba9681054569694b7a5c, the way fetch was called within process-substitution was changed. As call-with-cached-connection actually includes important error handling for the opening of a HTTP request, this change removed some error handling. This commit adds that back. Fixes <https://bugs.gnu.org/47157>. * guix/scripts/substitute.scm (call-with-cached-connection): New procedure. (with-cached-connection): New syntax rule. (process-substitution): Retry once for some errors when making HTTP requests to fetch substitutes. Christopher Baines
2021-03-09guix: substitutes: Make progress reporting configurable....Rather than always outputting to (current-error-port) in lookup-narinfos (which is called from within lookup-narinfos/diverse), take a procedure which should return a progress reporter, and defer any output to that. As this is now general purpose code, make the default behaviour to output nothing. Maintain the current behaviour of the substitute script by moving the progress reporter implementation there, and passing it in when calling lookup-narinfos/diverse. These changes should be generally useful, but I'm particularly looking at getting guix weather to do progress reporting differently, with this new flexibility. * guix/substitutes.scm (fetch-narinfos): Take a procedure to make a progress-reporter, and use that rather than the hardcoded behaviour. (lookup-narinfos): Add #:make-progress-reporter keyword argument, and pass this through to fetch-narinfos. (lookup-narinfos/diverse): Add a #:make-progress-reporter keyword argument, and pass this through to lookup-narinfos. * guix/scripts/substitute.scm (process-query): Pass a progress-reporter to lookup-narinfos/diverse. Christopher Baines
2021-03-05guix: Split (guix substitutes) from (guix scripts substitute)....This means there's a module for working with substitutes, rather than all the code sitting in the script. The need for this can be seen with the weather and challenge scripts, that now don't have to use code from the substitute script, but can instead use the substitute module. The separation here between the actual functionality of the substitute script and the underlying functionality used both there and elsewhere should make maintenance easier moving forward. This commit just moves code, none of the code should have been changed significantly. * guix/scripts/substitute.scm (%narinfo-cache-directory, %narinfo-ttl, %narinfo-negative-ttl, %narinfo-transient-error-ttl, %unreachable-hosts): Move variables to guix/substitutes.scm. (narinfo-cache-file, cached-narinfo, cache-narinfo!, narinfo-request, read-to-eof, call-with-connection-error-handling, fetch-narinfos, lookup-narinfos, lookup-narinfos/diverse): Move procedures to guix/substitutes.scm. * guix/substitutes.scm: New file. * Makefile.am: Add it. * guix/narinfo.scm: Remove redundant module. * guix/scripts/challenge.scm: Change (guix scripts substitute) to (guix substitutes). * guix/scripts/weather.scm: Change (guix scripts substitute) to (guix substitutes). Christopher Baines
2021-02-24substitute: Ensure backtraces go to file descriptor 4....Fixes <https://bugs.gnu.org/46362>. Reported by Zhu Zihao <all_but_last@163.com>. Previously, backtraces due to uncaught exceptions would always to go file descriptor 2; the daemon would read it and error out with something like: error: got unexpected path `Backtrace:' from substituter This patch fixes that by ensuring backtraces are properly displayed on file descriptor 4. * guix/scripts/substitute.scm (with-redirected-error-port): New macro. (guix-substitute): Use 'with-redirected-error-port' instead of 'parameterize'. Ludovic Courtès
2021-02-24substitute: Do not unwind stack traces upon uncaught exceptions....This is similar to commit a168c3e4f8d580f70e1c26bcdfc5b8378b2fa42d. * guix/scripts/substitute.scm (with-networking): Use 'with-throw-handler' instead of 'catch'. Ludovic Courtès
2021-02-22substitute: Rework connection error handling....This is part of trying to reduce the interdependency of code within the substitute module. This commit addresses some of the error handling that was performed through open-connection-for-uri/maybe. The new approach is to use call-with-connection-error-handling, and wrap calls to http-multiple-get and http-fetch with that procedure, which takes care of handling connection errors. I think this is even slightly more rigerous than the previous setup, because this approach handles connection errors that occur when http-multiple-get reconnects to a host. * guix/scripts/substitute.scm (open-connection-for-uri/maybe): Transform in to call-with-connection-error-handling. (fetch-narinfos): Use call-with-connection-error-handling. (process-query): Replace open-connection-for-uri/maybe with open-connection-for-uri/cached. (open-connection-for-uri/cached): Set a default timeout, matching the behaviour in open-connection-for-uri/maybe. (process-substitution): Use call-with-connection-error-handling. Christopher Baines
2021-02-22substitute: Remove fetch-narinfos use open-connection-for-uri/maybe....At least by default. Instead, make the open-connection procedure a parameter, and make the default guix:open-connection-for-uri. Do so similarly for lookup-narinfos and lookup-narinfos/diverse which work towards calling fetch-narinfos. This means this code can be moved to a different module, without having use/move the connection caching code. * guix/scripts/substitute.scm (fetch-narinfos): Add #:open-connection argument, and call http-multiple-get with it. (lookup-narinfos) Add #:open-connection argument, and call fetch-narinfos with it. (lookup-narinfos/diverse): Add #:open-connection argument, and call lookup-narinfos with it. (process-query): Call lookup-narinfos/diverse with #:open-connection open-connection-for-uri/maybe. Christopher Baines
2021-02-22substitute: Inline fetch in to process-substitutes....As it's only called in one place, and this should make the code easier to read. * guix/scripts/substitute.scm (fetch): Move procedure inside… (process-substitution): …here. Christopher Baines
2021-02-22substitute: Remove redundant fetch arguments....It's just called in one place, with hardcoded argument values, so just inline them. * guix/scripts/substitute.scm (fetch): Remove arguments that don't vary, copy the values from the call site in process-substitution. (process-substitution): Remove unnecessary argument values from fetch call. Christopher Baines
2021-02-22substitute: Remove now redundant connection caching helpers....Failures now should be handled where they occur, and if there's a problem that's symptomatic of an issue with the connection, the port should be closed. * guix/scripts/substitute.scm (call-with-cached-connection): Remove procedure. (with-cached-connection): Remove syntax rule. Christopher Baines
2021-02-22substitute: Change connection cache handling in process-substitution....Just pass open-connection-for-uri/maybe to http-fetch, this removes the need for with-cached-connection and passing the port in. * guix/scripts/substitute.scm (fetch): Don't take a port as an argument, and pass open-connection-for-uri/maybe to http-fetch. (process-substitution): Don't call fetch with with-cached-connection. Christopher Baines
2021-02-22substitute: Stop using call-with-cached-connection in fetch-narinfos....Instead, just pass open-connection-for-uri/maybe to http-multiple-get. This code should be functionaly similar to the previous code. The eventual aim of this is to make the connection caching not mandatory in fetch-narinfos. * guix/scripts/substitute.scm (fetch-narinfos): Remove use of call-with-cached-connection. Christopher Baines
2021-02-22substitute: open-connection-for-uri/maybe add #:verify-certificate?....As this is used by http-fetch and http-multiple-get when they call the specified open connection procedure. * guix/scripts/substitute.scm (open-connection-for-uri/maybe): Support #:verify-certificate?. Christopher Baines
2021-02-22guix: Move http-multiple-get to (guix http-client)....From (guix scripts substitute). This will make it easier to reuse this code. * guix/scripts/substitute.scm (http-multiple-get): Remove, and move to… * guix/http-client.scm (http-multiple-get): …here. Christopher Baines
2021-02-22substitute: Remove redundant let block from fetch....* guix/scripts/substitute.scm (fetch): Remove redundant let block. Christopher Baines
2021-02-22substitute: Remove connection handling from fetch....http-fetch does this, so just use that code instead. * guix/scripts/substitute.scm (fetch): Remove connection handling when the port is closed. Christopher Baines
2021-02-22substitute: Remove buffer handling from fetch....http-fetch does this, so just set the right option. * guix/scripts/substitute.scm (fetch): Remove buffering code, and pass #:buffered? to http-fetch. Christopher Baines
2021-01-21Revert "substitute: Remove extra 'newline' call."...This reverts commit d5a1b0e86f248edd53c785b910b5d51ee708b0fc. This change turned out to be misguided; it would lead to "substitute:" lines being printed on the client side, without anything beyond the colon sign. Ludovic Courtès
2021-01-16guix: Move narinfo code from substitute script to module....This separation between the code for dealing with narinfos from the code doing that for a purpose should make things clearer, and better support components other that the substitute script in using this code. This is just moving the code around, no code should have been significantly changed. * guix/scripts/substitute.scm (<narinfo>): Move record type to (guix narinfo). (fields->alist, narinfo-hash-algorithm+value, narinfo-hash->sha256, narinfo-signature->canonical-sexp, narinfo-maker, read-narinfo, narinfo-sha256, valid-narinfo?, write-narinfo, narinfo->string, string->narinfo, equivalent-narinfo?, supported-compression?, compresses-better?, narinfo-best-uri): Move procedures to (guix narinfo). (%compression-methods): Move variable to (guix narinfo). * guix/narinfo.scm: New file. * Makefile.am (MODULES): Add it. * po/guix/POTFILES.in: Add 'guix/narinfo.scm'. Christopher Baines
2021-01-16substitute: Untangle skipping authentication from valid-narinfo?....Rather than having valid-narinfo? evaluate to #t if %allow-unauthenticated-substitutes? is set to #t, just use (const #t) for valid-narinfo? when %allow-unauthenticated-substitutes? is set to #t. This will allow moving valid-narinfo? in to a (guix substitutes) module. * guix/scripts/substitute.scm (process-query, process-substitution): Change the authorized? argument to lookup-narinfo and lookup-narinfos/diverse based on %allow-unauthenticated-substitutes?. (valid-narinfo?): Remove use of %allow-unauthenticated-substitutes?. Christopher Baines
2021-01-13substitute: Add zstd support....* guix/scripts/substitute.scm (%compression-methods): Add zstd. (compresses-better?): "lzip" always wins. Ludovic Courtès
2021-01-13substitute: Remove extra 'newline' call....This led to extra "substitute:" lines since commit 79c6614f58a57b985daf8940766319e440311db0 or thereabouts. * guix/scripts/substitute.scm (fetch-narinfos): Remove extra 'newline' call, which can be traced back to d3a652037ef879f9279bc056c43d15ba7afcbb25 (2015). Ludovic Courtès
2021-01-04substitute: Handle "invalid session" GnuTLS errors on reused connections....Reported by Christopher Baines <mail@cbaines.net> at <https://issues.guix.gnu.org/45323#2>. * guix/scripts/substitute.scm (call-with-cached-connection): Handle 'gnutls-error and ERROR/INVALID-SESSION. Ludovic Courtès
2020-12-23substitute: Reuse connections for '--query'....This significantly speeds up things like substituting the closure of a .drv. This is a followup to 5ff521452b9ec2aae9ed8e4bb7bdc250a581f203. * guix/scripts/substitute.scm (http-multiple-get): Add #:open-connection and #:keep-alive? and honor them. (open-connection-for-uri/maybe): Use 'open-connection-for-uri/cached' instead of 'guix:open-connection-for-uri'. Call 'http-multiple-get' within 'call-with-cached-connection'. (open-connection-for-uri/cached): Add #:timeout and #:verify-certificate? and honor them. (call-with-cached-connection): Add 'open-connection' parameter and honor it. Ludovic Courtès
2020-12-19daemon: Delegate deduplication to 'guix substitute'....This removes the main source of latency between subsequent downloads. * nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a "deduplicate" key to ENV. (SubstitutionGoal::finished): Remove call to 'optimisePath'. * guix/scripts/substitute.scm (process-substitution)[destination-in-store?] [dump-file/deduplicate*]: New variables. Pass #:dump-file to 'restore-file'. * guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New variable. Pass #:deduplicate? to 'process-substitution'. * guix/serialization.scm (dump-file): Export and augment 'dump-file'. Ludovic Courtès
2020-12-19daemon: Let 'guix substitute' perform hash checks....This way, the hash of the store item can be computed as it is restored, thereby avoiding an additional file tree traversal ('hashPath' call) later on in the daemon. Consequently, it should reduce latency between subsequent substitute downloads. This is a followup to 5ff521452b9ec2aae9ed8e4bb7bdc250a581f203. * guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New procedure. (process-substitution): Wrap INPUT into a hash input port, 'hashed', and read from it. Compare the actual and expected hashes, and print a "hash-mismatch" status line when they differ. When they match, print not just "success" but also the nar hash and size. * nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]: Remove. (SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle "success" and "hash-mismatch" accordingly. Call 'hashPath' only when the returned hash is not SHA256. (SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr' handling. * tests/substitute.scm ("substitute, invalid hash"): Rename to... ("substitute, invalid narinfo hash"): ... this. ("substitute, invalid hash"): New test. Ludovic Courtès
2020-12-08substitute: Cache and reuse connections while substituting....That way, when fetching a series of substitutes from the same server(s), the connection is reused instead of being closed/opened for each substitutes, which saves on network round trips and TLS handshakes. * guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it. * guix/progress.scm (progress-report-port): Add #:close? parameter and honor it. * guix/scripts/substitute.scm (at-most): Return the tail as a second value. (fetch): Add #:port and #:keep-alive? and honor them. (%max-cached-connections): New variable. (open-connection-for-uri/cached, call-with-cached-connection): New procedures. (with-cached-connection): New macro. (process-substitution): Wrap 'fetch' call in 'with-cached-connection'. Pass #:close? to 'progress-report-port'. Ludovic Courtès
2020-12-08daemon: Run 'guix substitute --substitute' as an agent....This avoids spawning one substitute process per substitution. * nix/libstore/build.cc (class Worker)[substituter]: New field. [outPipe, logPipe, pid]: Remove. (class SubstitutionGoal)[expectedHashStr, status, substituter]: New fields. (SubstitutionGoal::timedOut): Adjust to check 'substituter'. (SubstitutionGoal::tryToRun): Remove references to 'outPipe' and 'logPipe'. Run "guix substitute --substitute" as an 'Agent'. Send the request with 'writeLine'. (SubstitutionGoal::finished): Likewise. (SubstitutionGoal::handleChildOutput): Change to fill in 'expectedHashStr' and 'status'. (SubstitutionGoal::handleEOF): Call 'wakeUp' unconditionally. (SubstitutionGoal::~SubstitutionGoal): Adjust to check 'substituter'. * guix/scripts/substitute.scm (process-substitution): Write "success\n" to stdout upon success. (%error-to-file-descriptor-4?): New variable. (guix-substitute): Set 'current-error-port' to file descriptor 4 unless (%error-to-file-descriptor-4?) is false. Remove "--substitute" arguments. Loop reading line from stdin. * tests/substitute.scm <top level>: Call '%error-to-file-descriptor-4?'. (request-substitution): New procedure. ("substitute, no signature") ("substitute, invalid hash") ("substitute, unauthorized key") ("substitute, authorized key") ("substitute, unauthorized narinfo comes first") ("substitute, unsigned narinfo comes first") ("substitute, first narinfo is unsigned and has wrong hash") ("substitute, first narinfo is unsigned and has wrong refs") ("substitute, two invalid narinfos") ("substitute, narinfo with several URLs"): Adjust to new "guix substitute --substitute" calling convention. Ludovic Courtès
2020-12-08daemon: Use 'Agent' to spawn 'guix substitute --query'....* nix/libstore/local-store.hh (RunningSubstituter): Remove. (LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>. [setSubstituterEnv, didSetSubstituterEnv]: Remove. [getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'. * nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove reference to 'runningSubstituter'. (LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove. (LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'. (LocalStore::querySubstitutablePaths): Spawn substituter agent if needed. Adjust to 'Agent' interface. (LocalStore::querySubstitutablePathInfos): Likewise. * nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to 'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead. (SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'. * guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?): Remove second argument to 'make-parameter'. (process-query): Call 'warn-about-missing-authentication' when (%allow-unauthenticated-substitutes?) is #t. (guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port' too. No longer exit when 'substitute-urls' returns the empty list. No longer print newline initially. * tests/substitute.scm (test-quit): Parameterize 'current-error-port' to account for the port changes in 'guix-substitute'. Ludovic Courtès
2020-12-04scripts: substitute: Fix discover option....The discover option can be unset or set to "yes/no". Handle both cases. * guix/scripts/substitute.scm (%local-substitute-urls): Fix discover option. Mathieu Othacehe
2020-11-29Make "guile-avahi" dependency optional....* configure.ac (HAVE_GUILE_AVAHI): New conditional. * Makefile.am (MODULES): Add "guix/avahi.scm" and "guix/scripts/discover.scm" if HAVE_GUILE_AVAHI is set. * guix/scripts/publish.scm: Autoload (guix avahi). * guix/scripts/substitute.scm: Autoload (guix scripts discovery). Mathieu Othacehe
2020-11-29Use substitute servers on the local network....* guix/scripts/discover.scm: New file. * Makefile.am (MODULES): Add it. * nix/nix-daemon/guix-daemon.cc (options): Add "discover" option, (parse-opt): parse it, (main): start "guix discover" process when the option is set. * guix/scripts/substitute.scm (%local-substitute-urls): New variable, (substitute-urls): add it. * gnu/services/base.scm (<guix-configuration>): Add "discover?" field, (guix-shepherd-service): honor it. * doc/guix.texi (Invoking guix-daemon): Document "discover" option, (Base Services): ditto. Mathieu Othacehe
2020-10-28substitute: Do not discard ttl info when the response code is 202....* guix/scripts/substitute.scm (fetch-narinfos) [handle-narinfo-response]: Add code 202 to the condition. This will allow in the future the use of 404 for nar requests which cannot be built in the present, leaving for the 202 to indicate that a request has triggered a nar generation that might be completed later, as specified in <https://www.ietf.org/rfc/rfc2616.txt>. Miguel Ángel Arruga Vivas
2020-10-12substitute: Reduce default narinfo negative TTL to 1h....* guix/scripts/substitute.scm (%narinfo-negative-ttl): Reduce to 1h. Ludovic Courtès
2020-09-10scripts: Use 'define-command' and have 'guix help' use that....This changes 'guix help' to print a short synopsis for each command and to group commands by category. * guix/scripts.scm (synopsis, category): New variables. (define-command-categories, define-command): New macros. (%command-categories): New variable. * guix/ui.scm (<command>): New record type. (source-file-command): New procedure. (command-files): Return absolute file names. (commands): Return a list of <command> records. (show-guix-help)[display-commands, category-predicate]: New procedures. Display commands grouped in three categories. * guix/scripts/archive.scm (guix-archive): Use 'define-command'. * guix/scripts/authenticate.scm (guix-authenticate): Likewise. * guix/scripts/build.scm (guix-build): Likewise. * guix/scripts/challenge.scm (guix-challenge): Likewise. * guix/scripts/container.scm (guix-container): Likewise. * guix/scripts/copy.scm (guix-copy): Likewise. * guix/scripts/deploy.scm (guix-deploy): Likewise. * guix/scripts/describe.scm (guix-describe): Likewise. * guix/scripts/download.scm (guix-download): Likewise. * guix/scripts/edit.scm (guix-edit): Likewise. * guix/scripts/environment.scm (guix-environment): Likewise. * guix/scripts/gc.scm (guix-gc): Likewise. * guix/scripts/git.scm (guix-git): Likewise. * guix/scripts/graph.scm (guix-graph): Likewise. * guix/scripts/hash.scm (guix-hash): Likewise. * guix/scripts/import.scm (guix-import): Likewise. * guix/scripts/install.scm (guix-install): Likewise. * guix/scripts/lint.scm (guix-lint): Likewise. * guix/scripts/offload.scm (guix-offload): Likewise. * guix/scripts/pack.scm (guix-pack): Likewise. * guix/scripts/package.scm (guix-package): Likewise. * guix/scripts/perform-download.scm (guix-perform-download): Likewise. * guix/scripts/processes.scm (guix-processes): Likewise. * guix/scripts/publish.scm (guix-publish): Likewise. * guix/scripts/pull.scm (guix-pull): Likewise. * guix/scripts/refresh.scm (guix-refresh): Likewise. * guix/scripts/remove.scm (guix-remove): Likewise. * guix/scripts/repl.scm (guix-repl): Likewise. * guix/scripts/search.scm (guix-search): Likewise. * guix/scripts/show.scm (guix-show): Likewise. * guix/scripts/size.scm (guix-size): Likewise. * guix/scripts/substitute.scm (guix-substitute): Likewise. * guix/scripts/system.scm (guix-system): Likewise. * guix/scripts/time-machine.scm (guix-time-machine): Likewise. * guix/scripts/upgrade.scm (guix-upgrade): Likewise. * guix/scripts/weather.scm (guix-weather): Likewise. Ludovic Courtès
2020-09-02substitute: Set LC_MESSAGES to the client's locale, not LC_ALL....Fixes <https://bugs.gnu.org/43039>. Reported by Adam Griffiths <adam.lw.griffiths@gmail.com>. Previously, a client could lead 'guix substitute' to run in a non-UTF-8 locale, such as the "C" locale. Consequently, 'guix substitute' would now fail to decode UTF-8 file names such as those that appear in the 'nss-certs' package. * guix/scripts/substitute.scm (guix-substitute): Set LC_MESSAGES, not LC_ALL. Ludovic Courtès
2020-08-24Use "guile-zlib" and "guile-lzlib" instead of (guix config)....* Makefile.am (MODULES): Remove guix/zlib.scm and guix/lzlib.scm, (SCM_TESTS): remove tests/zlib.scm, tests/lzlib.scm. * build-aux/build-self.scm (make-config.scm): Remove unused %libz variable. * configure.ac: Remove LIBZ and LIBLZ variables and check instead for Guile-zlib and Guile-lzlib. * doc/guix.texi ("Requirements"): Remove zlib requirement and add Guile-zlib and Guile-lzlib instead. * gnu/packages/package-management.scm (guix)[native-inputs]: Add "guile-zlib" and "guile-lzlib", [inputs]: remove "zlib" and "lzlib", [propagated-inputs]: ditto, [arguments]: add "guile-zlib" and "guile-lzlib" to Guile load path. * guix/config.scm.in (%libz, %liblz): Remove them. * guix/lzlib.scm: Remove it. * guix/man-db.scm: Use (zlib) instead of (guix zlib). * guix/profiles.scm (manual-database): Do not stub (guix config) in imported modules list, instead add "guile-zlib" to the extension list. * guix/scripts/publish.scm: Use (zlib) instead of (guix zlib) and (lzlib) instead of (guix lzlib), (string->compression-type, effective-compression): do not check for zlib and lzlib availability. * guix/scripts/substitute.scm (%compression-methods): Do not check for lzlib availability. * guix/self.scm (specification->package): Add "guile-zlib" and "guile-lzlib" and remove "zlib" and "lzlib", (compiled-guix): remove "zlib" and "lzlib" arguments and add guile-zlib and guile-lzlib to the dependencies, also do not pass "zlib" and "lzlib" to "make-config.scm" procedure, (make-config.scm): remove "zlib" and "lzlib" arguments as well as %libz and %liblz variables. * guix/utils.scm (lzip-port): Use (lzlib) instead of (guix lzlib) and do not check for lzlib availability. * guix/zlib.scm: Remove it. * m4/guix.m4 (GUIX_LIBZ_LIBDIR, GUIX_LIBLZ_FILE_NAME): Remove them. * tests/lzlib.scm: Use (zlib) instead of (guix zlib) and (lzlib) instead of (guix lzlib), and do not check for zlib and lzlib availability. * tests/publish.scm: Ditto. * tests/substitute.scm: Do not check for lzlib availability. * tests/utils.scm: Ditto. * tests/zlib.scm: Remove it. Mathieu Othacehe
2020-04-26substitute: Close port at the end of http-multiple-get....* guix/scripts/substitute.scm (http-multiple-get): Add close-port call. Christopher Baines
2020-04-26substitute: Make http-multiple-get batch size configurable....* guix/scripts/substitute.scm (http-multiple-get): Add batch-size parameter. Christopher Baines