summaryrefslogtreecommitdiff
path: root/guix/scripts/substitute.scm
AgeCommit message (Expand)Author
2017-05-03ui: Rename '_' to 'G_'....This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`". Ludovic Courtès
2017-05-02substitute: Validate substitute URLs....Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com> at <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00548.html>. * guix/scripts/substitute.scm (validate-uri): New procedure. (guix-substitute): Use it. Ludovic Courtès
2017-04-18Add (guix cache) and use it in (guix scripts substitute)....* guix/cache.scm, tests/cache.scm: New files. * Makefile.am (MODULES, SCM_TESTS): Add them. * guix/scripts/substitute.scm (obsolete?): Remove. (remove-expired-cached-narinfos): Rename to... (cached-narinfo-expiration-time): ... this. Remove the removal part and only keep the expiration time part. (narinfo-cache-directories): Add optional 'directory' parameter and honor it. (maybe-remove-expired-cached-narinfo): Remove. (cached-narinfo-files): New procedure. (guix-substitute): Use 'maybe-remove-expired-cache-entries' instead of 'maybe-remove-expired-cached-narinfo'. Ludovic Courtès
2017-03-31substitute: Send ‘User-Agent’ header....* guix/scripts/substitute.scm (narinfo-request): Pass ‘User-Agent’ #:headers to ‘build-request’. Tobias Geerinckx-Rice
2017-03-18http-client: Avoid name clash with 'open-connection-for-uri' in 2.2.0....* guix/build/download.scm (open-connection-for-uri): Add note about same-named binding in Guile 2.2.0. * guix/http-client.scm: Use 'guix:open-connection-for-uri' for the procedure coming from (guix build download). * guix/scripts/lint.scm: Likewise. * guix/scripts/substitute.scm: Likewise. Ludovic Courtès
2016-11-12substitute: Disable HTTPS certificate verification....Fixes a regression introduced in 9e4e431e049fae3f1121c3be22cf13b174404ba8 as a consequence of bc3c41ce36349ed4ec758c70b48a7059e363043a. Reported by Marius Bakke <mbakke@fastmail.com>. * guix/scripts/substitute.scm (fetch): Pass #:verify-certificate? #f to 'open-connection-for-uri' and 'http-fetch'. (download-cache-info): Likewise. (http-multiple-get): Add #:verify-certificate? and honor it. (fetch-narinfos): Pass #:verify-certificate? #f. Ludovic Courtès
2016-10-19Use (ice-9 binary-ports) instead of (rnrs io ports)....This reduces the closure of (guix ui) from 123 to 106 modules. * guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io ports). (map-derivation)[substitute-file]: Use 'read-string' instead of 'get-string-all'. * guix/ftp-client.scm: Likewise. * guix/hash.scm: Likewise. * guix/http-client.scm: Likewise. * guix/pki.scm (ensure-acl, current-acl): Likewise. * guix/scripts/archive.scm (authorize-key)[read-key]: Likewise. * guix/scripts/authenticate.scm (read-canonical-sexp) (read-hash-data): Likewise. * guix/scripts/download.scm: Likewise. * guix/scripts/offload.scm (register-gc-root, remove-gc-roots) (send-files): Likewise. * guix/scripts/publish.scm (lazy-read-file-sexp): Likewise. * guix/scripts/refresh.scm: Likewise. * guix/scripts/substitute.scm (check-acl-initialized): Likewise. * guix/serialization.scm (read-maybe-utf8-string): Likewise. * guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of 'flush-output-port'. * guix/store.scm (process-stderr): Likewise. * guix/tests.scm: Likewise. * guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports) for 'make-custom-binary-input-port'. Ludovic Courtès
2016-10-10substitute: Make download messages more concise....'guix substitute' now displays 2 lines per download instead of 4. Suggested by Ricardo Wurmus. * guix/scripts/substitute.scm (assert-valid-narinfo): Have #:verbose? default to #f. Remove leading newline in message. (process-substitution): Display the URI rather than the store file name. Display two newlines after the substitution. Ludovic Courtès
2016-09-20ui: Do not shadow '_' where it's used as a literal syntax match....Fixes compilation with Guile 2.1. Reported by Mu Lei. * guix/ui.scm (report-load-error) (warn-about-load-error, read/eval-package-expression): Use 'rest' instead of '_' as the pattern variable name. * gnu/packages.scm (%find-package): Likewise. * guix/scripts/build.scm (transform-package-inputs): Likewise. * guix/scripts/hash.scm (guix-hash): Likewise. * guix/scripts/import/gnu.scm (%options, guix-import-gnu): Likewise. * guix/scripts/import/nix.scm (guix-import-nix): Likewise. * guix/scripts/offload.scm (build-machines): Likewise. * guix/scripts/refresh.scm (%options): Likewise. * guix/scripts/substitute.scm (narinfo-signature->canonical-sexp): Likewise. Ludovic Courtès
2016-07-14substitute: Gracefully handle trailing slashes in URLs....Previously, using something like "--substitute-urls=http://example.org///" would lead to a 'cache-narinfo!' call with #f as its second argument. It would also do the wrong thing for URLs with a non-empty initial path component, such as "http://example.org/foo/bar". * guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]: Add call to 'basename' for PATH. Ludovic Courtès
2016-06-22substitute: Use ~/.cache when invoked by an unprivileged user....This is a followup to ea0c6e0507a6997f12a4f29d0445b51cf53bd81e. * guix/scripts/substitute.scm (%narinfo-cache-directory): Use 'cache-directory' when (getuid) returns non-zero. (cache-narinfo!): Remove 'catch'. Ludovic Courtès
2016-05-26substitute: Internationalize the "Downloading" message....* guix/scripts/substitute.scm (process-substitution): I18n "Downloading" message. Ludovic Courtès
2016-05-21substitute: Gracefully handle invalid store file names....Before, something like: echo have /gnu/foo | ./test-env guix substitute --query would lead to an ugly backtrace. * guix/scripts/substitute.scm (narinfo-cache-file): Call 'leave' when 'store-hash-part' returns #f. Ludovic Courtès
2016-05-04utils: Move combinators to (guix combinators)....* guix/utils.scm (compile-time-value, memoize, fold2) (fold-tree, fold-tree-leaves): Move to... * guix/combinators: ... here. New file. * tests/utils.scm ("fold2, 1 list", "fold2, 2 lists") (fold-tree tests): Move to... * tests/combinators.scm: ... here. New file. * Makefile.am (MODULES, SCM_TESTS): Add them. * gnu/packages.scm, gnu/packages/bootstrap.scm, gnu/services/herd.scm, guix/build-system/gnu.scm, guix/build-system/python.scm, guix/derivations.scm, guix/gnu-maintenance.scm, guix/import/elpa.scm, guix/scripts/archive.scm, guix/scripts/build.scm, guix/scripts/graph.scm, guix/scripts/lint.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/serialization.scm, guix/store.scm, guix/ui.scm: Adjust imports accordingly. Ludovic Courtès
2016-04-20substitute: Sanitize the client-provided column number....* guix/scripts/substitute.scm (client-terminal-columns): Sanitize the client's column number. Ludovic Courtès
2016-04-20substitute: Install the client's locale....* guix/store.scm (set-build-options): Add #:locale parameter and honor it. * guix/scripts/substitute.scm (guix-substitute): Install the client's locale. Ludovic Courtès
2016-04-20substitute: Internationalize signature and download messages....* guix/scripts/substitute.scm (assert-valid-narinfo): Use gettext for messages. Ludovic Courtès
2016-04-20substitute: Better abbreviate substitute URL in progress report....Suggested by Danny Milosavljevic <dannym@scratchpost.org>. * guix/build/download.scm (nar-uri-abbreviation): New procedure. * guix/scripts/substitute.scm (process-substitution): Use it instead of 'store-path-abbreviation'. Ludovic Courtès
2016-04-15substitute: Honor the number of columns of the client terminal....* guix/store.scm (set-build-options): Add #:terminal-columns parameter and honor it. * guix/scripts/substitute.scm (client-terminal-columns): New procedure. (guix-substitute): Use it to parameterize 'current-terminal-columns'. Ludovic Courtès
2016-03-23substitute: Gracefully handle TLS errors....* guix/scripts/substitute.scm (with-networking): Use 'match-lambda*' and add case for 'gnutls-error'. Ludovic Courtès
2016-03-17substitute: Do not leak file descriptors for TLS connections....Partially fixes <http://bugs.gnu.org/20145>. * guix/scripts/substitute.scm (fetch, download-cache-info): (http-multiple-get, fetch-narinfos, progress-report-port): Use 'close-connection' instead of 'close-port'. Ludovic Courtès
2016-03-17substitute: Cache transient HTTP errors for 10mn....* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]: Cache transient errors for 10mn. (%narinfo-transient-error-ttl): New variable. Ludovic Courtès
2016-03-17substitute: Update progress for responses different from 200/404....* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]: Add missing call to 'update-progress!'. Ludovic Courtès
2016-03-16substitute: Honor client-provided empty URL list....Before that, 'guix build --substitute-urls=""' would lead to using the daemon's own URL list instead of the empty list. The 'or*' hack, which is to blame, had become unnecessary since commit fb4bf72be3fbc23bca35ba4b842b7e1517ef0e3a. Reported by Mark H Weaver <mhw@netris.org>. * guix/scripts/substitute.scm (or*): Remove. (%cache-urls): Use 'or' instead of 'or*'. * tests/store.scm ("substitute query, alternating URLs"): Add test with empty URL list. * doc/guix.texi (Common Build Options): Mention the empty string. Ludovic Courtès
2016-03-16substitute: Honor the 'max-age' of 'Cache-Control' headers....This allows substitute servers to tell 'guix substitute' how long they can cache narinfo lookups. * guix/scripts/substitute.scm (cache-narinfo!): Add 'ttl' parameter. [cache-entry]: Honor it. (fetch-narinfos)[handle-narinfo-response]: Check the 'Cache-Control' header of RESPONSE and pass its 'max-age' value to 'cache-narinfo!'. Ludovic Courtès
2016-03-16substitute: Make room for a 'ttl' field in cached entries....* guix/scripts/substitute.scm (cached-narinfo): Expect 'narinfo' sexp version 2 with a 'ttl' field. (cache-narinfo!)[cache-entry]: Produce 'narinfo' sexp version 2 with a 'ttl' field. (remove-expired-cached-narinfos)[expired?]: Read 'narinfo' sexp version 2. Ludovic Courtès
2016-03-14substitute: Keep the initial connection alive....The connection used to fetch /nix-cache-info is now reused for the subsequent narinfo requests. * guix/scripts/substitute.scm (download-cache-info)[download]: Remove. [uri, read-cache-info]: New variables. Rewrite in terms of 'http-fetch' instead of 'fetch'. Return an open port in addition to a <cache-info>. * guix/scripts/substitute.scm (http-multiple-get): Add #:port parameter and honor it. (fetch-narinfos)[do-fetch]: Add 'port' parameter. Adjust to new 'download-cache-info' and 'do-fetch' signatures. Ludovic Courtès
2016-03-14substitute: Remove dead code....This parameter became unused with the switch to HTTP pipelining in commit d3a652037ef879f9279bc056c43d15ba7afcbb25. * guix/scripts/substitute.scm (fetch): Remove #:quiet-404? and adjust accordingly. Ludovic Courtès
2016-03-10substitute: Optimize HTTP pipelining over TLS....* guix/scripts/substitute.scm (http-multiple-get): Write the requests to a bytevector output port before sending them. Ludovic Courtès
2016-03-10substitute: Add HTTPS support....Fixes <http://bugs.gnu.org/22937>. Reported by Chris Marusich <cmmarusich@gmail.com>. * guix/scripts/substitute.scm (fetch): Add 'https' alongside 'http'. Use 'open-connection-for-uri' instead of 'open-socket-for-uri'. Call 'setvbuf' only when PORT matches 'file-port?'. (http-multiple-get): Likewise. Change 'base-url' parameter to 'base-uri'. (fetch-narinfos)[do-fetch]: Add 'https' case alongside 'http'. Pass URI instead of URL to 'http-multiple-get'. * doc/guix.texi (Requirements): Move GnuTLS one level higher and mention HTTPS substitutes. (Substitutes): Mention HTTPS and recommend it. Explain why servers are not authenticated. Add "On Trusting Binaries" subsection. Ludovic Courtès
2016-03-09substitute: Error out on unsupported URL schemes....Reported in <http://bugs.gnu.org/22937> by Chris Marusich <cmmarusich@gmail.com>. * guix/scripts/substitute.scm (fetch): Add 'else' case and call 'leave'. Ludovic Courtès
2015-11-28substitute: Warn upon store prefix mismatches....Suggested by Hynek Urban <hynek.urban@gmail.com>. * guix/scripts/substitute.scm (fetch-narinfos): Move body to... [do-fetch]: ... here. New procedure. Emit a warning when CACHE-INFO's prefix does not match. Ludovic Courtès
2015-10-28substitute: Honor all the specified server URLs....* guix/scripts/substitute.scm (lookup-narinfos/diverse): New procedure. (lookup-narinfo): Use it. (process-query): Change #:cache-url to #:cache-urls. [valid?]: Remove 'narinfo?' check, which is no longer necessary. Use 'lookup-narinfos/diverse' instead of 'lookup-narinfos'. (process-substitution): Change #:cache-url to #:cache-urls. (%cache-url): Rename to... (%cache-urls): ... this. Turn into a list. (guix-substitute): Remove 'getaddrinfo' test with early exit. Adjust calls to 'process-query' and 'process-substitution'. * tests/substitute.scm: Change '%cache-url' to '%cache-urls'. Ludovic Courtès
2015-10-28substitute: 'lookup-narinfos' returns exactly a list of narinfos....* guix/scripts/substitute.scm (lookup-narinfos): Filter out #f values from CACHED, such that the end result is exactly a list of narinfos, not interspersed with #f. * guix/scripts/challenge.scm (discrepancies): Assume REMOTE is a list of narinfos. Ludovic Courtès
2015-10-28substitute: 'http-multiple-get' follows 'fold' style....* guix/scripts/substitute.scm (http-multiple-get): Add 'seed' parameter. Call PROC in 'fold' style. (fetch-narinfos)[handle-narinfo-response]: Adjust accordingly. Update 'http-multiple-get' call accordingly. Ludovic Courtès
2015-10-19substitute: Expose narinfo access....* guix/scripts/substitute.scm: Export <narinfo> accessors. (narinfo-hash->sha256): New procedure. (cache-narinfo!): Ignore EACCES exceptions. Ludovic Courtès
2015-09-16substitute: Improve readability of download progress report....* guix/build/download.scm (string-pad-middle, store-url-abbreviation, store-path-abbreviation): New procedures. (progress-proc): Add #:abbreviation parameter and use it. Generate a better indeterminate progress string. * guix/scripts/substitute.scm (assert-valid-narinfo): Add newlines to output. (process-substitution): Use byte-count->string and store-path-abbreviation. Co-authored-by: Ludovic Courtès <ludo@gnu.org> Steve Sprang
2015-09-04publish: Gracefully handle the lack of a deriver....* guix/scripts/publish.scm (narinfo-string): Catch 'system-error' around 'load-derivation' call; return BASE-INFO upon ENOENT. This allows us to return the narinfo even if DERIVER is missing. Before that, the exception would be uncaught, leading to 500 Internal Error on the client side. Ludovic Courtès
2015-07-13substitute: Improve functional decomposition....* guix/scripts/substitute.scm (display-narinfo-data, process-query, process-substitution): New procedures. Code moved from... (guix-substitute): ... here. Use them. Ludovic Courtès
2015-07-13substitute: Honor "substitute-urls" option passed by "untrusted" clients....* guix/scripts/substitute.scm (or*): New macro. (%cache-url): Honor "untrusted-substitute-urls". * guix/tests.scm (%test-substitute-urls): New variable. (open-connection-for-tests): Use it. * tests/derivations.scm ("derivation-prerequisites-to-build and substitutes", "derivation-prerequisites-to-build and substitutes, non-substitutable build", "derivation-prerequisites-to-build and substitutes, local build"): Pass it to 'set-build-options'. * tests/guix-daemon.sh: Likewise. * tests/store.scm ("substitute query, alternating URLs"): New test. ("substitute query", "substitute", "substitute + build-things with output path", "substitute, corrupt output hash", "substitute --fallback"): Pass #:substitute-urls to 'set-build-options'. Ludovic Courtès
2015-07-13substitute: Store cached narinfo in cache-specific sub-directories....This ensures that switching between different substitute servers doesn't lead to a polluted narinfo cache. * guix/scripts/substitute.scm (narinfo-cache-file): Add 'cache-url' parameter. Add the base32 of CACHE-URL as a sub-directory under %NARINFO-CACHE-DIRECTORY. Update callers. (cached-narinfo): Likewise. Call 'mkdir-p' on the dirname of the cache file. Update callers. (remove-expired-cached-narinfos): Add 'directory' parameter and use it instead of %NARINFO-CACHE-DIRECTORY. (narinfo-cache-directories): New procedure. (maybe-remove-expired-cached-narinfo): Call 'remove-expired-cached-narinfos' for each item returned by 'narinfo-cache-directories'. Ludovic Courtès
2015-07-13substitute: Pass the cache URL instead of <cache> objects....* guix/scripts/substitute.scm (<cache>): Rename to... (<cache-info>): ... this. (open-cache): Rename to... (download-cache-info): ... this. Return a <cache-info> or #f. (open-cache*): Remove. (cache-narinfo!): Take a URL instead of a <cache> as the first parameter. (fetch-narinfos): Likewise. Call 'download-cache-info'. Remove use of 'force'. (guix-substitute): Replace calls to 'open-cache*' with %CACHE-URL. Ludovic Courtès
2015-07-13substitute: Remove unneeded conditionals....* guix/scripts/substitute.scm (guix-substitute): Remove unneeded (if cache ...) forms since CACHE is always true (it's a promise.) Ludovic Courtès
2015-07-07substitute: Avoid infinite loop when updating the substitute list....Reported at <http://lists.gnu.org/archive/html/guix-devel/2015-07/msg00119.html>. * guix/scripts/substitute.scm (http-multiple-get): When RESP has "Connection: close", consume HEAD anyway; always call PROC to read from BODY. Co-authored-by: Ludovic Courtès <ludo@gnu.org> Andy Patterson
2015-05-10Remove assorted Guile 2.0.5 workarounds....* guix/scripts/authenticate.scm (%default-port-conversion-strategy): Remove. * guix/scripts/substitute.scm (fetch): Remove 2.0.5 special cases. * guix/serialization.scm (write-file): Remove 'scandir' workaround. * guix/ui.scm (command-files): Likewise. Ludovic Courtès
2015-05-06download: Work around Guile small-receive-buffer bug....Previously, code using directly (guix build download) was still affected by <http://bugs.gnu.org/15368>. This includes source derivations, the 'guix download' command, and (guix gnu-maintenance). 'guix substitute' was unaffected since it used (guix http-client), which already had the fix. * guix/http-client.scm (open-socket-for-uri): Remove. (http-fetch): Remove #:buffered? argument to 'open-socket-for-uri'; use 'setvbuf' instead. * guix/scripts/substitute.scm (fetch): Likewise. * guix/build/download.scm (open-socket-for-uri): New procedure, taken from guix/http-client.scm, but without the #:buffered? parameter. Ludovic Courtès
2015-05-04substitute: Increase TTL from 24h to 36h....* guix/scripts/substitute.scm (%narinfo-ttl): Increase to 36h. Ludovic Courtès
2015-05-01substitute: Fix file descriptor leak in 'http-multiple-get'....In practice we would not leak much since we reconnect after ~100 requests (with nginx running on hydra.gnu.org.) * guix/scripts/substitute.scm (http-multiple-get): Call 'close-port' before 'connect'. Ludovic Courtès
2015-03-25substitute: Rename cache directory from "substitute-binary" to "substitute"....* guix/scripts/substitute.scm (%narinfo-cache-directory): Change "substitute-binary" to "substitute". * tests/store.scm ("substitute query"): Likewise. * tests/substitute.scm (call-with-narinfo): Likewise. Ludovic Courtès
2015-03-25Rename 'guix substitute-binary' to 'guix substitute'....* guix/scripts/substitute-binary.scm: Rename to... * guix/scripts/substitute.scm: ... this. Adjust module name, entry point, comments, and help string accordingly. * nix/scripts/substitute-binary.in: Rename to... * nix/scripts/substitute.in: ... this. * pre-inst-env.in (NIX_SUBSTITUTERS): Adjust accordingly. * tests/substitute-binary.scm: Rename to... * tests/substitute.scm: ... this. Adjust references to (guix scripts substitute) accordingly. * guix/ui.scm (show-guix-help)[internal?]: Change "substitute-binary" to "substitute". * Makefile.am (MODULES, SCM_TESTS): Adjust to file renames. * daemon.am (nodist_pkglibexec_SCRIPTS): Likewise. * config-daemon.ac: Likewise. * guix/tests.scm (call-with-derivation-narinfo): Adjust comments and docstring. Ludovic Courtès
generated by cgit v1.2.3 (git 2.46.2) at 2024-11-15 01:24:30 +0000