| Age | Commit message (Expand) | Author |
|---|
| 2017-10-25 | substitute: Don't send more than 1000 requests in a row....Fixes <https://bugs.gnu.org/28731>.
Reported by Jan Nieuwenhuizen <janneke@gnu.org>.
* guix/scripts/substitute.scm (at-most): New procedure.
(http-multiple-get): Use it to send at most 1000 requests at once.
| Ludovic Courtès |
| 2017-10-19 | Add (guix progress)....Among other things, this removes (guix utils), (guix ui), (guix config),
etc. from the closure of (guix build download), as was the case since
798648515b77507c242752457b4dc17c155bad6e.
* guix/utils.scm (<progress-reporter>, call-with-progress-reporter):
Move to...
* guix/progress.scm: ... here. New file.
* Makefile.am (MODULES): Add it.
* guix/build/download.scm (current-terminal-columns)
(nearest-exact-integer, duration->seconds, seconds->string)
(byte-count->string, progress-bar, string-pad-middle)
(rate-limited, progress-reporter/file, dump-port*)
(time-monotonic): Move to progress.scm.
* guix/scripts/download.scm: Adjust accordingly.
* guix/scripts/substitute.scm: Likewise.
| Ludovic Courtès |
| 2017-10-17 | guix: substitute: Report the last progress in the child process....Fixes <https://bugs.gnu.org/28756>.
* guix/utils.scm (filtered-port): Close the 'input' port for the current
process, and close it upon exit in the child process.
* guix/scripts/substitute.scm (progress-substitution): Display "\n\n" after
the reporter has finished.
| 宋文武 |
| 2017-10-12 | substitute: Close the progress port after substitute finished....Fixes <https://bugs.gnu.org/28756>.
* guix/scripts/substitute.scm (progress-substitution):
Call '(close-port progress)'.
| 宋文武 |
| 2017-09-20 | download: Don't report the progress too fast....* guix/utils.scm (<progress-reporter>): New record type.
(call-with-progress-reporter): New procedure.
* guix/build/download.scm (dump-port*, rate-limited, progress-reporter/file):
New procedures.
(ftp-fetch, http-fetch): Use 'dump-port*'.
(progress-proc): Remove procedure.
* guix/scripts/substitute.scm (progress-report-port): Rewrite in terms of
<progress-reporter>.
(process-substitution): Adjust accordingly.
| 宋文武 |
| 2017-09-18 | substitute: Close the download port after substitution finished....* guix/scripts/substitute.scm (progress-substitution): Call 'close-port'
after 'restore-file'.
| 宋文武 |
| 2017-09-11 | substitute: Download from unauthorized sources that provide the right content....This allows substitutes to be downloaded from unauthorized servers, as
long as they advertise the same hash and references as one of the
authorized servers.
* guix/scripts/substitute.scm (assert-valid-narinfo): Remove.
(valid-narinfo?): Add #:verbose?. Handle each case of
'signature-case'.
(equivalent-narinfo?): New procedure.
(lookup-narinfos/diverse): Add 'authorized?' parameter and honor it.
[select-hit]: New procedure.
(lookup-narinfo): Add 'authorized?' parameter and pass it.
(process-query): Adjust callers accordingly.
(process-substitution): Remove call to 'assert-valid-narinfo'. Check
whether 'lookup-narinfo' returns true and call 'leave' if not.
* tests/substitute.scm (%main-substitute-directory)
(%alternate-substitute-directory): New variables.
(call-with-narinfo): Make 'narinfo-directory' a parameter. Call
'mkdir-p' to create it. Change unwind handler to check whether
CACHE-DIRECTORY exists before deleting it.
(with-narinfo*): New macro.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key"): Change expected error message to "no
valid substitute".
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, unsigned narinfo comes first")
("substitute, two invalid narinfos"): New tests.
* doc/guix.texi (Substitutes): Explain the new behavior.
| Ludovic Courtès |
| 2017-09-11 | substitute: Make substitute URLs a SRFI-39 parameter....* guix/scripts/substitute.scm (%cache-urls): Rename to...
(%default-substitute-urls): ... this.
(substitute-urls): New variable.
(guix-substitute): Use it instead of %CACHE-URLS.
* tests/substitute.scm: Likewise.
| Ludovic Courtès |
| 2017-07-28 | utils: Factorize XDG directory handling....* guix/ui.scm (config-directory): Remove.
* guix/utils.scm (xdg-directory, config-directory): New procedures.
(cache-directory): Rewrite in terms of 'xdg-directory'.
* guix/scripts/substitute.scm (%narinfo-cache-directory): Pass #:ensure?
#f to 'cache-directory'.
| Ludovic Courtès |
| 2017-07-21 | substitute: Avoid repeated calls to 'length'....* guix/scripts/substitute.scm (fetch-narinfos)[update-progress!]: Move
'length' call outside of lambda.
| Ludovic Courtès |
| 2017-07-21 | substitute: Optimize hash-part-to-path conversion on non-200 responses....Previously this operation was linear in the number of requests and
involved costly calls to 'string-contains'.
* guix/scripts/substitute.scm (fetch-narinfos)[hash-part->path]: New
procedure.
[handle-narinfo-response]: Use it for caching when CODE is not 200.
| Ludovic Courtès |
| 2017-07-12 | substitute: Work around Guile 2.2 'time-monotonic' bug....Prior to this change, half of the cached narinfos would expire
immediately since they contained the number of nanoseconds instead of
the number of seconds as their date.
* guix/scripts/substitute.scm (time-monotonic) <guile-2.2>: Define, as a
workaround.
| Ludovic Courtès |
| 2017-06-02 | substitute: Do not display the installed size....* guix/scripts/substitute.scm (process-substitution): Do not show the
installed size in the "Downloading" message.
| Ludovic Courtès |
| 2017-05-28 | scripts: Set thread names....This allows 'guix publish' threads as well as 'guix substitute' and
'guix offload' processes to be properly labeled in 'top', 'pstree', etc.
* guix/workers.scm (worker-thunk): Add #:thread-name parameter and honor it.
(make-pool): Likewise.
* guix/scripts/publish.scm (http-write): Add calls to 'set-thread-name'
in bodies of 'call-with-new-thread'.
(guix-publish): Call 'set-thread-name'. Pass #:thread-name to 'make-pool'.
* guix/scripts/offload.scm (guix-offload): Call 'set-thread-name'.
* guix/scripts/substitute.scm (guix-substitute): Likewise.
| Ludovic Courtès |
| 2017-05-11 | substitute: Honor 'Cache-Control' on 404 responses....* guix/scripts/substitute.scm (cached-narinfo): When VALUE is #f, use
the TTL that is read instead of %NARINFO-NEGATIVE-TTL.
(cached-narinfo-expiration-time): Likewise.
| Ludovic Courtès |
| 2017-05-03 | ui: Rename '_' to 'G_'....This avoids collisions with '_' when the latter is used as a 'match'
pattern for instance. See
<https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>.
* guix/ui.scm: Rename '_' to 'G_'.
* po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly.
* build-aux/compile-all.scm (warnings): Remove 'format'.
* gnu/packages.scm,
gnu/services.scm,
gnu/services/shepherd.scm,
gnu/system.scm,
gnu/system/shadow.scm,
guix/gnupg.scm,
guix/http-client.scm,
guix/import/cpan.scm,
guix/import/elpa.scm,
guix/import/pypi.scm,
guix/nar.scm,
guix/scripts.scm,
guix/scripts/archive.scm,
guix/scripts/authenticate.scm,
guix/scripts/build.scm,
guix/scripts/challenge.scm,
guix/scripts/container.scm,
guix/scripts/container/exec.scm,
guix/scripts/copy.scm,
guix/scripts/download.scm,
guix/scripts/edit.scm,
guix/scripts/environment.scm,
guix/scripts/gc.scm,
guix/scripts/graph.scm,
guix/scripts/hash.scm,
guix/scripts/import.scm,
guix/scripts/import/cpan.scm,
guix/scripts/import/cran.scm,
guix/scripts/import/crate.scm,
guix/scripts/import/elpa.scm,
guix/scripts/import/gem.scm,
guix/scripts/import/gnu.scm,
guix/scripts/import/hackage.scm,
guix/scripts/import/nix.scm,
guix/scripts/import/pypi.scm,
guix/scripts/import/stackage.scm,
guix/scripts/lint.scm,
guix/scripts/offload.scm,
guix/scripts/pack.scm,
guix/scripts/package.scm,
guix/scripts/perform-download.scm,
guix/scripts/publish.scm,
guix/scripts/pull.scm,
guix/scripts/refresh.scm,
guix/scripts/size.scm,
guix/scripts/substitute.scm,
guix/scripts/system.scm,
guix/ssh.scm,
guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was
obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
| Ludovic Courtès |
| 2017-05-02 | substitute: Validate substitute URLs....Reported by Maxim Cournoyer <maxim.cournoyer@gmail.com>
at <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00548.html>.
* guix/scripts/substitute.scm (validate-uri): New procedure.
(guix-substitute): Use it.
| Ludovic Courtès |
| 2017-04-18 | Add (guix cache) and use it in (guix scripts substitute)....* guix/cache.scm, tests/cache.scm: New files.
* Makefile.am (MODULES, SCM_TESTS): Add them.
* guix/scripts/substitute.scm (obsolete?): Remove.
(remove-expired-cached-narinfos): Rename to...
(cached-narinfo-expiration-time): ... this. Remove the removal part and
only keep the expiration time part.
(narinfo-cache-directories): Add optional 'directory' parameter and
honor it.
(maybe-remove-expired-cached-narinfo): Remove.
(cached-narinfo-files): New procedure.
(guix-substitute): Use 'maybe-remove-expired-cache-entries' instead of
'maybe-remove-expired-cached-narinfo'.
| Ludovic Courtès |
| 2017-03-31 | substitute: Send ‘User-Agent’ header....* guix/scripts/substitute.scm (narinfo-request): Pass ‘User-Agent’ #:headers
to ‘build-request’.
| Tobias Geerinckx-Rice |
| 2017-03-18 | http-client: Avoid name clash with 'open-connection-for-uri' in 2.2.0....* guix/build/download.scm (open-connection-for-uri): Add note about
same-named binding in Guile 2.2.0.
* guix/http-client.scm: Use 'guix:open-connection-for-uri' for the
procedure coming from (guix build download).
* guix/scripts/lint.scm: Likewise.
* guix/scripts/substitute.scm: Likewise.
| Ludovic Courtès |
| 2016-11-12 | substitute: Disable HTTPS certificate verification....Fixes a regression introduced in
9e4e431e049fae3f1121c3be22cf13b174404ba8 as a consequence of
bc3c41ce36349ed4ec758c70b48a7059e363043a.
Reported by Marius Bakke <mbakke@fastmail.com>.
* guix/scripts/substitute.scm (fetch): Pass #:verify-certificate? #f to
'open-connection-for-uri' and 'http-fetch'.
(download-cache-info): Likewise.
(http-multiple-get): Add #:verify-certificate? and honor it.
(fetch-narinfos): Pass #:verify-certificate? #f.
| Ludovic Courtès |
| 2016-10-19 | Use (ice-9 binary-ports) instead of (rnrs io ports)....This reduces the closure of (guix ui) from 123 to 106 modules.
* guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io
ports).
(map-derivation)[substitute-file]: Use 'read-string' instead of
'get-string-all'.
* guix/ftp-client.scm: Likewise.
* guix/hash.scm: Likewise.
* guix/http-client.scm: Likewise.
* guix/pki.scm (ensure-acl, current-acl): Likewise.
* guix/scripts/archive.scm (authorize-key)[read-key]: Likewise.
* guix/scripts/authenticate.scm (read-canonical-sexp)
(read-hash-data): Likewise.
* guix/scripts/download.scm: Likewise.
* guix/scripts/offload.scm (register-gc-root, remove-gc-roots)
(send-files): Likewise.
* guix/scripts/publish.scm (lazy-read-file-sexp): Likewise.
* guix/scripts/refresh.scm: Likewise.
* guix/scripts/substitute.scm (check-acl-initialized): Likewise.
* guix/serialization.scm (read-maybe-utf8-string): Likewise.
* guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of
'flush-output-port'.
* guix/store.scm (process-stderr): Likewise.
* guix/tests.scm: Likewise.
* guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports)
for 'make-custom-binary-input-port'.
| Ludovic Courtès |
| 2016-10-10 | substitute: Make download messages more concise....'guix substitute' now displays 2 lines per download instead of 4.
Suggested by Ricardo Wurmus.
* guix/scripts/substitute.scm (assert-valid-narinfo): Have #:verbose?
default to #f. Remove leading newline in message.
(process-substitution): Display the URI rather than the store file name.
Display two newlines after the substitution.
| Ludovic Courtès |
| 2016-09-20 | ui: Do not shadow '_' where it's used as a literal syntax match....Fixes compilation with Guile 2.1.
Reported by Mu Lei.
* guix/ui.scm (report-load-error)
(warn-about-load-error, read/eval-package-expression): Use 'rest'
instead of '_' as the pattern variable name.
* gnu/packages.scm (%find-package): Likewise.
* guix/scripts/build.scm (transform-package-inputs): Likewise.
* guix/scripts/hash.scm (guix-hash): Likewise.
* guix/scripts/import/gnu.scm (%options, guix-import-gnu): Likewise.
* guix/scripts/import/nix.scm (guix-import-nix): Likewise.
* guix/scripts/offload.scm (build-machines): Likewise.
* guix/scripts/refresh.scm (%options): Likewise.
* guix/scripts/substitute.scm (narinfo-signature->canonical-sexp):
Likewise.
| Ludovic Courtès |
| 2016-07-14 | substitute: Gracefully handle trailing slashes in URLs....Previously, using something like
"--substitute-urls=http://example.org///" would lead to a
'cache-narinfo!' call with #f as its second argument.
It would also do the wrong thing for URLs with a non-empty initial path
component, such as "http://example.org/foo/bar".
* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]:
Add call to 'basename' for PATH.
| Ludovic Courtès |
| 2016-06-22 | substitute: Use ~/.cache when invoked by an unprivileged user....This is a followup to ea0c6e0507a6997f12a4f29d0445b51cf53bd81e.
* guix/scripts/substitute.scm (%narinfo-cache-directory): Use
'cache-directory' when (getuid) returns non-zero.
(cache-narinfo!): Remove 'catch'.
| Ludovic Courtès |
| 2016-05-26 | substitute: Internationalize the "Downloading" message....* guix/scripts/substitute.scm (process-substitution): I18n "Downloading"
message.
| Ludovic Courtès |
| 2016-05-21 | substitute: Gracefully handle invalid store file names....Before, something like:
echo have /gnu/foo | ./test-env guix substitute --query
would lead to an ugly backtrace.
* guix/scripts/substitute.scm (narinfo-cache-file): Call 'leave' when
'store-hash-part' returns #f.
| Ludovic Courtès |
| 2016-05-04 | utils: Move combinators to (guix combinators)....* guix/utils.scm (compile-time-value, memoize, fold2)
(fold-tree, fold-tree-leaves): Move to...
* guix/combinators: ... here. New file.
* tests/utils.scm ("fold2, 1 list", "fold2, 2 lists")
(fold-tree tests): Move to...
* tests/combinators.scm: ... here. New file.
* Makefile.am (MODULES, SCM_TESTS): Add them.
* gnu/packages.scm, gnu/packages/bootstrap.scm,
gnu/services/herd.scm, guix/build-system/gnu.scm,
guix/build-system/python.scm, guix/derivations.scm,
guix/gnu-maintenance.scm, guix/import/elpa.scm,
guix/scripts/archive.scm, guix/scripts/build.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/scripts/size.scm, guix/scripts/substitute.scm,
guix/serialization.scm, guix/store.scm, guix/ui.scm: Adjust imports
accordingly.
| Ludovic Courtès |
| 2016-04-20 | substitute: Sanitize the client-provided column number....* guix/scripts/substitute.scm (client-terminal-columns): Sanitize the
client's column number.
| Ludovic Courtès |
| 2016-04-20 | substitute: Install the client's locale....* guix/store.scm (set-build-options): Add #:locale parameter and honor
it.
* guix/scripts/substitute.scm (guix-substitute): Install the client's
locale.
| Ludovic Courtès |
| 2016-04-20 | substitute: Internationalize signature and download messages....* guix/scripts/substitute.scm (assert-valid-narinfo): Use gettext for
messages.
| Ludovic Courtès |
| 2016-04-20 | substitute: Better abbreviate substitute URL in progress report....Suggested by Danny Milosavljevic <dannym@scratchpost.org>.
* guix/build/download.scm (nar-uri-abbreviation): New procedure.
* guix/scripts/substitute.scm (process-substitution): Use it instead of
'store-path-abbreviation'.
| Ludovic Courtès |
| 2016-04-15 | substitute: Honor the number of columns of the client terminal....* guix/store.scm (set-build-options): Add #:terminal-columns parameter
and honor it.
* guix/scripts/substitute.scm (client-terminal-columns): New procedure.
(guix-substitute): Use it to parameterize 'current-terminal-columns'.
| Ludovic Courtès |
| 2016-03-23 | substitute: Gracefully handle TLS errors....* guix/scripts/substitute.scm (with-networking): Use 'match-lambda*' and
add case for 'gnutls-error'.
| Ludovic Courtès |
| 2016-03-17 | substitute: Do not leak file descriptors for TLS connections....Partially fixes <http://bugs.gnu.org/20145>.
* guix/scripts/substitute.scm (fetch, download-cache-info):
(http-multiple-get, fetch-narinfos, progress-report-port): Use
'close-connection' instead of 'close-port'.
| Ludovic Courtès |
| 2016-03-17 | substitute: Cache transient HTTP errors for 10mn....* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]:
Cache transient errors for 10mn.
(%narinfo-transient-error-ttl): New variable.
| Ludovic Courtès |
| 2016-03-17 | substitute: Update progress for responses different from 200/404....* guix/scripts/substitute.scm (fetch-narinfos)[handle-narinfo-response]:
Add missing call to 'update-progress!'.
| Ludovic Courtès |
| 2016-03-16 | substitute: Honor client-provided empty URL list....Before that, 'guix build --substitute-urls=""' would lead to using the
daemon's own URL list instead of the empty list. The 'or*' hack, which
is to blame, had become unnecessary since commit
fb4bf72be3fbc23bca35ba4b842b7e1517ef0e3a.
Reported by Mark H Weaver <mhw@netris.org>.
* guix/scripts/substitute.scm (or*): Remove.
(%cache-urls): Use 'or' instead of 'or*'.
* tests/store.scm ("substitute query, alternating URLs"): Add test with
empty URL list.
* doc/guix.texi (Common Build Options): Mention the empty string.
| Ludovic Courtès |
| 2016-03-16 | substitute: Honor the 'max-age' of 'Cache-Control' headers....This allows substitute servers to tell 'guix substitute' how long they
can cache narinfo lookups.
* guix/scripts/substitute.scm (cache-narinfo!): Add 'ttl' parameter.
[cache-entry]: Honor it.
(fetch-narinfos)[handle-narinfo-response]: Check the 'Cache-Control'
header of RESPONSE and pass its 'max-age' value to 'cache-narinfo!'.
| Ludovic Courtès |
| 2016-03-16 | substitute: Make room for a 'ttl' field in cached entries....* guix/scripts/substitute.scm (cached-narinfo): Expect 'narinfo' sexp
version 2 with a 'ttl' field.
(cache-narinfo!)[cache-entry]: Produce 'narinfo' sexp version 2 with a
'ttl' field.
(remove-expired-cached-narinfos)[expired?]: Read 'narinfo' sexp version 2.
| Ludovic Courtès |
| 2016-03-14 | substitute: Keep the initial connection alive....The connection used to fetch /nix-cache-info is now reused for the
subsequent narinfo requests.
* guix/scripts/substitute.scm (download-cache-info)[download]: Remove.
[uri, read-cache-info]: New variables.
Rewrite in terms of 'http-fetch' instead of 'fetch'. Return an open
port in addition to a <cache-info>.
* guix/scripts/substitute.scm (http-multiple-get): Add #:port parameter
and honor it.
(fetch-narinfos)[do-fetch]: Add 'port' parameter.
Adjust to new 'download-cache-info' and 'do-fetch' signatures.
| Ludovic Courtès |
| 2016-03-14 | substitute: Remove dead code....This parameter became unused with the switch to HTTP pipelining in
commit d3a652037ef879f9279bc056c43d15ba7afcbb25.
* guix/scripts/substitute.scm (fetch): Remove #:quiet-404? and adjust
accordingly.
| Ludovic Courtès |
| 2016-03-10 | substitute: Optimize HTTP pipelining over TLS....* guix/scripts/substitute.scm (http-multiple-get): Write the requests
to a bytevector output port before sending them.
| Ludovic Courtès |
| 2016-03-10 | substitute: Add HTTPS support....Fixes <http://bugs.gnu.org/22937>.
Reported by Chris Marusich <cmmarusich@gmail.com>.
* guix/scripts/substitute.scm (fetch): Add 'https' alongside 'http'.
Use 'open-connection-for-uri' instead of 'open-socket-for-uri'. Call
'setvbuf' only when PORT matches 'file-port?'.
(http-multiple-get): Likewise. Change 'base-url' parameter to
'base-uri'.
(fetch-narinfos)[do-fetch]: Add 'https' case alongside 'http'. Pass URI
instead of URL to 'http-multiple-get'.
* doc/guix.texi (Requirements): Move GnuTLS one level higher and mention
HTTPS substitutes.
(Substitutes): Mention HTTPS and recommend it. Explain why servers
are not authenticated. Add "On Trusting Binaries" subsection.
| Ludovic Courtès |
| 2016-03-09 | substitute: Error out on unsupported URL schemes....Reported in <http://bugs.gnu.org/22937>
by Chris Marusich <cmmarusich@gmail.com>.
* guix/scripts/substitute.scm (fetch): Add 'else' case and call 'leave'.
| Ludovic Courtès |
| 2015-11-28 | substitute: Warn upon store prefix mismatches....Suggested by Hynek Urban <hynek.urban@gmail.com>.
* guix/scripts/substitute.scm (fetch-narinfos): Move body to...
[do-fetch]: ... here. New procedure.
Emit a warning when CACHE-INFO's prefix does not match.
| Ludovic Courtès |
| 2015-10-28 | substitute: Honor all the specified server URLs....* guix/scripts/substitute.scm (lookup-narinfos/diverse): New procedure.
(lookup-narinfo): Use it.
(process-query): Change #:cache-url to #:cache-urls.
[valid?]: Remove 'narinfo?' check, which is no longer necessary.
Use 'lookup-narinfos/diverse' instead of 'lookup-narinfos'.
(process-substitution): Change #:cache-url to #:cache-urls.
(%cache-url): Rename to...
(%cache-urls): ... this. Turn into a list.
(guix-substitute): Remove 'getaddrinfo' test with early exit. Adjust
calls to 'process-query' and 'process-substitution'.
* tests/substitute.scm: Change '%cache-url' to '%cache-urls'.
| Ludovic Courtès |
| 2015-10-28 | substitute: 'lookup-narinfos' returns exactly a list of narinfos....* guix/scripts/substitute.scm (lookup-narinfos): Filter out #f values
from CACHED, such that the end result is exactly a list of narinfos,
not interspersed with #f.
* guix/scripts/challenge.scm (discrepancies): Assume REMOTE is a list of
narinfos.
| Ludovic Courtès |
| 2015-10-28 | substitute: 'http-multiple-get' follows 'fold' style....* guix/scripts/substitute.scm (http-multiple-get): Add 'seed'
parameter. Call PROC in 'fold' style.
(fetch-narinfos)[handle-narinfo-response]: Adjust accordingly.
Update 'http-multiple-get' call accordingly.
| Ludovic Courtès |
