summaryrefslogtreecommitdiff
path: root/guix/http-client.scm
AgeCommit message (Expand)Author
2022-05-22http-client: 'http-fetch/cached' converts strings to URIs....* guix/http-client.scm (http-fetch/cached): Use 'string->uri' if URI is a string, as with 'http-fetch'. Signed-off-by: Ludovic Courtès <ludo@gnu.org> Philip McGrath
2022-05-22http-client: Accept '#:headers' in 'http-fetched/cached'....Callers can supply alternative headers as with 'http-fetch'. * guix/http-client.scm (http-fetch/cached): Add '#:headers' argument. Signed-off-by: Ludovic Courtès <ludo@gnu.org> Philip McGrath
2022-04-11http-client: Fix redirection....* guix/http-client.scm (http-fetch): Use the right uri variable in case of redirection. Signed-off-by: Ludovic Courtès <ludo@gnu.org> Attila Lendvai
2022-03-06http-client: Correctly handle redirects when #:keep-alive? #t....Previously PORT would be closed unconditionally, which broke redirects when #:keep-alive? #t is given. * guix/http-client.scm (http-fetch): Make 'port' a parameter of 'loop'. Upon 3xx responses, do not close PORT is KEEP-ALIVE? is true, but consume RESP's body. Add second argument to 'loop'. Ludovic Courtès
2022-03-06http-client: Add response headers to '&http-get-error'....* guix/http-client.scm (&http-get-error)[headers]: New field. (http-fetch): Initialize the 'headers' field. Ludovic Courtès
2021-04-24http-client: Remove exception mishandling in 'http-multiple-get'....Partly fixes <https://bugs.gnu.org/47867>. Reported by Florian Pelz <pelzflorian@pelzflorian.de>. The non-tail recursive call to 'connect' could cause requests to be processed twice, with 'p' possibly closed the second time. Regression introduced in 205833b72c5517915a47a50dbe28e7024dc74e57 and carried over in 45fce38fb0b6c6796906149ade145b8d3594c1c6. * guix/http-client.scm (http-multiple-get): Remove call to 'close-port' and recursive call to 'connect' when the 'write-request' block returns #f. Ludovic Courtès
2021-04-24http-client, substitute: Gracefully handle GnuTLS EAGAIN/EINTR....Partly fixes <https://bugs.gnu.org/47867>. Reported by Florian Pelz <pelzflorian@pelzflorian.de>. In GnuTLS up to 3.7.1 included, GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED are not handled by 'write_to_session_record_port' and could be thrown at the caller. This patch works around that by dropping connections altogether and restarting when this happens. * guix/http-client.scm (false-if-networking-error): Swallow ERROR/AGAIN and ERROR/INTERRUPTED. * guix/scripts/substitute.scm (call-with-cached-connection): Likewise. Ludovic Courtès
2021-03-28http-client: Don't drop failed query when reconnecting....Reported by Christopher Baines <mail@cbaines.net>. * guix/http-client.scm (http-multiple-get): Change 2nd argument to 'drop' to PROCESSED when (false-if-networking-error ...) returns #f. Ludovic Courtès
2021-03-27http-client: 'http-multiple-get' is tail-recursive again....Fixes <https://bugs.gnu.org/47283>. Commit 205833b72c5517915a47a50dbe28e7024dc74e57 made 'http-multiple-get' non-tail-recursive. Each recursive call would install an exception handler. As the number of iterations grows beyond 1,000, quadratic complexity of 'raise-exception' would show and we'd spend most of our time there. * guix/http-client.scm (false-if-networking-error): New macro. (http-multiple-get): Use it around 'write-request' and 'put-bytevector' calls, and around 'read-response' call, in lieu of the inline 'catch' forms. Ludovic Courtès
2021-03-17http-client: 'http-fetch' and 'http-fetch/cached' accept #:log-port....* guix/http-client.scm (http-fetch, http-fetch/cached): Add #:log-port and honor it. Ludovic Courtès
2021-02-22http-client: Accept #:open-connection in http-fetch....So that an alternative procedure can be passed in, perhaps to perform connection caching. * guix/http-client.scm (http-fetch): Add an #:open-connection keyword argument. Christopher Baines
2021-02-22http-client: Add error handling to http-multiple-get....Making sure to close the port if it looks to be unusable. This closing of the port will allow for caching connections, without caching broken connections, as the cache can avoid handing out closed ports. * guix/http-client.scm (http-multiple-get): Try to catch exceptions that happen if the port is unusable, this is a adaptation of code within the (guix scripts substitute) module. Christopher Baines
2021-02-22guix: Move http-multiple-get to (guix http-client)....From (guix scripts substitute). This will make it easier to reuse this code. * guix/scripts/substitute.scm (http-multiple-get): Remove, and move to… * guix/http-client.scm (http-multiple-get): …here. Christopher Baines
2020-12-08substitute: Cache and reuse connections while substituting....That way, when fetching a series of substitutes from the same server(s), the connection is reused instead of being closed/opened for each substitutes, which saves on network round trips and TLS handshakes. * guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it. * guix/progress.scm (progress-report-port): Add #:close? parameter and honor it. * guix/scripts/substitute.scm (at-most): Return the tail as a second value. (fetch): Add #:port and #:keep-alive? and honor them. (%max-cached-connections): New variable. (open-connection-for-uri/cached, call-with-cached-connection): New procedures. (with-cached-connection): New macro. (process-substitution): Wrap 'fetch' call in 'with-cached-connection'. Pass #:close? to 'progress-report-port'. Ludovic Courtès
2020-10-12http-client: 'http-fetch' and 'http-fetch/cached' accept #:timeout....* guix/http-client.scm (http-fetch): Add #:timeout and pass it to 'guix:open-connection-for-uri'. (http-fetch/cached): Add #:timeout parameter and pass it to 'http-fetch'. Ludovic Courtès
2020-01-03download: Do not leak file descriptors on TLS ports....Fixes <https://bugs.gnu.org/20145>. * guix/build/download.scm (%tls-ports, register-tls-record-port): Remove. (tls-wrap): Remove call to 'register-tls-record-port'. Return a custom binary input/output port instead. This is a backport of what Guile 2.2's (web client) module has been doing. (close-connection): Define as an alias for 'close-port'. * guix/http-client.scm (http-fetch): Remove #:keep-alive? parameter, which was ignored and unused. Pass #:keep-alive? #f to 'http-get'. * guix/lint.scm (probe-uri): Use 'close-port' instead of 'close-connection'. * guix/scripts/substitute.scm (http-multiple-get): Likewise. Ludovic Courtès
2019-01-09Remove most uses of the _IO*F constants....These constants, for use with 'setvbuf', were deprecated in Guile 2.2 and disappeared in Guile 3.0. Here we keep these constants in build-side code where removing them is not feasible. * guix/build/download-nar.scm (download-nar): Adjust 'setvbuf' calls to the Guile 2.2+ API. * guix/build/download.scm (open-socket-for-uri): Likewise. (open-connection-for-uri, url-fetch): Likewise. * guix/build/make-bootstrap.scm (make-stripped-libc): Likewise. * guix/build/union.scm (setvbuf) [guile-2.0]: New conditional wrapper. (union-build): Adjust to new API. * guix/ftp-client.scm (ftp-open, ftp-list, ftp-retr): Likewise. * guix/http-client.scm (http-fetch): Likewise. * guix/inferior.scm (proxy): Likewise. * guix/scripts/substitute.scm (fetch, http-multiple-get): Likewise. * guix/self.scm (compiled-modules): Likewise. * guix/ssh.scm (remote-daemon-channel, store-import-channel) (store-export-channel): Likewise. * guix/ui.scm (initialize-guix): Likewise. * tests/publish.scm (http-get-port): Likewise. * guix/store.scm (%newlines): Adjust comment. Ludovic Courtès
2018-09-04Switch to Guile-Gcrypt....This removes (guix hash) and (guix pk-crypto), which now live as part of Guile-Gcrypt (version 0.1.0.) * guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm, tests/hash.scm, tests/pk-crypto.scm: Remove. * configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and LIBGCRYPT_LIBDIR assignments. * m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove. * README: Add Guile-Gcrypt to the dependencies; move libgcrypt as "required unless --disable-daemon". * doc/guix.texi (Requirements): Likewise. * gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm, guix/git.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm, guix/import/gnu.scm, guix/import/hackage.scm, guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm, guix/pki.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/download.scm, guix/scripts/hash.scm, guix/scripts/pack.scm, guix/scripts/publish.scm, guix/scripts/refresh.scm, guix/scripts/substitute.scm, guix/store.scm, guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/crate.scm, tests/derivations.scm, tests/gem.scm, tests/nar.scm, tests/opam.scm, tests/pki.scm, tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm, tests/store.scm, tests/substitute.scm: Adjust imports. * gnu/system/vm.scm: Likewise. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (expression->derivation-in-linux-vm)[config]: Remove. (iso9660-image)[config]: Remove. (qemu-image)[config]: Remove. (system-docker-image)[config]: Remove. * guix/scripts/pack.scm: Adjust imports. (guile-sqlite3&co): Rename to... (gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT. (self-contained-tarball)[build]: Call 'make-config.scm' without #:libgcrypt argument. (squashfs-image)[libgcrypt]: Remove. [build]: Call 'make-config.scm' without #:libgcrypt. (docker-image)[config, json]: Remove. [build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from the imported modules. * guix/self.scm (specification->package): Remove "libgcrypt", add "guile-gcrypt". (compiled-guix): Remove #:libgcrypt. [guile-gcrypt]: New variable. [dependencies]: Add it. [*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call. Add #:extensions. [*config*]: Remove #:libgcrypt from 'make-config.scm' call. (%dependency-variables): Remove %libgcrypt. (make-config.scm): Remove #:libgcrypt. * build-aux/build-self.scm (guile-gcrypt): New variable. (make-config.scm): Remove #:libgcrypt. (build-program)[fake-gcrypt-hash]: New variable. Add (gcrypt hash) to the imported modules. Adjust load path assignments. * gnu/packages/package-management.scm (guix)[propagated-inputs]: Add GUILE-GCRYPT. [arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search path. Ludovic Courtès
2018-05-26http-client: Send redirection messages to stderr....* guix/http-client.scm (http-fetch): Use CURRENT-ERROR-PORT instead of default output. Marius Bakke
2018-02-26build: Require Guile >= 2.0.13....* README, configure.ac, doc/guix.texi (Requirements): Increase minimum Guile version from 2.0.9 to 2.0.13. * config-daemon.ac: Remove use of 'GUIX_CHECK_UNBUFFERED_CBIP'. * m4/guix.m4 (GUIX_CHECK_UNBUFFERED_CBIP): Remove. * guix/build/download.scm (current-http-proxy): Remove. * guix/build/syscalls.scm (%libc-errno-pointer, errno): Remove. (syscall->procedure): Use #:return-errno unconditionally. * guix/hash.scm (open-sha256-input-port)[unbuffered]: Remove outdated comment. * guix/http-client.scm (when-guile<=2.0.5-or-otherwise-broken): Remove. <top level>: Remove 'when-guile<=2.0.5-or-otherwise-broken' block. * guix/scripts/substitute.scm (fetch): Remove 'guile-version>?' conditional. * tests/hash.scm (supports-unbuffered-cbip?): Remove. <top level>: Remove 'test-skip' call. Ludovic Courtès
2017-11-16cve: Use 'http-fetch/cached' instead of having custom caching....That way CVE fetching benefits from 'If-Modified-Since' handling. * guix/http-client.scm (http-fetch/cached): Add #:write-cache and #:cache-miss parameters and honor them. * guix/cve.scm (%current-year-ttl, %past-year-ttl): Reduce. (call-with-cve-port): Remove. (write-cache): New procedure. (fetch-vulnerabilities): Rewrite in terms of 'http-fetch/cached'. Ludovic Courtès
2017-09-26http-client: Reset cache TTL upon 304 "Not Modified" responses....* guix/http-client.scm (http-fetch/cached)[update-cache]: Add call to 'utime' in the 304 case. Ludovic Courtès
2017-09-19http-client: 'http-client/cached' uses 'If-Modified-Since'....* guix/http-client.scm (http-fetch/cached)[update-cache]: Add 'cache-port' parameter. Check its mtime and compute 'if-modified-since' header accordingly. Guard 'http-get-error?' and honor 304. Adjust callers of 'update-cache'. * guix/gnu-maintenance.scm (ftp.gnu.org-files): Set #:ttl to 15m. Ludovic Courtès
2017-09-05Handle the same HTTP redirects everywhere....* guix/build/download.scm (http-fetch): Complete the hard-coded list of HTTP redirection status codes. * guix/http-client.scm (http-fetch): Likewise. * guix/scripts/lint.scm (probe-uri): Likewise. Tobias Geerinckx-Rice
2017-05-03ui: Rename '_' to 'G_'....This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`". Ludovic Courtès
2017-03-18http-client: Avoid name clash with 'open-connection-for-uri' in 2.2.0....* guix/build/download.scm (open-connection-for-uri): Add note about same-named binding in Guile 2.2.0. * guix/http-client.scm: Use 'guix:open-connection-for-uri' for the procedure coming from (guix build download). * guix/scripts/lint.scm: Likewise. * guix/scripts/substitute.scm: Likewise. Ludovic Courtès
2017-03-18build: Require Guile >= 2.0.9....* configure.ac: Bump requirement to 2.0.9. * doc/guix.texi (Requirements): Adjust accordingly. * README (Requirements): Likewise. * build-aux/download.scm: Remove workaround for <http://bugs.gnu.org/13095>. * guix/build/download.scm: Likewise. (http-fetch)[post-2.0.7?]: Remove. Remove conditional code for not POST-2.0.7?. * guix/http-client.scm: Remove workaround for <http://bugs.gnu.org/13095>. (http-fetch)[post-2.0.7?]: Remove. Remove conditional code for not POST-2.0.7?. * guix/serialization.scm (read-latin1-string): Remove mention of 2.0.9. * tests/nar.scm: Use (ice-9 control). (let/ec): Remove. Ludovic Courtès
2017-01-13http-client: Provide 'User-Agent' header by default....* guix/http-client.scm (http-fetch): Add #:headers parameter and honor it. Rename 'auth-header' to 'headers'. * guix/import/github.scm (json-fetch*): Add comment about required User-Agent. Ludovic Courtès
2017-01-10http-client: Improve error reporting....* guix/http-client.scm (http-fetch): Change message in &message condition to include URI, CODE, and the reason phrase. Ludovic Courtès
2016-11-12http-client: Add #:verify-certificate? to 'http-fetch'....* guix/http-client.scm (http-fetch): Add #:verify-certificate? parameter and pass it to 'open-connection-for-uri'. Ludovic Courtès
2016-10-19Use (ice-9 binary-ports) instead of (rnrs io ports)....This reduces the closure of (guix ui) from 123 to 106 modules. * guix/derivations.scm: Use (ice-9 binary-ports) instead of (rnrs io ports). (map-derivation)[substitute-file]: Use 'read-string' instead of 'get-string-all'. * guix/ftp-client.scm: Likewise. * guix/hash.scm: Likewise. * guix/http-client.scm: Likewise. * guix/pki.scm (ensure-acl, current-acl): Likewise. * guix/scripts/archive.scm (authorize-key)[read-key]: Likewise. * guix/scripts/authenticate.scm (read-canonical-sexp) (read-hash-data): Likewise. * guix/scripts/download.scm: Likewise. * guix/scripts/offload.scm (register-gc-root, remove-gc-roots) (send-files): Likewise. * guix/scripts/publish.scm (lazy-read-file-sexp): Likewise. * guix/scripts/refresh.scm: Likewise. * guix/scripts/substitute.scm (check-acl-initialized): Likewise. * guix/serialization.scm (read-maybe-utf8-string): Likewise. * guix/scripts/hash.scm (guix-hash): Use 'force-output' instead of 'flush-output-port'. * guix/store.scm (process-stderr): Likewise. * guix/tests.scm: Likewise. * guix/utils.scm: Use (ice-9 binary-ports) and autoload (rnrs io ports) for 'make-custom-binary-input-port'. Ludovic Courtès
2016-03-16http-client: No 'setvbuf' for non-file ports....* guix/http-client.scm (http-fetch): Do not call 'setvbuf' on non-file ports. Ludovic Courtès
2016-03-14http-client: Add #:keep-alive? parameter....* guix/http-client.scm (http-fetch): Add #:keep-alive? parameter and pass it to 'http-get' or 'http-get*'. Ludovic Courtès
2016-02-25http-client: 'http-client/cached' uses unique cache file names....* guix/http-client.scm (cache-file-for-uri): New procedure. (http-fetch/cached): Use it. Remove 'directory' variable. [update-cache]: Make the 'dirname' of FILE. Ludovic Courtès
2016-02-25http-client: 'http-fetch/cached' updates the cache atomically....* guix/http-client.scm (http-fetch/cached)[update-cache]: Use 'with-atomic-file-output' instead of 'call-with-output-file'. Ludovic Courtès
2016-01-06http-client: Work around <http://bugs.gnu.org/22273>....* guix/http-client.scm (read-header-line): New procedure. Use it. Ludovic Courtès
2016-01-06http-client: Fix erroneous comment....* guix/http-client.scm: Fix Guile version number in comment for Guile commit 5a10e41. Ludovic Courtès
2015-12-21http-client: Support basic authentication....* guix/http-client.scm (http-fetch): Add Authorization header to request when the URI contains userinfo. Ricardo Wurmus
2015-11-29http-client: 'http-fetch' converts strings to URIs....* guix/http-client.scm (http-fetch): Use 'string->uri' if URI is a string. Fixes a regression introduced in 8a5063f. Ludovic Courtès
2015-11-26http-client: 'http-fetch' and 'http-fetch/cached' support HTTPS....* guix/http-client.scm (http-fetch): Use 'open-connection-for-uri', to support HTTPS. Ludovic Courtès
2015-10-21http-client: '%http-cache-ttl' is really a parameter....Fixes a typo in commit 739ab68 that made it a procedure returning a parameter. * guix/http-client.scm (%http-cache-ttl): Turn into a parameter. Ludovic Courtès
2015-10-17http-client: Add 'http-fetch/cached'....* guix/utils.scm (cache-directory): New procedure. * guix/http-client.scm (%http-cache-ttl): New variable. (http-fetch/cached): New procedure. Ludovic Courtès
2015-09-26http-client: Backport Guile fix for 'read-chunk-header'....Fixes the wrong-type-arg exception initially reported at <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19976#5> by Ricardo Wurmus <rekado@elephly.net>. * guix/http-client.scm (read-chunk-header): Backport Guile commit 53b8d5f. Ludovic Courtès
2015-09-10http-client: Backport delimited input port fix from Guile....* guix/http-client.scm (make-delimited-input-port): Backport Guile commit 5a10e41. Monkey-patch 'make-delimited-input-port' for any Guile <= 2.0.11. Ludovic Courtès
2015-09-10http-client: Remove redundant import clause....* guix/http-client.scm: Remove redundant #:use-module (guix utils). Ludovic Courtès
2015-05-10http-client: Remove monkey patching for 2.0.5....* guix/http-client.scm (read-response-body*): Remove. (http-fetch): Remove hacks for 2.0.5. Ludovic Courtès
2015-05-06download: Work around Guile small-receive-buffer bug....Previously, code using directly (guix build download) was still affected by <http://bugs.gnu.org/15368>. This includes source derivations, the 'guix download' command, and (guix gnu-maintenance). 'guix substitute' was unaffected since it used (guix http-client), which already had the fix. * guix/http-client.scm (open-socket-for-uri): Remove. (http-fetch): Remove #:buffered? argument to 'open-socket-for-uri'; use 'setvbuf' instead. * guix/scripts/substitute.scm (fetch): Likewise. * guix/build/download.scm (open-socket-for-uri): New procedure, taken from guix/http-client.scm, but without the #:buffered? parameter. Ludovic Courtès
2015-04-08http-client: Add workaround for HTTP pipelining on Guile <= 2.0.9....Reported by Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>. * guix/http-client.scm (make-delimited-input-port): New procedure. Install it in (web response) for Guile <= 2.0.9. Ludovic Courtès
2015-03-03http-client: Monkey-patch 'make-chunked-input-port' on Guile <= 2.0.11....Fixes <http://bugs.gnu.org/19976>. * guix/http-client.scm (when-guile<=2.0.5): Rename to... (when-guile<=2.0.5-or-otherwise-broken): ... this. (%web-http): New variable. Monkey-patch 'make-chunked-input-port' when %WEB-HTTP defines 'read-chunk-body'. Ludovic Courtès
2015-03-03http-client: Update backport of chunked encoding support to Guile 2.0.5....* guix/http-client.scm (read-chunk, read-chunk-body) [when-guile<=2.0.5]: Remove. (make-chunked-input-port) [when-guile<=2.0.5]: Update to Guile commit 00d3ecf2. Ludovic Courtès