summaryrefslogtreecommitdiff
path: root/guix/store.scm
AgeCommit message (Collapse)Author
2021-08-10store: Add 'map/accumulate-builds' cutoff to address pathological cases.Ludovic Courtès
Fixes <https://bugs.gnu.org/49439>. Reported by Ricardo Wurmus <rekado@elephly.net>. Previously, a command such as: guix environment pigx-scrnaseq could lead to unbounded memory growth and could even fail to complete when some items are missing from the store. This was because 'map/accumulate-builds' callees would keep making .drv build requests that were turned into <unresolved> nodes; in this case, there are often many identical build requests. Stopping accumulation earlier allows us to unlock the situation by proceeding with the first few build requests instead of spinning until we've accumulated all the build requests. * guix/store.scm (map/accumulate-builds): Define 'accumulation-cutoff'. Use a loop when iterating over LST and maintain a counter of unresolved nodes met so far; return when the counter exceeds ACCUMULATION-CUTOFF.
2021-06-18Start enabling substitutes from bordeaux.guix.gnu.org.Christopher Baines
In addition to substitutes from ci.guix.gnu.org. There are more changes that can be made in the future, but these changes seem like a good start. * config-daemon.ac (guix_substitute_urls): Add https://bordeaux.guix.gnu.org. * guix/scripts/substitute.scm (%default-substitute-urls): Add http://bordeaux.guix.gnu.org. * guix/store.scm (%default-substitute-urls): Add bordeaux.guix.gnu.org. * doc/guix.texi: Adjust accordingly. * doc/contributing.texi: Adjust accordingly.
2021-06-08store: 'references/cached' now uses a per-session cache.Ludovic Courtès
* guix/store.scm (%reference-cache): Remove. (%reference-cache-id): New variable. (references/cached): Rewrite in terms of it.
2021-06-08store: Remove 'references/substitutes'.Ludovic Courtès
This procedure lost its only user in commit 710854304b1ab29332edcb76f3de532e0724c197. * guix/store.scm (references/substitutes): Remove. * tests/store.scm ("references/substitutes missing reference info") ("references/substitutes with substitute info"): Remove.
2021-06-08store: Generalize cache lookup recording.Ludovic Courtès
* guix/store.scm (cache-lookup-recorder): New procedure. (record-cache-lookup!): Define in terms of it.
2021-06-08store: Support dynamic allocation of per-connection caches.Ludovic Courtès
* guix/store.scm (<store-connection>)[object-cache]: Remove. [caches]: New field. (open-connection, port->connection): Adjust '%make-store-connection' calls accordingly. (%store-connection-caches, %object-cache-id): New variables. (allocate-store-connection-cache, vector-set) (store-connection-cache, set-store-connection-cache) (set-store-connection-caches!, set-store-connection-cache!): New procedures. (cache-object-mapping): Add #:cache parameter. (set-store-connection-object-cache!): Remove. (lookup-cached-object): Use 'store-connection-cache'. (run-with-store): Use 'store-connection-caches' and 'set-store-connection-caches!'.
2021-06-01maint: Require Guile 3.0.Ludovic Courtès
* configure.ac: Require Guile 3.0. * doc/guix.texi (Requirements): Adjust accordingly. * gnu/packages/package-management.scm (guile2.2-guix): Remove. * guix/lint.scm (exception-with-kind-and-args?): Remove 'cond-expand'. * guix/scripts/deploy.scm (deploy-machine*): Likewise. * guix/store.scm (call-with-store): Likewise. * guix/swh.scm (http-get*, http-post*): Likewise. * guix/ui.scm (without-compiler-optimizations, guard*) (call-with-error-handling): Likewise.
2021-05-08store: Export 'connect-to-daemon'.Ludovic Courtès
* guix/store.scm (connect-to-daemon): Make public. Improve docstring.
2021-05-08store: 'open-connection' never returns #f.Ludovic Courtès
* guix/store.scm (open-connection)[handshake-error]: New procedure. Call it in code paths that would previously return #f.
2021-02-23store: Micro-optimize object cache lookup.Ludovic Courtès
This avoids a closure allocation when 'lookup-cached-object' is called. * guix/store.scm (lookup-cached-object): Avoid optional/keyword arguments and inline. (%mcached): Adjust accordingly.
2021-02-23store: Object cache profiling shows the number of entries.Ludovic Courtès
* guix/store.scm (record-cache-lookup!): Add 'size' variable; keep it up-to-date and display it.
2021-02-04store: 'store-path-hash-part' really returns false for invalid file names.Ludovic Courtès
The "store-path-hash-part #f", due to a SRFI-64 bug, was marked as successful even though 'store-path-hash-part' was throwing an exception. * guix/store.scm (store-path-hash-part): Really return #f.
2021-01-22store: Add 'find-roots' RPC.Ludovic Courtès
* guix/serialization.scm (read-string-pairs): New procedure. * guix/store.scm (read-arg): Add support for 'string-pairs'. (find-roots): New procedure. * tests/store.scm ("add-indirect-root and find-roots"): New test.
2020-11-23store: Fix ensure-path docstring.Mathieu Othacehe
Suggested by Ludovic Courtès. * guix/store.scm (ensure-path): Fix the docstring.
2020-11-22guix: store: Add ensure-path.Mathieu Othacehe
* guix/store.scm (ensure-path): New procedure.
2020-09-02store: 'set-build-options' sends LC_MESSAGES, not LC_ALL.Ludovic Courtès
* guix/store.scm (set-build-options): Change #:locale default value to (setlocale LC_MESSAGES).
2020-08-31ssh: 'send-files' displays a progress bar.Ludovic Courtès
* guix/store.scm (export-paths): Add #:start, #:progress, and #:finish parameters and honor them. * guix/ssh.scm (prepare-to-send, notify-transfer-progress) (notify-transfer-completion): New procedures. (send-files): Pass #:start, #:progress, and #:finish to 'export-paths'.
2020-08-28store: 'with-store' returns as many values as its body.Ludovic Courtès
Fixes <https://bugs.gnu.org/42912>. Reported by Ricardo Wurmus <rekado@elephly.net>. * guix/store.scm (call-with-store)[thunk]: Wrap call to PROC in 'call-with-values'. * tests/store.scm ("with-store, multiple values"): New test.
2020-06-26store: Use '=' instead of 'eqv?'.Ludovic Courtès
Fixes <https://bugs.gnu.org/42047>. Reported by Jan Nieuwenhuizen <janneke@gnu.org>. * guix/store.scm (open-connection): Use '=' instead of 'eqv?'. This works around <https://bugs.gnu.org/42060> while also being more accurate since the arguments are known to be integers.
2020-06-09store: Use buffered I/O for all protocol writesLars-Dominik Braun
* guix/store.scm (run-gc) Use buffered output port. (export-path) Same. (add-file-tree-to-store) Same. (set-build-options): Same. Add explicit flush. Signed-off-by: Ludovic Courtès <ludo@gnu.org>
2020-05-14store: 'mapm/accumulate-builds' preserves '%current-target-system'.Ludovic Courtès
Fixes <https://bugs.gnu.org/41182>. * guix/store.scm (mapm/accumulate-builds): Pass #:system and #:target to 'run-with-store'. * tests/store.scm ("mapm/accumulate-builds, %current-target-system"): New test. * tests/guix-pack.sh: Add 'guix pack -d --target' test.
2020-04-08Merge branch 'master' into core-updatesMarius Bakke
Conflicts: etc/news.scm gnu/local.mk gnu/packages/check.scm gnu/packages/cross-base.scm gnu/packages/gimp.scm gnu/packages/java.scm gnu/packages/mail.scm gnu/packages/sdl.scm gnu/packages/texinfo.scm gnu/packages/tls.scm gnu/packages/version-control.scm
2020-04-05store: 'with-store' uses 'with-exception-handler'.Ludovic Courtès
This ensures the stack is not unwound before the exception is re-thrown, as was the case since 8ed597f4a261fe188de82cd1f5daed83dba948eb, leading to '&store-protocol-error' being uncaught by 'with-error-handling' in (guix scripts build) & co. * guix/store.scm (call-with-store): Define 'thunk'. Add 'cond-expand' to use 'with-exception-handler' on 'guile-3' and 'catch' otherwise.
2020-04-04store: 'with-store' doesn't close the store upon abort.Ludovic Courtès
Fixes <https://bugs.gnu.org/40428>. Reported by Marius Bakke <mbakke@fastmail.com> and 白い熊. Regression introduced with the first uses of 'with-build-handler' in commit 62195b9a8fd6846117c5d7698842748300d13e31 and subsequent. * guix/store.scm (call-with-store): Use 'catch #t' instead of 'dynamic-wind'. This ensures STORE remains open when a non-local exit other than an exception occurs, such as an abort to the build handler prompt. * tests/store.scm ("with-build-handler + with-store"): New test.
2020-03-30Merge branch 'master' into core-updatesMarius Bakke
Conflicts: gnu/packages/admin.scm gnu/packages/commencement.scm gnu/packages/guile.scm gnu/packages/linux.scm gnu/packages/package-management.scm gnu/packages/pulseaudio.scm gnu/packages/web.scm
2020-03-29store: Add 'references/cached'.Ludovic Courtès
* guix/store.scm (references/cached): New procedure.
2020-03-29store: Add 'map/accumulate-builds'.Ludovic Courtès
* guix/store.scm (<unresolved>): New record type. (build-accumulator, map/accumulate-builds, mapm/accumulate-builds): New procedures. * tests/store.scm ("map/accumulate-builds", "mapm/accumulate-builds"): New tests.
2020-03-27Merge branch 'master' into core-updatesMarius Bakke
Conflicts: gnu/packages/icu4c.scm gnu/packages/man.scm gnu/packages/python-xyz.scm guix/scripts/environment.scm guix/scripts/pack.scm guix/scripts/package.scm guix/scripts/pull.scm guix/store.scm
2020-03-22store: Add 'with-build-handler'.Ludovic Courtès
* guix/store.scm (current-build-prompt): New variable. (call-with-build-handler, invoke-build-handler): New procedures. (with-build-handler): New macro. * tests/store.scm ("with-build-handler"): New test.
2020-03-18Merge branch 'master' into core-updatesMarius Bakke
2020-03-14store: Fix many guix commands failing on some locales.Florian Pelz
Partly fixes <https://bugs.gnu.org/39970>. At least 'guix environment', 'guix install' and 'guix pull' on 'az_AZ.utf8' and 'tr_TR.utf8' were affected. * guix/store.scm (store-path-hash-part): Move base path detection to ... (store-path-base): ... this new exported procedure. (store-path-package-name): Use it instead of locale-dependent regexps. (store-regexp*): Remove.
2020-03-08store: Add set-current-target procedure.Mathieu Othacehe
* guix/store.scm (set-current-target): New exported procedure.
2020-03-06store: Add set-current-target procedure.Mathieu Othacehe
* guix/store.scm (set-current-target): New exported procedure.
2020-01-30gexp: Move 'file-mapping->tree' to (guix store).Ludovic Courtès
* guix/gexp.scm (%not-slash): Remove. (file-mapping->tree): Move to... * guix/store.scm (file-mapping->tree): ... here.
2019-12-29profiles: Fix profile-derivation cross-compilation.Mathieu Othacehe
* guix/store.scm (current-target-system): New exported monadic procedure. * guix/profiles.scm (profile-derivation): Set target at bind time using the above procedure.
2019-12-26profiles: Fix profile-derivation cross-compilation.Mathieu Othacehe
* guix/store.scm (current-target-system): New exported monadic procedure. * guix/profiles.scm (profile-derivation): Set target at bind time using the above procedure.
2019-11-26Use 'offload?' instead of 'build-hook?' internally.Ludovic Courtès
* guix/scripts/archive.scm (%default-options): Replace 'build-hook?' with 'offload?'. * guix/scripts/build.scm (set-build-options-from-command-line): Pass #:offload? instead of #:use-build-hook?. (%standard-build-options): Use the 'offload? key instead of 'build-hook?. (%default-options): Replace 'build-hook?' with 'offload?'. * guix/scripts/copy.scm (%default-options): Likewise. * guix/scripts/deploy.scm (%default-options): Likewise. * guix/scripts/environment.scm (%default-options): Likewise. * guix/scripts/pack.scm (%default-options): Likewise. * guix/scripts/package.scm (%default-options): Likewise. * guix/scripts/pull.scm (%default-options): Likewise. * guix/scripts/system.scm (%default-options): Likewise. * guix/scripts/time-machine.scm (%default-options): Likewise. * guix/store.scm (set-build-options): Have #:use-build-hook? default to *unspecified*. Add #:offload?. Add call to 'warn-about-deprecation' when #:use-build-hook? is specified.
2019-10-27store: Allow objects in the cache to be inserted and search for with 'equal?'.Ludovic Courtès
* guix/store.scm (cache-object-mapping): Add #:vhash-cons parameter and honor it. (lookup-cached-object): Add #:vhash-fold* parameter and honor it. (%mcached): Add #:vhash-fold* and #:vhash-cons and honor them. (mcached): Add clauses with 'eq?' and 'equal?' as the first argument.
2019-10-16daemon: Make 'profiles/per-user' non-world-writable.Ludovic Courtès
Fixes <https://bugs.gnu.org/37744>. Reported at <https://www.openwall.com/lists/oss-security/2019/10/09/4>. Based on Nix commit 5a303093dcae1e5ce9212616ef18f2ca51020b0d by Eelco Dolstra <edolstra@gmail.com>. * nix/libstore/local-store.cc (LocalStore::LocalStore): Set 'perUserDir' to #o755 instead of #o1777. (LocalStore::createUser): New function. * nix/libstore/local-store.hh (LocalStore): Add it. * nix/libstore/store-api.hh (StoreAPI): Add it. * nix/nix-daemon/nix-daemon.cc (performOp): In 'wopSetOptions', add condition to handle "user-name" property and honor it. (processConnection): Add 'userId' parameter. Call 'store->createUser' when userId is not -1. * guix/profiles.scm (ensure-profile-directory): Note that this is now handled by the daemon. * guix/store.scm (current-user-name): New procedure. (set-build-options): Add #:user-name parameter and pass it to the daemon. * tests/guix-daemon.sh: Test the creation of 'profiles/per-user' when listening on a TCP socket. * tests/store.scm ("profiles/per-user exists and is not writable") ("profiles/per-user/$USER exists"): New tests.
2019-07-05store: 'run-with-store' gracefully deals with #f store.Ludovic Courtès
This is a followup to dd0ee954c4fa965023fd887452927c02edb8b52f, which introduced a failure in tests/graph.scm. * guix/store.scm (run-with-store): Check whether STORE and NEW-STORE are true before calling 'store-connection-object-cache' etc. Fixes a failure in tests/graph.scm related to %REVERSE-PACKAGE-NODE-TYPE, which uses #f as the store.
2019-07-03store: 'run-with-store' preserves the functional object cache.Ludovic Courtès
This increases the hit rate of the object cache from 56% to 85% when running something like 'guix build libreoffice -nd'. * guix/store.scm (set-store-connection-object-cache!): New procedure. (run-with-store): Copy 'object-cache' field of the new store into the original one.
2019-06-10store: 'build-things' accepts derivation/output pairs.Ludovic Courtès
This allows callers to request the substitution of a single derivation output. * guix/store.scm (build-things): Accept derivation/output pairs among THINGS. * guix/derivations.scm (build-derivations): Likewise. * tests/store.scm ("substitute + build-things with specific output"): New test. * tests/derivations.scm ("build-derivations with specific output"): New test. * doc/guix.texi (The Store): Adjust accordingly.
2019-05-26store: Fix 'hash-part->path' docstring.Ludovic Courtès
* guix/store.scm (hash-part->path): Adjust docstring to match reality.
2019-05-01build: Change default substitute server to "ci.guix.gnu.org".Ludovic Courtès
* config-daemon.ac: Replace "ci.guix.info" with "ci.guix.gnu.org". * doc/guix.texi (SUBSTITUTE-SERVER): Likewise. * etc/substitutes/ci.guix.gnu.org.pub: New file. * Makefile.am (dist_pkgdata_DATA): Add it. * guix/scripts/build.scm (%default-log-urls): Update. * guix/scripts/substitute.scm (%default-substitute-urls): Likewise. * guix/store.scm (%default-substitute-urls): Likewise. * guix/self.scm (miscellaneous-files): Add "ci.guix.gnu.org".
2019-04-16store: Add "add-data-to-store-cache" profiling component.Ludovic Courtès
* guix/store.scm (add-data-to-store): Define 'lookup' and use it instead of 'hash-ref'.
2019-04-16store: Memoize 'built-in-builders' call directly in <store-connection>.Ludovic Courtès
The caching strategy introduced in 40cc850aebb497faed0a11d867d8fcee729023df was ineffective since we regularly start from an empty object cache. For example, "guix build inkscape -n" would make 241 'built-in-builders' RPCs. * guix/store.scm (<store-connection>)[built-in-builders]: New field. (open-connection): Adjust '%make-store-connection' call accordingly. (port->connection): Likewise. (built-in-builders): Rename to... (%built-in-builders): ... this. (built-in-builders): New procedure. * guix/download.scm (built-in-builders*): Remove 'mcached' call.
2019-04-10store: 'with-store' expands to a single procedure call.Ludovic Courtès
* guix/store.scm (call-with-store): New procedure. (with-store): Write in terms of 'call-with-store'.
2019-02-04store: 'log-file' honors 'GUIX_LOG_DIRECTORY'.Ludovic Courtès
* guix/store.scm (derivation-log-file): Use %LOCALSTATEDIR or "GUIX_LOG_DIRECTORY" instead of (dirname %STATE-DIRECTORY).
2019-01-21store: Rename '&nix-error' to '&store-error'.Ludovic Courtès
* guix/store.scm (&nix-error): Rename to... (&store-error): ... this, and adjust users. (&nix-connection-error): Rename to... (&store-connection-error): ... this, and adjust users. (&nix-protocol-error): Rename to... (&store-protocol-error): ... this, adjust users. (&nix-error, &nix-connection-error, &nix-protocol-error): Define these condition types and their getters as deprecrated aliases. * build-aux/run-system-tests.scm, guix/derivations.scm, guix/grafts.scm, guix/scripts/challenge.scm, guix/scripts/graph.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/serialization.scm, guix/ssh.scm, guix/tests.scm, guix/ui.scm, tests/derivations.scm, tests/gexp.scm, tests/guix-daemon.sh, tests/packages.scm, tests/store.scm, doc/guix.texi: Adjust to use the new names.
2019-01-21store: Rename <nix-server> to <store-connection>.Ludovic Courtès
* guix/store.scm (<nix-server>): Rename to... (<store-connection>): ... this. Adjust users accordingly. (nix-server?, nix-server-major-version) (nix-server-minor-version, nix-server-socket) (nix-server-version): Define as deprecated aliases. * guix/inferior.scm: Adjust accordingly. * guix/ssh.scm: Likewise.
generated by cgit v1.2.3 (git 2.46.2) at 2024-11-05 11:28:05 +0000