| Age | Commit message (Expand) | Author |
|---|
| 2021-06-08 | grafts: Cache the derivation/graft mapping for the whole session....Partly fixes <https://bugs.gnu.org/41702>.
Reported by Lars-Dominik Braun <ldb@leibniz-psychology.org>.
Previously, 'graft-derivation' would start anew at every call. When
creating a profile with lots of packages, it would potentially do the
same work multiple times. The per-session cache addresses this. It
increases the derivation-graft-cache hit rate from 77.9% to 80.1% on:
GUIX_PROFILING="derivation-graft-cache" ./pre-inst-env \
guix environment --ad-hoc libreoffice inkscape krita darktable -n
The effect is more visible on the pathological case below, where cache
hit rate goes from 75% to 87% and wall-clock time from 5.0s to 3.5s:
GUIX_PROFILING="derivation-graft-cache" ./pre-inst-env \
guix environment --ad-hoc r-learnr --search-paths
* guix/grafts.scm (%graft-cache): New variable.
(graft-derivation): Add calls to 'store-connection-cache' and
'set-store-connection-cache!'.
| Ludovic Courtès |
| 2021-06-08 | grafts: Use SRFI-71 instead of SRFI-11....* guix/grafts.scm (reference-origins): Use SRFI-71 'let*'.
| Ludovic Courtès |
| 2021-06-08 | grafts: Record cache lookups for profiling....* guix/grafts.scm (record-cache-lookup!): New procedure.
(with-cache): Use it.
| Ludovic Courtès |
| 2021-02-23 | grafts: Inline 'grafting?' and 'set-grafting'....As for 'current-target-system' & co., this makes sure we don't
needlessly allocate closures.
* guix/grafts.scm (grafting?, set-grafting): Inline.
| Ludovic Courtès |
| 2020-06-06 | grafts: Improve performance for derivations with many inputs....Partly fixes <https://bugs.gnu.org/41702>.
Reported by Lars-Dominik Braun <ldb@leibniz-psychology.org>.
Previously we'd potentially traverse the same sub-graph of DEPS several
times.
With this patch, command:
guix environment --ad-hoc r-learnr --search-paths
goes from 11.3s to 4.6s.
* guix/grafts.scm (reference-origin): Rename to...
(reference-origins): ... this. Change 'item' parameter to 'items'.
[lookup-derivers]: New procedure.
(cumulative-grafts)[dependency-grafts]: Change 'item' to 'items' and use
'reference-origins'.
Remove 'mapm' around 'dependency-grafts' call.
| Ludovic Courtès |
| 2020-04-02 | grafts: Simplify access to store item references....This is a followup to 710854304b1ab29332edcb76f3de532e0724c197.
This also slightly reduces the number of 'query-references' RPCs, for
instance from 176 to 166 from "guix build emacs -d".
* guix/grafts.scm (references-oracle): Remove.
(non-self-references): Remove 'references' parameter and add 'store'.
Add 'references*' procedure and use it instead of 'references'. Adjust
caller accordingly.
(cumulative-grafts): Remove 'references' parameter and adjust caller
accordingly.
| Ludovic Courtès |
| 2020-03-29 | grafts: Don't rely on substitute info for missing store items....Fixes <https://bugs.gnu.org/22990>.
* guix/grafts.scm (references-oracle)[references*]: Remove call to
'substitution-oracle' and to 'references/substitutes'. Use
'references/cached' and 'build-derivations' right away instead.
| Ludovic Courtès |
| 2019-07-02 | grafts: 'references-oracle' now takes a derivation input....That way, if we end up calling 'build-derivations', we'll only build the
outputs that we really need.
* guix/grafts.scm (references-oracle): Rename 'drv' to 'input'.
[output-paths]: Remove.
Adjust accordingly.
(graft-derivation): Adjust call to 'references-oracle'.
| Ludovic Courtès |
| 2019-06-19 | grafts: Avoid 'query-valid-derivers' RPC....Previously we'd make 502 'query-valid-derivers' RPCs for
"guix build vim -d", and after this patch, we don't do any.
Furthermore, the previous strategy was "stateful" in the sense that
'item->deriver' could return a derivation that is not the one that was
actually computed by this process, but an "equivalent" one (due to
fixed-output derivations); which one is chosen would depend on the state
of the store.
This in turn means that we'd have to call 'read-derivation-from-file' to
actually read .drv files (as opposed to getting them from
%DERIVATION-CACHE). This is costly and doesn't work with
GUIX_DAEMON_SOCKET=ssh://….
* guix/grafts.scm (item->deriver): Remove.
(reference-origin): New procedure.
(cumulative-grafts): Use it instead of 'item->deriver'.
| Ludovic Courtès |
| 2019-01-21 | store: Rename '&nix-error' to '&store-error'....* guix/store.scm (&nix-error): Rename to...
(&store-error): ... this, and adjust users.
(&nix-connection-error): Rename to...
(&store-connection-error): ... this, and adjust users.
(&nix-protocol-error): Rename to...
(&store-protocol-error): ... this, adjust users.
(&nix-error, &nix-connection-error, &nix-protocol-error): Define these
condition types and their getters as deprecrated aliases.
* build-aux/run-system-tests.scm, guix/derivations.scm,
guix/grafts.scm, guix/scripts/challenge.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/scripts/offload.scm, guix/serialization.scm,
guix/ssh.scm, guix/tests.scm, guix/ui.scm,
tests/derivations.scm, tests/gexp.scm, tests/guix-daemon.sh,
tests/packages.scm, tests/store.scm, doc/guix.texi: Adjust to use the
new names.
| Ludovic Courtès |
| 2018-12-04 | grafts: Mark as non substitutable....* guix/grafts.scm (graft-derivation/shallow): Pass #:substitutable? to
'build-expression->derivation'.
| Ludovic Courtès |
| 2018-11-28 | grafts: Record metadata as derivation properties....* guix/grafts.scm (graft-derivation/shallow): Pass #:properties to
'build-expression->derivation'.
* tests/grafts.scm ("graft-derivation, grafted item is a direct
dependency"): Check the value returned by 'derivation-properties'.
| Ludovic Courtès |
| 2018-11-12 | store: Add a functional object cache and use it in 'lower-object'....This leads to ~25% improvements on things like:
guix system build desktop.tmpl --no-grafts -d
* guix/store.scm (<nix-server>)[object-cache]: New field.
* guix/store.scm (open-connection): Initialize it.
(cache-object-mapping, lookup-cached-object, %mcached): New procedures.
(mcached): New macro.
* guix/gexp.scm (lower-object): Use it.
* guix/grafts.scm (grafting?): New procedure.
| Ludovic Courtès |
| 2018-08-24 | grafts: Add (guix build debug-link) and use it....Fixes <https://bugs.gnu.org/19973>.
Reported by Mark H Weaver <mhw@netris.org>.
* guix/build/debug-link.scm: New file.
* guix/build/graft.scm (%graft-hooks): New variable.
(graft): Add #:hooks and honor it.
* guix/grafts.scm (graft-derivation/shallow): Add (guix build
debug-link) and (guix elf) to #:modules.
* tests/debug-link.scm: New file.
* Makefile.am (MODULES): Add guix/build/debug-link.scm.
(SCM_TESTS): Add tests/debug-link.scm.
| Ludovic Courtès |
| 2018-08-24 | grafts: Add high-level 'graft' procedure on the build side....* guix/build/graft.scm (graft): New procedure.
* guix/grafts.scm (graft-derivation/shallow)[build]: Use it instead of
inline code.
| Ludovic Courtès |
| 2017-06-12 | derivations: Introduce 'read-derivation-from-file'....This avoids the open/fstat/close syscalls upon a cache hit that we had
with the previous idiom:
(call-with-input-file file read-derivation)
where caching happened in 'read-derivation' itself.
* guix/derivations.scm (%read-derivation): Rename to...
(read-derivation): ... this.
(read-derivation-from-file): New procedure.
(derivation-prerequisites, substitution-oracle)
(derivation-prerequisites-to-build):
(derivation-path->output-path, derivation-path->output-paths):
(derivation-path->base16-hash, map-derivation): Use
'read-derivation-from-file' instead of (call-with-input-file …
read-derivation).
* guix/grafts.scm (item->deriver): Likewise.
* guix/scripts/build.scm (log-url, options->things-to-build): Likewise.
* guix/scripts/graph.scm (file->derivation): Remove.
(derivation-dependencies, %derivation-node-type): Use
'read-derivation-from-file' instead.
* guix/scripts/offload.scm (guix-offload): Likewise.
* guix/scripts/perform-download.scm (guix-perform-download): Likewise.
* guix/scripts/publish.scm (load-derivation): Remove.
(narinfo-string): Use 'read-derivation-from-file'.
| Ludovic Courtès |
| 2017-01-25 | grafts: Do not pull derivation outputs not depended on....Fixes <http://bugs.gnu.org/24886>.
Previously, the grafting derivation of, say, brdf-explorer would pull in
qt:doc even though brdf-explorer depends only on qt:out, not qt:doc.
* guix/grafts.scm (with-cache): Use 'vhash-assoc' and 'vhash-cons'
instead of 'vhash-assq' and 'vhash-consq'.
(cumulative-grafts): Pass #:outputs to 'graft-derivation/shallow'. Use
OUTPUTS instead of (derivation-output-names drv).
(graft-derivation): Add #:outputs parameter; pass it to
'cumulative-grafts'.
* tests/grafts.scm (make-derivation-input): New variable.
("graft-derivation, replaced derivation has multiple outputs"): Make
sure P2:zzz is not part of the outputs of P3D.
("graft-derivation with #:outputs")
("graft-derivation, unused outputs not depended on"): New tests.
| Ludovic Courtès |
| 2017-01-24 | grafts: Shallow grafting can be performed on a subset of the outputs....* guix/grafts.scm (graft-derivation/shallow): Add #:outputs parameter.
[outputs]: Rename to...
[output-pairs]: ... this. Adjust 'build-expression->derivation' call
accordingly.
| Ludovic Courtès |
| 2017-01-16 | grafts: Preserve the cache across recursive calls....Before this commit, we'd lose the cache across recursive calls to
'cumulative-grafts', which isn't great performance-wise.
This bug was already present before
d38bc9a9f6feefc465964531520fee5663a12f48.
* guix/grafts.scm (with-cache): In the miss case, call 'current-state'
after EXP has been evaluated.
| Ludovic Courtès |
| 2017-01-16 | grafts: Actually cache grafts during the derivation DAG traversal....This fixes a regression introduced in
d38bc9a9f6feefc465964531520fee5663a12f48 whereby the cache was
effectively disabled.
Reported by Thomas Danckaert <thomas.danckaert@gmail.com>.
* guix/grafts.scm (with-cache): In the cache miss case, wrap body in
'mbegin'.
| Ludovic Courtès |
| 2017-01-04 | grafts: Move caching to a new 'with-cache' macro....* guix/grafts.scm (with-cache): New macro.
(cumulative-grafts)[return/cache]: Remove.
Use 'with-cache' instead.
| Ludovic Courtès |
| 2016-10-17 | grafts: Apply the right grafts in the presence of multiple outputs....Fixes <http://bugs.gnu.org/24712>.
* guix/grafts.scm (cumulative-grafts): Add grafts for all the outputs of
DRV.
* tests/grafts.scm ("graft-derivation, replaced derivation has multiple
outputs"): New test.
| Ludovic Courtès |
| 2016-10-15 | grafts: Remove unused variables and confusing monad use....* guix/grafts.scm (cumulative-grafts)[return/cache]: Use %STATE-MONAD,
not %STORE-MONAD.
Remove unused 'origins' variable and unnecessary inner 'cache'
variable.
| Ludovic Courtès |
| 2016-10-14 | grafts: 'graft-derivation' does now introduce grafts that shadow other grafts....Partly fixes <http://bugs.gnu.org/24418>.
* guix/grafts.scm (cumulative-grafts)[graft-origin?]: New procedure.
[dependency-grafts]: Use it in new 'if' around recursive call.
* tests/grafts.scm ("graft-derivation, grafts are not shadowed"): New test.
| Ludovic Courtès |
| 2016-05-25 | grafts: Create only one grafted variant of each derivation....Currently, with several grafts applicable to Inkscape, this makes:
guix gc -R $(guix build inkscape -d) | wc -l
go from 2376 to 2266 (4.6%).
* guix/grafts.scm (cumulative-grafts): Pass 'graft-derivation/shallow'
the subset of GRAFTS that applies to DRV.
| Ludovic Courtès |
| 2016-03-14 | grafts: Update the narinfo cache before building a derivation....* guix/grafts.scm (references-oracle)[references*]: Add call to
'substitution-oracle'.
| Ludovic Courtès |
| 2016-03-05 | grafts: Memoize intermediate results in 'cumulative-grafts'....The time for:
guix build inkscape -n --no-substitutes
goes down by 30% (in the presence of 3 replacements among all the
packages.)
* guix/grafts.scm (cumulative-grafts): Turn into a monadic procedure in
%STATE-MONAD. Use the current state as a derivation-to-graft cache.
(graft-derivation): Call 'cumulative-grafts' within 'run-with-state'.
| Ludovic Courtès |
| 2016-03-05 | grafts: Use dependency information from substitutes when possible....This avoids starting derivation builds just for the sake of knowing the
references of their outputs, thereby restoring the expected behavior of
--dry-run when substitutes are available.
* guix/grafts.scm (non-self-references): Remove 'store' parameter, and
add 'references'. Use it. Update caller.
(references-oracle): New variable.
(cumulative-grafts): Add 'references' parameter and use it. Update
callers.
(graft-derivation): Remove 'build-derivations' call. Add call to
'references-oracle'.
| Ludovic Courtès |
| 2016-03-01 | grafts: Graft recursively....Fixes <http://bugs.gnu.org/22139>.
* guix/grafts.scm (graft-derivation): Rename to...
(graft-derivation/shallow): ... this.
(graft-origin-file-name, item->deriver, non-self-references)
(cumulative-grafts, graft-derivation): New procedures
* tests/grafts.scm ("graft-derivation, grafted item is a direct
dependency"): Clarify title. Use 'grafted' instead of 'graft' to refer
to the grafted derivation.
("graft-derivation, grafted item is an indirect dependency")
("graft-derivation, no dependencies on grafted output"): New tests.
* guix/packages.scm (input-graft): Change to take a package instead of
an input.
(input-cross-graft): Likewise.
(fold-bag-dependencies): New procedure.
(bag-grafts): Rewrite in terms of 'fold-bag-dependencies'.
* tests/packages.scm ("package-derivation, indirect grafts"): Comment out.
* doc/guix.texi (Security Updates): Mention run-time dependencies and
recursive grafting.
| Ludovic Courtès |
| 2016-02-27 | grafts: Consider all the outputs in the graft mapping....Before that, outputs of a derivation could be left referring to the
ungrafted version of the derivation.
* guix/grafts.scm (graft-derivation)[outputs]: Change to a list of
name/file pairs.
* guix/grafts.scm (graft-derivation)[build]: Add 'old-outputs' variable
and use it when computing 'mapping'. Use 'mapping' directly.
* tests/grafts.scm ("graft-derivation, multiple outputs"): New test.
| Ludovic Courtès |
| 2016-02-27 | grafts: Slight simplification....* guix/grafts.scm (graft-derivation)[output-names]: Use
'derivation-output-names'.
| Ludovic Courtès |
| 2016-02-26 | grafts: Add record type printer....* guix/grafts.scm (write-graft): New procedure. Register it as a
printer for <graft>.
| Ludovic Courtès |
| 2016-02-22 | grafts: 'name' parameter of 'graft-derivation' is now optional....* guix/grafts.scm (graft-derivation): Name 'name' a keyword parameter.
* guix/packages.scm (package-derivation, package-cross-derivation):
Adjust accordingly.
* tests/grafts.scm ("graft-derivation"): Likewise.
* tests/packages.scm ("package-derivation, indirect grafts"): Likewise.
| Ludovic Courtès |
| 2016-02-22 | derivations: Move grafts to (guix grafts)....* guix/derivations.scm (<graft>, graft-derivation, %graft?)
(set-grafting): Move to...
* guix/grafts.scm: ... here. New file.
* guix/gexp.scm, guix/packages.scm, tests/packages.scm,
guix/scripts/build.scm: Use it.
* Makefile.am (MODULES): Add it.
(SCM_TESTS): Add tests/grafts.scm.
* tests/derivations.scm ("graft-derivation"): Move to...
* tests/grafts.scm: ... here. New file.
| Ludovic Courtès |
