Age | Commit message (Collapse) | Author |
|
Previously the first argument to 'find' would show up high in profiles
of 'package-derivation'. This speeds things up a big, especially on
Guile 3.
* guix/derivations.scm (coalesce-duplicate-inputs)[find]: New
procedure.
|
|
* guix/derivations.scm (derivation-input-fold): New procedure.
(substitution-oracle)[closure]: Rewrite in terms of
'derivation-input-fold'.
* tests/derivations.scm ("derivation-input-fold"): New test.
|
|
The hit rate of the 'add-data-to-store' cache goes from 10% to 4% on:
guix build -e '(@@ (gnu packages libreoffice) libreoffice)' -nd
* guix/derivations.scm (%module-cache): Turn into a regular hash table.
It didn't make sense to use a weak-value hash table given that values
are pairs.
|
|
This is a followup to d727a9343d861cf775645df8be5bfefd43d6c6f0, which
broke 'hydra-jobs' from (gnu ci).
* guix/derivations.scm (invalidate-derivation-caches!): Remove call to
'invalidate-memoization!' for 'derivation->bytevector'.
|
|
Its hit rate was only 8%. Removing it reduces heap size of "guix build
libreoffice -nd" from 69MiB to 61MiB and the wall-clock time is unchanged.
* guix/derivations.scm (derivation->bytevector): Change from 'mlambda'
to 'lambda'.
|
|
This reduces the number of lookups in the 'add-data-to-store' cache from
7505 to 3329 (hit rate from 68% to 27%) when running:
GUIX_PROFILING=add-data-to-store-cache guix build libreoffice -nd
The execution time of "guix build libreoffice -nd" goes from 2.12s to 1.87s.
* guix/derivations.scm (%module-cache): New variable.
(imported+compiled-modules)[key]: New variable.
Lookup KEY in %MODULE-CACHE and populate %MODULE-CACHE upon cache miss.
|
|
* guix/derivations.scm (imported+compiled-modules): New procedure.
(build-expression->derivation): Use it instead of separate calls to
'%imported-modules' and '%compiled-modules'.
|
|
* guix/derivations.scm (derivation-build-plan): Fix typo in the
docstring.
|
|
Fixes <https://bugs.gnu.org/36777>.
Reported by Carl Dong <contact@carldong.me>.
* guix/derivations.scm (derivation/masked-inputs): Call
'delete-duplicates' on INPUTS.
* tests/derivations.scm ("derivation with duplicate fixed-output inputs"):
New test.
|
|
* guix/derivations.scm (derivation-input-output-path): New procedure.
* guix/gexp.scm (lower-gexp): Wrap GUILE in a <derivation-input>.
(gexp->derivation): Adjust accordingly.
* guix/remote.scm (remote-pipe-for-gexp, remote-eval): Adjust
accordingly.
* tests/gexp.scm ("lower-gexp"): Adjust accordingly.
|
|
We will eventually require #:inputs to be a list of <derivation-input>;
store items will have to be passed as #:sources, already interned.
* guix/derivations.scm (warn-about-derivation-deprecation): New procedure.
(derivation): Add #:%deprecation-warning? parameter.
[warn-deprecation]: New macro.
[input->derivation-input, input->source]: Use it.
(build-expression->derivation): Pass #:%deprecation-warning?.
* po/guix/POTFILES.in: Add guix/derivations.scm.
|
|
* guix/derivations.scm (map-derivation)[input->output-paths]: Adjust to
deal with an argument that's either 'derivation-input?' or a string.
[rewritten-input]: Return a <derivation-input> or a string.
Pass #:inputs and #:sources to 'derivation'.
|
|
This brings us closer to the <derivation> data type.
* guix/derivations.scm (derivation): Add #:sources parameter.
[input->derivation-input]: Add clause for 'derivation-input?'.
Honor SOURCES.
|
|
* guix/derivations.scm (substitution-oracle)[valid?, dependencies]:
Remove.
[closure]: New procedure.
Rename parameter from 'drv' to 'inputs-or-drv' and adjust accordingly.
(derivation-build-plan): Pass INPUTS directly to 'substitution-oracle'.
* guix/ui.scm (show-what-to-build)[substitutable-info]: Likewise.
|
|
This fixes a bug whereby "guix build texlive -n" would report:
0.0 MB would be downloaded:
/gnu/store/…-texlive-20180414
instead of:
The following derivation would be built:
/gnu/store/…-texlive-texmf-20180414.drv
2,595.2 MB would be downloaded:
/gnu/store/…-texlive-20180414-texmf.tar.xz
/gnu/store/…-texlive-20180414
where 'texlive-texmf' is a non-substitutable dependency of 'texlive'.
* guix/derivations.scm (dependencies-of-substitutables): New procedure.
(derivation-build-plan): When 'input-substitutable-info' returns true,
append the subset of DEPS that corresponds to SUBSTITUABLES to the first
argument of 'loop'.
* guix/ui.scm (show-what-to-build): Remove half-baked traversal of
DOWNLOAD.
* tests/derivations.scm ("derivation-build-plan and substitutes,
non-substitutable dep"): New test.
|
|
This is a followup to 5cf4b26d52bcea382d98fb4becce89be9ee37b55.
* guix/derivations.scm (substitution-oracle): Use
'derivation-input-derivation' instead of calling
'read-derivation-from-file'.
|
|
This is a followup to 5cf4b26d52bcea382d98fb4becce89be9ee37b55.
* guix/derivations.scm (derivation-path->base16-hash): Rename to...
(derivation-base16-hash): ... this, and take a <derivation> rather than
a file name. Use 'mlambdaq' instead of 'mlambda'.
(derivation/masked-inputs): Adjust accordingly.
(invalidate-derivation-caches!): Likewise.
|
|
This is a followup to 5cf4b26d52bcea382d98fb4becce89be9ee37b55.
* guix/derivations.scm (map-derivation)[rewritten-input]: Avoid call to
'read-derivation-from-file'.
|
|
This works around <https://bugs.gnu.org/36487>.
* guix/derivations.scm (invalidate-derivation-caches!): Comment out
'hash-clear!' call.
|
|
This is a followup to 7c690a47381f645ec5ec0a1fd6ffc34dba1b69c2.
* guix/derivations.scm (build-derivations): Update docstring.
|
|
* guix/derivations.scm (build-derivations): Accept <derivation-input>
records among DERIVATIONS.
* tests/derivations.scm ("build-derivations with specific output"): Test
it.
|
|
* guix/derivations.scm (derivation): Check if FILE is already in
%DERIVATION-CACHE and return it if it is.
|
|
Consequently, the whole graph of <derivation> object is readily
available without having to go through 'read-derivation-from-file',
which could have cache misses if the requested <derivation> object had
been GC'd in the meantime. This is an important property for the
performance of things like 'derivation-build-plan' that traverse the
derivation graph.
* guix/derivations.scm (<derivation-input>): Replace 'path' field by
'derivation'.
(derivation-input-path): Adjust accordingly.
(derivation-input-key): New procedure.
(derivation-input-output-paths): Adjust accordingly.
(coalesce-duplicate-inputs): Likewise.
(derivation-prerequisites): Use 'derivation-input-key' to compute keys
for INPUT-SET.
(derivation-build-plan): Likewise.
(read-derivation): Add optional 'read-derivation-from-file' parameter.
[make-input-drvs]: Call it.
(write-derivation)[write-input]: Adjust to new <derivation-input>.
(derivation/masked-inputs): Likewise, and remove redundant
'coalesce-duplicate-inputs' call.
(derivation)[input->derivation-input]: Change to consider only the
derivation case. Update call to 'make-derivation-input'.
[input->source]: New procedure.
Separate sources from inputs.
(map-derivation): Adjust to new <derivation-input>.
* tests/derivations.scm ("parse & export"): Pass a second argument to
'read-derivation'.
("build-expression->derivation and derivation-prerequisites")
("derivation-prerequisites and valid-derivation-input?"): Adjust to new
<derivation-input>.
|
|
The new 'derivation-build-plan' procedure has a more appropriate
signature: it takes a list of <derivation-inputs> instead of taking one
<derivation>. Its body is also much simpler.
* guix/derivations.scm (derivation-build-plan): New procedure.
(derivation-prerequisites-to-build): Express in terms of
'derivation-build-plan' and mark as deprecated.
* tests/derivations.scm: Change 'derivation-prerequisites-to-build'
tests to 'derivation-build-plan' and adjust accordingly.
|
|
* guix/derivations.scm (derivation-input): New procedure.
* tests/grafts.scm (make-derivation-input): Remove.
("graft-derivation, unused outputs not depended on"): Use
'derivation-input'.
|
|
* guix/derivations.scm (derivation-input-derivation): New procedure.
(derivation-prerequisites): Use it.
|
|
This allows callers to request the substitution of a single derivation
output.
* guix/store.scm (build-things): Accept derivation/output pairs among
THINGS.
* guix/derivations.scm (build-derivations): Likewise.
* tests/store.scm ("substitute + build-things with specific output"):
New test.
* tests/derivations.scm ("build-derivations with specific output"):
New test.
* doc/guix.texi (The Store): Adjust accordingly.
|
|
On a profile with 280 packages, this reduces the number of
'valid-paths?' RPCs made by 'guix package -nu' from 6K to 500.
* guix/derivations.scm (derivation-prerequisites-to-build)[built?]:
Memoize 'valid-path?' calls.
|
|
* guix/store.scm (&nix-error): Rename to...
(&store-error): ... this, and adjust users.
(&nix-connection-error): Rename to...
(&store-connection-error): ... this, and adjust users.
(&nix-protocol-error): Rename to...
(&store-protocol-error): ... this, adjust users.
(&nix-error, &nix-connection-error, &nix-protocol-error): Define these
condition types and their getters as deprecrated aliases.
* build-aux/run-system-tests.scm, guix/derivations.scm,
guix/grafts.scm, guix/scripts/challenge.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/scripts/offload.scm, guix/serialization.scm,
guix/ssh.scm, guix/tests.scm, guix/ui.scm,
tests/derivations.scm, tests/gexp.scm, tests/guix-daemon.sh,
tests/packages.scm, tests/store.scm, doc/guix.texi: Adjust to use the
new names.
|
|
* guix/derivations.scm (derivation): Add #:properties parameter.
[user+system-env-vars]: Honor it.
(derivation-properties): New procedure.
(build-expression->derivation): Add #:properties and pass it to
'derivation'.
* guix/gexp.scm (gexp->derivation): Likewise.
* tests/derivations.scm ("derivation-properties"): New test.
* tests/gexp.scm ("gexp->derivation properties"): New test.
* doc/guix.texi (Derivations, G-Expressions): Adjust accordingly.
|
|
This removes (guix hash) and (guix pk-crypto), which now live as part of
Guile-Gcrypt (version 0.1.0.)
* guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm,
tests/hash.scm, tests/pk-crypto.scm: Remove.
* configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and
LIBGCRYPT_LIBDIR assignments.
* m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove.
* README: Add Guile-Gcrypt to the dependencies; move libgcrypt as
"required unless --disable-daemon".
* doc/guix.texi (Requirements): Likewise.
* gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm,
guix/git.scm, guix/http-client.scm, guix/import/cpan.scm,
guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm,
guix/import/gnu.scm, guix/import/hackage.scm,
guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm,
guix/pki.scm, guix/scripts/archive.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/scripts/pack.scm,
guix/scripts/publish.scm, guix/scripts/refresh.scm,
guix/scripts/substitute.scm, guix/store.scm,
guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm,
tests/builders.scm, tests/challenge.scm, tests/cpan.scm,
tests/crate.scm, tests/derivations.scm, tests/gem.scm,
tests/nar.scm, tests/opam.scm, tests/pki.scm,
tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm,
tests/store.scm, tests/substitute.scm: Adjust imports.
* gnu/system/vm.scm: Likewise.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT.
(expression->derivation-in-linux-vm)[config]: Remove.
(iso9660-image)[config]: Remove.
(qemu-image)[config]: Remove.
(system-docker-image)[config]: Remove.
* guix/scripts/pack.scm: Adjust imports.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT.
(self-contained-tarball)[build]: Call 'make-config.scm' without
#:libgcrypt argument.
(squashfs-image)[libgcrypt]: Remove.
[build]: Call 'make-config.scm' without #:libgcrypt.
(docker-image)[config, json]: Remove.
[build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from
the imported modules.
* guix/self.scm (specification->package): Remove "libgcrypt", add
"guile-gcrypt".
(compiled-guix): Remove #:libgcrypt.
[guile-gcrypt]: New variable.
[dependencies]: Add it.
[*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call.
Add #:extensions.
[*config*]: Remove #:libgcrypt from 'make-config.scm' call.
(%dependency-variables): Remove %libgcrypt.
(make-config.scm): Remove #:libgcrypt.
* build-aux/build-self.scm (guile-gcrypt): New variable.
(make-config.scm): Remove #:libgcrypt.
(build-program)[fake-gcrypt-hash]: New variable.
Add (gcrypt hash) to the imported modules. Adjust load path
assignments.
* gnu/packages/package-management.scm (guix)[propagated-inputs]: Add
GUILE-GCRYPT.
[arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search
path.
|
|
* guix/derivations.scm (derivation-prerequisites): Fix typo in docstring.
|
|
This reduces max RSS from 1.3G to 1.0G.
* guix/derivations.scm (invalidate-derivation-caches!): New procedure.
* build-aux/hydra/gnu-system.scm (hydra-jobs): Use it. Add 'format'
call.
|
|
This has little or no run-time impact and slightly reduces the memory
footprint.
* guix/derivations.scm (derivation-hash): Replace 'mlambda' with
'lambda'.
|
|
* guix/derivations.scm (derivation/masked-inputs): New procedure.
(derivation-hash): Use it instead of the inline code.
|
|
This reverts commit 1d008d9f8c44dfdb808235d451b72f255e72f103.
Reported by Rutger Helling <rhelling@mykolab.com>.
Fixes <https://bugs.gnu.org/29689>.
|
|
* guix/derivations.scm (derivation-hash): Remove
redundant 'coalesce-duplicate-inputs' call.
|
|
This is a followup to dc673fa1131fb5d1e5ca29acb4a693cfb906986f.
* guix/derivations.scm (<derivation-output>, <derivation-input>): Use
'define-immutable-record-type'.
|
|
This avoids the open/fstat/close syscalls upon a cache hit that we had
with the previous idiom:
(call-with-input-file file read-derivation)
where caching happened in 'read-derivation' itself.
* guix/derivations.scm (%read-derivation): Rename to...
(read-derivation): ... this.
(read-derivation-from-file): New procedure.
(derivation-prerequisites, substitution-oracle)
(derivation-prerequisites-to-build):
(derivation-path->output-path, derivation-path->output-paths):
(derivation-path->base16-hash, map-derivation): Use
'read-derivation-from-file' instead of (call-with-input-file …
read-derivation).
* guix/grafts.scm (item->deriver): Likewise.
* guix/scripts/build.scm (log-url, options->things-to-build): Likewise.
* guix/scripts/graph.scm (file->derivation): Remove.
(derivation-dependencies, %derivation-node-type): Use
'read-derivation-from-file' instead.
* guix/scripts/offload.scm (guix-offload): Likewise.
* guix/scripts/perform-download.scm (guix-perform-download): Likewise.
* guix/scripts/publish.scm (load-derivation): Remove.
(narinfo-string): Use 'read-derivation-from-file'.
|
|
* guix/derivations.scm (derivation-prerequisites-to-build): Rename
#:substitutable? to #:substitutable-info.
[derivation-substitutable?]: Rename to...
[derivation-substitutable-info]: ... this. Return a list of <substitutable>.
Second return value is now a list of <substitutable> instead of a list
of strings.
* guix/ui.scm (show-what-to-build)[substitutable?]: Rename to...
[substitutable-info]: ... this.
Adjust to new 'derivation-prerequisites-to-build' return value type.
* tests/derivations.scm ("derivation-prerequisites-to-build and
substitutes"): Adjust.
("derivation-prerequisites-to-build and substitutes, local build"):
Likewise.
|
|
* guix/derivations.scm (substitution-oracle): Use
'substitution-path-info' instead of 'substitution-paths'. Turn SUBST
into a vhash from path to <substitutable>. Change the returned
procedure to provide a <substitutable> instead of a Boolean.
* tests/derivations.scm ("substitution-oracle and #:substitute? #f"):
Mock 'substitutable-path-info' instead of 'substitutable-paths'.
|
|
* guix/derivations.scm (compressed-hash, store-path)
(output-path, fixed-output-path): Move to...
* guix/store.scm: ... here.
|
|
Reported by Mark H Weaver <mhw@netris.org>
at <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00457.html>.
* guix/derivations.scm (build-expression->derivation): Use a string
output port for the expression. This reverts part of
2dce88d5bbe7a65e101c0734d1c6db44ecc8c299.
* tests/derivations.scm ("build-expression->derivation and builder
encoding"): New test.
|
|
On Guile 2.2.1, this yields a 5% speedup on:
guix build libreoffice xmonad certbot -n --no-substitutes --no-build-hook
* guix/derivations.scm (derivation->string): Rename to...
(derivation->bytevector): ... this. Use 'open-bytevector-output-port'
instead of 'call-with-output-string'.
(derivation-hash): Remove string-to-bytevector conversion before
'sha256' call.
(build-expression->derivation): Use 'add-data-to-store' and an
bytevector port instead of a string port for the expression.
|
|
This avoids connections to substitute servers for derivations that are
not substitutable anyway, such as profiles.
Reported by Andy Wingo.
* guix/derivations.scm (substitution-oracle): Skip derivations that do
not pass 'substitutable-derivation?'.
* tests/derivations.scm ("substitution-oracle and #:substitute? #f"):
New test.
|
|
* guix/utils.scm (bytevector->base16-string, base16-string->bytevector):
Move to...
* guix/base16.scm: ... here. New file.
* tests/utils.scm ("bytevector->base16-string->bytevector"): Move to...
* tests/base16.scm: ... here. New file.
* Makefile.am (MODULES): Add guix/base16.scm.
(SCM_TESTS): Add tests/base16.scm.
* build-aux/download.scm, guix/derivations.scm,
guix/docker.scm, guix/import/snix.scm, guix/pk-crypto.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/store.scm, tests/hash.scm,
tests/pk-crypto.scm: Adjust imports accordingly.
|
|
* gnu/packages.scm (find-newest-available-packages): Use 'mlambda'
instead of (memoize (lambda ...) ...).
* gnu/packages/bootstrap.scm (package-with-bootstrap-guile): Likewise.
* guix/build-system/gnu.scm (package-with-explicit-inputs)[rewritten-input]:
Likewise.
* guix/build-system/python.scm (package-with-explicit-python)[transform]:
Likewise.
* guix/derivations.scm (derivation->string): Likewise.
* guix/gnu-maintenance.scm (gnu-package?): Likewise.
* guix/modules.scm (module-file-dependencies): Likewise.
* guix/scripts/graph.scm (standard-package-set): Likewise.
* guix/scripts/lint.scm (official-gnu-packages*): Likewise.
* guix/store.scm (store-regexp*): Likewise.
* guix/utils.scm (location): Likewise.
|
|
* guix/combinators.scm (memoize): Remove.
* guix/memoization.scm: New file.
* Makefile.am (MODULES): Add it.
* gnu/packages.scm, gnu/packages/bootstrap.scm,
guix/build-system/gnu.scm, guix/build-system/python.scm,
guix/derivations.scm, guix/gnu-maintenance.scm,
guix/import/cran.scm, guix/import/elpa.scm,
guix/modules.scm, guix/scripts/build.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/store.scm, guix/utils.scm: Adjust imports accordingly.
|
|
* guix/derivations.scm (<derivation>): Make it immutable.
(derivation): Use generic 'set-field' instead of ad-hoc functional setter.
|
|
This leads a 13% speedup on 'guix build libreoffice -d' and 18% on
'guix build gnome -d'.
* guix/derivations.scm (%derivation-cache): New variable.
(read-derivation): Use it instead of the private 'cache' variable.
(derivation): Populate %DERIVATION-CACHE before returning.
|