Age | Commit message (Collapse) | Author |
|
This increases the hit rate of the object cache from 56% to 85% when
running something like 'guix build libreoffice -nd'.
* guix/store.scm (set-store-connection-object-cache!): New procedure.
(run-with-store): Copy 'object-cache' field of the new store into the
original one.
|
|
This allows callers to request the substitution of a single derivation
output.
* guix/store.scm (build-things): Accept derivation/output pairs among
THINGS.
* guix/derivations.scm (build-derivations): Likewise.
* tests/store.scm ("substitute + build-things with specific output"):
New test.
* tests/derivations.scm ("build-derivations with specific output"):
New test.
* doc/guix.texi (The Store): Adjust accordingly.
|
|
* guix/store.scm (hash-part->path): Adjust docstring to match reality.
|
|
* config-daemon.ac: Replace "ci.guix.info" with "ci.guix.gnu.org".
* doc/guix.texi (SUBSTITUTE-SERVER): Likewise.
* etc/substitutes/ci.guix.gnu.org.pub: New file.
* Makefile.am (dist_pkgdata_DATA): Add it.
* guix/scripts/build.scm (%default-log-urls): Update.
* guix/scripts/substitute.scm (%default-substitute-urls): Likewise.
* guix/store.scm (%default-substitute-urls): Likewise.
* guix/self.scm (miscellaneous-files): Add "ci.guix.gnu.org".
|
|
* guix/store.scm (add-data-to-store): Define 'lookup' and use it instead
of 'hash-ref'.
|
|
The caching strategy introduced in
40cc850aebb497faed0a11d867d8fcee729023df was ineffective since we
regularly start from an empty object cache. For example, "guix build
inkscape -n" would make 241 'built-in-builders' RPCs.
* guix/store.scm (<store-connection>)[built-in-builders]: New field.
(open-connection): Adjust '%make-store-connection' call accordingly.
(port->connection): Likewise.
(built-in-builders): Rename to...
(%built-in-builders): ... this.
(built-in-builders): New procedure.
* guix/download.scm (built-in-builders*): Remove 'mcached' call.
|
|
* guix/store.scm (call-with-store): New procedure.
(with-store): Write in terms of 'call-with-store'.
|
|
* guix/store.scm (derivation-log-file): Use %LOCALSTATEDIR or
"GUIX_LOG_DIRECTORY" instead of (dirname %STATE-DIRECTORY).
|
|
* guix/store.scm (&nix-error): Rename to...
(&store-error): ... this, and adjust users.
(&nix-connection-error): Rename to...
(&store-connection-error): ... this, and adjust users.
(&nix-protocol-error): Rename to...
(&store-protocol-error): ... this, adjust users.
(&nix-error, &nix-connection-error, &nix-protocol-error): Define these
condition types and their getters as deprecrated aliases.
* build-aux/run-system-tests.scm, guix/derivations.scm,
guix/grafts.scm, guix/scripts/challenge.scm,
guix/scripts/graph.scm, guix/scripts/lint.scm,
guix/scripts/offload.scm, guix/serialization.scm,
guix/ssh.scm, guix/tests.scm, guix/ui.scm,
tests/derivations.scm, tests/gexp.scm, tests/guix-daemon.sh,
tests/packages.scm, tests/store.scm, doc/guix.texi: Adjust to use the
new names.
|
|
* guix/store.scm (<nix-server>): Rename to...
(<store-connection>): ... this. Adjust users accordingly.
(nix-server?, nix-server-major-version)
(nix-server-minor-version, nix-server-socket)
(nix-server-version): Define as deprecated aliases.
* guix/inferior.scm: Adjust accordingly.
* guix/ssh.scm: Likewise.
|
|
These constants, for use with 'setvbuf', were deprecated in Guile 2.2
and disappeared in Guile 3.0. Here we keep these constants in
build-side code where removing them is not feasible.
* guix/build/download-nar.scm (download-nar): Adjust 'setvbuf' calls to
the Guile 2.2+ API.
* guix/build/download.scm (open-socket-for-uri): Likewise.
(open-connection-for-uri, url-fetch): Likewise.
* guix/build/make-bootstrap.scm (make-stripped-libc): Likewise.
* guix/build/union.scm (setvbuf) [guile-2.0]: New conditional wrapper.
(union-build): Adjust to new API.
* guix/ftp-client.scm (ftp-open, ftp-list, ftp-retr): Likewise.
* guix/http-client.scm (http-fetch): Likewise.
* guix/inferior.scm (proxy): Likewise.
* guix/scripts/substitute.scm (fetch, http-multiple-get): Likewise.
* guix/self.scm (compiled-modules): Likewise.
* guix/ssh.scm (remote-daemon-channel, store-import-channel)
(store-export-channel): Likewise.
* guix/ui.scm (initialize-guix): Likewise.
* tests/publish.scm (http-get-port): Likewise.
* guix/store.scm (%newlines): Adjust comment.
|
|
Note: Leave 'cond-expand' forms used in the build-side modules that can
run on %BOOTSTRAP-GUILE, which is currently Guile 2.0.
* guix/build/compile.scm: Move 'use-modules' clause from 'cond-expand'
to 'define-module' form.
(%default-optimizations): Remove 'cond-expand'.
* guix/build/download.scm (tls-wrap): Remove 'cond-expand'.
* guix/build/syscalls.scm: Remove 'cond-expand' form around
'%set-automatic-finalization-enabled?!' and
'without-automatic-finalization'.
* guix/inferior.scm (port->inferior): Remove 'cond-expand'.
* guix/scripts/pack.scm (wrapped-package)[build]: Remove 'cond-expand'.
* guix/status.scm (build-event-output-port): Remove 'cond-expand'.
* guix/store.scm (open-inet-socket): Remove 'cond-expand'.
* guix/ui.scm (install-locale): Remove 'cond-expand'.
* tests/status.scm ("current-build-output-port, UTF-8 + garbage"):
Remove 'cond-expand'.
* tests/store.scm ("current-build-output-port, UTF-8 + garbage"):
Remove 'cond-expand'.
|
|
* guix/store.scm (profiled?): New procedure.
(record-operation): Use it.
(record-cache-lookup!): New procedure.
(lookup-cached-object): Use it.
|
|
* config-daemon.ac (guix_substitute_urls): Always default to
"https://ci.guix.info".
* doc/guix.texi (SUBSTITUTE-SERVER): Switch to ci.guix.info.
* guix/scripts/build.scm (%default-log-urls): Likewise.
* guix/scripts/substitute.scm (%default-substitute-urls): Likewise.
* guix/store.scm (%default-substitute-urls): Likewise.
|
|
This leads to ~25% improvements on things like:
guix system build desktop.tmpl --no-grafts -d
* guix/store.scm (<nix-server>)[object-cache]: New field.
* guix/store.scm (open-connection): Initialize it.
(cache-object-mapping, lookup-cached-object, %mcached): New procedures.
(mcached): New macro.
* guix/gexp.scm (lower-object): Use it.
* guix/grafts.scm (grafting?): New procedure.
|
|
This allows clients to tell whether output comes from the daemon or, if
it comes from a builder, from which builder it comes. The latter is
particularly useful when MAX-BUILD-JOBS > 1.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook)
(DerivationGoal::startBuilder): Print the child's PID in "@ build-started"
traces.
(DerivationGoal::handleChildOutput): Define 'prefix', pass it to
'writeToStderr'.
* nix/libstore/globals.cc (Settings:Settings): Initialize
'multiplexedBuildOutput'.
(Settings::update): Likewise.
* nix/libstore/globals.hh (Settings)[multiplexedBuildOutput]: New field.
Update 'printBuildTrace' documentation.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0.163.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: Special-case
"multiplexed-build-output" and remove "use-ssh-substituter".
* guix/store.scm (set-build-options): Add #:multiplexed-build-output?
and honor it.
(%protocol-version): Bump to #x163.
* tests/store.scm ("multiplexed-build-output"): New test.
fixlet
|
|
* guix/store.scm (derivation-log-file): New procedure.o
(log-file): Use it.
* guix/status.scm (print-build-event): Use 'derivation-log-file' instead
of 'log-file'. Check wheter the return value is #f.
|
|
* guix/progress.scm (progress-reporter/trace): New procedure.
(%progress-interval): New variable.
(progress-reporter/file): Use it.
* guix/scripts/build.scm (set-build-options-from-command-line): Pass
#:print-extended-build-trace?.
(%default-options): Add 'print-extended-build-trace?'.
(guix-build): Parameterize CURRENT-TERMINAL-COLUMNS. Use
'build-status-updater'.
* guix/scripts/environment.scm (%default-options): Add
'print-extended-build-trace?'.
(guix-environment): Wrap body in 'with-status-report'.
* guix/scripts/pack.scm (%default-options): Add 'print-build-trace?' and
'print-extended-build-trace?'.
(guix-pack): Wrap body in 'with-status-report'.
* guix/scripts/package.scm (%default-options, guix-package): Likewise.
* guix/scripts/system.scm (%default-options, guix-system): Likewise.
* guix/scripts/pull.scm (%default-options, guix-pull): Likewise.
* guix/scripts/substitute.scm (progress-report-port): Don't call STOP
when TOTAL is zero.
(process-substitution): Add #:print-build-trace? and honor it.
(guix-substitute)[print-build-trace?]: New variable.
Pass #:print-build-trace? to 'process-substitution'.
* guix/status.scm: New file.
* guix/store.scm (set-build-options): Add #:print-extended-build-trace?;
pass it into PAIRS.
(%protocol-version): Bump.
(protocol-version, nix-server-version): New procedures.
(current-store-protocol-version): New variable.
(with-store, build-things): Parameterize it.
* guix/ui.scm (build-output-port): Remove.
(colorize-string): Export.
* po/guix/POTFILES.in: Add guix/status.scm.
* tests/status.scm: New file.
* Makefile.am (SCM_TESTS): Add it.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x162.
* nix/libstore/build.cc (DerivationGoal::registerOutputs)
(SubstitutionGoal::finished): Print a "@ hash-mismatch" trace before
throwing.
|
|
This is a followup to 3809824199cdd52446176c9cd4761dd09f732542.
* guix/store.scm (add-file-tree-to-store): Add 'write-buffered-output'
call.
|
|
This could result in deadlock in unusual situations, whereby we'd start
waiting for a reply while the query hasn't been flushed to the socket.
* guix/store.scm (buffering-output-port)[flush]: Add call to
'force-output'.
(add-to-store): Add call to 'write-buffered-output'.
|
|
This removes (guix hash) and (guix pk-crypto), which now live as part of
Guile-Gcrypt (version 0.1.0.)
* guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm,
tests/hash.scm, tests/pk-crypto.scm: Remove.
* configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and
LIBGCRYPT_LIBDIR assignments.
* m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove.
* README: Add Guile-Gcrypt to the dependencies; move libgcrypt as
"required unless --disable-daemon".
* doc/guix.texi (Requirements): Likewise.
* gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm,
guix/git.scm, guix/http-client.scm, guix/import/cpan.scm,
guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm,
guix/import/gnu.scm, guix/import/hackage.scm,
guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm,
guix/pki.scm, guix/scripts/archive.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/scripts/pack.scm,
guix/scripts/publish.scm, guix/scripts/refresh.scm,
guix/scripts/substitute.scm, guix/store.scm,
guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm,
tests/builders.scm, tests/challenge.scm, tests/cpan.scm,
tests/crate.scm, tests/derivations.scm, tests/gem.scm,
tests/nar.scm, tests/opam.scm, tests/pki.scm,
tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm,
tests/store.scm, tests/substitute.scm: Adjust imports.
* gnu/system/vm.scm: Likewise.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT.
(expression->derivation-in-linux-vm)[config]: Remove.
(iso9660-image)[config]: Remove.
(qemu-image)[config]: Remove.
(system-docker-image)[config]: Remove.
* guix/scripts/pack.scm: Adjust imports.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT.
(self-contained-tarball)[build]: Call 'make-config.scm' without
#:libgcrypt argument.
(squashfs-image)[libgcrypt]: Remove.
[build]: Call 'make-config.scm' without #:libgcrypt.
(docker-image)[config, json]: Remove.
[build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from
the imported modules.
* guix/self.scm (specification->package): Remove "libgcrypt", add
"guile-gcrypt".
(compiled-guix): Remove #:libgcrypt.
[guile-gcrypt]: New variable.
[dependencies]: Add it.
[*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call.
Add #:extensions.
[*config*]: Remove #:libgcrypt from 'make-config.scm' call.
(%dependency-variables): Remove %libgcrypt.
(make-config.scm): Remove #:libgcrypt.
* build-aux/build-self.scm (guile-gcrypt): New variable.
(make-config.scm): Remove #:libgcrypt.
(build-program)[fake-gcrypt-hash]: New variable.
Add (gcrypt hash) to the imported modules. Adjust load path
assignments.
* gnu/packages/package-management.scm (guix)[propagated-inputs]: Add
GUILE-GCRYPT.
[arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search
path.
|
|
* guix/store.scm (%not-slash): New variable.
(add-file-tree-to-store, interned-file-tree): New procedures.
* tests/store.scm ("add-file-tree-to-store"): New test.
|
|
* guix/store.scm (binary-file): New function.
* doc/guix.texi (The Store Monad): Describe binary-file.
|
|
* guix/scripts/size.scm (query-path-info*): Move to...
* guix/store.scm (query-path-info*): ... here.
|
|
* guix/store.scm (port->connection): New procedure.
|
|
* guix/store.scm (register-path): Remove.
* guix/nar.scm: Use (guix store database).
* guix/scripts/system.scm: Likewise.
* tests/store-database.scm: Remove #:hide (register-path).
* tests/store.scm ("register-path"): Remove.
|
|
Previously 'process-stderr' would always pass a bytevector of MAX-LEN to
then daemon in the %stderr-read case (i.e., 'import-paths'), instead of
LEN (where LEN <= MAX-LEN).
In practice the extra bytes didn't cause a protocol violation or
anything because they happen at the end of the stream, which typically
contains the canonical sexp of the signature, and the extra zeros were
just ignored.
* guix/serialization.scm (write-bytevector): Add optional 'l' parameter
and honor it.
* guix/store.scm (process-stderr): Pass LEN to 'write-bytevector'.
|
|
* nix/nix-daemon/guix-daemon.cc (GUIX_OPT_LOG_COMPRESSION): New macro.
(options): Mark "disable-log-compression" as hidden and add
"log-compression".
(parse_opt): Handle GUIX_OPT_LOG_COMPRESSION.
* nix/libstore/build.cc (DerivationGoal): Add 'gzLogFile'.
(openLogFile): Initialize it when 'logCompression' is COMPRESSION_GZIP.
(closeLogFile, handleChildOutput): Honor 'gzLogFile'.
* nix/libstore/globals.hh (Settings)[compressLog]: Remove.
[logCompression]: New field.
(CompressionType): New enum.
* nix/libstore/globals.cc (Settings::Settings): Initialize it.
(update): Remove '_get' call for 'compressLog'.
* nix/local.mk (guix_daemon_LDADD, guix_register_LDADD): Add -lz.
* guix/store.scm (log-file): Handle '.gz' log files.
* tests/guix-daemon.sh: Add test with '--log-compression=gzip'.
* doc/guix.texi (Invoking guix-daemon): Adjust accordingly.
* config-daemon.ac: Check for libz and zlib.h.
|
|
* guix/profiling.scm: New file.
* Makefile.am (MODULES): Add it.
* guix/store.scm (record-operation): Use 'profiled?' and
'register-profiling-hook!'.
|
|
* guix/store.scm: Use (ice-9 format).
|
|
* guix/store.scm (run-with-store): Add #:target and honor it.
|
|
When current-processor-count is used without (ice-9 threads) being used, Guile
complains with the following warning:
Import (ice-9 threads) to have access to `current-processor-count'.
* guix/store.scm: Use (ice-9 threads).
|
|
* guix/store.scm (store-path-hash-part): Rewrite without using a
regexp. This speeds up 'guix substitute'.
|
|
* guix/store.scm (add-to-store): Add call to 'record-operation'.
|
|
Until now the cache was always empty because 'for-each' was passed ITEMS
as its second argument, and ITEMS was the empty list at that point.
* guix/store.scm (references/substitutes): Add 'requested' variable.
Use it as second argument of 'for-each' in base case.
|
|
* guix/store.scm (references/substitutes): Save a
'substitutable-path-info' call when MISSING is empty.
|
|
* guix/store.scm (%default-guix-port): New variable.
(connect-to-daemon)[connect]: Use it when (uri-port uri) is #f.
* doc/guix.texi (The Store): Mention the default port number.
|
|
* guix/store.scm (open-inet-socket): Pass hints in the 'getaddrinfo'
call.
|
|
For a command like:
guix build python2-numpy -n
this reduces the number of 'write' syscalls from 9.5K to 2.0K.
* guix/store.scm (<nix-server>)[buffer, flush]: New fields.
(open-connection): Adjust accordingly. Call 'buffering-output-port' to
compute the two new fields.
(write-buffered-output, buffering-output-port): New procedures.
(operation): Write to (nix-server-output-port server). Call
'write-buffered-output'.
|
|
* guix/store.scm (%rpc-calls): New variable.
(show-rpc-profile, record-operation): New procedures.
(operation): Add call to 'record-operation'.
* guix/ui.scm (run-guix-command): Wrap COMMAND-MAIN in 'dynamic-wind'.
Run EXIT-HOOK.
|
|
* guix/store.scm (add-to-store): Remove 'lstat' call.
|
|
* guix/store.scm (open-inet-socket): Add 'cond-expand' form to define
'TCP_NODELAY' when needed. Add call to 'setsockopt' after 'connect'.
|
|
* guix/store.scm (process-stderr): Use 'write-bytevector' in the
%STDERR-READ case.
|
|
* guix/derivations.scm (compressed-hash, store-path)
(output-path, fixed-output-path): Move to...
* guix/store.scm: ... here.
|
|
* guix/monads.scm (%templates, %template-instances): New variables.
(register-template!, register-template-instance!): New procedures.
(template-directory, define-template): New macro.
(foldm, sequence, anym): Define using 'define-template'. Avoid replace
ellipses with dots.
(mapm): Likewise, but do not use 'foldm'.
* guix/store.scm: Add 'template-directory' invocation.
|
|
This allows 'guix' commands to talk to a remote store over SSH.
* guix/store.scm (connect-to-daemon)[connect]: Call 'resolve-interface'
for unknown URI schemes.
* guix/store/ssh.scm: New file.
* Makefile.am (MODULES): Add it.
* doc/guix.texi (The Store): Document it. Mark remote access as
experimental.
|
|
* guix/store.scm (open-inet-socket): New procedure.
(connect-to-daemon): Support the 'guix' URI scheme.
* doc/guix.texi (The Store): Document it.
|
|
* guix/store.scm (system-error-to-connection-error): New macro.
(open-unix-domain-socket): Use it instead of 'catch'.
|
|
* guix/store.scm (%daemon-socket-file): Rename to...
(%daemon-socket-uri): ... this.
(connect-to-daemon): New procedure.
(open-connection): Rename 'file' to 'uri'. Use 'connect-to-daemon'
instead of 'open-unix-domain-socket'.
* guix/tests.scm (open-connection-for-tests): Rename 'file' to 'uri'.
* tests/guix-build.sh: Add tests.
* tests/store.scm ("open-connection with file:// URI"): New tests.
|
|
* guix/utils.scm (bytevector->base16-string, base16-string->bytevector):
Move to...
* guix/base16.scm: ... here. New file.
* tests/utils.scm ("bytevector->base16-string->bytevector"): Move to...
* tests/base16.scm: ... here. New file.
* Makefile.am (MODULES): Add guix/base16.scm.
(SCM_TESTS): Add tests/base16.scm.
* build-aux/download.scm, guix/derivations.scm,
guix/docker.scm, guix/import/snix.scm, guix/pk-crypto.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/store.scm, tests/hash.scm,
tests/pk-crypto.scm: Adjust imports accordingly.
|