| Age | Commit message (Expand) | Author |
|---|
| 2020-06-10 | database: separate transaction-handling and retry-handling....Previously call-with-transaction would both retry when SQLITE_BUSY errors were
thrown and do what its name suggested (start and rollback/commit a
transaction). This changes it to do only what its name implies, which
simplifies its implementation. Retrying is provided by the new
call-with-SQLITE_BUSY-retrying procedure.
* guix/store/database.scm (call-with-transaction): no longer restarts, new
#:restartable? argument controls whether "begin" or "begin immediate" is
used.
(call-with-SQLITE_BUSY-retrying, call-with-retrying-transaction,
call-with-retrying-savepoint): new procedures.
(register-items): use call-with-retrying-transaction to preserve old
behavior.
* .dir-locals.el (call-with-retrying-transaction,
call-with-retrying-savepoint): add indentation information.
| Caleb Ristvedt |
| 2020-06-10 | database: ensure update-or-insert is run within a transaction...update-or-insert can break if an insert occurs between when it decides whether
to update or insert and when it actually performs that operation. Putting the
check and the update/insert operation in the same transaction ensures that the
update/insert will only succeed if no other write has occurred in the middle.
* guix/store/database.scm (call-with-savepoint): new procedure.
(update-or-insert): use call-with-savepoint to ensure the read and the
insert/update occur within the same transaction.
| Caleb Ristvedt |
| 2020-06-10 | database: rewrite query procedures in terms of with-statement....Most of our queries would fail to finalize their statements properly if sqlite
returned an error during their execution. This resolves that, and also makes
them somewhat more concise as a side-effect.
This also makes some small changes to improve certain queries where behavior
was strange or overly verbose.
* guix/store/database.scm (call-with-statement): new procedure.
(with-statement): new macro.
(last-insert-row-id, path-id, update-or-insert, add-references): rewrite to
use with-statement.
(update-or-insert): factor last-insert-row-id out of the end of both
branches.
(add-references): remove pointless last-insert-row-id call.
* .dir-locals.el (with-statement): add indenting information.
| Caleb Ristvedt |
| 2020-06-10 | database: work around guile-sqlite3 bug preventing statement reset...guile-sqlite3 provides statement caching, making it unnecessary for sqlite to
keep re-preparing statements that are frequently used. Unfortunately it
doesn't quite emulate the semantics of sqlite_finalize properly, because it
doesn't cause a commit if the statement being finalized is the last "active"
statement (see https://notabug.org/guile-sqlite3/guile-sqlite3/issues/12). We
work around this by wrapping sqlite-finalize with our own version that ensures
sqlite-reset is called, which does The Right Thing™.
* guix/store/database.scm (sqlite-finalize): new procedure that shadows the
sqlite-finalize from (sqlite3).
| Caleb Ristvedt |
| 2020-04-11 | database: 'reset-timestamps' can optionally preserve permissions....* guix/store/database.scm (reset-timestamps): Add
#:preserve-permissions? and honor it.
| Ludovic Courtès |
| 2020-02-22 | deduplication: Use nix-base32 encoding for link names....Fixes <https://bugs.gnu.org/39725>.
* guix/store/deduplication.scm (deduplicate): Use
'bytevector->nix-base32-string' instead of 'bytevector->base16-string'.
| Ludovic Courtès |
| 2019-09-08 | daemon: Invoke 'guix gc --list-busy' instead of 'list-runtime-roots'....* nix/scripts/list-runtime-roots.in: Remove.
* guix/store/roots.scm (%proc-directory): New variable.
(proc-file-roots, proc-exe-roots, proc-cwd-roots)
(proc-fd-roots, proc-maps-roots, proc-environ-roots)
(referenced-files, canonicalize-store-item, busy-store-items): New
procedures, taken from 'list-runtime-roots.in'.
* nix/libstore/globals.hh (Settings)[guixProgram]: New field.
* nix/libstore/globals.cc (Settings::processEnvironment): Initialize
'guixProgram'.
* nix/libstore/gc.cc (addAdditionalRoots): Drop code related to
'NIX_ROOT_FINDER'. Run "guix gc --list-busy".
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove
'scripts/list-runtime-roots'.
* config-daemon.ac: Don't output nix/scripts/list-runtime-roots.
* build-aux/pre-inst-env.in: Don't set 'NIX_ROOT_FINDER'.
Set 'GUIX'.
* doc/guix.texi (Invoking guix gc): Document '--list-busy'.
* guix/scripts/gc.scm (show-help, %options): Add "--list-busy".
(guix-gc)[list-busy]: New procedure.
Handle the 'list-busy' action.
| Ludovic Courtès |
| 2019-04-25 | gnu, guix: Yearly ritual purging of the filesystems....* gnu/packages/android.scm (android-ext4-utils)[synopsis]: Fix ‘file
system’ spelling.
* gnu/packages/disk.scm (rmlint)[synopsis, description]: Likewise.
* gnu/packages/golang.scm (go-github-com-kr-fs)[synopsis, description]:
Likewise & edit for grammar.
* gnu/packages/ipfs.scm (gx, go-ipfs)[description]: Likewise.
* /gnu/packages/java.scm (java-commons-vfs)[synopsis]: Likewise.
* gnu/packages/linux.scm (fuseiso)[description]: Likewise.
(genext2fs)[synopsis, description]: Likewise.
* gnu/packages/package-management.scm (libostree)[description]: Likewise.
* gnu/packages/python-xyz.scm (python-requests-file)[description]:
Likewise & mark up.
* gnu/packages/rails.scm (ruby-with-advisory-lock)[description]:
Likewise.
* gnu/packages/ruby.scm (ruby-rerun)[description]: Likewise.
* guix/build/go-build-system.scm (setup-go-environment)<docstring>:
Likewise.
* guix/store/deduplication.scm (get-temp-link)<docstring>: Likewise.
| Tobias Geerinckx-Rice |
| 2019-04-10 | Add (guix store roots)....* guix/store/roots.scm, tests/store-roots.scm: New files.
* Makefile.am (STORE_MODULES): Add guix/store/roots.scm.
(SCM_TESTS): Add tests/store-roots.scm.
| Ludovic Courtès |
| 2019-03-07 | database: Make 'register-items' transactional....* guix/store/database.scm (SQLITE_BUSY, register-output-sql): New variables.
(add-references): Don't try finalizing after each use, only after all the
uses (otherwise a finalized statement would be used if #:cache? was #f).
(call-with-transaction): New procedure.
(register-items): Use call-with-transaction to prevent broken intermediate
states from being visible.
* .dir-locals.el (call-with-transaction): indent it.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
| Caleb Ristvedt |
| 2019-01-23 | deduplication: Ignore EMLINK....Until now 'guix offload' would fail (transient failure) upon EMLINK.
* guix/store/deduplication.scm (replace-with-link)
(deduplicate): Ignore EMLINK.
| Ludovic Courtès |
| 2018-12-21 | database: Use "write-ahead log" mode and set a long "busy timeout"....This should avoid "database is locked" errors when there's a lot of
concurrency, for instance when offloading simultaneously a lot of
builds.
* guix/store/database.scm (call-with-database): Add two 'sqlite-exec'
calls to set 'journal_mode' and 'busy_timeout'.
| Ludovic Courtès |
| 2018-12-14 | deduplication: Gracefully handle ENOSPC raised by 'link' calls....Reported by Andreas Enge <andreas@enge.fr>
in <https://bugs.gnu.org/33676>.
* guix/store/deduplication.scm (replace-with-link): Catch ENOSPC around
'get-temp-link'. Do nothing when 'get-temp-link' throws ENOSPC. Move
code to restore PARENT's permissions outside of 'catch'.
* tests/store-deduplication.scm ("deduplicate, ENOSPC"): New test.
| Ludovic Courtès |
| 2018-11-23 | Update Guile-SQLite3 URL everywhere....* README: Update Guile-SQLite3 URL.
* doc/guix.texi (Requirements): Likewise.
* guix/store/database.scm (sqlite-exec): Likewise.
* m4/guix.m4 (GUIX_CHECK_GUILE_SQLITE3): Likewise.
| Ludovic Courtès |
| 2018-11-13 | deduplication: Restore directory mtime and permissions after deduplication....Fixes <https://bugs.gnu.org/33361>.
* guix/store/deduplication.scm (replace-with-link): Call 'set-file-time'
and 'chmod' after 'rename-file'.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
New test.
| Ludovic Courtès |
| 2018-11-13 | nar: Access the database instead of connecting to the daemon....* guix/store/database.scm (%default-database-file): New variable.
(path-id): Export.
* guix/nar.scm (finalize-store-file): Use 'with-database' instead of
'with-store', and use 'path-id' instead of 'valid-path?'.
| Ludovic Courtès |
| 2018-09-23 | database: Register each store item only once....Fixes <https://bugs.gnu.org/32600>.
Reported by Leo Famulari.
* guix/store/database.scm (register-items): Check whether TO-REGISTER is
in DB by calling 'path-id', and skip the reset-timestamps,
registration, and deduplication phases when it is.
| Ludovic Courtès |
| 2018-09-23 | database: 'register-items' shows a progress bar....* guix/store/database.scm (register-items): Add #:log-port. Use
'progress-reporter/bar' to show a progress report.
(register-path): Pass #:log-port to 'register-items'.
| Ludovic Courtès |
| 2018-09-04 | Switch to Guile-Gcrypt....This removes (guix hash) and (guix pk-crypto), which now live as part of
Guile-Gcrypt (version 0.1.0.)
* guix/gcrypt.scm, guix/hash.scm, guix/pk-crypto.scm,
tests/hash.scm, tests/pk-crypto.scm: Remove.
* configure.ac: Test for Guile-Gcrypt. Remove LIBGCRYPT and
LIBGCRYPT_LIBDIR assignments.
* m4/guix.m4 (GUIX_ASSERT_LIBGCRYPT_USABLE): Remove.
* README: Add Guile-Gcrypt to the dependencies; move libgcrypt as
"required unless --disable-daemon".
* doc/guix.texi (Requirements): Likewise.
* gnu/packages/bash.scm, guix/derivations.scm, guix/docker.scm,
guix/git.scm, guix/http-client.scm, guix/import/cpan.scm,
guix/import/cran.scm, guix/import/crate.scm, guix/import/elpa.scm,
guix/import/gnu.scm, guix/import/hackage.scm,
guix/import/texlive.scm, guix/import/utils.scm, guix/nar.scm,
guix/pki.scm, guix/scripts/archive.scm,
guix/scripts/authenticate.scm, guix/scripts/download.scm,
guix/scripts/hash.scm, guix/scripts/pack.scm,
guix/scripts/publish.scm, guix/scripts/refresh.scm,
guix/scripts/substitute.scm, guix/store.scm,
guix/store/deduplication.scm, guix/tests.scm, tests/base32.scm,
tests/builders.scm, tests/challenge.scm, tests/cpan.scm,
tests/crate.scm, tests/derivations.scm, tests/gem.scm,
tests/nar.scm, tests/opam.scm, tests/pki.scm,
tests/publish.scm, tests/pypi.scm, tests/store-deduplication.scm,
tests/store.scm, tests/substitute.scm: Adjust imports.
* gnu/system/vm.scm: Likewise.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT.
(expression->derivation-in-linux-vm)[config]: Remove.
(iso9660-image)[config]: Remove.
(qemu-image)[config]: Remove.
(system-docker-image)[config]: Remove.
* guix/scripts/pack.scm: Adjust imports.
(guile-sqlite3&co): Rename to...
(gcrypt-sqlite3&co): ... this. Add GUILE-GCRYPT.
(self-contained-tarball)[build]: Call 'make-config.scm' without
#:libgcrypt argument.
(squashfs-image)[libgcrypt]: Remove.
[build]: Call 'make-config.scm' without #:libgcrypt.
(docker-image)[config, json]: Remove.
[build]: Add GUILE-GCRYPT to the extensions Remove (guix config) from
the imported modules.
* guix/self.scm (specification->package): Remove "libgcrypt", add
"guile-gcrypt".
(compiled-guix): Remove #:libgcrypt.
[guile-gcrypt]: New variable.
[dependencies]: Add it.
[*core-modules*]: Remove #:libgcrypt from 'make-config.scm' call.
Add #:extensions.
[*config*]: Remove #:libgcrypt from 'make-config.scm' call.
(%dependency-variables): Remove %libgcrypt.
(make-config.scm): Remove #:libgcrypt.
* build-aux/build-self.scm (guile-gcrypt): New variable.
(make-config.scm): Remove #:libgcrypt.
(build-program)[fake-gcrypt-hash]: New variable.
Add (gcrypt hash) to the imported modules. Adjust load path
assignments.
* gnu/packages/package-management.scm (guix)[propagated-inputs]: Add
GUILE-GCRYPT.
[arguments]: In 'wrap-program' phase, add GUILE-GCRYPT to the search
path.
| Ludovic Courtès |
| 2018-07-20 | database: Reset timestamps to one second after the Epoch....Previously, store items registered in the database by this code (for
instance, store items retrieved by 'guix offload' and passed to
'restore-file-set') would have an mtime of 0 instead of 1.
This would cause problems for things like .go files: Guile would
consider them to be older than the corresponding .scm file, and
consequently it would ignore them and possibly use another (incorrect)
.go file.
Reported by Ricardo Wurmus.
* guix/store/database.scm (reset-timestamps): Pass 1, not 0, to
'utime'.
* tests/store-database.scm ("register-path"): Check the mtime of FILE
and REF.
| Ludovic Courtès |
| 2018-07-20 | deduplication: Work around Guile bug in 'seek'....Fixes <https://bugs.gnu.org/32161>.
Reported by Ricardo Wurmus <rekado@elephly.net>.
This mostly reverts 83099892e0cf0d9c59f5e1a0774331026e48baa8.
* guix/store/deduplication.scm (counting-wrapper-port): New procedure.
(nar-sha256): Use it.
| Ludovic Courtès |
| 2018-07-19 | deduplication: Remove 'counting-wrapper-port'....* guix/store/deduplication.scm (counting-wrapper-port): Remove.
(nar-sha256): Call 'port-position' directly on PORT.
| Ludovic Courtès |
| 2018-07-03 | database: 'reset-timestamps' now correctly handles symlinks....* guix/store/database.scm (reset-timestamps): Use 'utime' with
AT_SYMLINK_NOFOLLOW for symlinks.
| Ludovic Courtès |
| 2018-07-03 | deduplication: Remove 'false-if-system-error', now unused....* guix/store/deduplication.scm (false-if-system-error): Remove.
| Ludovic Courtès |
| 2018-07-03 | deduplication: Place link files under /gnu/store/.links....Previously they'd always be placed next to TO-REPLACE, which would lead
to EPERM in some cases.
* guix/store/deduplication.scm (replace-with-link): Add #:swap-directory
parameter and honor it. Add call to 'make-file-writable'. Catch
'system-error' around 'rename-file'.
(deduplicate): Pass #:swap-directory and remove uses of
'false-if-system-error'.
* tests/store-deduplication.scm ("deduplicate"): Add 'chmod' call.
| Ludovic Courtès |
| 2018-07-03 | deduplication: Fix incorrect use of 'throw'....* guix/store/deduplication.scm (get-temp-link): In handler, fix call to
'throw'.
| Ludovic Courtès |
| 2018-06-14 | database: Allow for deterministic database construction....Fixes <https://bugs.gnu.org/21073>.
* guix/store/database.scm (sqlite-register): Add #:time.
(%epoch): New variable.
(register-items): Add #:registration-time. Pass #:time to
'sqlite-register'.
* gnu/build/install.scm (register-closure): Pass #:registration-time.
| Ludovic Courtès |
| 2018-06-14 | install: Use 'reset-timestamps' from (guix store database)....* gnu/build/install.scm (reset-timestamps): Remove.
* gnu/build/vm.scm: Use 'reset-timestamps' from (guix store database).
| Ludovic Courtès |
| 2018-06-14 | database: Add 'register-items'....* guix/build/store-copy.scm (store-info): Export.
* guix/store/database.scm (register-items): New procedure.
(register-path): Implement in terms of 'register-items'.
* gnu/build/install.scm (register-closure): Use 'register-items' instead
of 'for-each' and 'register-path'.
| Ludovic Courtès |
| 2018-06-14 | database: 'sqlite-register' takes a database, not a file name....* guix/store/database.scm (sqlite-register): Remove #:db-file and add
'db' parameter. Remove #:schema and 'parameterize'.
(register-path): Wrap 'sqlite-register' call in 'with-database' and in
'parameterize'.
* tests/store-database.scm ("new database")
("register-path with unregistered references"): Adjust accordingly.
| Ludovic Courtès |
| 2018-06-14 | database: 'reset-timestamps' sets file permissions as well....* guix/store/database.scm (reset-timestamps): Add 'chmod' calls.
| Ludovic Courtès |
| 2018-06-14 | database: Replace existing entries in Refs....* guix/store/database.scm (add-reference-sql): Add "OR REPLACE".
| Ludovic Courtès |
| 2018-06-14 | database: Add #:reset-timestamps? to 'register-path'....* guix/store/database.scm (register-path): Add #:reset-timestamps? and
honor it.
| Ludovic Courtès |
| 2018-06-14 | database: Remove extra SQL parameter in 'update-or-insert'....* guix/store/database.scm (update-or-insert): Remove extra #:path
parameter.
| Ludovic Courtès |
| 2018-06-14 | deduplicate: Fix a couple of thinkos....* guix/store/deduplication.scm (get-temp-link): Turn 'args' in the 'catch'
handler into a rest argument.
(deduplicate): Use 'lstat' instead of 'file-is-directory?' to properly
handle symlinks. When iterating over the result of 'scandir', exclude
the ".links" sub-directory.
* tests/store-deduplication.scm ("deduplicate"): Create sub-directories
and call 'deduplicate' directly on STORE.
| Ludovic Courtès |
| 2018-06-14 | database: 'register-path' creates the database directory if needed....* guix/store/database.scm (register-path): Call 'mkdir-p'.
| Ludovic Courtès |
| 2018-06-14 | database: Provide a way to specify the schema location....* guix/store/database.scm (sqlite-register): Add #:schema. Parameterize
'sql-schema' based on this.
(register-path): Add #:schema and pass it to 'sqlite-register'.
| Ludovic Courtès |
| 2018-06-14 | database: Fail registration when encountering unregistered references....* guix/store/database.scm (add-reference-sql): Remove nested SELECT.
(add-references): Expect REFERENCES to be a list of ids.
(sqlite-register): Call 'path-id' for each of REFERENCES and pass it to
'add-references'.
* tests/store-database.scm ("register-path with unregistered references"):
New test.
| Ludovic Courtès |
| 2018-06-14 | database: 'with-database' can now initialize new databases....* nix/libstore/schema.sql: Rename to...
* guix/store/schema.sql: ... this.
* Makefile.am (nobase_dist_guilemodule_DATA): Add it.
* nix/local.mk (%D%/libstore/schema.sql.hh): Adjust accordingly.
* guix/store/database.scm (sql-schema): New variable.
(sqlite-exec, initialize-database, call-with-database): New procedures.
(with-database): Rewrite in terms of 'call-with-database'.
* tests/store-database.scm ("new database"): New test.
* guix/self.scm (compiled-guix)[*core-modules*]: Add 'schema.sql' to
#:extra-files.
| Ludovic Courtès |
| 2018-06-01 | Add (guix store deduplication)....* guix/store/database.scm (register-path): Add #:deduplicate? and call
'deduplicate' when it's true.
(counting-wrapper-port, nar-sha256): Move to...
* guix/store/deduplication.scm: ... here. New file.
* tests/store-deduplication.scm: New file.
* Makefile.am (STORE_MODULES): Add deduplication.scm.
(SCM_TESTS) [HAVE_GUILE_SQLITE3]: Add store-deduplication.scm.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
| Caleb Ristvedt |
| 2018-06-01 | database: 'register-path' resets timestamps....* guix/store/database.scm (reset-timestamps): New procedure.
(register-path): Use it.
| Ludovic Courtès |
| 2018-06-01 | Add (gnu store database)....* guix/config.scm.in (%store-database-directory): New variable.
* guix/store/database.scm: New file.
* tests/store-database.scm: New file.
* Makefile.am (STORE_MODULES): New variable.
(MODULES, MODULES_NOT_COMPILED): Adjust accordingly.
(SCM_TESTS) [HAVE_GUILE_SQLITE3]: Add tests/store-database.scm.
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
| Caleb Ristvedt |
| 2017-04-21 | store: Support 'ssh://' URIs in 'GUIX_DAEMON_SOCKET'....This allows 'guix' commands to talk to a remote store over SSH.
* guix/store.scm (connect-to-daemon)[connect]: Call 'resolve-interface'
for unknown URI schemes.
* guix/store/ssh.scm: New file.
* Makefile.am (MODULES): Add it.
* doc/guix.texi (The Store): Document it. Mark remote access as
experimental.
| Ludovic Courtès |
