summaryrefslogtreecommitdiff
path: root/guix/import/pypi.scm
AgeCommit message (Collapse)Author
2020-09-08Remove (guix json) and require Guile-JSON 4.3.0+.Ludovic Courtès
This is a followup to 4071879c86d059ee087c8986915ea72b8c742b72. * guix/json.scm: Remove. * Makefile.am (MODULES): Adjust accordingly. * m4/guix.m4 (GUIX_CHECK_GUILE_JSON): Check for 'define-json-mapping'. * doc/guix.texi (Requirements): Require Guile-JSON 4.3.0+. * guix/ci.scm, guix/cve.scm, guix/import/cpan.scm, guix/import/crate.scm, guix/swh.scm: Remove (guix json) import. * guix/import/gem.scm, guix/import/pypi.scm: Likewise, and import (json). * guix/self.scm (specification->package): Switch to GUILE-JSON-4. * guix/git-download.scm (git-fetch): Likewise.
2020-07-25utils: Remove compatibility re-export of 'memoize'.Ludovic Courtès
The 'memoize' binding was re-exported in 2016, commit 19e1d5f7f90194f1ac7e783b28a688ce1441786d, as a backwards-compatibility measure that makes little sense now. * guix/utils.scm: Don't re-export 'memoize'. * guix/import/pypi.scm: Adjust imports. * tests/pypi.scm: Remove duplicate (guix memoization) import.
2020-07-05import: pypi: Handle 'null instead of #nil.Arun Isaac
* guix/import/pypi.scm (non-empty-string-or-false): guile-json now returns 'null instead of #nil for null JSON values. Handle it.
2020-07-03upstream: Define 'url-predicate' and use it.Ludovic Courtès
* guix/upstream.scm (url-predicate): New procedure. (url-prefix-predicate): Define in terms of 'url-predicate'. * guix/import/cpan.scm (cpan-package?): Use 'url-predicate'. * guix/import/cran.scm (cran-package?) (bioconductor-package?) (bioconductor-data-package?) (bioconductor-experiment-package?): Likewise. * guix/import/crate.scm (crate-package?): Likewise. * guix/import/elpa.scm (package-from-gnu.org?): Likewise. * guix/import/hackage.scm (hackage-package?): Likewise. * guix/import/pypi.scm (pypi-package?): Likewise. * guix/import/gem.scm (gem-package?): Use 'url-prefix-predicate'.
2020-03-12import: pypi: Rewrite to use 'define-json-mapping'.Ludovic Courtès
* guix/import/pypi.scm (non-empty-string-or-false): New procedure. (<pypi-project>, <project-info>, <distribution>): New record types. (pypi-fetch): Call 'json->pypi-project'. (latest-source-release, latest-wheel-release): Use the new record accessors instead of 'assoc-ref*'. (pypi->guix-package, latest-release): Likewise. * tests/pypi.scm (test-json): Add mandatory fields.
2020-03-04import: pypi: Add more licensesLars-Dominik Braun
* guix/import/pypi.scm (string->license): Add the BSD 2-clause and MPL 2.0 licenses, and add more strings for BSD 3-clause and Expat license. Signed-off-by: Leo Famulari <leo@famulari.name>
2020-02-05import: pypi: Support exporting packages with .zip source.Jakub Kądziołka
* guix/import/pypi.scm (make-pypi-sexp): Rename test-inputs to native-inputs. Restructure the way pypi-uri parameters are generated. Use pypi-uri's extension parameter when required. Add "unzip" to native inputs when the package source is a zip file. Signed-off-by: Marius Bakke <mbakke@fastmail.com>
2019-09-18import: pypi: Refresher recognizes pythonhosted.org source URLs.Ludovic Courtès
This is a followup to a5376200541abf8245973e601be246bf65b8b6c7. Since that commit, 'pypi-package?' would return false for most Python packages, and thus "guix refresh python-xxx" would report that no updaters apply to the package. * guix/import/pypi.scm (pypi-package?)[pypi-url?]: Recognize "files.pythonhosted.org" URLs.
2019-07-25maint: Switch to Guile-JSON 3.x.Ludovic Courtès
Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * m4/guix.m4 (GUIX_CHECK_GUILE_JSON): New macro. * configure.ac: Use it. * doc/guix.texi (Requirements): Mention the Guile-JSON version. * guix/git-download.scm (git-fetch)[guile-json]: Use GUILE-JSON-3. * guix/import/cpan.scm (string->license): Expect vectors instead of lists. (module->dist-name): Use 'json-fetch' instead of 'json-fetch-alist'. (cpan-fetch): Likewise. * guix/import/crate.scm (crate-fetch): Likewise, and call 'vector->list' for DEPS. * guix/import/gem.scm (rubygems-fetch): Likewise. * guix/import/json.scm (json-fetch-alist): Remove. * guix/import/pypi.scm (pypi-fetch): Use 'json-fetch' instead of 'json-fetch-alist'. (latest-source-release, latest-wheel-release): Call 'vector->list' on RELEASES. * guix/import/stackage.scm (stackage-lts-info-fetch): Use 'json-fetch' instead of 'json-fetch-alist'. (lts-package-version): Use 'vector->list'. * guix/import/utils.scm (hash-table->alist): Remove. (alist->package): Pass 'vector->list' on the inputs fields, and default to the empty vector. * guix/scripts/import/json.scm (guix-import-json): Remove call to 'hash-table->alist'. * guix/swh.scm (define-json-reader): Expect pair? or null? instead of hash-table?. [extract-field]: Use 'assoc-ref' instead of 'hash-ref'. (json->branches): Use 'map' instead of 'hash-map->list'. (json->checksums): Likewise. (json->directory-entries, origin-visits): Call 'vector->list' on the result of 'json->scm'. * tests/import-utils.scm ("alist->package with dependencies"): New test. * gnu/installer.scm (build-compiled-file)[builder]: Use GUILE-JSON-3. * gnu/installer.scm (installer-program)[installer-builder]: Likewise. * gnu/installer/locale.scm (iso639->iso639-languages): Use 'assoc-ref' instead of 'hash-ref', and pass vectors through 'vector->list'. (iso3166->iso3166-territories): Likewise. * gnu/system/vm.scm (system-docker-image)[build]: Use GUILE-JSON-3. * guix/docker.scm (manifest, config): Adjust for Guile-JSON 3. * guix/scripts/pack.scm (docker-image)[build]: Use GUILE-JSON-3. * guix/import/github.scm (fetch-releases-or-tags): Update docstring. (latest-released-version): Use 'assoc-ref' instead of 'hash-ref'. Pass the result of 'fetch-releases-or-tags' to 'vector->list'. * guix/import/launchpad.scm (latest-released-version): Likewise.
2019-07-02import: pypi: Preserve package name case when forming pypi-uri.Maxim Cournoyer
Fixes <https://bugs.gnu.org/33046>. * guix/build-system/python.scm (pypi-uri): Update the host URI to "files.pythonhosted.org". * guix/import/pypi.scm (make-pypi-sexp): Preserve the package name case when the source URL calls for it.
2019-07-02import: pypi: Include optional test inputs as native-inputs.Maxim Cournoyer
* guix/import/pypi.scm (maybe-inputs): Add INPUT-TYPE argument, and use it. (test-section?): New predicate. (parse-requires.txt): Collect the optional test inputs, and return them as the second element of the returned list. (parse-wheel-metadata): Likewise. (guess-requirements): Adapt. (make-pypi-sexp): Likewise, and include the test inputs requirements as native inputs in the returned package expression. * tests/pypi.scm (test-requires.txt): Include a test section in the test-requires.txt data. (test-requires.txt-beaker): New variable. ("parse-requires.txt"): Adapt. ("parse-requires.txt - Beaker"): New test. ("parse-wheel-metadata, with extras"): Adapt. ("parse-wheel-metadata, with extras - Jedi"): Adapt. ("pypi->guix-package, no wheel"): Re-indent, and add the expected native-inputs. ("pypi->guix-package, wheels"): Likewise.
2019-07-02import: pypi: Completely mute the output of the "unzip" command.Maxim Cournoyer
* guix/import/pypi.scm (guess-requirements): Completely mute the output of the "unzip" command.
2019-07-02import: pypi: Fix typo in docstring.Maxim Cournoyer
* guix/import/pypi.scm (guess-requirements): Fix typo.
2019-07-02import: pypi: Parse wheel METADATA instead of metadata.json.Maxim Cournoyer
With newer Wheel releases, there is no more metadata.json file; the METADATA file should be used instead (see: https://github.com/pypa/wheel/issues/195). This change updates our PyPI importer so that it uses the latter. * guix/import/pypi.scm (define-module): Remove unnecessary modules and export the PARSE-WHEEL-METADATA procedure. (parse-wheel-metadata): Add procedure. (guess-requirements): Use it. * tests/pypi.scm (test-metadata): Test it.
2019-07-02import: pypi: Support more types of archives.Maxim Cournoyer
This change enables the PyPI importer to look for requirements in a source archive of a different type than "tar.gz" or "tar.bz2". Also, scan the source archive to find a requires.txt file. * guix/import/pypi.scm: (guess-requirements)[tarball-directory]: Remove procedure. [guess-requirements-from-source]: Use COMRESSED-FILE? to determine if an archive type is supported, and some file extension logic that chooses either "tar" or "unzip" as the extractor. Search for the requires.txt file in the archive instead of using a static, expected location. (guess-requirements): Rename the TARBALL argument to ARCHIVE, to denote the archive format is no longer bound specifically to the Tar format. (compute-inputs): Likewise. * tests/pypi.scm ("pypi->guix-package, no wheel"): Mock the requires.txt at a non-standard location. ("pypi->guix-package, no usable requirement file."): New test.
2019-07-02import: pypi: Deduplicate requirements.Maxim Cournoyer
* guix/import/pypi.scm (parse-requires.txt): Remove potential duplicates.
2019-07-02import: pypi: Improve parsing of requirement specifications.Maxim Cournoyer
The previous solution was fragile and could leave unwanted characters in a requirement name, such as '[' or ']'. Partially fixes <https://bugs.gnu.org/33047>. * guix/import/pypi.scm (use-modules): Export SPECIFICATION->REQUIREMENT-NAME (%requirement-name-regexp): New variable. (clean-requirement): Rename to... (specification->requirement-name): this, which now uses %requirement-name-regexp to select the requirement name from the requirement specification. (parse-requires.txt): Adapt.
2019-07-02import: pypi: Do not parse optional requirements from source.Maxim Cournoyer
* guix/import/pypi.scm: Export PARSE-REQUIRES.TXT. (clean-requirement): Move procedure to the top level. (guess-requirements): Move the READ-REQUIREMENTS procedure to the top level, and rename it to PARSE-REQUIRES.TXT. Move the CLEAN-REQUIREMENT procedure to the top level. Move the COMMENT? functions inside the PARSE-REQUIRES.TXT procedure. (parse-requires.txt): Add a SECTION-HEADER? predicate, and use it to prevent parsing optional requirements. * tests/pypi.scm (test-requires-with-sections): New variable. ("parse-requires.txt, with sections"): New test.
2019-07-02import: pypi: Do not consider requirements.txt files.Maxim Cournoyer
PyPI packages are mandated to have a setup.py file, which contains a listing of the required dependencies. The setuptools/distutils machinery embed metadata in the archives they produce, which contains this information. There is no need nor gain to collect the requirements from a "requirements.txt" file, as it is not the true record of dependencies for PyPI packages and may contain extraneous requirements or not exist at all. * guix/import/pypi.scm (guess-requirements): Update comment. [guess-requirements-from-source]: Do not attempt to parse the file requirements.txt. Streamline logic. * tests/pypi.scm (test-requires.txt): Rename from test-requirements, to hint at the file being tested. ("pypi->guix-package"): Adapt so that the fake package contains a requires.txt file rather than a requirements.txt file. ("pypi->guix-package, wheels"): Likewise.
2018-10-18gnu: Use pypi.org.Leo Famulari
<pypi.io> redirects to <pypi.org>. * guix/build-system/python.scm (pypi-uri): Replace pypi.io with pypi.org. * guix/import/pypi.scm (pypi-url?): Likewise. * tests/pypi.scm: Likewise.
2018-08-30import: pypi: Support recursive importing.Ricardo Wurmus
* guix/import/pypi.scm (guess-requirements): Use upstream names. (compute-inputs): Return the upstream dependency names as an additional value. (make-pypi-sexp): Likewise. (pypi->guix-package): Memoize it. (pypi-recursive-import): New procedure. * guix/scripts/import/pypi.scm (show-help, %options): Accept "recursive" option. (guix-import-pypi): Use pypi-recursive-import. * doc/guix.texi (Invoking guix import): Document it.
2018-08-30import: pypi: Also guess dependencies from *.egg-info/requires.txt.Ricardo Wurmus
* guix/import/pypi.scm (guess-requirements): Extract "requires.txt" from the egg-info directory in addition to "requirements.txt"; strip off version constraints; use call-with-temporary-directory.
2018-07-24import: PyPI: Update redirected URL.Marius Bakke
* guix/import/pypi.scm (guix-package->pypi-name, pypi->guix-package): Update docstrings. (pypi-package?): Test for pypi.org, too. (pypi-fetch): s/pypi.python.org/pypi.org/ * tests/pypi.scm ("guix-package->pypi-name, new URL style", "pypi->guix-package", "pypi->guix-package, wheels"): Likewise.
2018-06-10import: json: Consolidate duplicate json-fetch functionality.Jelle Licht
* guix/import/json.scm (json-fetch): Return a list or hash table. (json-fetch-alist): New procedure. * guix/import/github.scm (json-fetch*): Remove. (latest-released-version): Use json-fetch. * guix/import/cpan.scm (module->dist-name): Use json-fetch-alist. (cpan-fetch): Likewise. * guix/import/crate.scm (crate-fetch): Likewise. * guix/import/gem.scm (rubygems-fetch): Likewise. * guix/import/pypi.scm (pypi-fetch): Likewise. * guix/import/stackage.scm (stackage-lts-info-fetch): Likewise.
2017-10-10import: pypi: Remove unneeded import.Ludovic Courtès
* guix/import/pypi.scm: Remove unneeded import.
2017-06-15import: pypi: Always use pypi.io URL with downcased package name.Danny Milosavljevic
* guix/import/pypi.scm (make-pypi-sexp): Always use pypi.io URL with downcased package name.
2017-05-13import: pypi: Robustify latest-release.Mathieu Othacehe
* guix/import/pypi.scm (latest-release): Check if pypi-fetch has failed. If so return #f, else construct the <upstream-source>.
2017-05-03ui: Rename '_' to 'G_'.Ludovic Courtès
This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`".
2017-02-08import: pypi: Correctly handle multiple-URL origins.Ludovic Courtès
Fixes <https://bugs.gnu.org/25635>. Reported by Sergei Trofimovich <slyfox@inbox.ru>. * guix/import/pypi.scm (guix-package->pypi-name)[url->pypi-name]: New procedure. Rewrite body to match lists in addition to strings. * tests/pypi.scm ("guix-package->pypi-name, several URLs"): New test.
2017-01-26import: pypi: Don't add setuptools to propagated-inputs.Carlo Zancanaro
* guix/import/pypi.scm (compute-inputs): Don't add setuptools to the imported package's propagated-inputs.
2016-12-20import: pypi: Match new 'pypi-uri' domain in updater.Marius Bakke
* guix/import/pypi.scm (pypi-package?): Match pypi.io domain.
2016-12-12import: json: Silence json-fetch output.Eric Bavier
* guix/import/json.scm (json-fetch): Use http-fetch instead of url-fetch to avoid writing to stdout and a temporary file for each invocation. * guix/import/gem.scm (rubygems-fetch): Do not redirect json-fetch output to /dev/null. * guix/import/pypi.scm (pypi-fetch): Likewise.
2016-10-23import: pypi: All inputs are propagated-inputs by default.Hartmut Goebel
* guix/import/pypi.scm (maybe-inputs): Return inputs as "propagated-inputs".
2016-09-28import: Move string->license to importers.David Craven
* guix/import/gem.scm (string->license): Move from (guix import utils). * guix/import/pypi.scm (string->license): Move from (guix import utils).
2016-07-26import: pypi: Correctly handle new-style URLs.Ludovic Courtès
Fixes <http://bugs.gnu.org/23997>. * guix/import/pypi.scm (guix-package->pypi-name): Rewrite using 'basename' and 'hyphen-package-name->name+version'. * tests/pypi.scm ("guix-package->pypi-name, old URL style") ("guix-package->pypi-name, new URL style"): New tests.
2016-06-25import: pypi: do not fail when 'run_requires' is missing from the metadata.Cyril Roelandt
* guix/import/pypi.scm (read-wheel-metadata): do not crash when 'run_requires' is missing from the metadata.
2016-06-14import: pypi: read requirements from wheels.Cyril Roelandt
* doc/guix.tex (Invoking guix import): Mention that the pypi importer works better with "unzip". * guix/import/pypi.scm (latest-wheel-release, wheel-url->extracted-directory): New procedures. * tests/pypi.scm (("pypi->guix-package, wheels"): New test.
2016-04-15upstream: Pass a package object to updaters.Ludovic Courtès
* guix/upstream.scm (package-update-path): Pass PACKAGE to 'latest-release'. * guix/gnu-maintenance.scm (latest-release*) (latest-gnome-release, latest-xorg-release): Adjust accordingly. * guix/import/cran.scm (latest-cran-release): (latest-bioconductor-release): Likewise. * guix/import/elpa.scm (latest-release): Likewise. * guix/import/gem.scm (latest-release): Likewise. * guix/import/github.scm (latest-release): Likewise. * guix/import/hackage.scm (latest-release): Likewise. * guix/import/pypi.scm (latest-release): Likewise.
2016-03-19import: pypi: Emit 'pypi-uri' only when it yields the right URL.Ludovic Courtès
Fixes <http://bugs.gnu.org/23062>. Reported by Danny Milosavljevic <dannym@scratchpost.org>. * guix/import/pypi.scm (make-pypi-sexp): Check whether 'pypi-uri' returns SOURCE-URL and fall back to the full URL otherwise. * tests/pypi.scm ("pypi->guix-package"): Adjust expected URI accordingly. Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
2015-12-01import: pypi: Updater silently ignores packages without source.Ludovic Courtès
Reported by Andreas Enge <andreas@enge.fr> at <https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00764.html>. * guix/import/pypi.scm (&missing-source-error): New error type. (latest-source-release): Raise it instead of using 'error'. (pypi->guix-package): Guard against it and use 'leave' to report the error. (latest-release): Guard against it and return #f when caught.
2015-11-04import: pypi: Make downloads silent.Ludovic Courtès
* guix/import/pypi.scm (pypi-fetch): Wrap body in 'call-with-output-file' and 'with-error-to-port'.
2015-11-04import: pypi: Add missing copyright line.Ludovic Courtès
* guix/import/pypi.scm: Add missing copyright line for bab020d.
2015-11-03import: pypi: add updaterCyril Roelandt
* guix/import/pypi.scm (guix-package->pypi-name, latest-release): New procedures. (%pypi-updater): New variable. * guix/scripts/refresh.scm (%updaters): Add %PYPI-UPDATER. * doc/guix.texi (Invoking guix refresh): Mention PyPI
2015-11-03import: pypi: Use "pypi-uri" instead of building the URL manually.Cyril Roelandt
* guix/import/pypi.scm (make-pypi-sexp): Use "pypi-uri". * tests/pypi.scm: Update the tests accordingly.
2015-10-05import: pypi: Remove unused procedure.Ludovic Courtès
* guix/import/pypi.scm (join): Remove.
2015-08-27import: pypi: Move generally useful procedures to utils module.David Thompson
* guix/import/pypi.scm (make-pypi-sexp): Factorize license to symbol conversion code. (string->license, snake-case, guix-hash-url): Move from here... * guix/import/utils.scm: ... to here. (license->symbol): New procedure.
2015-07-21import: pypi: Improve warning message.Ludovic Courtès
* guix/import/pypi.scm (guess-requirements): Add missing newline and quotes in warning.
2015-06-24import: pypi: Detect inputs.Cyril Roelandt
* guix/import/pypi.scm (python->package-name, maybe-inputs, compute-inputs, guess-requirements): New procedures. * guix/import/pypi.scm (guix-hash-url): Now takes a filename instead of an URL as input. * guix/import/pypi.scm (make-pypi-sexp): Now tries to generate the inputs automagically. * tests/pypi.scm: Update the test.
2015-01-09import: Factorize utility functions.Eric Bavier
* guix/import/pypi.scm (hash-table->alist, flatten, assoc-ref*, url-fetch, json-fetch): Pull procedures from here into... * guix/import/utils.scm: Here and... * guix/import/json.scm: Here. New file. * Makefile.am (MODULE)[HAVE_GUILE_JSON]: Add it. * guix/import/gnu.scm (file-sha256): Move from here to... * guix/hash.scm: Here. * tests/pypi.scm (pypi->guix-package): Update mock module reference.
2014-11-05import: pypi: Gracefully handle non-existent packages.Ludovic Courtès
Fixes <http://bugs.gnu.org/18831>. Reported by Ian Denhardt <ian@zenhack.net>. * guix/import/pypi.scm (url-fetch, json-fetch, pypi-fetch): Augment docstring to mention #f on failure. (pypi->guix-package): Likewise, and actually return #f on failure. * guix/scripts/import/pypi.scm (guix-import-pypi): Call 'leave' when 'pypi->guix-package' returns #f.
generated by cgit v1.2.3 (git 2.46.2) at 2024-11-14 00:16:52 +0000