summaryrefslogtreecommitdiff
path: root/guix/import/pypi.scm
AgeCommit message (Expand)Author
2020-12-02import: utils: 'recursive-import' accepts an optional version parameter....This adds a key VERSION to 'recursive-import' and moves the parameter REPO to a key. This also changes all the places that rely on 'recursive-import'. * guix/import/utils.scm (recursive-import): Add the VERSION key. Make REPO a key. (package->definition): Add optional 'append-version?'. * guix/scripts/import/crate.scm (guix-import-crate): Add the VERSION key. * guix/import/crate.scm (crate->guix-package): Add the VERSION key. (crate-recursive-import): Pass VERSION to recursive-import, remove now unnecessary code. * guix/import/cran.scm (cran->guix-package, cran-recursive-import): Change the REPO parameter to a key. * guix/import/elpa.scm (elpa->guix-package, elpa-recursive-import): Likewise. * guix/import/gem.scm (gem->guix-package, recursive-import): Likewise. * guix/import/opam.scm (opam-recurive-import): Likewise. * guix/import/pypi.scm (pypi-recursive-import): Likewise. * guix/import/stackage.scm (stackage-recursive-import): Likewise. * guix/scripts/import/cran.scm (guix-import-cran): Likewise. * guix/scripts/import/elpa.scm (guix-import-elpa): Likewise. * tests/elpa.scm (eval-test-with-elpa): Likewise. * tests/import-utils.scm (recursive-import): Likewise. Co-authored-by: Hartmut Goebel <h.goebel@crazy-compilers.com> Martin Becze
2020-09-08Remove (guix json) and require Guile-JSON 4.3.0+....This is a followup to 4071879c86d059ee087c8986915ea72b8c742b72. * guix/json.scm: Remove. * Makefile.am (MODULES): Adjust accordingly. * m4/guix.m4 (GUIX_CHECK_GUILE_JSON): Check for 'define-json-mapping'. * doc/guix.texi (Requirements): Require Guile-JSON 4.3.0+. * guix/ci.scm, guix/cve.scm, guix/import/cpan.scm, guix/import/crate.scm, guix/swh.scm: Remove (guix json) import. * guix/import/gem.scm, guix/import/pypi.scm: Likewise, and import (json). * guix/self.scm (specification->package): Switch to GUILE-JSON-4. * guix/git-download.scm (git-fetch): Likewise. Ludovic Courtès
2020-07-25utils: Remove compatibility re-export of 'memoize'....The 'memoize' binding was re-exported in 2016, commit 19e1d5f7f90194f1ac7e783b28a688ce1441786d, as a backwards-compatibility measure that makes little sense now. * guix/utils.scm: Don't re-export 'memoize'. * guix/import/pypi.scm: Adjust imports. * tests/pypi.scm: Remove duplicate (guix memoization) import. Ludovic Courtès
2020-07-05import: pypi: Handle 'null instead of #nil....* guix/import/pypi.scm (non-empty-string-or-false): guile-json now returns 'null instead of #nil for null JSON values. Handle it. Arun Isaac
2020-07-03upstream: Define 'url-predicate' and use it....* guix/upstream.scm (url-predicate): New procedure. (url-prefix-predicate): Define in terms of 'url-predicate'. * guix/import/cpan.scm (cpan-package?): Use 'url-predicate'. * guix/import/cran.scm (cran-package?) (bioconductor-package?) (bioconductor-data-package?) (bioconductor-experiment-package?): Likewise. * guix/import/crate.scm (crate-package?): Likewise. * guix/import/elpa.scm (package-from-gnu.org?): Likewise. * guix/import/hackage.scm (hackage-package?): Likewise. * guix/import/pypi.scm (pypi-package?): Likewise. * guix/import/gem.scm (gem-package?): Use 'url-prefix-predicate'. Ludovic Courtès
2020-03-12import: pypi: Rewrite to use 'define-json-mapping'....* guix/import/pypi.scm (non-empty-string-or-false): New procedure. (<pypi-project>, <project-info>, <distribution>): New record types. (pypi-fetch): Call 'json->pypi-project'. (latest-source-release, latest-wheel-release): Use the new record accessors instead of 'assoc-ref*'. (pypi->guix-package, latest-release): Likewise. * tests/pypi.scm (test-json): Add mandatory fields. Ludovic Courtès
2020-03-04import: pypi: Add more licenses...* guix/import/pypi.scm (string->license): Add the BSD 2-clause and MPL 2.0 licenses, and add more strings for BSD 3-clause and Expat license. Signed-off-by: Leo Famulari <leo@famulari.name> Lars-Dominik Braun
2020-02-05import: pypi: Support exporting packages with .zip source....* guix/import/pypi.scm (make-pypi-sexp): Rename test-inputs to native-inputs. Restructure the way pypi-uri parameters are generated. Use pypi-uri's extension parameter when required. Add "unzip" to native inputs when the package source is a zip file. Signed-off-by: Marius Bakke <mbakke@fastmail.com> Jakub Kądziołka
2019-09-18import: pypi: Refresher recognizes pythonhosted.org source URLs....This is a followup to a5376200541abf8245973e601be246bf65b8b6c7. Since that commit, 'pypi-package?' would return false for most Python packages, and thus "guix refresh python-xxx" would report that no updaters apply to the package. * guix/import/pypi.scm (pypi-package?)[pypi-url?]: Recognize "files.pythonhosted.org" URLs. Ludovic Courtès
2019-07-25maint: Switch to Guile-JSON 3.x....Guile-JSON 3.x is incompatible with Guile-JSON 1.x, which we relied on until now: it maps JSON dictionaries to alists (instead of hash tables), and JSON arrays to vectors (instead of lists). This commit is about adjusting all the existing code to this new mapping. * m4/guix.m4 (GUIX_CHECK_GUILE_JSON): New macro. * configure.ac: Use it. * doc/guix.texi (Requirements): Mention the Guile-JSON version. * guix/git-download.scm (git-fetch)[guile-json]: Use GUILE-JSON-3. * guix/import/cpan.scm (string->license): Expect vectors instead of lists. (module->dist-name): Use 'json-fetch' instead of 'json-fetch-alist'. (cpan-fetch): Likewise. * guix/import/crate.scm (crate-fetch): Likewise, and call 'vector->list' for DEPS. * guix/import/gem.scm (rubygems-fetch): Likewise. * guix/import/json.scm (json-fetch-alist): Remove. * guix/import/pypi.scm (pypi-fetch): Use 'json-fetch' instead of 'json-fetch-alist'. (latest-source-release, latest-wheel-release): Call 'vector->list' on RELEASES. * guix/import/stackage.scm (stackage-lts-info-fetch): Use 'json-fetch' instead of 'json-fetch-alist'. (lts-package-version): Use 'vector->list'. * guix/import/utils.scm (hash-table->alist): Remove. (alist->package): Pass 'vector->list' on the inputs fields, and default to the empty vector. * guix/scripts/import/json.scm (guix-import-json): Remove call to 'hash-table->alist'. * guix/swh.scm (define-json-reader): Expect pair? or null? instead of hash-table?. [extract-field]: Use 'assoc-ref' instead of 'hash-ref'. (json->branches): Use 'map' instead of 'hash-map->list'. (json->checksums): Likewise. (json->directory-entries, origin-visits): Call 'vector->list' on the result of 'json->scm'. * tests/import-utils.scm ("alist->package with dependencies"): New test. * gnu/installer.scm (build-compiled-file)[builder]: Use GUILE-JSON-3. * gnu/installer.scm (installer-program)[installer-builder]: Likewise. * gnu/installer/locale.scm (iso639->iso639-languages): Use 'assoc-ref' instead of 'hash-ref', and pass vectors through 'vector->list'. (iso3166->iso3166-territories): Likewise. * gnu/system/vm.scm (system-docker-image)[build]: Use GUILE-JSON-3. * guix/docker.scm (manifest, config): Adjust for Guile-JSON 3. * guix/scripts/pack.scm (docker-image)[build]: Use GUILE-JSON-3. * guix/import/github.scm (fetch-releases-or-tags): Update docstring. (latest-released-version): Use 'assoc-ref' instead of 'hash-ref'. Pass the result of 'fetch-releases-or-tags' to 'vector->list'. * guix/import/launchpad.scm (latest-released-version): Likewise. Ludovic Courtès
2019-07-02import: pypi: Preserve package name case when forming pypi-uri....Fixes <https://bugs.gnu.org/33046>. * guix/build-system/python.scm (pypi-uri): Update the host URI to "files.pythonhosted.org". * guix/import/pypi.scm (make-pypi-sexp): Preserve the package name case when the source URL calls for it. Maxim Cournoyer
2019-07-02import: pypi: Include optional test inputs as native-inputs....* guix/import/pypi.scm (maybe-inputs): Add INPUT-TYPE argument, and use it. (test-section?): New predicate. (parse-requires.txt): Collect the optional test inputs, and return them as the second element of the returned list. (parse-wheel-metadata): Likewise. (guess-requirements): Adapt. (make-pypi-sexp): Likewise, and include the test inputs requirements as native inputs in the returned package expression. * tests/pypi.scm (test-requires.txt): Include a test section in the test-requires.txt data. (test-requires.txt-beaker): New variable. ("parse-requires.txt"): Adapt. ("parse-requires.txt - Beaker"): New test. ("parse-wheel-metadata, with extras"): Adapt. ("parse-wheel-metadata, with extras - Jedi"): Adapt. ("pypi->guix-package, no wheel"): Re-indent, and add the expected native-inputs. ("pypi->guix-package, wheels"): Likewise. Maxim Cournoyer
2019-07-02import: pypi: Completely mute the output of the "unzip" command....* guix/import/pypi.scm (guess-requirements): Completely mute the output of the "unzip" command. Maxim Cournoyer
2019-07-02import: pypi: Fix typo in docstring....* guix/import/pypi.scm (guess-requirements): Fix typo. Maxim Cournoyer
2019-07-02import: pypi: Parse wheel METADATA instead of metadata.json....With newer Wheel releases, there is no more metadata.json file; the METADATA file should be used instead (see: https://github.com/pypa/wheel/issues/195). This change updates our PyPI importer so that it uses the latter. * guix/import/pypi.scm (define-module): Remove unnecessary modules and export the PARSE-WHEEL-METADATA procedure. (parse-wheel-metadata): Add procedure. (guess-requirements): Use it. * tests/pypi.scm (test-metadata): Test it. Maxim Cournoyer
2019-07-02import: pypi: Support more types of archives....This change enables the PyPI importer to look for requirements in a source archive of a different type than "tar.gz" or "tar.bz2". Also, scan the source archive to find a requires.txt file. * guix/import/pypi.scm: (guess-requirements)[tarball-directory]: Remove procedure. [guess-requirements-from-source]: Use COMRESSED-FILE? to determine if an archive type is supported, and some file extension logic that chooses either "tar" or "unzip" as the extractor. Search for the requires.txt file in the archive instead of using a static, expected location. (guess-requirements): Rename the TARBALL argument to ARCHIVE, to denote the archive format is no longer bound specifically to the Tar format. (compute-inputs): Likewise. * tests/pypi.scm ("pypi->guix-package, no wheel"): Mock the requires.txt at a non-standard location. ("pypi->guix-package, no usable requirement file."): New test. Maxim Cournoyer
2019-07-02import: pypi: Deduplicate requirements....* guix/import/pypi.scm (parse-requires.txt): Remove potential duplicates. Maxim Cournoyer
2019-07-02import: pypi: Improve parsing of requirement specifications....The previous solution was fragile and could leave unwanted characters in a requirement name, such as '[' or ']'. Partially fixes <https://bugs.gnu.org/33047>. * guix/import/pypi.scm (use-modules): Export SPECIFICATION->REQUIREMENT-NAME (%requirement-name-regexp): New variable. (clean-requirement): Rename to... (specification->requirement-name): this, which now uses %requirement-name-regexp to select the requirement name from the requirement specification. (parse-requires.txt): Adapt. Maxim Cournoyer
2019-07-02import: pypi: Do not parse optional requirements from source....* guix/import/pypi.scm: Export PARSE-REQUIRES.TXT. (clean-requirement): Move procedure to the top level. (guess-requirements): Move the READ-REQUIREMENTS procedure to the top level, and rename it to PARSE-REQUIRES.TXT. Move the CLEAN-REQUIREMENT procedure to the top level. Move the COMMENT? functions inside the PARSE-REQUIRES.TXT procedure. (parse-requires.txt): Add a SECTION-HEADER? predicate, and use it to prevent parsing optional requirements. * tests/pypi.scm (test-requires-with-sections): New variable. ("parse-requires.txt, with sections"): New test. Maxim Cournoyer
2019-07-02import: pypi: Do not consider requirements.txt files....PyPI packages are mandated to have a setup.py file, which contains a listing of the required dependencies. The setuptools/distutils machinery embed metadata in the archives they produce, which contains this information. There is no need nor gain to collect the requirements from a "requirements.txt" file, as it is not the true record of dependencies for PyPI packages and may contain extraneous requirements or not exist at all. * guix/import/pypi.scm (guess-requirements): Update comment. [guess-requirements-from-source]: Do not attempt to parse the file requirements.txt. Streamline logic. * tests/pypi.scm (test-requires.txt): Rename from test-requirements, to hint at the file being tested. ("pypi->guix-package"): Adapt so that the fake package contains a requires.txt file rather than a requirements.txt file. ("pypi->guix-package, wheels"): Likewise. Maxim Cournoyer
2018-10-18gnu: Use pypi.org....<pypi.io> redirects to <pypi.org>. * guix/build-system/python.scm (pypi-uri): Replace pypi.io with pypi.org. * guix/import/pypi.scm (pypi-url?): Likewise. * tests/pypi.scm: Likewise. Leo Famulari
2018-08-30import: pypi: Support recursive importing....* guix/import/pypi.scm (guess-requirements): Use upstream names. (compute-inputs): Return the upstream dependency names as an additional value. (make-pypi-sexp): Likewise. (pypi->guix-package): Memoize it. (pypi-recursive-import): New procedure. * guix/scripts/import/pypi.scm (show-help, %options): Accept "recursive" option. (guix-import-pypi): Use pypi-recursive-import. * doc/guix.texi (Invoking guix import): Document it. Ricardo Wurmus
2018-08-30import: pypi: Also guess dependencies from *.egg-info/requires.txt....* guix/import/pypi.scm (guess-requirements): Extract "requires.txt" from the egg-info directory in addition to "requirements.txt"; strip off version constraints; use call-with-temporary-directory. Ricardo Wurmus
2018-07-24import: PyPI: Update redirected URL....* guix/import/pypi.scm (guix-package->pypi-name, pypi->guix-package): Update docstrings. (pypi-package?): Test for pypi.org, too. (pypi-fetch): s/pypi.python.org/pypi.org/ * tests/pypi.scm ("guix-package->pypi-name, new URL style", "pypi->guix-package", "pypi->guix-package, wheels"): Likewise. Marius Bakke
2018-06-10import: json: Consolidate duplicate json-fetch functionality....* guix/import/json.scm (json-fetch): Return a list or hash table. (json-fetch-alist): New procedure. * guix/import/github.scm (json-fetch*): Remove. (latest-released-version): Use json-fetch. * guix/import/cpan.scm (module->dist-name): Use json-fetch-alist. (cpan-fetch): Likewise. * guix/import/crate.scm (crate-fetch): Likewise. * guix/import/gem.scm (rubygems-fetch): Likewise. * guix/import/pypi.scm (pypi-fetch): Likewise. * guix/import/stackage.scm (stackage-lts-info-fetch): Likewise. Jelle Licht
2017-10-10import: pypi: Remove unneeded import....* guix/import/pypi.scm: Remove unneeded import. Ludovic Courtès
2017-06-15import: pypi: Always use pypi.io URL with downcased package name....* guix/import/pypi.scm (make-pypi-sexp): Always use pypi.io URL with downcased package name. Danny Milosavljevic
2017-05-13import: pypi: Robustify latest-release....* guix/import/pypi.scm (latest-release): Check if pypi-fetch has failed. If so return #f, else construct the <upstream-source>. Mathieu Othacehe
2017-05-03ui: Rename '_' to 'G_'....This avoids collisions with '_' when the latter is used as a 'match' pattern for instance. See <https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00464.html>. * guix/ui.scm: Rename '_' to 'G_'. * po/guix/Makevars (XGETTEXT_OPTIONS): Adjust accordingly. * build-aux/compile-all.scm (warnings): Remove 'format'. * gnu/packages.scm, gnu/services.scm, gnu/services/shepherd.scm, gnu/system.scm, gnu/system/shadow.scm, guix/gnupg.scm, guix/http-client.scm, guix/import/cpan.scm, guix/import/elpa.scm, guix/import/pypi.scm, guix/nar.scm, guix/scripts.scm, guix/scripts/archive.scm, guix/scripts/authenticate.scm, guix/scripts/build.scm, guix/scripts/challenge.scm, guix/scripts/container.scm, guix/scripts/container/exec.scm, guix/scripts/copy.scm, guix/scripts/download.scm, guix/scripts/edit.scm, guix/scripts/environment.scm, guix/scripts/gc.scm, guix/scripts/graph.scm, guix/scripts/hash.scm, guix/scripts/import.scm, guix/scripts/import/cpan.scm, guix/scripts/import/cran.scm, guix/scripts/import/crate.scm, guix/scripts/import/elpa.scm, guix/scripts/import/gem.scm, guix/scripts/import/gnu.scm, guix/scripts/import/hackage.scm, guix/scripts/import/nix.scm, guix/scripts/import/pypi.scm, guix/scripts/import/stackage.scm, guix/scripts/lint.scm, guix/scripts/offload.scm, guix/scripts/pack.scm, guix/scripts/package.scm, guix/scripts/perform-download.scm, guix/scripts/publish.scm, guix/scripts/pull.scm, guix/scripts/refresh.scm, guix/scripts/size.scm, guix/scripts/substitute.scm, guix/scripts/system.scm, guix/ssh.scm, guix/upstream.scm: Use 'G_' instead of '_'. Most of this change was obtained by running: "sed -i -e's/(_ "/(G_ "/g' `find -name \*.scm`". Ludovic Courtès
2017-02-08import: pypi: Correctly handle multiple-URL origins....Fixes <https://bugs.gnu.org/25635>. Reported by Sergei Trofimovich <slyfox@inbox.ru>. * guix/import/pypi.scm (guix-package->pypi-name)[url->pypi-name]: New procedure. Rewrite body to match lists in addition to strings. * tests/pypi.scm ("guix-package->pypi-name, several URLs"): New test. Ludovic Courtès
2017-01-26import: pypi: Don't add setuptools to propagated-inputs....* guix/import/pypi.scm (compute-inputs): Don't add setuptools to the imported package's propagated-inputs. Carlo Zancanaro
2016-12-20import: pypi: Match new 'pypi-uri' domain in updater....* guix/import/pypi.scm (pypi-package?): Match pypi.io domain. Marius Bakke
2016-12-12import: json: Silence json-fetch output....* guix/import/json.scm (json-fetch): Use http-fetch instead of url-fetch to avoid writing to stdout and a temporary file for each invocation. * guix/import/gem.scm (rubygems-fetch): Do not redirect json-fetch output to /dev/null. * guix/import/pypi.scm (pypi-fetch): Likewise. Eric Bavier
2016-10-23import: pypi: All inputs are propagated-inputs by default....* guix/import/pypi.scm (maybe-inputs): Return inputs as "propagated-inputs". Hartmut Goebel
2016-09-28import: Move string->license to importers....* guix/import/gem.scm (string->license): Move from (guix import utils). * guix/import/pypi.scm (string->license): Move from (guix import utils). David Craven
2016-07-26import: pypi: Correctly handle new-style URLs....Fixes <http://bugs.gnu.org/23997>. * guix/import/pypi.scm (guix-package->pypi-name): Rewrite using 'basename' and 'hyphen-package-name->name+version'. * tests/pypi.scm ("guix-package->pypi-name, old URL style") ("guix-package->pypi-name, new URL style"): New tests. Ludovic Courtès
2016-06-25import: pypi: do not fail when 'run_requires' is missing from the metadata....* guix/import/pypi.scm (read-wheel-metadata): do not crash when 'run_requires' is missing from the metadata. Cyril Roelandt
2016-06-14import: pypi: read requirements from wheels....* doc/guix.tex (Invoking guix import): Mention that the pypi importer works better with "unzip". * guix/import/pypi.scm (latest-wheel-release, wheel-url->extracted-directory): New procedures. * tests/pypi.scm (("pypi->guix-package, wheels"): New test. Cyril Roelandt
2016-04-15upstream: Pass a package object to updaters....* guix/upstream.scm (package-update-path): Pass PACKAGE to 'latest-release'. * guix/gnu-maintenance.scm (latest-release*) (latest-gnome-release, latest-xorg-release): Adjust accordingly. * guix/import/cran.scm (latest-cran-release): (latest-bioconductor-release): Likewise. * guix/import/elpa.scm (latest-release): Likewise. * guix/import/gem.scm (latest-release): Likewise. * guix/import/github.scm (latest-release): Likewise. * guix/import/hackage.scm (latest-release): Likewise. * guix/import/pypi.scm (latest-release): Likewise. Ludovic Courtès
2016-03-19import: pypi: Emit 'pypi-uri' only when it yields the right URL....Fixes <http://bugs.gnu.org/23062>. Reported by Danny Milosavljevic <dannym@scratchpost.org>. * guix/import/pypi.scm (make-pypi-sexp): Check whether 'pypi-uri' returns SOURCE-URL and fall back to the full URL otherwise. * tests/pypi.scm ("pypi->guix-package"): Adjust expected URI accordingly. Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org> Ludovic Courtès
2015-12-01import: pypi: Updater silently ignores packages without source....Reported by Andreas Enge <andreas@enge.fr> at <https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00764.html>. * guix/import/pypi.scm (&missing-source-error): New error type. (latest-source-release): Raise it instead of using 'error'. (pypi->guix-package): Guard against it and use 'leave' to report the error. (latest-release): Guard against it and return #f when caught. Ludovic Courtès
2015-11-04import: pypi: Make downloads silent....* guix/import/pypi.scm (pypi-fetch): Wrap body in 'call-with-output-file' and 'with-error-to-port'. Ludovic Courtès
2015-11-04import: pypi: Add missing copyright line....* guix/import/pypi.scm: Add missing copyright line for bab020d. Ludovic Courtès
2015-11-03import: pypi: add updater...* guix/import/pypi.scm (guix-package->pypi-name, latest-release): New procedures. (%pypi-updater): New variable. * guix/scripts/refresh.scm (%updaters): Add %PYPI-UPDATER. * doc/guix.texi (Invoking guix refresh): Mention PyPI Cyril Roelandt
2015-11-03import: pypi: Use "pypi-uri" instead of building the URL manually....* guix/import/pypi.scm (make-pypi-sexp): Use "pypi-uri". * tests/pypi.scm: Update the tests accordingly. Cyril Roelandt
2015-10-05import: pypi: Remove unused procedure....* guix/import/pypi.scm (join): Remove. Ludovic Courtès
2015-08-27import: pypi: Move generally useful procedures to utils module....* guix/import/pypi.scm (make-pypi-sexp): Factorize license to symbol conversion code. (string->license, snake-case, guix-hash-url): Move from here... * guix/import/utils.scm: ... to here. (license->symbol): New procedure. David Thompson
2015-07-21import: pypi: Improve warning message....* guix/import/pypi.scm (guess-requirements): Add missing newline and quotes in warning. Ludovic Courtès
2015-06-24import: pypi: Detect inputs....* guix/import/pypi.scm (python->package-name, maybe-inputs, compute-inputs, guess-requirements): New procedures. * guix/import/pypi.scm (guix-hash-url): Now takes a filename instead of an URL as input. * guix/import/pypi.scm (make-pypi-sexp): Now tries to generate the inputs automagically. * tests/pypi.scm: Update the test. Cyril Roelandt
2015-01-09import: Factorize utility functions....* guix/import/pypi.scm (hash-table->alist, flatten, assoc-ref*, url-fetch, json-fetch): Pull procedures from here into... * guix/import/utils.scm: Here and... * guix/import/json.scm: Here. New file. * Makefile.am (MODULE)[HAVE_GUILE_JSON]: Add it. * guix/import/gnu.scm (file-sha256): Move from here to... * guix/hash.scm: Here. * tests/pypi.scm (pypi->guix-package): Update mock module reference. Eric Bavier