Audits Python environments and dependency trees for known vulnerabilities

Last update: Dec 28, 2022

Overview

pip-audit

pip-audit is a prototype tool for scanning Python environments for packages with known vulnerabilities. It uses the Python Packaging Advisory Database (https://github.com/pypa/advisory-db) as a source of vulnerability reports.

This project is developed by Trail of Bits with support from Google. This is not an official Google product.

Development steps

git clone https://github.com/trailofbits/pip-audit && cd pip-audit
make dev && source env/bin/activate
pip-audit --help

Release process

Releases of pip-audit are managed via bump and GitHub Actions.

# default release (patch bump)
make release

# override the default
# vX.Y.Z -> vX.Y.Z-rc.0
make release BUMP_ARGS="--pre rc.0"

# vX.Y.Z -> vN.0.0
make release BUMP_ARGS="--major"

make release will fail if there are any untracked changes in the source tree.

If make release succeeds, you'll see an output like this:

RUN ME MANUALLY: git push origin main && git push origin vX.Y.Z

Run that last command sequence to complete the release.

Comments

0.0.4 test failure: fixture 'vuln_service' not found

When running the self tests for pip-audit-0.0.4 using py.test, I see:

___________________________________________________________ ERROR at setup of test_audit ___________________________________________________________
file /scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py, line 10
  def test_audit(vuln_service, dep_source):                                                                                                         
E       fixture 'vuln_service' not found                                  
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, 
record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory                                        
>       use 'pytest --fixtures [testpath]' for help on them.              
                                                                                                                                                    
/scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py:10                                                                                
_______________________________________________________ ERROR at setup of test_audit_dry_run _______________________________________________________
file /scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py, line 32                                                                     
  def test_audit_dry_run(monkeypatch, vuln_service, dep_source):                                                                                    
E       fixture 'vuln_service' not found                                                                                                            
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, 
record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.                                                                                        
                                                                                                                                                    
/scratch/wip/py-pip-audit/work/pip-audit-0.0.4/test/test_audit.py:32

bug

opened by 0-wiz-0 35

scan without resolving/downloading dependencies
maybe i'm misunderstanding, but it seems that pip-audit cannot scan the contents of a requirements file in the simplest way imaginable: for each entry, check if the specified version has known vulnerabilities.

currently, running against a requirements file with exclusively exact version matches (foo==x.y), like this

$ pip-audit -r requirements.txt

… downloads and (temporarily) installs various packages, which not only takes forever, but also eventually fails because some packages cannot be found – they come from a private index server, which is a different issue.

as a work-around, installing everything (also downloads and installs) and then using pip-audit --local seems to work, but this is not a convenient solution.

i understand that in the general case, requirements files may not be exhaustive and may not have exact versions, in which case running a resolver etc makes total sense, but that's not always the case.

some background: the use case is that my requirements files are produced by pip-tools and hence constitute the exact list of pinned versions used by my application, and i would like to integrate pip-audit into a continuous integration job. i also run pip-compile (from pip-tools) in that job to warn about potential package upgrades, some of which come from private index servers, and it would be great if pip-audit can be added to it.
enhancement
opened by wbolster 19

Fails to parse data-requires-python with a `*` char

Bug description

pip-audit raises an InvalidSpecifier error when trying to parse a data-requires-python key containing a * from pypi. Seems related to this old issue : https://github.com/pypa/pip-audit/issues/138

Reproduction steps

I found the nltk package wich contains this link in its pypi page raising the mentioned error:

<a 
  href="https://files.pythonhosted.org/packages/98/06/de681159e6750d0a215c2126e784504e177896afddf5a68cba42ebe42355/nltk-3.6-py3-none-any.whl#sha256=718de6908f538db19a77f96b9e6f5f586b0892d7de5eea32e71f2a2535ed8657" 
  data-requires-python="&gt;=3.5.*" 
>
  nltk-3.6-py3-none-any.whl
</a><br />

To reproduce, we only need to launch a pip-audit with this dependency.

echo "nltk" > requirements.txt
pip-audit -v --requirement requirements.txt

Screenshots and logs

DEBUG:pip_audit._cli:parsed arguments: Namespace(cache_dir=None, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, dry_run=False, extra_index_urls=[], fix=False, format=<OutputFormatChoice.Columns: 'columns'>, ignore_vulns=[], index_url='https://pypi.org/simple/', local=False, no_deps=False, output=PosixPath('stdout'), paths=[], progress_spinner=<ProgressSpinnerChoice.On: 'on'>, project_path=None, require_hashes=False, requirements=[<_io.TextIOWrapper name='test_require.txt' mode='r' encoding='UTF-8'>], skip_editable=False, strict=False, timeout=15, verbose=True, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/nltk/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 1186
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/nltk/ HTTP/1.1" 304 0
Traceback (most recent call last):
  File "/usr/local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_cli.py", line 448, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_audit.py", line 67, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_service/interface.py", line 155, in query_all
    for spec in specs:
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 116, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/requirement.py", line 328, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/interface.py", line 88, in resolve_all
    yield (req, self.resolve(req))
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 77, in resolve
    result = self.resolver.resolve([req])
  File "/usr/local/lib/python3.8/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.8/site-packages/resolvelib/resolvers.py", line 372, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.8/site-packages/resolvelib/resolvers.py", line 168, in _add_to_criteria
    candidates=build_iter_view(matches),
  File "/usr/local/lib/python3.8/site-packages/resolvelib/structs.py", line 169, in build_iter_view
    matches = list(matches)
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 362, in find_matches
    [
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 362, in <listcomp>
    [
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 203, in get_project_from_indexes
    yield from get_project_from_index(index_url, session, project, extras, timeout, state)
  File "/usr/local/lib/python3.8/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 249, in get_project_from_index
    spec = SpecifierSet(py_req)
  File "/usr/local/lib/python3.8/site-packages/packaging/specifiers.py", line 700, in __init__
    parsed.add(Specifier(specifier))
  File "/usr/local/lib/python3.8/site-packages/packaging/specifiers.py", line 234, in __init__
    raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
packaging.specifiers.InvalidSpecifier: Invalid specifier: '>=3.5.*'

Platform information

Linux
pip-audit 2.4.9
Python 3.8.2
pip 22.3.1

bug component:dep-sources

opened by j0ack 18

Incompatibility with packaging>=22.0: cannot import name 'LegacyVersion' from 'packaging.version'

Bug description

pip-audit is incompatible with latest version of packaging, due to removed LegacyVersion

Reproduction steps

pip install pip-audit
Check that packaging version is >= 22.0
Run pip freeze | pip-audit --no-deps -r - to check the currently installed packages

Expected behavior

pip-audit installs a compatible version of packaging

Screenshots and logs

Traceback (most recent call last):
  File "/usr/local/bin/pip-audit", line 5, in <module>
    from pip_audit._cli import audit
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_cli.py", line 17, in <module>
    from pip_audit._audit import AuditOptions, Auditor
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_audit.py", line 10, in <module>
    from pip_audit._dependency_source import DependencySource
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/__init__.py", line 14, in <module>
    from .requirement import RequirementSource
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 19, in <module>
    from pip_requirements_parser import InstallRequirement, InvalidRequirementLine, RequirementsFile
  File "/usr/local/lib/python3.9/site-packages/pip_requirements_parser.py", line 73, in <module>
    from packaging.version import LegacyVersion
ImportError: cannot import name 'LegacyVersion' from 'packaging.version' (/usr/local/lib/python3.9/site-packages/packaging/version.py)

Platform information

OS name and version: docker, python:3.9-alpine
pip-audit version (pip-audit -V): 2.4.7
Python version (python -V or python3 -V): 3.9.15
pip version (pip -V or pip3 -V): 22.0.4

bug

opened by haydngreatnews 15

pip-audit failure using --index-url

Bug description

Hey guys sorry I'm actually not sure if this is a bug or I'm doing something wrong. pip-audit is failing with a custom pypi index-url option

Reproduction steps

pip-audit -r requirements.txt --index-url https://custom-repo.com/api/pypi/pypi-all/simple

Expected behavior

Expect this to download the packages from the custom pypi repo as expected. for example pip install -r requirements.txt --index-url https://custom-repo.com/api/pypi/pypi-all/simple works as expected

Screenshots and logs

$ pip-audit -v \
    --format json \
    --progress-spinner off \
    --requirement ./_pip_audit.txt \
    --index-url https://custom-repo.com/api/pypi/pypi-all/simple \
    --output /report/pip-audit.json
DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='./_pip_audit.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Json: 'json'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=PosixPath('/cache'), progress_spinner=<ProgressSpinnerChoice.Off: 'off'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://custom-repo.com/api/pypi/pypi-all/simple', extra_index_urls=[], skip_editable=False, no_deps=False, output=<_io.TextIOWrapper name='/report/pip-audit.json' mode='w' encoding='UTF-8'>, ignore_vulns=[])
DEBUG:cachecontrol.controller:Looking up "https://custom-repo.com/api/pypi/pypi-all/simple/django" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): custom-repo.com:443
DEBUG:urllib3.connectionpool:https://custom-repo.com:443 "GET /api/pypi/pypi-all/simple/django HTTP/1.1" 302 0
DEBUG:cachecontrol.controller:Status code 302 not in (200, 203, 300, 301, 308)
DEBUG:cachecontrol.controller:Looking up "https://custom-repo.com:443/api/pypi/pypi-all/simple/django/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://custom-repo.com:443 "GET /api/pypi/pypi-all/simple/django/ HTTP/1.1" 200 None
DEBUG:cachecontrol.controller:Updating cache with response from "https://custom-repo.com:443/api/pypi/pypi-all/simple/django/"
Traceback (most recent call last):
  File "/usr/local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_cli.py", line 432, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 150, in query_all
    for spec in specs:
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 114, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 326, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
    yield (req, self.resolve(req))
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 75, in resolve
    result = self.resolver.resolve([req])
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 402, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 384, in get_dependencies
    return candidate.dependencies
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 121, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 103, in _get_dependencies
    deps: List[str] = self.metadata.get_all("Requires-Dist", [])
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in metadata
    self._metadata = self._get_metadata_for_wheel()
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 128, in _get_metadata_for_wheel
    data = self._session.get(self.url, timeout=self._timeout).content
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 484, in prepare_request
    p.prepare(
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 368, in prepare
    self.prepare_url(url, params)
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 439, in prepare_url
    raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL '../../packages/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl#sha256=6b1de6886cae14c7c44d188f580f8ba8da05750f544c80ae5ad43375ab293cd5': No scheme supplied. Perhaps you meant http://../../packages/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl#sha256=6b1de6886cae14c7c44d188f580f8ba8da05750f544c80ae5ad43375ab293cd5?

Platform information

Linux Centos 7, Docker python:3.9-alpine
pip-audit 2.4.5
Python 3.9.15
pip 22.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

Additional context

Running without the index-url doesn't work either

$ pip-audit -v \
    --format json \
    --progress-spinner off \
    --cache-dir /cache \
    --requirement _pip_audit.txt \
    --output /report/pip-audit.json
DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='_pip_audit.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Json: 'json'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=PosixPath('/cache'), progress_spinner=<ProgressSpinnerChoice.Off: 'off'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://pypi.org/simple', extra_index_urls=[], skip_editable=False, no_deps=False, output=<_io.TextIOWrapper name='/report/pip-audit.json' mode='w' encoding='UTF-8'>, ignore_vulns=[])
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/django HTTP/1.1" 301 112
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/django"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/django/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/django/ HTTP/1.1" 200 48676
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/django/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl HTTP/1.1" 200 8094697
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/4f/be/cd28514516e66c40b87e487be68cf61a1637bd3ec60a2db90bb5075a2df5/Django-4.1.3-py3-none-any.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/asgiref HTTP/1.1" 301 113
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/asgiref"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/asgiref/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/asgiref/ HTTP/1.1" 200 10369
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/asgiref/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/sqlparse" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/sqlparse HTTP/1.1" 301 114
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/sqlparse"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/sqlparse/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/sqlparse/ HTTP/1.1" 200 4081
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/sqlparse/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/af/6d/ea3a5c3027c3f14b0321cd4f7e594c776ebe64e4b927432ca6917512a4f7/asgiref-3.5.2-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/af/6d/ea3a5c3027c3f14b0321cd4f7e594c776ebe64e4b927432ca6917512a4f7/asgiref-3.5.2-py3-none-any.whl HTTP/1.1" 200 22881
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/af/6d/ea3a5c3027c3f14b0321cd4f7e594c776ebe64e4b927432ca6917512a4f7/asgiref-3.5.2-py3-none-any.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/97/d3/31dd2c3e48fc2060819f4acb0686248250a0f2326356306b38a42e059144/sqlparse-0.4.3-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/97/d3/31dd2c3e48fc2060819f4acb0686248250a0f2326356306b38a42e059144/sqlparse-0.4.3-py3-none-any.whl HTTP/1.1" 200 42768
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/97/d3/31dd2c3e48fc2060819f4acb0686248250a0f2326356306b38a42e059144/sqlparse-0.4.3-py3-none-any.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/django/4.1.3/json" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/django/4.1.3/json HTTP/1.1" 200 2287
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/django/4.1.3/json"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/asgiref/3.5.2/json" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/asgiref/3.5.2/json HTTP/1.1" 200 4383
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/asgiref/3.5.2/json"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/sqlparse/0.4.3/json" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/sqlparse/0.4.3/json HTTP/1.1" 200 2044
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/sqlparse/0.4.3/json"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dvm" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dvm HTTP/1.1" 301 109
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dvm"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dvm/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dvm/ HTTP/1.1" 200 463
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dvm/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/f7/fd/2b16390eae49db436cab7ce06b2f5b05a2e27527c4b6df867cb6dc277d92/dvm-1.0.0-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/f7/fd/2b16390eae49db436cab7ce06b2f5b05a2e27527c4b6df867cb6dc277d92/dvm-1.0.0-py3-none-any.whl HTTP/1.1" 200 18333
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/f7/fd/2b16390eae49db436cab7ce06b2f5b05a2e27527c4b6df867cb6dc277d92/dvm-1.0.0-py3-none-any.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/numpy HTTP/1.1" 301 111
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/numpy"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/numpy/ HTTP/1.1" 200 211789
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/numpy/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pymc3" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pymc3 HTTP/1.1" 301 111
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pymc3"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pymc3/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pymc3/ HTTP/1.1" 200 6667
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pymc3/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pandas HTTP/1.1" 301 112
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pandas"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pandas/ HTTP/1.1" 200 129114
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/pandas/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tqdm" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tqdm HTTP/1.1" 301 110
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tqdm"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tqdm/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tqdm/ HTTP/1.1" 200 26943
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tqdm/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow-probability" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow-probability HTTP/1.1" 301 128
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow-probability"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow-probability/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow-probability/ HTTP/1.1" 200 4916
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow-probability/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow HTTP/1.1" 301 116
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/tensorflow/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/tensorflow/ HTTP/1.1" 200 97476
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/tensorflow/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/seaborn" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/seaborn HTTP/1.1" 301 113
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/seaborn"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/seaborn/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/seaborn/ HTTP/1.1" 200 4702
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/seaborn/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/f1/55/14b80a052fc3aee2f8834fd5e4859299abdddd4aa82a92c18c3d869b1ff6/pymc3-3.11.5-py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/f1/55/14b80a052fc3aee2f8834fd5e4859299abdddd4aa82a92c18c3d869b1ff6/pymc3-3.11.5-py3-none-any.whl HTTP/1.1" 200 872225
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/f1/55/14b80a052fc3aee2f8834fd5e4859299abdddd4aa82a92c18c3d869b1ff6/pymc3-3.11.5-py3-none-any.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/arviz" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/arviz HTTP/1.1" 301 111
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/arviz"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/arviz/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/arviz/ HTTP/1.1" 200 5657
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/arviz/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cachetools" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachetools HTTP/1.1" 301 116
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cachetools"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cachetools/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachetools/ HTTP/1.1" 200 6943
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cachetools/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/deprecat" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/deprecat HTTP/1.1" 301 114
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/deprecat"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/deprecat/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/deprecat/ HTTP/1.1" 200 1768
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/deprecat/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dill" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dill HTTP/1.1" 301 110
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dill"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/dill/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/dill/ HTTP/1.1" 200 3943
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/dill/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/fastprogress" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/fastprogress HTTP/1.1" 301 118
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/fastprogress"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/fastprogress/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/fastprogress/ HTTP/1.1" 200 5148
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/fastprogress/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/numpy/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 23
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 23
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas" in the cache
DEBUG:cachecontrol.controller:Returning cached permanent redirect response (ignoring date and etag information)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/pandas/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 23
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 23
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/patsy" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/patsy HTTP/1.1" 301 111
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/patsy"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/patsy/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/patsy/ HTTP/1.1" 200 2061
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/patsy/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/scipy" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/scipy HTTP/1.1" 301 111
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/scipy"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/scipy/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/scipy/ HTTP/1.1" 200 114256
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/scipy/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/semver" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/semver HTTP/1.1" 301 112
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/semver"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/semver/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/semver/ HTTP/1.1" 200 7095
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/semver/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/theano-pymc" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/theano-pymc HTTP/1.1" 301 117
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/theano-pymc"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/theano-pymc/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/theano-pymc/ HTTP/1.1" 200 1513
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/theano-pymc/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/typing-extensions" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/typing-extensions HTTP/1.1" 301 123
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/typing-extensions"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/typing-extensions/" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/typing-extensions/ HTTP/1.1" 200 4800
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/typing-extensions/"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/0b/70/b84f9944a03964a88031ef6ac219b6c91e8ba2f373362329d8770ef36f02/semver-2.13.0-py2.py3-none-any.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/0b/70/b84f9944a03964a88031ef6ac219b6c91e8ba2f373362329d8770ef36f02/semver-2.13.0-py2.py3-none-any.whl HTTP/1.1" 200 12901
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/0b/70/b84f9944a03964a88031ef6ac219b6c91e8ba2f373362329d8770ef36f02/semver-2.13.0-py2.py3-none-any.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/01/26/ee0f0a4c2d18d6a7058c71e3cfed21b31a209979e7d8191dbc990c542a61/Theano-PyMC-1.1.2.tar.gz" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/01/26/ee0f0a4c2d18d6a7058c71e3cfed21b31a209979e7d8191dbc990c542a61/Theano-PyMC-1.1.2.tar.gz HTTP/1.1" 200 1810180
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/01/26/ee0f0a4c2d18d6a7058c71e3cfed21b31a209979e7d8191dbc990c542a61/Theano-PyMC-1.1.2.tar.gz"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 104, in post_setup
    run(package_install_cmd, state=self._state)
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_subprocess.py", line 51, in run
    raise CalledProcessError(f"{pretty_args} exited with {process.returncode}")
pip_audit._subprocess.CalledProcessError: python -m pip install /tmp/tmpft40uuu8/Theano-PyMC-1.1.2.tar.gz exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_cli.py", line 432, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 150, in query_all
    for spec in specs:
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 114, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 326, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
    yield (req, self.resolve(req))
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 75, in resolve
    result = self.resolver.resolve([req])
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 402, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/usr/local/lib/python3.9/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 384, in get_dependencies
    return candidate.dependencies
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 121, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 103, in _get_dependencies
    deps: List[str] = self.metadata.get_all("Requires-Dist", [])
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 96, in metadata
    self._metadata = self._get_metadata_for_sdist()
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 165, in _get_metadata_for_sdist
    ve.create(ve_dir)
  File "/usr/local/lib/python3.9/venv/__init__.py", line 78, in create
    self.post_setup(context)
  File "/usr/local/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 106, in post_setup
    raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpbniri7fx/bin/python', '-m', 'pip', 'install', '/tmp/tmpft40uuu8/Theano-PyMC-1.1.2.tar.gz']

bug component:dep-sources

opened by fi0rini 15

osv reports pyyaml needs upgrade when it is already upgraded
Hello, I can't tell is this is because osv is corrupt and y'all can't do anything about it or if pip-audit can't do version number comparisons. I'm still learning, but I can't find a way to skip this.

It only takes one unresolvable false positive and I have to disable a tool like this from the build server. safety has a --ignore switch, does pip-audit have one?

$ pip-audit -s osv Found 1 known vulnerabilities in 1 packages Name Version ID Fix Versions ------ ------- -------------- ------------ pyyaml 5.4.1 PYSEC-2021-142 5.4 $ pip freeze | grep PyY PyYAML==5.4.1
upstream
opened by matthewdeanmartin 15

json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 13246)

I just ran pip-audit-0.0.4 for the first time and got an error:

Traceback (most recent call last):
  File "/usr/pkg/lib/python3.9/site-packages/pip_audit/dependency_source/pip.py", line 64, in collect
    for (_, dist) in pip_api.installed_distributions(local=self._local).items():
  File "/usr/pkg/lib/python3.9/site-packages/pip_api/_installed_distributions.py", line 83, in installed_distributions
    return _new_installed_distributions(local)
  File "/usr/pkg/lib/python3.9/site-packages/pip_api/_installed_distributions.py", line 71, in _new_installed_distributions
    for raw_dist in json.loads(result):
  File "/usr/pkg/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/pkg/lib/python3.9/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 13246)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/pkg/bin/pip-audit-3.9", line 33, in <module>
    sys.exit(load_entry_point('pip-audit==0.0.4', 'console_scripts', 'pip-audit')())
  File "/usr/pkg/lib/python3.9/site-packages/pip_audit/cli.py", line 196, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/usr/pkg/lib/python3.9/site-packages/pip_audit/audit.py", line 60, in audit
    yield from self._service.query_all(specs)
  File "/usr/pkg/lib/python3.9/site-packages/pip_audit/service/interface.py", line 82, in query_all
    for spec in specs:
  File "/usr/pkg/lib/python3.9/site-packages/pip_audit/dependency_source/pip.py", line 72, in collect
    raise PipSourceError("failed to list installed distributions") from e
pip_audit.dependency_source.pip.PipSourceError: failed to list installed distributions

bug

opened by 0-wiz-0 15

Crash on `package @ git+...` dependencies
Bug description

Using package @ git+... dependencies crashes pip-audit with a traceback, when I'd expect it to output the usual report with those packages listed by name and skip-reason if unauditable.

Reproduction steps

# Some packages exist, and all is well shed == 0.10.5 # Others *don't* exist or can't be fetched, that's reported nicely this_might_exist_off_pypi == 0.0.1 # But if you have this awful kind of dep, you'll get a traceback! hypothesis @ git+https://github.com/HypothesisWorks/[email protected]#subdirectory=hypothesis-python

pip-audit --no-deps -r requirements.txt

Platform information

OS name and version: macOS Monterey

pip-audit version (pip-audit -V): pip-audit 2.4.4

Python version (python -V or python3 -V): Python 3.10.6

pip version (pip -V or pip3 -V): pip 22.3

bug component:dep-sources
opened by Zac-HD 14

Dependency resolution fails to install `cyclonedx-python-lib-0.11.1`

Bug description

If I audit a requirements file that transitively depends on cyclonedx-python-lib, I see a pip install failure.

Reproduction steps

Make a requirements.txt that looks like this (or any dependency that relies on cyclonedx-python-lib):

pip-audit==1.0.0

Run pip-audit -r requirements.txt
Observe the failure

Expected behavior

All dependencies are audited properly.

Screenshots and logs

(.ve) [email protected] test % pip-audit -r requirements.txt
- Installing package in isolated environment
Traceback (most recent call last):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 90, in post_setup
    subprocess.run(
  File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmppf9hai1y/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpykqq2afm/cyclonedx-python-lib-0.11.1/LICENSE']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/tetsuo/Workspace/test/.ve/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_cli.py", line 240, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_audit.py", line 60, in audit
    yield from self._service.query_all(specs)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 115, in query_all
    for spec in specs:
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 62, in collect
    for _, deps in self.resolver.resolve_all(iter(req_values)):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 67, in resolve_all
    yield (req, self.resolve(req))
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 41, in resolve
    result = self.resolver.resolve([req])
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 293, in get_dependencies
    return candidate.dependencies
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 112, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in _get_dependencies
    deps: List[str] = self.metadata.get_all("Requires-Dist", [])
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 87, in metadata
    self._metadata = self._get_metadata_for_sdist()
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 170, in _get_metadata_for_sdist
    ve.create(ve_dir)
  File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/venv/__init__.py", line 78, in create
    self.post_setup(context)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 97, in post_setup
    raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmppf9hai1y/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpykqq2afm/cyclonedx-python-lib-0.11.1/LICENSE']
(.ve) [email protected] test % vim requirements.txt
(.ve) [email protected] test % PIP_AUDIT_LOGLEVEL=debug pip-audit -r requirements.txt
DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], format=<OutputFormatChoice.Columns: 'columns'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=None, progress_spinner=<ProgressSpinnerChoice.On: 'on'>, timeout=15)
- Installing package in isolated environment
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-audit HTTP/1.1" 301 215
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-audit/ HTTP/1.1" 200 2843
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/16/b2/c8a8208030b89825cdc1a4f2edaabf2076ba38ab07e77213fc299cf74d99/pip_audit-1.0.0-py3-none-any.whl HTTP/1.1" 200 49177
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-api HTTP/1.1" 301 213
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip-api/ HTTP/1.1" 200 4058
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packaging HTTP/1.1" 301 215
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packaging/ HTTP/1.1" 200 7379
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress HTTP/1.1" 301 214
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress/ HTTP/1.1" 200 1044
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/resolvelib HTTP/1.1" 301 216
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/resolvelib/ HTTP/1.1" 200 3179
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/html5lib HTTP/1.1" 301 214
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/html5lib/ HTTP/1.1" 200 3097
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachecontrol HTTP/1.1" 301 218
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cachecontrol/ HTTP/1.1" 200 4358
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/lockfile HTTP/1.1" 301 214
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/lockfile/ HTTP/1.1" 200 990
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cyclonedx-python-lib HTTP/1.1" 301 226
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cyclonedx-python-lib/ HTTP/1.1" 200 4903
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/2a/68/d8412d1e0d70edf9791cbac5426dc859f4649afc22f2abbeb0d947cf70fd/progress-1.6.tar.gz HTTP/1.1" 200 7842
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip HTTP/1.1" 301 109
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip/ HTTP/1.1" 200 18015
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress HTTP/1.1" 301 214
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/progress/ HTTP/1.1" 200 1044
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools HTTP/1.1" 301 216
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools/ HTTP/1.1" 200 104565
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/e8/28/b6b91e2dd3dbc964db396650703b922d5f0047a32c7f1af5c3c6c662b5cb/pip_api-0.0.23-py3-none-any.whl HTTP/1.1" 200 107323
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip HTTP/1.1" 301 109
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/pip/ HTTP/1.1" 200 18015
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/6c/dd/a834df6482147d48e225a49515aabc28974ad5a4ca3215c18a882565b028/html5lib-1.1-py2.py3-none-any.whl HTTP/1.1" 200 112173
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/six HTTP/1.1" 301 109
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/six/ HTTP/1.1" 200 4429
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/webencodings HTTP/1.1" 301 218
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/webencodings/ HTTP/1.1" 200 891
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/d3/39/b7cd9ef1be03ac33e71f76837a23d59842b016e5159cf5aff30c0b340907/CacheControl-0.12.10-py2.py3-none-any.whl HTTP/1.1" 200 20297
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requests HTTP/1.1" 301 114
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requests/ HTTP/1.1" 200 18544
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/msgpack HTTP/1.1" 301 213
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/msgpack/ HTTP/1.1" 200 28426
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/c8/22/9460e311f340cb62d26a38c419b1381b8593b0bb6b5d1f056938b086d362/lockfile-0.12.2-py2.py3-none-any.whl HTTP/1.1" 200 13564
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/0c/51/51b9438d2813d8252cade62650347b38fbf692e761cc0511d8c81adfdd31/cyclonedx_python_lib-0.11.1-py3-none-any.whl HTTP/1.1" 200 127708
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packageurl-python HTTP/1.1" 301 223
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/packageurl-python/ HTTP/1.1" 200 3928
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requirements-parser HTTP/1.1" 301 225
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/requirements-parser/ HTTP/1.1" 200 822
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools HTTP/1.1" 301 216
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/setuptools/ HTTP/1.1" 200 104565
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/13/ea/5720270a0e984e5dd238ec8e75a8acccb042ac97bcb215be9e2cb339dc1f/cyclonedx-python-lib-0.11.1.tar.gz HTTP/1.1" 200 98048
Traceback (most recent call last):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 90, in post_setup
    subprocess.run(
  File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpapwlw08o/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpz7uf8d_w/cyclonedx-python-lib-0.11.1/LICENSE']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/tetsuo/Workspace/test/.ve/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_cli.py", line 240, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_audit.py", line 60, in audit
    yield from self._service.query_all(specs)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_service/interface.py", line 115, in query_all
    for spec in specs:
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/requirement.py", line 62, in collect
    for _, deps in self.resolver.resolve_all(iter(req_values)):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/interface.py", line 67, in resolve_all
    yield (req, self.resolve(req))
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 41, in resolve
    result = self.resolver.resolve([req])
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 293, in get_dependencies
    return candidate.dependencies
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 112, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in _get_dependencies
    deps: List[str] = self.metadata.get_all("Requires-Dist", [])
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 87, in metadata
    self._metadata = self._get_metadata_for_sdist()
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 170, in _get_metadata_for_sdist
    ve.create(ve_dir)
  File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/venv/__init__.py", line 78, in create
    self.post_setup(context)
  File "/Users/tetsuo/Workspace/test/.ve/lib/python3.9/site-packages/pip_audit/_virtual_env.py", line 97, in post_setup
    raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpapwlw08o/bin/python', '-m', 'pip', 'install', '-e', '/var/folders/mq/25w5gx6x0w7c5hq2l6tz8k_40000gn/T/tmpz7uf8d_w/cyclonedx-python-lib-0.11.1/LICENSE']

Platform information

OS name and version: macOS 12.0.1
pip-audit version (pip-audit -V): 1.0.0
Python version (python -V or python3 -V): 3.9.7
pip version (pip -V or pip3 -V): 21.3.1

bug component:dep-sources

opened by tetsuo-cpp 13

`pip` sometimes includes debug messages on `stdout`
This is an abbreviated copy of #115 and #116, since those have been filled up with additional debugging.

Summary: In rare cases (currently N=1), we fail to collect environmental dependencies from PipSource. PipSource uses pip-api internally, which boils down to pip list --format=json.

What seems to happen is that the pip list --format=json command doesn't emit just JSON. Instead, it also emits a trailing log message that looks like this:

Given no hashes to check 181 links for project 'pip': discarding no candidates

According to pip's source code, that message is produced by a call to log.debug, which should be going to stderr instead of stdout. pip-api only uses stdout from pip subprocesses, so this is a strong indicator that the two streams are being mixed and/or confused somewhere, potentially with a logging override. The original reporter also traced the process and confirmed that the log was produced on stdout.

The original reporter found this on Python 3.9, pip version 20.3.3. I was unable to reproduce it locally, and they were also unable to reliably reproduce it locally.
bug
opened by woodruffw 13

pip-audit not auditing the same package versions as pip installs.

Bug description

My understanding of pip-audit is that, when given a requirements.txt file, it would audit the same versions of the same packages that pip would install. This doesn't seem to be the case.

I have a requirements.txt file containing just the cryptography module with no version constraint specified.

If I create a venv and install the package then it installs:

cryptography 36.0.2
cffi 1.15.0 (as cryptography has a requirement for cffi>=1.1.2)
pcparser 2.21

If I run pip-audit -r requirement.txt then it tries to install cffi version 1.0.2-2 rather than 1.15.0 and fails with:

pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpffo5omkp/bin/python3', '-m', 'pip', 'install', '/tmp/tmp4dbeewpp/cffi-1.0.2-2.tar.gz']

If I change the requirements.txt file to contain cryptography==36.0.2, the same thing happens.

Likewise, if I append cffi==1.15.0 to the requirements.txt file pip-audit still tries, and fails, to use cffi 1.0.2-2

Reproduction steps

Running in Ubuntu 22.04 Docker container with just python3, venv, git installed. Running as a non-root user in the container I install pip-audit, either via pip, or directly from the git repo Create requirements.txt containing just cryptography Create a venv, install cryptography package from requirements.txt using pip Run pip list to confirm package versions installed as listed above (specifically cffi 1.15.0) Run pip-audit -r requirements.txt pip-audit tries to install cffi 1.0.2.2 and fails, as above

Expected behavior

pip-audit should audit the same packages and the same versions of the packages as pip install installs pip-audit should not fail

Screenshots and logs

Dockerfile used:

# Start with up-to-date Ubuntu

FROM ubuntu:22.04

# User to run with

ARG BUILD_USER=build
ARG BUILD_UID=1000
ARG BUILD_GID=1000
ARG HOME_DIR=/home/build

# Update & upgrade, install minimal Python setup

RUN apt update
RUN apt upgrade --yes
RUN apt install --yes apt-utils
RUN apt install --yes python3 python3-pip python3-venv git

# Add the user

RUN adduser --disabled-password --gecos '' ${BUILD_USER}
USER ${BUILD_UID}

WORKDIR ${HOME_DIR}

# Install pip-audit

#RUN pip install --upgrade --no-warn-script-location pip-audit
RUN python3 -m pip install git+https://github.com/trailofbits/pip-audit

Contents of requirements.txt file:

cryptography

Output from pip list in venv after installing cryptography package:

Package      Version
------------ -------
cffi         1.15.0
cryptography 36.0.2
pip          22.0.2
pycparser    2.21
setuptools   59.6.0

Output from pip-audit -v -r requirements.txt:

DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Columns: 'columns'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=None, progress_spinner=<ProgressSpinnerChoice.On: 'on'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://pypi.org/simple', extra_index_urls=[], skip_editable=False)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cryptography" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cryptography HTTP/1.1" 301 118
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cryptography"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cryptography/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 153
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 153
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl HTTP/1.1" 200 4716191
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cffi" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cffi HTTP/1.1" 301 110
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cffi"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cffi/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 153
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 153
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz HTTP/1.1" 200 317417
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
Traceback (most recent call last):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_virtual_env.py", line 103, in post_setup
    run(package_install_cmd, state=self._state)
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_subprocess.py", line 51, in run
    raise CalledProcessError(f"{pretty_args} exited with {process.returncode}")
pip_audit._subprocess.CalledProcessError: python3 -m pip install /tmp/tmps9lu0wkz/cffi-1.0.2-2.tar.gz exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/build/.local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_cli.py", line 357, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_service/interface.py", line 142, in query_all
    for spec in specs:
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 98, in collect
    for _, deps in self._resolver.resolve_all(iter(req_values)):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
    yield (req, self.resolve(req))
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 73, in resolve
    result = self.resolver.resolve([req])
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 341, in get_dependencies
    return candidate.dependencies
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 119, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 101, in _get_dependencies
    deps: List[str] = self.metadata.get_all("Requires-Dist", [])
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in metadata
    self._metadata = self._get_metadata_for_sdist()
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 162, in _get_metadata_for_sdist
    ve.create(ve_dir)
  File "/usr/lib/python3.10/venv/__init__.py", line 78, in create
    self.post_setup(context)
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_virtual_env.py", line 105, in post_setup
    raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpjk63y53b/bin/python3', '-m', 'pip', 'install', '/tmp/tmps9lu0wkz/cffi-1.0.2-2.tar.gz']

Platform information

OS name and version: Ubuntu 22.04 (in Docker container)
pip-audit version (pip-audit -V): pip-audit 2.1.0
Python version (python -V or python3 -V): Python 3.10.3
pip version (pip -V or pip3 -V): pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)

bug component:dep-sources upstream

opened by skilleter 12

Environmental audits: run `pip list` in the local virtual environment, if present
This is a follow-on to #450.

#451 "resolves" the issue by emitting a warning on the current ambiguity, but we can do better than that: instead of just warning, we can use ${VIRTUAL_ENV}/bin/python as pip_api's Python base, which in turn will allow globally installed copies of pip-audit to audit local virtual environments (as users might intuitively expect).

To do that, we'll need to re-think how we use pip_api, and possibly upstream some changes:

We can't rely on pip_api.PIP_VERSION, since it's computed eagerly from whatever pip was found at module import time. Maybe we should just use pip_api.version().

It might make sense to change how pip_api currently supports overrides -- PIPAPI_PYTHON_LOCATION in the environment could probably be replaced with something more ergonomic.

enhancement
opened by woodruffw 0
Post-mortem: regression in 2.4.9

We had a regression in 2.4.9, caused by a type error: #439.

In particular, we refactored a file-like type into a pathlib.Path, but forgot to update one of the use sites. This should have been caught by mypy, so I'm filing this post-mortem to figure out why it wasn't (and what we can do to fix that).

opened by woodruffw 5

Hash checking: respect third party indices

Is your feature request related to a problem? Please describe. Yes.

Currently if you are using a privately hosted repo and building your own wheels for various projects, you cannot use pip-audit -s pypi, while only listing the hashes of your privately hosted wheels. This leaves us with 2 options. Adding the upstream digest hash ( that we dont use ) to our hash list for each package. Or using -s osv.

Currently we are using osv. And though I am unsure if this is inferior. We would like to use the default where we can.

Describe the solution you'd like

Ideally - https://github.com/pypa/pip-audit/blob/76e4fa4a0ed005f543701d4b60c02c75c5b539d2/pip_audit/_service/pypi.py#L89

Would have logic have a flag around it to disable hash checking for situations like this.

Describe alternatives you've considered

As mentioned above using -s osv

Additional context

Example of current failure:

DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/adal/1.2.2/json HTTP/1.1" 200 1128
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/adal/1.2.2/json"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/aenum/3.1.0/json" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/aenum/3.1.0/json HTTP/1.1" 200 2235
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/aenum/3.1.0/json"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/pypi/aiobotocore/1.2.2/json" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /pypi/aiobotocore/1.2.2/json HTTP/1.1" 200 5390
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/pypi/aiobotocore/1.2.2/json"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
Traceback (most recent call last):
  File "/Users/carl.gill/.pyenv/versions/3.7.3/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_cli.py", line 439, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_audit.py", line 67, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_service/interface.py", line 156, in query_all
    yield self.query(spec)
  File "/Users/carl.gill/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip_audit/_service/pypi.py", line 101, in query
    f"Mismatched hash for {spec.canonical_name} ({spec.version}): listed "
pip_audit._service.interface.ServiceError: Mismatched hash for aiobotocore (1.2.2): listed 290833e77f3992cf947279106e43f97dc66f72731cf6818d89a171295bda79ba of type sha256 could not be found in PyPI releases```

bug question component:dep-sources

opened by carl-armis 12

=" in requirements.txt">

Cannot handle ">=" in requirements.txt

The vast vast majority of Python packages use >= in their requirements.txt not ==. But pip-audit is not flagging vulnerabilities when >= used.

E.g., take this example requirements.txt:

lxml>=4.5.1

lxml 4.5.1 contains a vulnerability but is not flagged by pip-audit. Only flagged if >= replaced with ==
discussion

opened by ValueRaider 7

`--requirement -` does not work: `No such file or directory: ''`

Bug description

When running pip-audit with --requirement - does not read the requirements from stdin like --requirement /dev/stdin does but instead raises an exception pip_requirements_parser.InstallationError: Could not open requirements file: <stdin>|n[Errno 2] No such file or directory: '<stdin>'.

Reproduction steps

echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement -

Expected behavior

I expect this to do the same as:

echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement /dev/stdin

i.e.

$ echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement /dev/stdin
⚠️  pip-audit is already on your PATH and installed at /home/iwana/.local/bin/pip-audit. Downloading and running anyway.
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
WARNING:cachecontrol.controller:Cache entry deserialization failed, entry ignored
Found 1 known vulnerability in 1 package
Name Version ID                  Fix Versions
---- ------- ------------------- ------------
cleo 1.0.0a5 GHSA-2p9h-ccw7-33gf

Screenshots and logs

instead this is what happens:

$ echo 'poetry' | pipx run --spec=pip-audit==2.4.6 pip-audit --requirement -
⚠️  pip-audit is already on your PATH and installed at /home/iwana/.local/bin/pip-audit. Downloading and running anyway.
Traceback (most recent call last):
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1577, in get_file_content
    with open(filename, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '<stdin>'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_cli.py", line 434, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_service/interface.py", line 155, in query_all
    for spec in specs:
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 82, in collect
    rf = RequirementsFile.from_file(filename)
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 233, in from_file
    for parsed in cls.parse(
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 277, in parse
    for parsed in parse_requirements(
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1152, in parse_requirements
    for parsed_line in parser.parse(
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1329, in parse
    yield from self._parse_and_recurse(
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1351, in _parse_and_recurse
    for line in self._parse_file(filename=filename, is_constraint=is_constraint):
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1404, in _parse_file
    content = get_file_content(filename)
  File "/home/iwana/.local/pipx/.cache/5da69b18b537a22/lib64/python3.10/site-packages/pip_requirements_parser.py", line 1580, in get_file_content
    raise InstallationError(
pip_requirements_parser.InstallationError: Could not open requirements file: <stdin>|n[Errno 2] No such file or directory: '<stdin>'

Platform information

OS name and version: Fedora Linux 36 (Xfce)
pip-audit version (pip-audit -V): 2.4.6
Python version (python -V or python3 -V): 3.10.8
pip version (pip -V or pip3 -V): Using pipx, so none.

Additional context

I'm trying to use pip-audit with poetry, and to do this I run:

poetry export --without-hashes --with dev --format requirements.txt | \
  poetry run python -m pip_audit --requirement /dev/stdin --no-deps --strict --desc on

However this will probably not work great on windows, and I would prefer a less platform dependent way of specifying stdin, like --requirement -.

enhancement upstream

opened by aucampia 2

Add support for Poetry

If the project path is specified in CLI, read dependencies from poetry.lock file (if available).

I'm quite happy with collect. Since this is a lock file, we can safely skip dependency resolution, as we do for pip source. And the format of the file is quite simple, so we don't depend on poetry for reading it. Cool!

Autofix is not supported yet because poetry needs hashes to be updated, see review comments.

I checked the change manually on a few quite big poetry projects, works like a charm.

Close #84

opened by orsinium 2

Releases(v2.4.12)

v2.4.12(Dec 29, 2022)
Fixed

Fixed pip-audit's virtual environment creation and upgrade behavior, preventing spurious vulnerability reports (#454)

Users are now warned if a pip-audit invocation is ambiguous, e.g. if they've installed pip-audit globally but are asking for an audit of a loaded virtual environment (#451)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.12-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.12-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.12.tar.gz.crt(1.30 KB)
pip_audit-2.4.12.tar.gz.sig(141 bytes)
v2.4.11(Dec 28, 2022)
Fixed

Fixed a crash triggered when a package specifies an invalid version specifier for its requires-python version (#447)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.11-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.11-py3-none-any.whl.sig(137 bytes)
pip_audit-2.4.11.tar.gz.crt(1.30 KB)
pip_audit-2.4.11.tar.gz.sig(137 bytes)
v2.4.10(Dec 15, 2022)
[2.4.10]

Fixed

Fixed a crash triggered when no vulnerabilities are found with some configurations (#437)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.10-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.10-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.10.tar.gz.crt(1.30 KB)
pip_audit-2.4.10.tar.gz.sig(141 bytes)
v2.4.9(Dec 14, 2022)
[2.4.9]

Fixed

The --output flag will no longer produce an empty file in the event of a failure within pip-audit itself, making it easier to distinguish between audit failures being reported by pip-audit and pip-audit's own errors (#432)

Removed pin on packaging now that our dependency pins it for us (#429)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.9-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.9-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.9.tar.gz.crt(1.30 KB)
pip_audit-2.4.9.tar.gz.sig(137 bytes)
v2.4.8(Dec 8, 2022)
[2.4.8]

Fixed

Pin maximum version of packaging dependency to avoid installing the new 22.0 version which is incompatible with pip-requirements-parser (#427)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.8-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.8-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.8.tar.gz.crt(1.30 KB)
pip_audit-2.4.8.tar.gz.sig(141 bytes)
v2.4.7(Nov 28, 2022)
Fixed

Fixed a timestamp parsing bug that occurred with some vulnerability reports provided by the OSV service (#416)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.7-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.7-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.7.tar.gz.crt(1.30 KB)
pip_audit-2.4.7.tar.gz.sig(141 bytes)
v2.4.6(Nov 21, 2022)
Fixed

Fixed an incorrect interaction between --desc=auto and --format=json; --desc=auto now includes the description in the generated JSON report, as intended (#399)

Fixed a bug in dependency resolution with third-party indices where relative URLs were not resolved correctly (#411, #412)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.6-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.6-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.6.tar.gz.crt(1.30 KB)
pip_audit-2.4.6.tar.gz.sig(141 bytes)
v2.4.5(Oct 31, 2022)
Fixed

Fixed an issue where audits done with the PyPI vulnerability service (the default) were not correctly filtered by "withdrawn" status; "withdrawn" vulnerabilities are now excluded (#393)

Fixed an issue where audits done with the OSV vulnerability service (-s osv) were not correctly filtered by "withdrawn" status; "withdrawn" vulnerabilities are now excluded (#386)

Fixed pip-audit's handling of URL-style requirements in --no-deps mode (URL requirements are now treated as skipped, rather than producing an error due to a lack of pinning) (#395)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.5-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.5-py3-none-any.whl.sig(137 bytes)
pip_audit-2.4.5.tar.gz.crt(1.30 KB)
pip_audit-2.4.5.tar.gz.sig(141 bytes)
v2.4.4(Sep 1, 2022)
Changed

pip-audit is now a PyPA member project, and lives under pypa/pip-audit!

Improved error message for when unpinned URL requirements are found during an audit with the --no-deps flag (#355)

Fixed

Fixed an issue where packages on PyPI with no published versions trigger a dependency resolution failure instead of being skipped (#357)

Fixed an incorrect assertion triggering for non-editable URL requirements that don't have an egg fragment (#359)

Source code(tar.gz)
Source code(zip)
pip_audit-2.4.4-py3-none-any.whl.crt(1.30 KB)
pip_audit-2.4.4-py3-none-any.whl.sig(141 bytes)
pip_audit-2.4.4.tar.gz.crt(1.30 KB)
pip_audit-2.4.4.tar.gz.sig(141 bytes)
v2.4.3(Jul 25, 2022)
Fixed

Fixed a regression in requirements auditing that was introduced during the move from pip-api to pip-requirements-parser where editable installs without an egg fragment would cause audits to crash (https://github.com/trailofbits/pip-audit/pull/331)

Source code(tar.gz)
Source code(zip)
v2.4.2(Jul 21, 2022)
Fixed

CLI: the --format=markdown and --format=columns output formats are no longer broken by long vulnerability descriptions from the OSV and PyPI vulnerability sources (https://github.com/trailofbits/pip-audit/pull/323)

Source code(tar.gz)
Source code(zip)
v2.4.1(Jul 7, 2022)
[2.4.1]

Fixed

Fixed a breakage in hash-checking mode caused by a change to the PyPI JSON API (#318)

Source code(tar.gz)
Source code(zip)
v2.4.0(Jun 30, 2022)
[2.4.0]

Added

Output formats: pip-audit now supports a Markdown format (--format=markdown) which renders results as a set of Markdown tables. (#312)

Source code(tar.gz)
Source code(zip)
v2.3.4(Jun 24, 2022)

Source code(tar.gz)
Source code(zip)
v2.3.3(Jun 15, 2022)
Changed

CLI: pip-audit now warns on the combination of -s osv and --require-hashes, notifying users that only the PyPI service can fully verify hashes (#298)

Fixed

CLI/Dependency sources: --cache-dir=... and other flags that affect dependency resolver behavior now work correctly when auditing a pyproject.toml dependency source (#300)

Source code(tar.gz)
Source code(zip)
v2.3.2(Jun 14, 2022)
Changed

CLI: pip-audit's progress spinner has been refactored to make it faster and more responsive (#283)

CLI, Vulnerability sources: the error message used to report connection failures to vulnerability sources was improved (#287)

Vulnerability sources: the OSV service is now more resilient to schema changes (#288)

Vulnerability sources: the PyPI service provides a better error message during some cases of service degradation (#294)

Fixed

Vulnerability sources: a bug stemming from an incorrect assumption about OSV's schema guarantees was fixed (#284)

Caching: pip-audit now respects pip's PIP_NO_CACHE_DIR and will not attempt to use the pip cache if present (#290)

Source code(tar.gz)
Source code(zip)
v2.3.1(May 24, 2022)
2.3.1 - 2022-05-24

Fixed

CLI: A bug causing the terminal's cursor to disappear on some versions of CPython was fixed (https://github.com/trailofbits/pip-audit/issues/280)

Source code(tar.gz)
Source code(zip)
v2.3.0(May 18, 2022)

2.3.0 - 2022-05-18

Added

CLI: The --ignore-vuln option has been added, allowing users to specify vulnerability IDs to ignore during the final report (https://github.com/trailofbits/pip-audit/pull/275)

CLI: The --no-deps flag has been added, allowing users to skip dependency resolution entirely when pip-audit is used in requirements mode (https://github.com/trailofbits/pip-audit/pull/255)
Source code(tar.gz)
Source code(zip)
v2.2.1(May 2, 2022)

Source code(tar.gz)
Source code(zip)
v2.2.0(May 2, 2022)
2.2.0 - 2022-05-02

Added

CLI: The --output option has been added, allowing users to specify a file to write output to. The default behavior of writing to stdout is unchanged (#262)

Fixed

Vulnerability sources: A bug caused by insufficient version normalization was fixed (#263)

Source code(tar.gz)
Source code(zip)
v2.1.1(Mar 29, 2022)
2.1.1 - 2022-03-29

Fixed

Dependency sources: A bug caused by ambiguous parses of source distribution files was fixed (#249)

Source code(tar.gz)
Source code(zip)
v2.1.0(Mar 11, 2022)
2.1.0 - 2022-03-11

Added

CLI: The --skip-editable flag has been added, allowing users to skip local packages or parsed requirements (via -r) that are marked as editable (https://github.com/trailofbits/pip-audit/pull/244)

CLI: pip-audit can audit projects that list their dependencies in pyproject.toml files, via pip-audit <dir> (https://github.com/trailofbits/pip-audit/pull/246)

Source code(tar.gz)
Source code(zip)
v2.0.0(Feb 18, 2022)
2.0.0 - 2022-02-18

Added

CLI: The --fix flag has been added, allowing users to attempt to automatically upgrade any vulnerable dependencies to the first safe version available (#212, #222)

CLI: The combination of --fix and --dry-run is now supported, causing pip-audit to perform the auditing step but not any resulting fix steps (#223)

CLI: The --require-hashes flag has been added which can be used in conjunction with -r to check that all requirements in the file have an associated hash (#229)

CLI: The --index-url flag has been added, allowing users to use custom package indices when running with the -r flag (#238)

CLI: The --extra-index-url flag has been added, allowing users to use multiple package indices when running with the -r flag (#238 )

Changed

pip-audit's minimum Python version is now 3.7.

CLI: The default output format is now correctly pluralized (#221)

Output formats: The SBOM output formats (--format=cyclonedx-xml and --format=cyclonedx-json) now use CycloneDX Schema 1.4 (#216)

Vulnerability sources: When using PyPI as a vulnerability service, any hashes provided in a requirements file are checked against those reported by PyPI (#229)

Vulnerability sources: pip-audit now uniques each result based on its alias set, reducing the amount of duplicate information in the default columnar output format (#232)

CLI: pip-audit now prints its output more frequently, including when there are no discovered vulnerabilities but packages were skipped. Similarly, "manifest" output formats (JSON, CycloneDX) are now emitted unconditionally (#240 )

Fixed

CLI: A regression causing excess output during pip audit -r was fixed (#226)

Source code(tar.gz)
Source code(zip)
v1.1.2(Jan 13, 2022)

Source code(tar.gz)
Source code(zip)
v1.1.1(Dec 7, 2021)

Source code(tar.gz)
Source code(zip)
v1.1.0(Dec 6, 2021)

Source code(tar.gz)
Source code(zip)
v1.0.1(Dec 2, 2021)

Source code(tar.gz)
Source code(zip)
v1.0.0(Dec 1, 2021)

Source code(tar.gz)
Source code(zip)
v0.0.9(Dec 1, 2021)

Source code(tar.gz)
Source code(zip)
v0.0.8(Nov 29, 2021)

Source code(tar.gz)
Source code(zip)

Audits Python environments and dependency trees for known vulnerabilities

Related tags

Overview

pip-audit

Development steps

Release process

Comments

Bug description

Reproduction steps

Screenshots and logs

Platform information

Bug description

Reproduction steps

Expected behavior

Screenshots and logs

Platform information

Bug description

Reproduction steps

Expected behavior

Screenshots and logs

Platform information

Additional context

Bug description

Reproduction steps

Platform information

Bug description

Reproduction steps

Expected behavior

Screenshots and logs

Platform information

Bug description

Reproduction steps

Expected behavior

Screenshots and logs

Platform information

Bug description

Reproduction steps

Expected behavior

Screenshots and logs

Platform information

Additional context

Releases(v2.4.12)

v2.4.12(Dec 29, 2022)

Fixed

v2.4.11(Dec 28, 2022)

Fixed

v2.4.10(Dec 15, 2022)

[2.4.10]

Fixed

v2.4.9(Dec 14, 2022)

[2.4.9]

Fixed

v2.4.8(Dec 8, 2022)

[2.4.8]

Fixed

v2.4.7(Nov 28, 2022)

Fixed

v2.4.6(Nov 21, 2022)

Fixed

v2.4.5(Oct 31, 2022)

Fixed

v2.4.4(Sep 1, 2022)

Changed

Fixed

v2.4.3(Jul 25, 2022)

Fixed

v2.4.2(Jul 21, 2022)

Fixed

v2.4.1(Jul 7, 2022)

[2.4.1]

Fixed

v2.4.0(Jun 30, 2022)

[2.4.0]

Added

v2.3.4(Jun 24, 2022)

v2.3.3(Jun 15, 2022)

Changed

Fixed

v2.3.2(Jun 14, 2022)

Changed