This is in reference to Issue #220

I've implemented a pipxrc file in each venv directory that contains the following:

• package_or_url - keeping track of the origin of the venv package
• venv_metadata - keeping track of venv package metadata so uninstall is possible even if the python pointer in the venv bin/ directory is no longer valid
• injected_packages - keeping track of all injected packages, allowing reinstall to re-inject them all

I realize that I don't have this in a proper named branch, and the commit log is a bit messy and winding (I spent time figuring out the best way to implement things.) I'm happy to fix all of the above, but I thought it best to submit the code I have to allow for discussion.

Having a pipxrc really helps a lot! I modified install, uninstall, upgrade, reinstall-all, upgrade-all: all of these commands behave much better, especially with URL-based packages. In this code I also modified reinstall-all to re-inject all injected packages, which fixes Issue #63.

I also tested reinstall-all with broken symlinks to python inside of the venv, and it works beautifully. This would allow this pull request to fix Issue #146 by allowing reinstall-all to update all the venvs after a system python upgrade. Note: I would recommend that reinstall-all use DEFAULT_PYTHON as the default for the python argument instead of requiring that argument.

Some outstanding issues:

• I uncovered a bug by looking at the code (#221). I believe pipxrc can help with this, because it remembers both the injected packages and their inject flags. But I have not addressed this Issue with this pull request
• I believe that upgrade can drop the '--spec' option now that each venv remembers its own package_or_url, but I didn't code this.
• I did not handle an exception if pipx is unable to write pipxrc. I was unsure if this was necessary
• I am not sure exactly whether _run_post_install_actions() affects the venv metadata. If it does, then pipxrc venv_metadata should be updated afterwards. (This is not done in my current pull request)

Describe the bug

pipx fails to install a local package.

How to reproduce

pipx install .
Cannot determine package name from spec '.'. Check package spec for errors.

pipx --version
0.15.1.3

Expected behavior

The package should be installed.

Fixes #267

This PR allows all arguments after pip run <app_name> to be sent verbatim to the <app_name> app. Currently a -- argument is possible to be removed by argparse before sending the list of arguments to the app.

The code included in a new parse_args() function doubles a -- argument if it will be removed by argparse. argparse will only remove the first -- in a line, so this ensures that the appargs specified by the user will all be sent to the app.

I ran

$ pipx install poetry

just yesterday, and everything worked fine. Then I ran

$ brew upgrade

which upgraded my system Python from 3.7.2 to 3.7.3. Now pipx no longer works:

% poetry --version
zsh: /Users/raxod502/.local/bin/poetry: bad interpreter: /Users/raxod502/.local/pipx/venvs/poetry/bin/python: no such file or directory

% pipx run poetry --version
⚠️  poetry is already on your PATH and installed at /Users/raxod502/.local/bin/poetry. Downloading and running anyway.
Traceback (most recent call last):
  File "/Users/raxod502/.local/bin/pipx", line 10, in <module>
    sys.exit(cli())
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 525, in cli
    exit(run_pipx_command(parsed_pipx_args, binary_args))
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 141, in run_pipx_command
    use_cache,
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 101, in run
    retval = venv.run_binary(binary, binary_args)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/Venv.py", line 106, in run_binary
    return _run(cmd, check=False)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/Venv.py", line 127, in _run
    returncode = subprocess.run(cmd_str_list).returncode
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/.cache/d7fe01e92227b36/bin/poetry': '/Users/raxod502/.local/pipx/.cache/d7fe01e92227b36/bin/poetry'

% pipx upgrade poetry
Traceback (most recent call last):
  File "/Users/raxod502/.local/bin/pipx", line 10, in <module>
    sys.exit(cli())
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 525, in cli
    exit(run_pipx_command(parsed_pipx_args, binary_args))
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 184, in run_pipx_command
    include_deps=args.include_deps,
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 213, in upgrade
    old_version = venv.get_venv_metadata_for_package(package).package_version
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/Venv.py", line 74, in get_venv_metadata_for_package
    stdout=subprocess.PIPE,
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/venvs/poetry/bin/python': '/Users/raxod502/.local/pipx/venvs/poetry/bin/python'

% pipx list
venvs are in /Users/raxod502/.local/pipx/venvs
binaries are exposed on your $PATH at /Users/raxod502/.local/bin
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 538, in _get_package_summary
    metadata = venv.get_venv_metadata_for_package(package)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/Venv.py", line 74, in get_venv_metadata_for_package
    stdout=subprocess.PIPE,
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/venvs/poetry/bin/python': '/Users/raxod502/.local/pipx/venvs/poetry/bin/python'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/raxod502/.local/bin/pipx", line 10, in <module>
    sys.exit(cli())
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 525, in cli
    exit(run_pipx_command(parsed_pipx_args, binary_args))
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 187, in run_pipx_command
    commands.list_packages(PIPX_LOCAL_VENVS)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 599, in list_packages
    for package_summary in p.map(_get_package_summary, dirs):
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
FileNotFoundError: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/venvs/poetry/bin/python': '/Users/raxod502/.local/pipx/venvs/poetry/bin/python'

% pipx install poetry
'poetry' already seems to be installed. Not modifying existing installation in '/Users/raxod502/.local/pipx/venvs/poetry'. Pass '--force' to force installation

% pipx install poetry --force
Installing to existing directory '/Users/raxod502/.local/pipx/venvs/poetry'
Error: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/venvs/poetry/bin/python3.7': '/Users/raxod502/.local/pipx/venvs/poetry/bin/python3.7'
'/usr/local/opt/python/bin/python3.7 -m venv /Users/raxod502/.local/pipx/venvs/poetry' failed

% pipx reinstall-all python3
Traceback (most recent call last):
  File "/Users/raxod502/.local/bin/pipx", line 10, in <module>
    sys.exit(cli())
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 516, in cli
    exit(run_pipx_command(parsed_pipx_args, binary_args))
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 209, in run_pipx_command
    skip=args.skip,
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 467, in reinstall_all
    uninstall(venv_dir, package, local_bin_dir, verbose)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 426, in uninstall
    metadata = venv.get_venv_metadata_for_package(package)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/Venv.py", line 85, in get_venv_metadata_for_package
    stdout=subprocess.PIPE,
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/venvs/poetry/bin/python': '/Users/raxod502/.local/pipx/venvs/poetry/bin/python'

% pipx uninstall-all
Traceback (most recent call last):
  File "/Users/raxod502/.local/bin/pipx", line 10, in <module>
    sys.exit(cli())
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 516, in cli
    exit(run_pipx_command(parsed_pipx_args, binary_args))
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/main.py", line 191, in run_pipx_command
    commands.uninstall_all(PIPX_LOCAL_VENVS, LOCAL_BIN_DIR, verbose)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 449, in uninstall_all
    uninstall(venv_dir, package, local_bin_dir, verbose)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/commands.py", line 426, in uninstall
    metadata = venv.get_venv_metadata_for_package(package)
  File "/Users/raxod502/.local/lib/python/site-packages/pipx/Venv.py", line 85, in get_venv_metadata_for_package
    stdout=subprocess.PIPE,
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/raxod502/.local/pipx/venvs/poetry/bin/python': '/Users/raxod502/.local/pipx/venvs/poetry/bin/python'

I assume the problem is that when pipx creates the virtual environments, it resolves /usr/local/bin/python3 as a symbolic link, which is problematic since the destination of that symlink changes every time Homebrew upgrades Python (thus making the old virtualenv's Python executable no longer work). I have experienced this problem with Pip as well, but presumably pipx could wrap this behavior somehow to make it work, or at least display a useful error message. At the least, it would be nice to have pipx reinstall-all python3 work properly.

Workaround is something like the following:

% mv ~/.local/pipx /tmp/pipx
% ls /tmp/pipx/venvs | xargs -L 1 pipx install

and manually prune any broken symlinks from ~/.local/bin.

Describe the bug

I have installed a new Debian (unstable) system. On one of my installations pipx runs without problems. The newly installed system behaves differently and I do not know where to look.

How to reproduce

$ pipx install --verbose youtube-dl
pipx > (_package_name_from_spec:93): Determined package name: youtube-dl
pipx > (_package_name_from_spec:94): Package name determined in 0.0s
pipx > (run_subprocess:112): running /usr/bin/python3 -m venv --without-pip /home/kai/.local/pipx/venvs/youtube-dl
pipx > (run_subprocess:112): running /home/kai/.local/pipx/venvs/youtube-dl/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
pipx > (run_subprocess:112): running /home/kai/.local/pipx/shared/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
pipx > (run_subprocess:112): running /home/kai/.local/pipx/venvs/youtube-dl/bin/python --version
pipx > (run_subprocess:112): running /home/kai/.local/pipx/venvs/youtube-dl/bin/python -m pip install youtube-dl
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/kai/.local/pipx/shared/lib/python3.8/site-packages/pip/__main__.py", line 16, in <module>
    from pip._internal.cli.main import main as _main  # isort:skip # noqa
  File "/home/kai/.local/pipx/shared/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/home/kai/.local/pipx/shared/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/home/kai/.local/pipx/shared/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
    from pip._internal.cli import cmdoptions
  File "/home/kai/.local/pipx/shared/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 24, in <module>
    from pip._internal.exceptions import CommandError
  File "/home/kai/.local/pipx/shared/lib/python3.8/site-packages/pip/_internal/exceptions.py", line 10, in <module>
    from pip._vendor.six import iteritems
ModuleNotFoundError: No module named 'pip._vendor.six'
pipx > (install_package:189): '/home/kai/.local/pipx/venvs/youtube-dl/bin/python -m pip install youtube-dl' failed

pipx > (rmdir:18): removing directory /home/kai/.local/pipx/venvs/youtube-dl
Error installing youtube-dl.

If I run python3 from the command line I can import the module:

$ python3 -c "import pip._vendor.six"
$

I have no idea what could be different. I already checked the list of python3-* packages on both systems but that seems to be the same. Any idea what to look for or how to debug the issue?

Expected behavior

The package should be installed.

Fixes #113.

This keeps sys.executable as the default, but will use the PIPX_DEFAULT_PYTHON environment variable, if defined. This mean no change in functionality for any users until they define the variable.

I'm suggesting the var PIPX_DEFAULT_PYTHON (no PYENV even though the original issue asked for pyenv support), because this approach will work with any python manager, and I think it might be confusing to enshrine one manager in the var name over others.

Also, it didn't look like the constants are tested, so I didn't update any tests. But let me know if I missed something and I'll be happy to add.

How would this feature be useful?

Hopefully this will synthesize (and possibly supersede) Issues #390 and #109. It addresses the reason for PRs #572 and #392.

This Issue is to track exactly what users are looking for with different pipx list output formats.

"Import / Export" set of pipx packages case

In #109, it appears that one primary motivator is the ability to export a file that specifies a set of pipx-installable packages that pipx can then install (possibly on another machine) in the future.

How many users desire this or would find it useful?

It would probably best be done by making a file in json format.

Within this there are some further points for those desiring this functionality:

Do people consider it desired (or necessary?) to freeze all versions of all dependencies for this to be useful? If so then this might be very difficult, based on my investigation creating test code for this.

Simply re-creating a pipx environment, and pinning the version of the main package and any injected packages, and any pipx options is very doable. I have a branch of doing this basically working. In this case, the output would be equivalent to what you would end up with on one machine executing pipx reinstall-all. You can be sure the main package and any injected packages will stay with their specified versions, but you cannot be sure that the versions of any dependencies that still satisfy all package requirements will be the same.

Archive / Documentation case

In #390 it appears that one motivation is simply a concise version of pipx list for "keeping track" or for archival or documentation purposes.

How many users desire this or would find it useful?

Is this type of list output (that is merely more concise) still desired if the "Import / Export" case is available?

What are the goals of this format? What are the use cases specifically? How would one know if the output of this was "successful"?

I must confess that I need this one explained to me, it doesn't naturally occur to me what is desired. If all that's needed is documentation of what is currently installed, then the current pipx list seems to cover that. If what is desired is a structured output that would allow one to recreate a pipx environment, then the "Import / Export" json scenario seems to satisfy the need.

@comkieffer @gvoysey @sanketdg @zachvalenta

Describe the solution you'd like

Describe alternatives you've considered

Working on #339, I discovered this use of the deprecated pkg_resources module. This change adds a dependency on importlib_metadata for python 3.7 and earlier and uses the stdlib version on Python 3.8 and later.

I haven't tested this behavior, so I could use some help validating that it does what's expected.

This might already be possible, I couldn't find out how however.

How would this feature be useful? Currently when installing a package all of it's included apps (dependencies?) are exposed to $PATH, even though when I for instance want to install organize using pipx install organize-tool I really only want the organize binary to be exposed. I assume (with my limited knowledge of python packages) that these apps are actually required for the main app but could also be contained to the venv and don't need to be exposed to $PATH.

What is actually installed is the following:

   package organize-tool 1.9.1, Python 3.8.5
    - EXIF.py
    - activate-global-python-argcomplete
    - docx2txt
    - dumppdf.py
    - latin2ascii.py
    - organize
    - pdf2txt.py
    - python-argcomplete-check-easy-install-script
    - python-argcomplete-tcsh
    - register-python-argcomplete
    - textract

I looked for a way to do this and thought I might be able to do so by editing the pipx_metadata.json but unfortunately that doesn't seem to be the case.

Describe the solution you'd like Given that it's probably not practical to determine which of the apps included is the main one it'd probably be better to make this configurable by the user themselves in for instance the pipx_metadata.json

It's not clear from the docs if pipx supports Windows, but given that it relies on symlinks, I assume not? (Symlinks typically require extra privileges on Windows, that the average user doesn't have). Are there plans to support Windows?

Describe the bug pkg_resources, which is part of setuptools, is used to parse the pipx version number and is also used extensively in the venv_metadata_inspector. setuptools should not be assumed to be available in the runtime environment unless explicitly required.

How to reproduce Install pipx in a virtual environment without setuptools:

$ python -m venv .venv
$ source .venv/bin/activate.fish
$ python -m pip uninstall setuptools
Uninstalling setuptools-41.2.0:
  Would remove:
    .venv/bin/easy_install
    .venv/bin/easy_install-3.8
    .venv/lib/python3.8/site-packages/easy_install.py
    .venv/lib/python3.8/site-packages/pkg_resources/*
    .venv/lib/python3.8/site-packages/setuptools-41.2.0.dist-info/*
    .venv/lib/python3.8/site-packages/setuptools/*
Proceed (y/n)? y
  Successfully uninstalled setuptools-41.2.0
$ python -m pip list
Package Version
------- -------
pip     19.2.3
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ python -m pip install pipx
Collecting pipx
  Downloading https://files.pythonhosted.org/packages/27/75/7a5762963703df84edc13c16ee6aaa3bcf559887a24c38d59d5e8a900ffa/pipx-0.15.1.2-py3-none-any.whl (43kB)
     |████████████████████████████████| 51kB 522kB/s
Collecting argcomplete<2.0,>=1.9.4 (from pipx)
  Downloading https://files.pythonhosted.org/packages/82/7d/455e149c28c320044cb763c23af375bd77d52baca041f611f5c2b4865cf4/argcomplete-1.11.1-py2.py3-none-any.whl
Collecting userpath (from pipx)
  Downloading https://files.pythonhosted.org/packages/8a/72/07927efb668a0aa0cef502dbe4da442ac9598f19344bca9a3eb9bd062ec1/userpath-1.3.0-py2.py3-none-any.whl
Collecting click (from userpath->pipx)
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Installing collected packages: argcomplete, click, userpath, pipx
Successfully installed argcomplete-1.11.1 click-7.0 pipx-0.15.1.2 userpath-1.3.0
WARNING: You are using pip version 19.2.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ pipx --version
Traceback (most recent call last):
  File ".venv/bin/pipx", line 6, in <module>
    from pipx.main import cli
  File ".venv/lib/python3.8/site-packages/pipx/main.py", line 9, in <module>
    from pkg_resources import parse_version
ModuleNotFoundError: No module named 'pkg_resources'

Expected behavior As above.

Describe the bug

This works correctly

pipx install "mycli[extras]"

but this doesn't

pipx install "mycli[extras]"
pipx upgrade "mycli[extras]"

and it fails with

Package is not installed. Expected to find /path/to/pipx/venvs/mycli[isolated], but it does not exist.

How to reproduce For reproduction, I suggest

pipx install "coverage[toml]"
pipx upgrade "coverage[toml]"

Expected behavior I'm not 100% sure what should happen actually, but the current error message seems broken (of course a directory with [ and ] in its name wasn't found). I think the upgrade should go through wether or not the extra targets match the ones I used (or didn't use) the first time I installed the CLI (here coverage).

I'd be happy to contribute a patch, but I'd appreciate some guidance regarding what the expected behaviour should be.

Bumps pypa/gh-action-pypi-publish from 1.5.2 to 1.6.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

How would this feature be useful?

I have a library package, and an app that uses the library package. Both are local-only, ie. I install them with pip install /path/to/library and pip install /path/to/app

I want to install app with pipx, but if I run pipx install /path/to/app it fails because library is not available.

I can't see how I can install library into the app venv before installing app.

Describe the solution you'd like

One possible workflow might be:

• create an empty venv with a specified name
• inject any dependencies into the venv
• inject the app into the existing venv, using --include-apps to create symlink(s) for the app into $HOME/.local/bin

In my use-case:

• pipx init foo
• pipx inject foo /path/to/library
• pipx inject --include-apps foo /path/to/app

So, what I need is the some command to create an empty venv into which I can inject dependencies and apps (the pipx init foo in the example above)

Describe alternatives you've considered I have hacked a solution like this:

• Install an arbitrary app to create a venv (I used cowsay)
• Inject library into the cowsay venv
• Inject app into the cowsay venv with --include-apps

app is now available

Suggestions for better ideas would be much appreciated.

Describe the bug The instructions to install an application globally from https://pypa.github.io/pipx/installation/ does not recognize pipx

How to reproduce Here are the steps I took:

python3 -m pip install --user pipx
python3 -m pipx ensurepath
sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install cowsay

Where the last command returns the error sudo: pipx: command not found

I am running on Rocky Linux 8

Fixes #913

TODO

• [x] I have added an entry to docs/changelog.md
• [x] Documentation updates
• [x] Additional tests for successfully running scripts with dependencies
• [x] Refactor our use of the Venv class to handle app-less venvs properly
• [x] Clean up the implementation

Summary of changes

This PR adds functionality to the pipx run <script> command to allow running scripts that have non-stdlib dependencies.

The script is parsed for a comment block that has the form

# Requirements:
# req1
# req2

The block is terminated by a blank comment line, or by a non-comment line. Requirements may be anything that can be a top-level requirement for pip.

Before running a script, the code checks for a requirements block. If one is present, it creates a temporary virtual environment (managed the same way as the existing pip run app handles its venvs), installs the requirements there, and runs the script in that environment.

Implementation issues

The existing implementation of the Venv class in pipx assumes every environment will have a "main app". The code to write the pipx metadata file enforces this. This is not the case for temporary environments supporting scripts.

Rather than changing the implementation of Venv to allow environments without apps (which would be an extensive change, as it changes a fundamental invariant of the existing class) I plan on splitting the Venv class into two subclasses - AppVenv and ScriptVenv. All existing uses would become AppVenv instances, with ScriptVenv being for the new functionality.

One subtlety here is that the code that lists venvs simply scans directories and assumes what it finds is a venv (VenvContainer.iter_venv_dirs). In practice that isn't an issue, as ScriptVenv environments will only be found in the cache directory, which is never scanned in this way (as far as I can tell) but it is a potential risk. I will decide how to handle this when I do the refactoring.

Test plan

The PR includes tests of the functionality.

Also, it can be manually tested by creating a file test.py containing

# Requirements:
#     requests

import sys
import subprocess
print(sys.executable)
subprocess.run([sys.executable, "-m", "pip", "list"])

Then run the command

pipx run file:test.py

Output should show the executable being in a temporary virtual environment, and the pip output will show requests being present (with its dependencies).

How would this feature be useful? If I want to run e.g. pipx run pylint, it would be nice if it could be tied to any version requirements I may have specified in my pyproject.toml file for installing pylint for development.

Describe the solution you'd like If something specified by pipx run is found in pyproject.toml's optional-dependencies, use that dependency specification when installing the tool into the temporary virtual environment. Probably would want to just grab the first requirement found instead of trying to be clever about it.

Describe alternatives you've considered I've considered not caring. 😉 Honestly it probably isn't worth worrying about this idea, but I at least wanted it written down after I thought about it in case anyone else also realized that when you are running a tool for a project there may be certain installation requirements for it.