I'm developing a custom plugin and can't get pybuilder to pull new versions when they're available. How do I specify min and/or locked version for a plugin pulled from a pypi repository?

I have installed Python 3 on macOS 11.3 (Big Sur) with brew install [email protected]. Homebrew installs Python 3.9 under /usr/local/Cellar/[email protected], and symlinks /usr/local/opt/[email protected] -> ../Cellar/[email protected]/3.9.4.

python3 is a symlink /usr/local/bin/python3 -> ../Cellar/[email protected]/3.9.4/bin/python3. pip3 is a symlink /usr/local/bin/pip3 -> ../Cellar/[email protected]/3.9.4/bin/pip3.

$ head -1 $(which pip3)
#!/usr/local/opt/[email protected]/bin/python3.9

I used pip3 install --user pybuilder to install pybuilder:

$ which pyb
/Users/username/Library/Python/3.9/bin/pyb
$ head -1 $(which pyb)
#!/usr/local/opt/[email protected]/bin/python3.9

$ pyb
PyBuilder version 0.12.10
Build started at 2021-04-28 10:36:15
------------------------------------------------------------
------------------------------------------------------------
BUILD FAILED - AssertionError: Filename /usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py does not start with any of these prefixes: ['/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/Extras/lib/python', '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9', '/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9', '/Users/username/Library/Python/3.9/lib/python/site-packages', '/Users/username/.local/lib/python/3.9/site-packages', '/Users/username/Library/Python/3.9/site-packages', '/Library/Python/3.9/site-packages'] (pybuilder/_vendor/virtualenv.py:1272)
------------------------------------------------------------
Build finished at 2021-04-28 10:36:15
Build took 0 seconds (64 ms)

AssertionError is raised from change_prefix:

https://github.com/pybuilder/pybuilder/blob/6ac1ef4e3f504fec7d4f86378fde1c5420d6a261/src/main/python/pybuilder/_vendor/virtualenv.py#L1272

As a python developer I want to create a nice description for PyPI. Nearly every GitHub project comes with a README.md (or something like that). It would be nice to have a simple way to generate the PyPI description from the GitHub flavored markdown / text file. Within build.py

description = description_from_text_file("README")

or

description = description_from_github_markdown_file("README.md")

[INFO]  Writing MANIFEST.in as /build/python-pybuilder/src/pybuilder-0.11.12-py2/target/dist/pybuilder-0.11.12/MANIFEST.in
[INFO]  Writing setup.py as /build/python-pybuilder/src/pybuilder-0.11.12-py2/target/dist/pybuilder-0.11.12/setup.py
------------------------------------------------------------
BUILD FAILED - 'ascii' codec can't encode character u'\u201c' in position 5932: ordinal not in range(128)
------------------------------------------------------------
Build finished at 2017-11-30 23:14:44
Build took 3 seconds (3610 ms)

Python 3 installation works fine, though. The installation command I used was:

python2 setup.py install --root="$pkgdir" -O1

As suggested by @esc if our automated tests pass on travis then we can do a pre-release so users can pip install --pre pybuilder.

Ideally there is a way to do it only if tests on all python versions passed... I'm not sure if there is though.

Hello,

I think the new coverage plugin should be refactored, as it's a potential source of problems. Looks like it's deleting a bunch of global modules, sparing only the ones that it consider "essential". We have a custom python installation, the coverage plugin gets a bit confused and considers non essential things like datetime. We end up with errors like:

Traceback (most recent call last): File "/tmp/test/env-infra/bin/pyb", line 25, in sys.exit(pybuilder.cli.main(*sys.argv[1:])) File "/tmp/test/env-infra/lib/python2.7/site-packages/pybuilder/cli.py", line 366, in main end = datetime.datetime.now() AttributeError: 'module' object has no attribute 'datetime'

Because of this issue we are unable to update to 0.11.1 I know there should be a way to fix on our side by reverse-engineering your code and put modules in strategic locations, but honestly nothing in pybuilder (nothing in anything IMHO) should play with sys.modules, as it will cause issues that are a nightmare to debug.

Downgrading to 0.10.63 fixes the issue obviously, but the install task would be nice to have for our purposes.

BUILD FAILED - [Errno 22] invalid mode ('w') or filename: 'c:\Users\jcorson\pythonWorkspace\projectNameHere\target/reports\distutils\sdist__upload__-r__http:123.123.120.117:8080'

I am on windows working in the Git Bash. pybuilder version 0.11.8. I saw another closed issue about pathing and you made him upgrade but this is very recent

This issue didn't exist when it was the public PyPI server but I changed it to a local pypi that I need to store my project in. As a result, it seems to append the -r and the URL I provided

project.set_property("distutils_upload_repository", 'http://123.123.120.117:8080')

This is a release prep for milestone 0.11.3.

• [x] Bugs and features reviewed and signed off
• [x] Do we want anything else for this release?

@mriehl @esc @locolupo

I think we should get a new release out - the improvements by @arcivanov are too sweet to pass on (xml reports are a boon for jenkins users) and there are a couple of bug fixes too.

I also think we should go for 0.11.0 since there are some deprecated properties and coverage might be affected if some task was using library functions from the covered project.

@locolupo @arcivanov what do you think? It would be great if you want to push out the new release, the release is process is very simple:

• bump the version, create a new tag
• build and upload to PyPI
• add release notes to the documentation

Cram is a functional testing framework for command line applications. This pull-requests presents a pybuilder plugin for it.

Cram and the details can be found on: https://pypi.python.org/pypi/cram

There are also two dog-fooding test files in src/cmdlinetest which is the default location for cram tests. The location can be configured, as can the glob used for discovering cram tests. The plugin respects the verbose setting.

One thing I wasn't able to work out, was how to launch the plugin as part of publish, so remains as its own task for now.

Another issue is that the unit test is not exactly DRY right now.

Anyway, I wanted to submit this for review now, because it is already in a working state and just needs some more polishing.

Hi,

This is a question regarding using one python project as a dependency in other project.

I have a project with test related utilities which is kept in separate repo. As of now, we were doing "git submodule" command to get this project in our main project and use its files in our tests. Now we are trying to create a separate package (tar.gz and .whl) of utilities project and upload to Artifactory. For our main project, we want to give utlities project as dependency in build.py so that when unit tests are executed during build, main project should be able to resolved those dependencies.

Any thoughts?

I had flake8 installed, but due to the dependency of the flake8-plugin, 4.0.1 was installed.

My question is here: should this still be pinned? flake8 is at v6 already.

The reason I noticed the failure was because I have flake8-pyproject installed and it started to fail all of a sudden...

I am using Pybuilder to develop a project that depends on a library that is being developed concurrently. In the requirements.txt file I am imposing a condition like this:

sample-library~=0.2.0

Currently, the only 0.2.0 version available on our PyPI clone is 0.2.0.dev20220210180228.

I can install the project's dependencies via pip using the --pre option, but I can't use this feature with Pybuilder: using the python.install_dependencies plugin and the pyb install_dependencies command, this is the output I get:

[INFO]  Going to execute task install_dependencies
[INFO]  Processing plugin packages 'flake8~=4.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'pypandoc~=1.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'setuptools>=38.6.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'twine>=1.15.0' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'unittest-xml-reporting~=3.0.4' to be installed with {'upgrade': True}
[INFO]  Processing plugin packages 'wheel>=0.34.0' to be installed with {'upgrade': True}
[INFO]  Installing all dependencies
[INFO]  Processing dependency packages 'requirements.txt' to be installed with {}
------------------------------------------------------------
BUILD FAILED - Unable to install dependency packages into <project-dir>\.venv. Please see '<project-dir>\target\logs\install_dependencies\install_batch' for full details:
        Looking in indexes: http://<our-pypi-instance>/repository/pypi-group/simple
        ERROR: Could not find a version that satisfies the requirement sample-library~=0.2.0 (from versions: 0.1.0, 0.2.0.dev20220210180228)
        ERROR: No matching distribution found for sample-library~=0.2.0

Would it be possible to add an option to pyb that allows pip to be internally invoked with the --pre option?

After upgrading from v0.12.10 to v0.13.3, we noticed a new error: `
File "/ws/.venv/lib/python3.8/site-packages/pybuilder/_vendor/importlib_metadata/init.py", line 6, in from .. import zipp ValueError: attempted relative import beyond top-level package

--------------------------------------------------------------------------------`

This importlib_metadata seems to have been added in v0.13.0

Looks like without -v the default is to suppress the traceback of unit test errors:

[INFO]  Executed 1 unit tests
Error:  Test has error: unittest.loader._FailedTest.xxx
------------------------------------------------------------
BUILD FAILED - There were 1 error(s) and 0 failure(s) in unit tests (pybuilder/plugins/python/unittest_plugin.py:109)
------------------------------------------------------------

I think the default should be to show the traceback that caused the build to fail, regardless of whether -v is used.

This is pretty much an edge case but it seems that some projects need to be built with a pip constraints file, such as Airflow (detailed description as to why here).

The equivalent on the command line with pip would be to run e.g.:

pip install "apache-airflow[celery]==2.2.2" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.6.txt"

I'm unsure which syntax / design would make most sense here. In our use-case, the dependency is required at build time as well in order to use the library in tests.