Here's a take on https://github.com/darrenburns/ward/discussions/197 , using a Rich progress bar displayed below the running tests.

Putting this up for an early look (@darrenburns) because there are a few problems with it that might need discussion...

• Ward's own suite is so fast that you can barely see the progress bar appear, at least on my terminal 🤪 . I've taken to adding a sleep right here so that I can actually see it while playing around...
• It flickers pretty badly (at least on my terminal...).
• I don't think it is compatible with either of the dots output displays, because it seems like Progress is expecting you to not print with end="" while it's running. Perhaps we can make some improvements to Rich itself to make that work better.

Following up on https://github.com/darrenburns/ward/pull/242 , this PR is a take on:

• Introducing pretty output for all comparison failures, in the same titled-panel style we landed on in the other PR.
• Introducing a consistent, extensible way to generate and combine parts of the comparison failure pretty output (it is very easy to add/change parts in the source, and should hopefully be easy to hookify it).
• Revising the existing == output to use the titled-panel style.

Incidentally, I extended the output to include assertion messages if present, and allowed for assertion messages that aren't plain strings.

Here's what it looks like right now on this example: test_examples.txt:

I decided to make the two operands be displayed side-by-side to save vertical space on wide terminals. If the terminal is narrow, Rich will automatically stack them vertically instead!

There is a clear distinction between describing a test and classifying it. They are semantically different things. The current method of 'tagging' or 'marking' tests is by patterns in the string descriptions. This not only mixes two things that probably shouldn't be but also causes some usability issues.

Even just a couple 'tags' can drown out the test descriptions, especially since they have to be formatted for reliable parsing for text search. They don't end up looking great in the output as the tags don't line up for easy human parsing. As alluded to before, text searching is clunky at best or unreliable.

So, I suggest adding a tags or similar attribute to tests and parameter to the test decorator to supplant the idea of tagging the descriptions. The --search CLI option could remain for text searching the descriptions, but a new --tags (or whatever name is chosen) option could be added. An implementation detail that would need to be ironed out is whether to support both 'or' and 'and' cases as well as how for multiple tags.

It might be nice to have a JSON writer if this is part of a pipeline. It might also be nice to have a non-colored output. Currently the fields are easy to see on a colored terminal, because they're different colors. However, in some cases (maybe in Jenkins) you may not have access to colors, so it'd be easier to read with clear separators and no color codes.

This was discovered after merging #80, which adds a Makefile which creates a venv inside the project directory. During test collection, ward recurses directories and may stumble across test modules not intended for it. Looking inside the virtualenv caused ward to fail. Running ward --path tests worked because it excluded the virtualenv from test collection.

I'm wondering whether it makes sense for ward to ignore virtualenvs or folders in the .gitignore when it looks for tests?

Changes:

1. Adding Location footer in AssertionErrors.

Concerns:

1. Didn't add tests for the change as I can't find other tests for the Locations.
2. Approach doesn't seem clean to me, not sure if how we can improve it.

issue #126

If you're interested in switching from setup.py to pyproject.toml (with flit), take a look. I thought this might be a good step toward #68.

Notes:

• flit looks for a project description at the start of __init__.py, so I moved it there
• I added a dev-requirements.txt though I don't know if you want that
• Someone with Windows would have to test if a workaround is still necessary here ("Work around encoding errors when installing on Windows." from setup.py)
• I don't know if you want to relax the dev dependency versions, if you want to keep them at all

Putting this up for some early feedback. Have not tested all the corner cases of #205 yet.

@darrenburns I did what I described in #227 and it works great! I think I came up with a fairly clean way of abstracting away the actual test running while giving the styles a fair bit of control over what the output looks like. I've got the "live" output style you describe in the issue, plus a "none" output style that prints nothing, PLUS dots work with progress bars! Some of these are even a little faster in wall clock time because more of the printing is happening on a background thread.

... but none of them will work in CI except test-per-line output with inline progress, because the others all use Lives. In fact, since I'm always starting the live right now, it may mess up CI even with the components that would use it turned off....

Threw this together re: https://github.com/darrenburns/ward/discussions/214#discussioncomment-767992 , since I guess I have Opinions about which hooks to use 😛

Depends on #217 to actually pass...

@darrenburns if you want to use pre-commit.ci , can you opt-in at https://pre-commit.ci/ ? It should work, since this branch has a .pre-commit-config.yaml. If you don't want to use the external service, we can set up the checks as a GitHub Actions workflow.

WIP...

So, this turned into a doozy. I think it's ready for a round of review.

Addresses #159

• Test and Fixture are now hashable. This was a little tricky for Fixture, since it seems like the underlying function's hash is different in different places (I suspect this is because they aren't actually the same function when inside the Suite because of assertion rewriting?).
• Test now has a helper class, TestArgumentResolver, where are all of the methods that make that happen are stored. I actually didn't really need to modify them at all, although I changed how information flows through them a little and added a helper method to get fixture arguments. This made the rest of the logic a little easier for me to grok and provides an obvious place to hang more code in the future. More data could be passed to the resolver directly to lessen the amount of self.test.<whatever> going on in it.
• ward fixtures can now display three kinds of dependency information: parent fixtures, child fixtures, and direct test usages. Parents and children can be displayed as trees.
• Rewrote most of the fixture information display from #154 to accommodate the new functionality.

I have ward installed in a virtual environment on Python 3.7.5, and I'm getting the following error when executing the ward command in the root of my repository. My code lives in a package omission directly in this repository, and that package contains subpackages and submodules configured as tests.

Here are the versions of the packages installed in my virtual environment:

• GitPython-3.1.0
• PyYAML-5.3.1
• appdirs-1.4.3
• bandit-1.6.2
• click-7.1.1
• colorama-0.3.9
• cucumber-tag-expressions-2.0.4
• entrypoints-0.3
• flake8-3.7.9
• gitdb-4.0.2
• mccabe-0.6.1
• pbr-5.4.4
• pprintpp-0.4.0
• pycodestyle-2.5.0
• pyflakes-2.1.1
• pygments-2.6.1
• pyside2-5.14.1
• shiboken2-5.14.1
• six-1.14.0
• smmap-3.0.1
• stevedore-1.32.0
• termcolor-1.1.0
• toml-0.9.6
• ward-0.42.0b0

Here's the error I'm encountering:

(venv) [email protected]:omission$ ward
Traceback (most recent call last):
  File "/home/jason/Code/Repositories/omission/venv/bin/ward", line 10, in <module>
    sys.exit(run())
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/ward/run.py", line 115, in run
    modules = list(load_modules(mod_infos))
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/ward/collect.py", line 109, in load_modules
    m.__loader__.exec_module(m)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "venv/lib/python3.7/site-packages/git/test/test_diff.py", line 7, in <module>
    import ddt
ModuleNotFoundError: No module named 'ddt'

After installing ddt manually (ddt-1.3.1), I get this error instead when running ward:

Traceback (most recent call last):
  File "/home/jason/Code/Repositories/omission/venv/bin/ward", line 10, in <module>
    sys.exit(run())
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/ward/run.py", line 115, in run
    modules = list(load_modules(mod_infos))
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/ward/collect.py", line 109, in load_modules
    m.__loader__.exec_module(m)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "venv/lib/python3.7/site-packages/git/test/performance/test_commit.py", line 11, in <module>
    from .lib import TestBigRepoRW
ImportError: attempted relative import with no known parent package

Interestingly, if I install pytest as well, the error on ward changes yet again:

Traceback (most recent call last):
  File "/home/jason/Code/Repositories/omission/venv/bin/ward", line 10, in <module>
    sys.exit(run())
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/ward/run.py", line 115, in run
    modules = list(load_modules(mod_infos))
  File "/home/jason/Code/Repositories/omission/venv/lib/python3.7/site-packages/ward/collect.py", line 109, in load_modules
    m.__loader__.exec_module(m)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "venv/lib/python3.7/site-packages/importlib_metadata/tests/test_api.py", line 5, in <module>
    from . import fixtures
ImportError: attempted relative import with no known parent package

For the record, installing pytest as well installs the following:

• attrs-19.3.0
• importlib-metadata-1.6.0
• more-itertools-8.2.0
• packaging-20.3
• pluggy-0.13.1
• py-1.8.1
• pyparsing-2.4.6
• pytest-5.4.1
• wcwidth-0.1.9
• zipp-3.1.0

This may be something to think about for more than just rich, but rich in particular is:

1. seeing rapid development and releases
2. used by a whole lot of projects (which may be co-installed with ward)

Currently ward prohibits installation of the newest rich releases, which means if a project is requiring that new version of rich, an older version of ward must be used, which doesn't restrict the rich version as much.

But this doesn't actually improve compatibility -- in fact it creates a cascading effect of installing/requiring older versions of more tools, increasing probability of in-practice incompatibility (as opposed to explicitly version-specified incompatibility) between new PyPI releases and very old ones.

See also:

• #323
• https://github.com/darrenburns/ward/issues/300#issuecomment-1371304154

Bumps actions/setup-python from 4.0.0 to 4.4.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps certifi from 2021.10.8 to 2022.12.7.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Currently type information in API is not complete. It makes it hard to use the library with mypy's strict mode and Pyright.

Also see: https://github.com/darrenburns/ward/issues/261#issuecomment-1301968435

This merge request adds support to run tests using an async library other than the standard asyncio. This can be configured globally for all tests by setting async-library in the project's pyproject.toml or per test by setting async_library in the test decorator.
Example: @test('my test that uses curio', async_library="curio")

Would you be willing to add support for Curio and/or Trio? I would submit a PR myself but I see in https://github.com/darrenburns/ward/blob/master/ward/testing.py#L200 that asyncio.run(coro) is hardcoded and I'm not sure how you would go about to make that configurable. If you could check on the user supplied config before calling asyncio.run(coro) you could do something like:

if self.is_async_test:
    coro = self.fn(**resolved_args)
    if config.asynclib == 'curio':
        import curio
        curio.run(coro)
    elif config.asynclib == 'trio':
        import trio
        trio.run(coro)
    else:
        asyncio.run(coro)
else:
    self.fn(**resolved_args)