💡 Type hints for Numpy

Last update: Dec 28, 2022

Related tags

Overview

Type hints with dynamic checks for Numpy!

(❒) Installation

pip install nptyping

(❒) Usage

(❒) NDArray

nptyping.NDArray lets you define the shape and type of your numpy.ndarray.

You can:

specify the number of dimensions;
specify the size per dimension;
specify the type of the array;
instance check your array with your nptying type.

(❒) Examples

An Array with any dimensions of any size and any type:

>>> from nptyping import NDArray
>>> from typing import Any


>>> NDArray
NDArray[(typing.Any, ...), typing.Any]

>>> NDArray[(Any, ...)]
NDArray[(typing.Any, ...), typing.Any]

>>> NDArray[(Any, ...), Any]
NDArray[(typing.Any, ...), typing.Any]

An array with 1 dimension of any size and any type:

>>> NDArray[Any]
NDArray[(typing.Any,), typing.Any]

>>> NDArray[(Any,)]
NDArray[(typing.Any,), typing.Any]

>>> NDArray[Any, Any]
NDArray[(typing.Any,), typing.Any]

>>> NDArray[(Any,), Any]
NDArray[(typing.Any,), typing.Any]

An array with 1 dimension of size 3 and any type:

>>> NDArray[3]
NDArray[(3,), typing.Any]

>>> NDArray[(3,)]
NDArray[(3,), typing.Any]

>>> NDArray[(3,), Any]
NDArray[(3,), typing.Any]

An array with 3 dimensions of size 3, 3 and any and any type:

>>> NDArray[3, 3, Any]
NDArray[(3, 3, typing.Any), typing.Any]

>>> NDArray[(3, 3, Any)]
NDArray[(3, 3, typing.Any), typing.Any]

>>> NDArray[(3, 3, Any), Any]
NDArray[(3, 3, typing.Any), typing.Any]

An array with any dimensions of any size and type int:

>>> import numpy as np

>>> NDArray[np.int32]
NDArray[(typing.Any, ...), Int[32]]

>>> NDArray[(Any, ...), np.int32]
NDArray[(typing.Any, ...), Int[32]]

Note that provided types are translated to nptyping types. Pure Python types (.e.g int or float are supported as well). You can also provide nptyping types yourself: NDArray[(Any, ...), Int[64]].

An array with 1 dimension of size 3 and type int:

>>> NDArray[3, np.int32]
NDArray[(3,), Int[32]]

>>> NDArray[(3,), np.int32]
NDArray[(3,), Int[32]]

An array with any dimensions of size 3 and type int:

>>> NDArray[(3, ...), np.int32]
NDArray[(3, ...), Int[32]]

An array with 3 dimensions of sizes 3, 3, 5 and type int:

>>> NDArray[(3, 3, 5), np.int32]
NDArray[(3, 3, 5), Int[32]]

A structured array:

>>> import numpy as np

>>> NDArray[(Any,...), np.dtype([('x',np.int32), ('y',np.int32)])]
NDArray[(typing.Any, ...), StructuredType[Int[32], Int[32]]]

(❒) Checking your instances

You can use NDArray with isinstance to dynamically check your arrays.

>>> import numpy as np

>>> arr = np.array([[1, 2, 3],
...                 [4, 5, 6]])

>>> isinstance(arr, NDArray[(2, 3), int])
True
>>> isinstance(arr, NDArray[(2, 3), float])
False
>>> isinstance(arr, NDArray[(2, 3, 1), int])
False

(❒) Finding the right annotation

You can use NDArray to find the type of a numpy array for you using NDArray.type_of:

>>> NDArray.type_of(np.array([[1, 2], [3, 4.0]]))
NDArray[(2, 2), Float[64]]

See also nptyping.get_type (documented below).

(❒) Int

An nptyping equivalent of numpy signed integers.

>>> from nptyping import Int

>>> Int[32]
Int[32]

You can also use one of these:

>>> from nptyping import Int8, Int16, Int32, Int64

(❒) UInt

An nptyping equivalent of numpy unsigned integers.

>>> from nptyping import UInt

>>> UInt[64]
UInt[64]

You can also use one of these:

>>> from nptyping import UInt8, UInt16, UInt32, UInt64

(❒) Float

An nptyping equivalent of numpy floats.

>>> from nptyping import Float

>>> Float[64]
Float[64]

You can also use one of these:

>>> from nptyping import Float16, Float32, Float64

(❒) Unicode

An nptyping equivalent of numpy unicodes.

>>> from nptyping import Unicode

>>> Unicode[100]
Unicode[100]

(❒) Bool

An nptyping equivalent of numpy bool.

>>> from nptyping import Bool

>>> Bool
Bool

(❒) Complex128

An nptyping equivalent of numpy complex128.

>>> from nptyping import Complex128

>>> Complex128
Complex128

(❒) Datetime64

An nptyping equivalent of numpy datetime64.

>>> from nptyping import Datetime64

>>> Datetime64
Datetime64

(❒) Timedelta64

An nptyping equivalent of numpy timedelta64.

>>> from nptyping import Timedelta64

>>> Timedelta64
Timedelta64

(❒) Object

An nptyping equivalent of numpy objects.

>>> from nptyping import Object

>>> Object
Object

(❒) StructuredType

An nptyping equivalent of numpy structured dtypes.

>>> from nptyping import StructuredType, Int

>>> StructuredType[Int[32], Int[32]]
StructuredType[Int[32], Int[32]]

(❒) SubArrayType

An nptyping equivalent of numpy subarray dtypes.

>>> from nptyping import SubArrayType, Int

>>> SubArrayType[Int[16], (4,2)]
SubArrayType[Int[16], (4, 2)]

(❒) get_type

With get_type you can get nptyping equivalent types for your arguments:

>>> from nptyping import get_type

>>> get_type(np.int32)
Int[32]
>>> get_type('some string')
Unicode[11]
>>> get_type(np.dtype([('x', np.int32), ('y', np.int32)]))
StructuredType[Int[32], Int[32]]

(❒) py_type

With py_type you can get the Python builtin type that corresponds to a Numpy dtype:

>>> from nptyping import py_type

>>> py_type(np.int32)
<class 'int'>

Comments

Support static type checker
An often asked question is whether nptyping supports a static type checker such as mypy, which at the moment isn't the case. Implementing this feature will be difficult (if even possible for some type checkers).

Here is an example of a challenge:

import numpy as np import nptyping as npt arr2x2: npt.NDArray[(2, 2), int] = np.array([[1, 2], [3, 4]]) arr4: npt.NDArray[4, int] = np.reshape(arr2x2, 4) # How is nptyping supposed to reassure that the type is correct (with no influence on np.reshape)?

There are several type checkers available such as mypy, pyright, pyre-check, pytype. Some support plugins (e.g. mypy) which may be something to look into.

The most widely used type checker would be mypy, so it would seem reasonable to start with that particular one.
help wanted feature
opened by ramonhagenaars 34
Allow for more expressive Array signatures

See also issues #9, #10 and #11.

There have been several requests to extend the expressiveness of Array. I don't feel much for a sudden signature change of Array. Rather, I'd like to introduce a new type NDArray (which name I like more than Array anyway) that will "slowly" replace Array.

I have the following signature in mind:

Signature design NDArray any dimension of any size of any type NDArray[...] 1 dimension of any size of any type NDArray[3] 1 dimension of size 3 of any type NDArray[(3, 3, 5)] 3 dimensions (3 x 3 x 5) of any type NDArray[(3, ..., 5)] 3 dimensions (3 x ? x 5) of any type NDArray[(D1, 3, D1)] 3 dimensions (D1 x 3 x D1 where D1 is an nptyping constant that can be imported to express a dimension variable, see #9 and #11) of any type

NDArray[int] any dimension of any size of type int NDArray[..., int] 1 dimension of any size of type int NDArray[(3, 3, 5), int] 3 dimensions (3 x 3 x 5) of type int NDArray[(3, 3, 5), np.dtype('int16')] 3 dimensions (3 x 3 x 5) of type int16 NDArray[(3, 3), np.dtype([('f1', np.int16), ('f2', np.int16)])] 2 dimensions (3 x 3) with structured types

Process The new NDArray is to replace the current Array. Once introduced, the original Array will become deprecated to be removed upon the minor release that follows next.

Before I start investing time into this, I'd love to hear your opinion on this. Please leave any feedback, any comments, any suggestions.
feature WIP

opened by ramonhagenaars 9
conda release

Hi there, Thanks for your efforts with nptyping, it's perfect for its use! We want to use it in our project (https://github.com/openclimatedata/openscm) but are also thinking about releasing our work with conda. To do that, all the dependencies need to be available on conda too. Would you be open to releasing nptyping on conda? I'd be happy to do the setup work for you/talk you through it, it's super simple for Python projects like this (once you've seen it done before). Cheers, Zeb
infra

opened by znicholls 6
Caching instantiations? (e.g. make `NDArray[int] is NDArray[int]` work?)
When using nptyping (c813f6d2), I observe the following:

>>> from nptyping import NDArray >>> NDArray is NDArray True >>> NDArray[int] is NDArray[int] False

Some possible approaches to remedy:

(Naive) Memoize based on item in typish.SubscriptableType.__getitem__. However, that means NDArray[int] is NDArray[Int[64]] will not work.

Memoize based on the simplified args computed. May require some reshuffling; from what it looks like, the current flow is SubscriptableType creates the class with (__args__, __origin__), then _NDarray._after_subscription sets (_shape, _type) based on the method dispatch.

improvement
opened by EricCousineau-TRI 5
Number is not Number
Just simple example:

In[1]: isinstance(np.array([1.0, -2]),NDArray[Number]) Out[2]: False In[3]: isinstance(np.array([1.0, -2]),NDArray[float]) Out[4]: True

Why does float is not considered to be a Number or Real? It's definitely a bug.

See also python/mypy#3186

Guess this is not so hard to fix when nptyping uses runtime checks. Also guess I supposed to use libraries' Number. But is it as full featured as built-in?
wontfix unreproducible
opened by baterflyrity 4
adding __version__ parameter

Great project!

I added a version parameter to the init.py file. That's the standard name and location for that variable.

I tried making the setup.py references that, but the init.py file imports typish, so it unfortunately requires putting the version in 2 places.

I also added python 3.8 to the build matrix.

It also looks like there is a requirement for typish = 1.2 on Python 3.5, which is causing the failed test. That failing test is not new.

opened by SteveDoyle2 4
PyCharm reports syntax error when using Shape['*'] in variable declarations
The following will run correctly:

a: NDArray[Shape['*']] = np.array([1, 2, 3]) isinstance(a, NDArray[Shape['*'], Int])

But causes PyCharm to issue the syntax error:

Statement expected, found Py:MULT
opened by nikolajsheller 3

Why does NDArray[Any] not work anymore?

[/usr/local/lib/python3.7/dist-packages/pyterrier/apply.py](https://localhost:8080/#) in <module>()
     79     return ApplyDocumentScoringTransformer(fn, *args, **kwargs)
     80 
---> 81 def doc_features(fn : Callable[..., NDArray[Any]], *args, **kwargs) -> Transformer:
     82     """
     83         Create a transformer that takes as input a ranked documents dataframe, and applies the supplied function to each document to compute feature scores.

[/usr/local/lib/python3.7/dist-packages/nptyping/ndarray.py](https://localhost:8080/#) in __getitem__(cls, item)
     70             raise NPTypingError(f"Type {cls} is already parameterized")
     71         if not isinstance(item, tuple):
---> 72             raise InvalidArgumentsError(f"Unexpected argument of type {type(item)}")
     73         shape_expression, dtype = _get_from_tuple(item)
     74         validate_dtype(dtype)

InvalidArgumentsError: Unexpected argument of type <class 'typing._SpecialForm'>

That code worked fine on previous nptyping

question

opened by cmacdonald 3

support pyright?
After installing v2.0.0, I found it works with mypy while it doesn't work with pyright.

For example, following code pass examimation by mypy, but it can't pass that by pyright.

from nptyping import Shape

Mypy gives no error, pyright gives that

feature
opened by wrvsrx 3

Implement structured types

This implements two additional NPTypes: StructuredType and SubArrayType.

It allows for defining NDArrays of structured types, which can in turn contain subarrays, if necessary:

>>> NDArray[(Any, ...), np.dtype([('x',int), ('y',int)])]
NDArray[(typing.Any, ...), StructuredType[Int[32],Int[32]]]

>>> NDArray[(Any, ...), np.dtype((int,4))]                
NDArray[(typing.Any, ...), SubArrayType[Int[32], (4,)]]

>>> NDArray[(Any, ...), np.dtype([('x',int), ('y',int, 3)])] 
NDArray[(typing.Any, ...), StructuredType[Int[32],SubArrayType[Int[32], (3,)]]]

I hope I've included all the tests necessary so that these can be used and instance and equality checks work as expected.

Resolves #39

opened by lyckantropen 3

Should rescursive nature of `SubscriptableTypes` allow `NDArray[float][int][bool]`?
When looking through the typing setup, it seems like this is possible (on c813f6d):

>>> from typing import NDArray >>> NDArray[2][int] NDArray[(2,), Int[64]] # Cool! >>> NDArray[float][int][bool] NDArray[(typing.Any, ...), Bool] # Er...

Dunno if that's a bug or feature?
opened by EricCousineau-TRI 3
How to get better __repr__ / autogenerated docstring for functions with type hints?

Hi there,

Would it be possible to get the type hints in a function definition automatically integrated into the function's docstring and/or string representation? Using the example from the github page, the function def plan_route(locations: NDArray[Shape["[from, to], [x, y]"], Float]) -> NDArray[Shape["* stops, [x, y]"], Float]: pass currently has the following help string: plan_route(locations: nptyping.base_meta_classes.NDArray) -> nptyping.base_meta_classes.NDArray which is not very helpful. The desired behavior would be something like: plan_route(locations: NDArray[Shape["[from, to], [x, y]"], Float]) -> NDArray[Shape["* stops, [x, y]"], Float]

Or is this already supported?

opened by nikolas-claussen 0
PYPI tarball does not include dependencies dir
Hey folks, is the PYPI release maintained by you too? Apparently it does not contain dependencies/build-requirements.txt, using the PYPI release will fail setup.py that explicitly required dependencies/build-requirements.txt

FileNotFoundError: [Errno 2] No such file or directory: '.../nptyping-2.3.1/dependencies/build-requirements.txt'
opened by MatthewZMD 1

Wildcard ellipsis ... matching incorrect?

As expected

>>> isinstance(random.randn(3, 2, 55), NDArray[Shape["3, *, ..."], Any])
True

These two I would expect the ellipsis to match the trailing dimensions - but they don't.

>>> isinstance(random.randn(3, 2, 55), NDArray[Shape["3, 2, ..."], Any])
False  

>>> isinstance(random.randn(3, 2, 55), NDArray[Shape["3,  ..."], Any])
False

Then finally, the ellipsis must only exist at the end.

>>> isinstance(random.randn(3, 2, 55), NDArray[Shape[" ..., 55"], Any])
nptyping.error.InvalidShapeError: '..., 55' is not a valid shape expression.

Am I just failing to understand how the ellipsis is used here? Which as far as I can tell is the usual usage in terms of array indexing, where it can match zero or more dimensions.. for example, these are all valid numpy indexing.

x = random.randn(2, 3, 55)
>>> x[1,...].shape
(3, 55)
>>> 
>>> x[1,...].shape
(3, 55)
>>> x[1, 1, ...].shape
(55,)
>>> x[1, 1, 1, ...].shape
()
>>> x[..., 1, 1].shape
(2,)
>>> x[1, ...,  1].shape
(3,)

opened by oliver-batchelor 1

Allow simplified syntax: `arr: NDArray["2, 2", int]`?
NPTyping is great, but could be more concise.

The main value of NDArray to me is to have a very concise way of documenting shapes.

I would propose

Just allowing us to enter a string or tuple for the shape, e.g. NDArray["2,2", Int] or NDArray[(2, 2), Int], as opposed to the more verbose NDArray[Shape["2,2"], Int]

Allowing us to just use built-in numeric types instead of importing them from nptyping. E.g. int vs nptyping.Int, etc.

I'd be happy to make a PR to this effect if it is approved in spirit.
opened by petered 3

2.3.0: pytest is failing

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

python3 -sBm build -w --no-isolation
because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
install .whl file in </install/prefix>
run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-nptyping-2.3.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-nptyping-2.3.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --ignore tests/test_package_info.py
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/nptyping-2.3.0
plugins: typeguard-2.13.3
collected 91 items

tests/test_assert_isinstance.py ...                                                                                                                                  [  3%]
tests/test_base_meta_classes.py .........                                                                                                                            [ 13%]
tests/test_beartype.py ....                                                                                                                                          [ 17%]
tests/test_lib_export.py .                                                                                                                                           [ 18%]
tests/test_mypy.py FFFFFFFFFF.                                                                                                                                       [ 30%]
tests/test_ndarray.py ..........................                                                                                                                     [ 59%]
tests/test_performance.py .                                                                                                                                          [ 60%]
tests/test_pyright.py ..                                                                                                                                             [ 62%]
tests/test_recarray.py ...                                                                                                                                           [ 65%]
tests/test_shape.py .....                                                                                                                                            [ 71%]
tests/test_shape_expression.py ...                                                                                                                                   [ 74%]
tests/test_structure.py ........                                                                                                                                     [ 83%]
tests/test_structure_expression.py ........                                                                                                                          [ 92%]
tests/test_typeguard.py ....                                                                                                                                         [ 96%]
tests/test_wheel.py F..                                                                                                                                              [100%]

================================================================================= FAILURES =================================================================================
_______________________________________________________________ MyPyTest.test_mypy_accepts_ndarray_with_any ________________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_ndarray_with_any>

    def test_mypy_accepts_ndarray_with_any(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            from nptyping import NDArray


            NDArray[Any, Any]
        """
        )
>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:26: AssertionError
______________________________________________________________ MyPyTest.test_mypy_accepts_ndarray_with_shape _______________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_ndarray_with_shape>

    def test_mypy_accepts_ndarray_with_shape(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            from nptyping import NDArray, Shape


            NDArray[Shape["3, 3"], Any]
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:39: AssertionError
____________________________________________________________ MyPyTest.test_mypy_accepts_ndarray_with_structure _____________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_ndarray_with_structure>

    def test_mypy_accepts_ndarray_with_structure(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            from nptyping import NDArray, RecArray, Structure


            NDArray[Any, Structure["x: Float, y: Int"]]
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:52: AssertionError
________________________________________________________ MyPyTest.test_mypy_accepts_ndarrays_as_function_arguments _________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_ndarrays_as_function_arguments>

    def test_mypy_accepts_ndarrays_as_function_arguments(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            import numpy as np
            from nptyping import NDArray, Shape


            def func(_: NDArray[Shape["2, 2"], Any]) -> None:
                ...


            func(np.array([1, 2]))  # (Wrong shape though)
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:90: AssertionError
__________________________________________________________ MyPyTest.test_mypy_accepts_ndarrays_as_variable_hints ___________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_ndarrays_as_variable_hints>

    def test_mypy_accepts_ndarrays_as_variable_hints(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            import numpy as np
            from nptyping import NDArray


            arr: NDArray[Any, Any] = np.array([1, 2, 3])
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:104: AssertionError
________________________________________________________________ MyPyTest.test_mypy_accepts_nptyping_types _________________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_nptyping_types>

    def test_mypy_accepts_nptyping_types(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            import numpy as np
            import numpy.typing as npt
            from nptyping import (
                NDArray,
                Number,
                Bool,
                Bool8,
                Object,
                Object0,
                Datetime64,
                Integer,
                SignedInteger,
                Int8,
                Int16,
                Int32,
                Int64,
                Byte,
                Short,
                IntC,
                IntP,
                Int0,
                Int,
                LongLong,
                Timedelta64,
                UnsignedInteger,
                UInt8,
                UInt16,
                UInt32,
                UInt64,
                UByte,
                UShort,
                UIntC,
                UIntP,
                UInt0,
                UInt,
                ULongLong,
                Inexact,
                Floating,
                Float16,
                Float32,
                Float64,
                Half,
                Single,
                Double,
                Float,
                LongDouble,
                LongFloat,
                ComplexFloating,
                Complex64,
                Complex128,
                CSingle,
                SingleComplex,
                CDouble,
                Complex,
                CFloat,
                CLongDouble,
                CLongFloat,
                LongComplex,
                Flexible,
                Void,
                Void0,
                Character,
                Bytes,
                String,
                Bytes0,
                Unicode,
                Str0,
            )

            NDArray[Any, Number]
            NDArray[Any, Bool]
            NDArray[Any, Bool8]
            NDArray[Any, Object]
            NDArray[Any, Object0]
            NDArray[Any, Datetime64]
            NDArray[Any, Integer]
            NDArray[Any, SignedInteger]
            NDArray[Any, Int8]
            NDArray[Any, Int16]
            NDArray[Any, Int32]
            NDArray[Any, Int64]
            NDArray[Any, Byte]
            NDArray[Any, Short]
            NDArray[Any, IntC]
            NDArray[Any, IntP]
            NDArray[Any, Int0]
            NDArray[Any, Int]
            NDArray[Any, LongLong]
            NDArray[Any, Timedelta64]
            NDArray[Any, UnsignedInteger]
            NDArray[Any, UInt8]
            NDArray[Any, UInt16]
            NDArray[Any, UInt32]
            NDArray[Any, UInt64]
            NDArray[Any, UByte]
            NDArray[Any, UShort]
            NDArray[Any, UIntC]
            NDArray[Any, UIntP]
            NDArray[Any, UInt0]
            NDArray[Any, UInt]
            NDArray[Any, ULongLong]
            NDArray[Any, Inexact]
            NDArray[Any, Floating]
            NDArray[Any, Float16]
            NDArray[Any, Float32]
            NDArray[Any, Float64]
            NDArray[Any, Half]
            NDArray[Any, Single]
            NDArray[Any, Double]
            NDArray[Any, Float]
            NDArray[Any, LongDouble]
            NDArray[Any, LongFloat]
            NDArray[Any, ComplexFloating]
            NDArray[Any, Complex64]
            NDArray[Any, Complex128]
            NDArray[Any, CSingle]
            NDArray[Any, SingleComplex]
            NDArray[Any, CDouble]
            NDArray[Any, Complex]
            NDArray[Any, CFloat]
            NDArray[Any, CLongDouble]
            NDArray[Any, CLongFloat]
            NDArray[Any, LongComplex]
            NDArray[Any, Flexible]
            NDArray[Any, Void]
            NDArray[Any, Void0]
            NDArray[Any, Character]
            NDArray[Any, Bytes]
            NDArray[Any, String]
            NDArray[Any, Bytes0]
            NDArray[Any, Unicode]
            NDArray[Any, Str0]
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:312: AssertionError
__________________________________________________________________ MyPyTest.test_mypy_accepts_numpy_types __________________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_numpy_types>

    def test_mypy_accepts_numpy_types(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            from nptyping import NDArray
            import numpy as np


            NDArray[Any, np.dtype[np.int_]]
            NDArray[Any, np.dtype[np.float_]]
            NDArray[Any, np.dtype[np.uint8]]
            NDArray[Any, np.dtype[np.bool_]]
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:134: AssertionError
____________________________________________________________ MyPyTest.test_mypy_accepts_recarray_with_structure ____________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_accepts_recarray_with_structure>

    def test_mypy_accepts_recarray_with_structure(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            from nptyping import RecArray, Structure


            RecArray[Any, Structure["x: Float, y: Int"]]
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:117: AssertionError
___________________________________________________ MyPyTest.test_mypy_disapproves_ndarray_with_wrong_function_arguments ___________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_disapproves_ndarray_with_wrong_function_arguments>

    def test_mypy_disapproves_ndarray_with_wrong_function_arguments(self):
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            import numpy as np
            from nptyping import NDArray, Shape


            def func(_: NDArray[Shape["2, 2"], Any]) -> None:
                ...


            func("Not an array...")
        """
        )

        self.assertIn('Argument 1 to "func" has incompatible type "str"', stdout)
        self.assertIn('expected "ndarray[Any, Any]"', stdout)
>       self.assertIn("Found 1 error in 1 file", stdout)
E       AssertionError: 'Found 1 error in 1 file' not found in 'nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]\nnptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment\n/tmp/tmpmb3qsf1a/test_file.py:10: error: Argument 1 to "func" has incompatible type "str"; expected "ndarray[Any, Any]"  [arg-type]\nFound 2 errors in 2 files (checked 1 source file)\n'

tests/test_mypy.py:72: AssertionError
_______________________________________________________________ MyPyTest.test_mypy_knows_of_ndarray_methods ________________________________________________________________

self = <tests.test_mypy.MyPyTest testMethod=test_mypy_knows_of_ndarray_methods>

    def test_mypy_knows_of_ndarray_methods(self):
        # If MyPy knows of some arbitrary ndarray methods, we can assume that
        # code completion works.
        exit_code, stdout, stderr = _check_mypy_on_code(
            """
            from typing import Any
            from nptyping import NDArray


            arr: NDArray[Any, Any]
            arr.shape
            arr.size
            arr.sort
            arr.squeeze
            arr.transpose
        """
        )

>       self.assertEqual(0, exit_code, stdout)
E       AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  [misc]
E       nptyping/structure.pyi:37: note: Error code "misc" not covered by "type: ignore" comment
E       Found 1 error in 1 file (checked 1 source file)

tests/test_mypy.py:171: AssertionError
_________________________________________________________________ WheelTest.test_wheel_is_built_correctly __________________________________________________________________

self = <tests.test_wheel.WheelTest testMethod=test_wheel_is_built_correctly>

    def test_wheel_is_built_correctly(self):
        with working_dir(_ROOT):
>           subprocess.check_output(f"{sys.executable} -m invoke wheel", shell=True)

tests/test_wheel.py:83:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib64/python3.8/subprocess.py:415: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, capture_output = False, timeout = None, check = True, popenargs = ('/usr/bin/python3 -m invoke wheel',), kwargs = {'shell': True, 'stdout': -1}
process = <subprocess.Popen object at 0x7fe71e6e7af0>, stdout = b'', stderr = None, retcode = 1

    def run(*popenargs,
            input=None, capture_output=False, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.

        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.

        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.

        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.

        There is an optional argument "input", allowing you to
        pass bytes or a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.

        By default, all communication is in bytes, and therefore any "input" should
        be bytes, and the stdout and stderr will be bytes. If in text mode, any
        "input" should be a string, and stdout and stderr will be strings decoded
        according to locale encoding, or by "encoding" if set. Text mode is
        triggered by setting any of text, encoding, errors or universal_newlines.

        The other arguments are the same as for the Popen constructor.
        """
        if input is not None:
            if kwargs.get('stdin') is not None:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE

        if capture_output:
            if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
                raise ValueError('stdout and stderr arguments may not be used '
                                 'with capture_output.')
            kwargs['stdout'] = PIPE
            kwargs['stderr'] = PIPE

        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired as exc:
                process.kill()
                if _mswindows:
                    # Windows accumulates the output in a single blocking
                    # read() call run on child threads, with the timeout
                    # being done in a join() on those threads.  communicate()
                    # _after_ kill() is required to collect that and add it
                    # to the exception.
                    exc.stdout, exc.stderr = process.communicate()
                else:
                    # POSIX _communicate already populated the output so
                    # far into the TimeoutExpired exception.
                    process.wait()
                raise
            except:  # Including KeyboardInterrupt, communicate handled that.
                process.kill()
                # We don't call process.wait() as .__exit__ does that for us.
                raise
            retcode = process.poll()
            if check and retcode:
>               raise CalledProcessError(retcode, process.args,
                                         output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '/usr/bin/python3 -m invoke wheel' returned non-zero exit status 1.

/usr/lib64/python3.8/subprocess.py:516: CalledProcessError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
/usr/bin/python3: No module named invoke
============================================================================= warnings summary =============================================================================
../../../../../usr/lib64/python3.8/unittest/loader.py:66
  /usr/lib64/python3.8/unittest/loader.py:66: PytestCollectionWarning: cannot collect test class 'TestLoader' because it has a __init__ constructor (from: tests/test_wheel.py)
    class TestLoader(object):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_ndarray_with_any - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dty...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_ndarray_with_shape - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "d...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_ndarray_with_structure - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final clas...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_ndarrays_as_function_arguments - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from fi...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_ndarrays_as_variable_hints - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final ...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_nptyping_types - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_numpy_types - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dtype"  ...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_accepts_recarray_with_structure - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final cla...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_disapproves_ndarray_with_wrong_function_arguments - AssertionError: 'Found 1 error in 1 file' not found in 'nptyping/struc...
FAILED tests/test_mypy.py::MyPyTest::test_mypy_knows_of_ndarray_methods - AssertionError: 0 != 1 : nptyping/structure.pyi:37: error: Cannot inherit from final class "dty...
FAILED tests/test_wheel.py::WheelTest::test_wheel_is_built_correctly - subprocess.CalledProcessError: Command '/usr/bin/python3 -m invoke wheel' returned non-zero exit s...
================================================================ 11 failed, 80 passed, 1 warning in 41.36s =================================================================

Here is list of installed modules in build env

Package           Version
----------------- --------------
appdirs           1.4.4
attrs             22.1.0
beartype          0.10.4
Brlapi            0.8.3
build             0.8.0
codespell         2.1.0
cssselect         1.1.0
distro            1.7.0
extras            1.0.0
fixtures          4.0.0
gpg               1.17.1-unknown
iniconfig         1.1.1
libcomps          0.1.18
louis             3.22.0
lxml              4.9.1
mypy              0.971
mypy-extensions   0.4.3
nodeenv           1.7.0
numpy             1.23.1
packaging         21.3
pbr               5.9.0
pep517            0.12.0
pip               22.2.1
pluggy            1.0.0
py                1.11.0
PyGObject         3.42.2
pyparsing         3.0.9
pyright           1.1.268
pytest            7.1.2
python-dateutil   2.8.2
rpm               4.17.0
scour             0.38.2
setuptools        65.3.0
six               1.16.0
testtools         2.5.0
tomli             2.0.1
typeguard         2.13.3
typing_extensions 4.2.0
wheel             0.37.1

opened by kloczek 20

More documentation on the extent to which mypy enforces the types
Thanks for this impressive library. I feel it is really important for maintainable array-heavy codebases. I am running into the issue however that mypy does not seem to do anything with the type hints. I would expect it to complain if I gave a variable annotated with the type NDArray[Shape["2", "2"], UInt16] to a function with the signature func4(param1: NDArray[Shape["2", "3"], Float32]) -> ....

In the FAQ / documentation I could not find anywhere to what extent mypy actually enforces correct usage. Could you add a word on that?

Below I added a script that I would expect mypy to complain about. This is with Python 3.8, nptyping 2.2.0 and mypy 0.971.

from nptyping import NDArray, Shape, Float32, UInt16 import numpy as np var1: NDArray[Shape["2", "2"], UInt16] = np.array([[1,2], [3,4]]) def func1(param1: NDArray[Shape["2", "2"], UInt16]) -> NDArray[Shape["2", "2"], UInt16]: return param1 def func2(param1: NDArray[Shape["2", "2"], UInt16]) -> NDArray[Shape["2", "3"], UInt16]: return param1 def func3(param1: NDArray[Shape["2", "3"], UInt16]) -> NDArray[Shape["2", "3"], UInt16]: return param1 def func4(param1: NDArray[Shape["2", "3"], Float32]) -> NDArray[Shape["2", "3"], Float32]: return param1 func1(var1) func2(var1) func3(var1) func4(var1)

Output (I expected at least 3 issues):

❯ mypy test.py Success: no issues found in 1 source file
opened by anieuwland 2

Releases(v2.4.1)

v2.4.1(Nov 16, 2022)
Fixed compatibility with mypy==0.991.

Source code(tar.gz)
Source code(zip)
v2.4.0(Nov 14, 2022)
Added hint for pandas DataFrame.

Fixed bug for checking against a Structure where a different number of fields did not fail the check.

Changed nptyping.Int pointing to the more generic numpy.integer rather than numpy.int32.

Added support for Python 3.11 with the exception of pandas.DataFrame.

Source code(tar.gz)
Source code(zip)
v2.3.1(Aug 30, 2022)
Fixed mypy error of inheriting final dtype as of numpy==1.23.1.

Allowed for quotes in shape expressions to appease PyCharm.

Source code(tar.gz)
Source code(zip)
v2.3.0(Aug 28, 2022)
Added support for subarrays with shape expressions inside structure expressions.

Added support for wildcards in structure expressions.

Source code(tar.gz)
Source code(zip)
v2.2.0(Jun 26, 2022)
Added support for expressing "at least N dimensions".

Source code(tar.gz)
Source code(zip)
v2.1.3(Jun 19, 2022)
Fixed typing issue with Pyright/Pylance that caused the message: "Literal" is not a class

Fixed wrong error message when an invalid Structure was provided to NDArray.

Source code(tar.gz)
Source code(zip)
v2.1.2(Jun 8, 2022)
Fixed bug that caused MyPy to fail with the message: Value of type variable "_DType_co" of "ndarray" cannot be "floating[Any]"

Source code(tar.gz)
Source code(zip)
v2.1.1(Jun 1, 2022)
Fixed bug that numpy ndarrays were incorrectly instance checked against RecArray.

Source code(tar.gz)
Source code(zip)
v2.1.0(Jun 1, 2022)
Added Structure and "structure expressions" to support structured arrays.

Added RecArray.

Source code(tar.gz)
Source code(zip)
v2.0.1(Apr 28, 2022)
Thanks to Jasha10 for this release.

Added an improved default message for assert_isinstance.

Also some typos in README, in test_mypy.py and some style corrections.
Source code(tar.gz)
Source code(zip)
v2.0.0(Apr 7, 2022)
Changes since 1.4.4:

Changed the interface of NDArray into NDArray[SHAPE, DTYPE]

Added MyPy-acceptance (limited static type checking)

Added support for variables

Added support for labels and named dimensions

Added support for all numpy dtypes with NDArray

Added support for dynamic type checker: beartype

Added support for dynamic type checker: typeguard

Added autocompletion for all attributes of ndarray

Added CONTRIBUTING.md

Removed support for Python 3.5 and Python 3.6

Source code(tar.gz)
Source code(zip)
v2.0.0a2(Mar 27, 2022)
Changed the interface of NDArray: switched the order to NDArray[SHAPE, DTYPE] to be compatible to numpy.ndarray.pyi

Added autocompletion for all attributes of ndarray by changing the implementation of NDArray

Added CONTRIBUTING.md

Added support for dynamic type checker: beartype

Added support for dynamic type checker: typeguard

Source code(tar.gz)
Source code(zip)
v2.0.0a1(Mar 19, 2022)
Changed the interface of NDArray

Added MyPy-acceptance (limited static type checking)

Added support for variables

Added support for labels and named dimensions

Added support for all numpy dtypes with NDArray

Removed support for Python 3.5 and Python 3.6

Source code(tar.gz)
Source code(zip)
v1.4.4(Sep 10, 2021)
Fixed instance checks with 0d arrays.

Source code(tar.gz)
Source code(zip)
v1.4.3(Aug 5, 2021)
Fixed setup.py to exclude test(-resources) in the wheel.

Source code(tar.gz)
Source code(zip)
v1.4.2(May 8, 2021)
Fixed instance check that was incompatible with typish==1.9.2.

Source code(tar.gz)
Source code(zip)
v1.4.1(Mar 23, 2021)
Fixed instance checks of some types that did not properly respond to non-numpy types.

Fixed instance checks with nptyping.Object.

Fixed identities of NPTyping instances: NDArray[(3,), int] is NDArray[(3,), int].

Source code(tar.gz)
Source code(zip)
v1.4.0(Dec 23, 2020)
Added SubArrayType

Added StructuredType

Added support for unsigned integers with py_type.

Source code(tar.gz)
Source code(zip)
v1.3.0(Jul 21, 2020)
Added Bool

Added Datetime64

Added Timedelta64

Source code(tar.gz)
Source code(zip)
v1.1.0(May 31, 2020)
Removed Array

Added get_type

Added Int

Added UInt

Added Float

Added Unicode

Added Number

Added NPType

Source code(tar.gz)
Source code(zip)
v1.0.1(Apr 5, 2020)
Added a hash function to _NDArrayMeta.

Source code(tar.gz)
Source code(zip)
v1.0.0(Apr 4, 2020)
Added NDArray

Deprecated Array

Source code(tar.gz)
Source code(zip)

💡 Type hints for Numpy

Related tags

Overview

(❒) Installation

(❒) Usage

(❒) NDArray

(❒) Examples

(❒) Checking your instances

(❒) Finding the right annotation

(❒) Int

(❒) UInt

(❒) Float

(❒) Unicode

(❒) Bool

(❒) Complex128

(❒) Datetime64

(❒) Timedelta64

(❒) Object

(❒) StructuredType

(❒) SubArrayType

(❒) get_type

(❒) py_type

Comments

Releases(v2.4.1)

v2.4.1(Nov 16, 2022)

v2.4.0(Nov 14, 2022)

v2.3.1(Aug 30, 2022)

v2.3.0(Aug 28, 2022)

v2.2.0(Jun 26, 2022)

v2.1.3(Jun 19, 2022)

v2.1.2(Jun 8, 2022)

v2.1.1(Jun 1, 2022)

v2.1.0(Jun 1, 2022)

v2.0.1(Apr 28, 2022)

v2.0.0(Apr 7, 2022)

v2.0.0a2(Mar 27, 2022)

v2.0.0a1(Mar 19, 2022)

v1.4.4(Sep 10, 2021)

v1.4.3(Aug 5, 2021)

v1.4.2(May 8, 2021)

v1.4.1(Mar 23, 2021)

v1.4.0(Dec 23, 2020)

v1.3.0(Jul 21, 2020)

v1.1.0(May 31, 2020)

v1.0.1(Apr 5, 2020)

v1.0.0(Apr 4, 2020)

Owner

Ramon Hagenaars

Converting CPT to bert form for use

Taichi Course Homework Template

A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Official code for "Maximum Likelihood Training of Score-Based Diffusion Models", NeurIPS 2021 (spotlight)

Code for Deep Single-image Portrait Image Relighting

Implementation of Heterogeneous Graph Attention Network

Predicting Semantic Map Representations from Images with Pyramid Occupancy Networks

YOLOX-RMPOLY

Inferred Model-based Fuzzer

"Projelerle Yapay Zeka Ve Bilgisayarlı Görü" Kitabımın projeleri

Pytorch implementation of paper Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Github project for Attention-guided Temporal Coherent Video Object Matting.

Code to reproduce the results for Compositional Attention

Cognition-aware Cognate Detection

PyTorch implementation of SwAV (Swapping Assignments between Views)

Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

Code for our WACV 2022 paper "Hyper-Convolution Networks for Biomedical Image Segmentation"

Tacotron 2 - PyTorch implementation with faster-than-realtime inference

A Pytorch Implementation of a continuously rate adjustable learned image compression framework.