@zifeo added the ability for nested dataclass methods (sealed trait pureconfig functionality in Scala), updated README, incremented to 0.1.6, and created version.py which will provide version and get the latest from the pyproject.toml

Also, on my local, the following test always fails. How do I get it to pass?

tests/test_parser.py:186 (TestParser.test_missing_type)
self = <tests.test_parser.TestParser object at 0x7fe398c417c0>

    def test_missing_type(self) -> None:
    
        with pytest.raises(MissingTypeException):
>           loads("", Dict)
E           Failed: DID NOT RAISE <class 'dataconf.exceptions.MissingTypeException'>

I discovered this while trying to use dataconf to parse a YAML config file with one level of nesting.

For example, adding the following to test_parse.py:

    def test_yaml_nested(self) -> None:

        @dataclass
        class B:
            c: Text

        @dataclass
        class A:
            b: B

        conf = """
        b:
          c: test
        """
        assert loads(conf, A) == A(b=B(c="test"))

This test should pass, but results in the following test failure:

=================================================== FAILURES ====================================================
__________________________________________ TestParser.test_yaml_nested __________________________________________

self = <tests.test_parse.TestParser object at 0x7f440c099f90>

    def test_yaml_nested(self) -> None:
    
        @dataclass
        class B:
            c: Text
    
        @dataclass
        class A:
            b: B
    
        conf = """
        b:
          c: test
        """
>       assert loads(conf, A) == A(b=B(c="test"))

tests/test_parse.py:298: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
dataconf/main.py:102: in loads
    return string(s, clazz, **kwargs)
dataconf/main.py:82: in string
    return multi.string(s, **kwargs).on(clazz)
dataconf/main.py:67: in on
    return parse(conf, clazz, self.strict, **self.kwargs)
dataconf/main.py:17: in parse
    return utils.__parse(conf, clazz, "", strict, ignore_unexpected)
dataconf/utils.py:76: in __parse
    fs[f.name] = __parse(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

value = 'c: test', clazz = <class 'tests.test_parse.TestParser.test_yaml_nested.<locals>.B'>, path = '.b'
strict = True, ignore_unexpected = False

    def __parse(value: any, clazz: Type, path: str, strict: bool, ignore_unexpected: bool):
    
        if is_dataclass(clazz):
    
            if not isinstance(value, ConfigTree):
>               raise TypeConfigException(
                    f"expected type {clazz} at {path}, got {type(value)}"
                )
E               dataconf.exceptions.TypeConfigException: expected type <class 'tests.test_parse.TestParser.test_yaml_nested.<locals>.B'> at .b, got <class 'str'>

dataconf/utils.py:55: TypeConfigException
============================================ short test summary info ============================================
FAILED tests/test_parse.py::TestParser::test_yaml_nested - dataconf.exceptions.TypeConfigException: expected t...
========================================= 1 failed, 30 passed in 0.40s ==========================================

Changing the input to:

        conf = """
        b:
            {c: test}
        """

causes the test to pass, but I suspect this is because it is coincedentally also valid HOCON.

@zifeo looks like the python on github ci isn't updated for python 3.10 release. I just pushed a dev branch to test if everything would work but issues with python 3.10 being looked up as 3.1

In the rare case that there is more than one subtype that matches the given fields:

class AmbigImplBase:
    pass


@dataclass(init=True, repr=True)
class AmbigImplOne(AmbigImplBase):
    bar: str


@dataclass(init=True, repr=True)
class AmbigImplTwo(AmbigImplBase):
    bar: str

With config:

{
    a: Europe
    foo {
        bar: Baz
    }
}

Add support to select which subclass to use based on just the class name or the module path using the _type field.

{
    a: Europe
    foo {
        _type: AmbigImplTwo
        bar: Baz
    }
}

The module path is tail-matched, meaning a class with the fully qualified name of a.b.c.d.AmbigImplTwo can be matched by:

• a.b.c.d.AmbigImplTwo
• c.d.AmbigImplTwo
• d.AmbigImplTwo
• AmbigImplTwo

If the _type field matches don't narrow it down enough it will still throw an error.

It appears that Any can cause some problems.

from dataclasses import dataclass, field
import dataconf
from typing import Any, Dict, Text


@dataclass
class Test:
    name: Text
    items: Dict[Text, Any] = field(default_factory=dict)


config = """
name: letters
items: {
    a: d, 
    b: e, 
    c: f
}
"""

conf = dataconf.string(config, Test)

Traceback:

AttributeError                            Traceback (most recent call last)
/var/folders/kh/n0p_nl6d7sg0hfqljfwmlhcr0000gq/T/ipykernel_6851/3484491678.py in <module>
----> 1 conf = dataconf.string(config, Test)

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/main.py in string(s, clazz)
     65 
     66 def string(s: str, clazz):
---> 67     return multi.string(s).on(clazz)
     68 
     69 

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/main.py in on(self, clazz)
     50         for nxt in nxts:
     51             conf = ConfigTree.merge_configs(conf, nxt)
---> 52         return parse(conf, clazz)
     53 
     54 

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/main.py in parse(conf, clazz)
     12 def parse(conf: ConfigTree, clazz):
     13     try:
---> 14         return utils.__parse(conf, clazz, "")
     15     except pyparsing.ParseSyntaxException as e:
     16         raise MalformedConfigException(

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/utils.py in __parse(value, clazz, path)
     66 
     67             if not isinstance(val, _MISSING_TYPE):
---> 68                 fs[f.name] = __parse(val, f.type, f"{path}.{f.name}")
     69 
     70             elif is_optional(f.type):

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/utils.py in __parse(value, clazz, path)
    107         left, right = args
    108         try:
--> 109             return __parse(value, left if right is NoneType else right, path)
    110         except TypeConfigException:
    111             # cannot parse Optional

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/utils.py in __parse(value, clazz, path)
    101             )
    102         if value is not None:
--> 103             return {k: __parse(v, args[1], f"{path}.{k}") for k, v in value.items()}
    104         return None
    105 

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/utils.py in <dictcomp>(.0)
    101             )
    102         if value is not None:
--> 103             return {k: __parse(v, args[1], f"{path}.{k}") for k, v in value.items()}
    104         return None
    105 

~/opt/anaconda3/lib/python3.8/site-packages/dataconf/utils.py in __parse(value, clazz, path)
    155 
    156     child_failures = []
--> 157     for child_clazz in sorted(clazz.__subclasses__(), key=lambda c: c.__name__):
    158         if is_dataclass(child_clazz):
    159             try:

AttributeError: '_SpecialForm' object has no attribute '__subclasses__'

With Python 3.10, we encounter an import error related to collections.

Similar issues: https://github.com/rmartin16/qbittorrent-api/issues/45 https://github.com/carbonblack/cbapi-python/issues/298

[ImportError]
cannot import name 'Mapping' from 'collections' (/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/collections/__init__.py)

From this issue, they found an issue with

*rmartin Feb 2021
AttrDict finally broke with Python 3.10 since abstract base classes can no
longer be imported from collections but should use collections.abc instead.
Since AttrDict is abandoned, I've consolidated the code here for future use.
AttrMap and AttrDefault are left for posterity but commented out.
"""

from abc import ABCMeta
from abc import abstractmethod
from re import match as re_match

try:  # python 3
    from collections.abc import Mapping
    from collections.abc import MutableMapping
    from collections.abc import Sequence
except ImportError:  # python 2
    from collections import Mapping
    from collections import MutableMapping
    from collections import Sequence

I can not pass below sample codes from README.md.

@dataclass
class Example:
    hello: string
    world: string

os.environ['DC_WORLD'] = 'monde'

print(
    dataconf
    .multi
    .url('https://raw.githubusercontent.com/zifeo/dataconf/master/confs/simple.hocon')
    .env('DC')
    .on(Example)
)
# Example(hello='bonjour',world='monde')

First error is about below line. hello: string =>no string

After replace string to str, I got below error msgs.

" dataconf ../../../opt/anaconda3/envs/test/lib/python3.8/site-packages/dataconf/main.py:89: in on conf = ConfigTree.merge_configs(conf, nxt) ../../../opt/anaconda3/envs/test/lib/python3.8/site-packages/pyhocon/config_tree.py:61: in merge_configs a[key] = value E TypeError: list indices must be integers or slices, not str "

Description

mypy report error when importing dataconf

Expected Behavior

mypy checks pass without any error without the need to use ignore_missing_imports = True option

Actual Behavior

error: Skipping analyzing "dataconf": module is installed, but missing library stubs or py.typed marker  [import]

In your parsing operation, you dont allow for optional parameters that are list or dicts.

    if origin is list:
        if len(args) != 1:
            raise MissingTypeException("excepted list with type information: List[?]")
        return [__parse(v, args[0], f"{path}[]") for v in value]

    if origin is dict:
        if len(args) != 2:
            raise MissingTypeException(
                "excepted dict with type information: Dict[?, ?]"
            )
        return {k: __parse(v, args[1], f"{path}.{k}") for k, v in value.items()}

Consider the following dataclass:

        @dataclass
        class Base:
            data_root: Text
            pipeline_name: Text
            data_type: Text
            production: bool
            conn: Optional[Conn] = None
            data_split: Optional[Dict[Text, int]] = None
            tfx_root: Optional[Text] = None
            metadata_root: Optional[Text] = None
            beam_args: Optional[List[Text]] = field(
                default_factory=lambda: ["--direct_running_mode=multi_processing", "--direct_num_workers=0"]
            )

An optional parameter with None shouldn't return a failure. We use HOCON configs all the time with case classes in JVM languages and optional parameter parsing should be able to be by passed if None is specified.

This allows to disable the strict mode (off by default when using .env as a source). Not sure this is the best solution, I will let open for a while to see if a better solution can be designed for #35.

Fix https://github.com/zifeo/dataconf/issues/35.

I replicated the behavior of using seal trait case classes with Scala and pureconfig. It may not be the cleanest but it is working. I have some Todo notes in case you have any ideas.

from __future__ import annotations

import dataconf

from dataclasses import dataclass


@dataclass
class Model():
    token: str
    
dataconf.env("TEST_", Model)

Error:

Traceback (most recent call last):
  File "main.py", line 13, in <module>
    dataconf.env("ITBUTKA_", Model)
  File "/.venv/lib/python3.9/site-packages/dataconf/main.py", line 64, in env
    return multi.env(prefix, **kwargs).on(clazz)
  File "/.venv/lib/python3.9/site-packages/dataconf/main.py", line 57, in on
    return parse(conf, clazz, self.strict, **self.kwargs)
  File "/.venv/lib/python3.9/site-packages/dataconf/main.py", line 16, in parse
    return utils.__parse(conf, clazz, "", strict, ignore_unexpected)
  File "/.venv/lib/python3.9/site-packages/dataconf/utils.py", line 68, in __parse
    fs[f.name] = __parse(
  File "/.venv/lib/python3.9/site-packages/dataconf/utils.py", line 186, in __parse
    for child_clazz in sorted(clazz.__subclasses__(), key=lambda c: c.__name__):
AttributeError: 'str' object has no attribute '__subclasses__'

Process finished with exit code 1