Hi! I've been using configobj a for while in my project. Due to other dependencies (twisted mainly), the stable release still depends on python v2.7. Recently, an issue was opened regarding utf8 encoded symbols in the config file: https://github.com/pazz/alot/issues/693 It turns out that this is due to configobj v5.0.x not behaving as previous versions when it comes to utf8 encoded symbols: Previously, those were always passed on as unicode strings, e.g. 🐜 became u'\U0001f41c'. Now, the same symbol is read as '\xf0\x9f\x90\x9c'. Obvously this makes trouble. Of course i can patch around this, but still, i think configobj should behave exactly the same as prevous versions here. Thanks!

I'm opening this PR early for review and commenting, this is intended to handle #32 and associated tickets. Will comment when things are ready for a merge.

When 5.0.0 is put on pypi, it's reasonable to expect that people would like to start using the github repo as a reference, and if that's the case, the README file should have the appropriate information to support that.

For your consideration...

If ConfigObj is instantiated with the on_change parameter set to a callable, on_change is called when changed data is detected in the root Section or any subsection. This is helpful for those cases where you may want a configuration change to trigger some other action.

Here is an example of how to use on_change:

>>> def my_on_change_handler(name, key, old_value, value):
>>>     print("Detected change in [{0}]:{1} from {2} to {3}".format(name, key, old_value, value)
>>> my_configobj = ConfigObj(on_change=my_on_change_handler)
>>> my_configobj["section"] = {}
Detected change in [section]:None from None to {}
>>> my_configobj["section"]["parameter"] = 1
Detected change in [section]:parameter from None to 1

*** I looked at the tests but couldn't get them to run, and I'm not sure which files are acutally used for the tests. Because of this, I didn't add in a test for on_change. If you could point me in the right direction, I'd happily write the tests as well.

Thanks! -Kris

This is an over-simplified version of zypper.conf:

## Configuration file for Zypper.

[main]

## Show repository alias instead of name.
# showAlias = false

## Columns to show in repository list printed by repos (lr) command by default.
# repoListColumns = Anr

[solver]

## Do not install soft dependencies (recommended packages)
# installRecommends = yes

[color]

## Whether to use colors
# useColors = never

When read with configobj, comments are assigned to sections in an awkward fashion:

>>> config = configobj.ConfigObj('zypper.conf')
>>> config.comments
{'color': ['',
  '## Do not install soft dependencies (recommended packages)',
  '# installRecommends = yes',
  ''],
 'main': [],
 'solver': ['',
  '## Show repository alias instead of name.',
  '# showAlias = false',
  '',
  '## Columns to show in repository list printed by repos (lr) command by default.',
  '# repoListColumns = Anr',
  '']}

As a consequence, if I run del config['solver'] I am deleting comments of main and preserving comments of solver, leading to a very confusing configuration file:

--- zypper.before.conf   2014-04-06 00:20:16.432652464 +0200
+++ zypper.after.conf    2014-04-06 00:26:16.569526812 +0200
@@ -2,14 +2,6 @@

 [main]

-## Show repository alias instead of name.
-# showAlias = false
-
-## Columns to show in repository list printed by repos (lr) command by default.
-# repoListColumns = Anr
-
-[solver]
-
 ## Do not install soft dependencies (recommended packages)
 # installRecommends = yes

Using configobj (5.0.6), I tried the following and got a DeprecationWarning:

Python 2.7.5 (default, Aug 18 2016, 15:58:25)
Type "copyright", "credits" or "license" for more information.

IPython 5.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from configobj import ConfigObj
   ...: from configobj import ConfigObjError
   ...:
   ...:     cfg = [str('thing  {"a": 1}')]
   ...:     c = ConfigObj(cfg)
   ...: except ConfigObjError as ex:
   ...:     print(ex.errors)
   ...:     for error in ex.errors:
   ...:         print(error.message)
   ...:
[ParseError('Invalid line (\'thing  {"a": 1}\') (matched as neither section nor keyword) at line 1.',)]
/home/yann/.virtualenvs/rprs_bootstrap/bin/ipython:7: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  from IPython import start_ipython
Invalid line ('thing  {"a": 1}') (matched as neither section nor keyword) at line 1.

Same with plain Python:

Python 2.7.5 (default, Aug 18 2016, 15:58:25)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from configobj import ConfigObj
>>> from configobj import ConfigObjError
>>>
>>> try:
...     cfg = [str('thing  {"a": 1}')]
...     c = ConfigObj(cfg)
... except ConfigObjError as ex:
...     print(ex.errors)
...     for error in ex.errors:
...         print(error.message)
...
[ParseError('Invalid line (\'thing  {"a": 1}\') (matched as neither section nor
keyword) at line 1.',)]
Invalid line ('thing  {"a": 1}') (matched as neither section nor keyword) at line 1.

Am I doing the catching wrong somehow?

I am not sure if this is a bug or just my not understanding. This is under Python 2.7, using v5.0.4 from pip. I can't seem to write out a CfgObj that has unicode strings. Here's a simple scenario:

File: 'in.ini'

folder="Arbeitsfläche"

code:

from  configobj import ConfigObj
cfg = ConfigObj("in.ini", encoding="UTF8")

print u"folder string[{0}] and repr [{1}]".format(cfg['folder'], repr(cfg['folder']))

cfg.filename = "out.ini"
cfg.write()

produces this:

folder string[Arbeitsfläche] and repr [u'Arbeitsfl\xe4che']
Traceback (most recent call last):
  File "stackoverflow.py", line 7, in <module>
    cfg.write()
  File "/usr/local/lib/python2.7/site-packages/configobj.py", line 2124, in write
    'ascii'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)

There doesn't seem to be any problem reading the file in. Also, if I change the input file to have the value "Desktop" instead of "Arbeitsfläche", no problem.

What I can't understand is why it wants to use the ascii codec, when I've explicitly set the UTF8 encoder.

Any help would be appreciated!

Version 5.0.0.

If one open a utf-8 encoded file ( with filename in variable fn below ), add an encoding parameter when opening the ConfigObj, the returned object does not convert the keys and values to unicode. The version 4.7.2 did..

In [6]: o = ConfigObj( infile = fn , encoding = "utf8" )

In [7]: t = o["root"]["title"]

In [8]: t Out[8]: 'Planl\xc3\xa6gning og data'

In [9]: isinstance( t , str ) Out[9]: True

In [10]: isinstance( t , unicode ) Out[10]: False

[]
a = b
[]
c = d

In [1]: from configobj import ConfigObj

In [2]: c = ConfigObj('./test.ini')
  File "<string>", line unknown
ConfigObjError: Parsing failed with several errors.
First error at line 1.

If I modify the first section name to be non-empty, the message is ok:

[s]
a = b
[]
c = d

In [1]: from configobj import ConfigObj

In [2]: c = ConfigObj('./test.ini')
  File "<string>", line unknown
ParseError: Invalid line ('[]') (matched as neither section nor keyword) at line 4.

Trac uses still supports Python 2.5 on the current stable release. easy_install ConfigObj in a Python 2.5 environment results in an installation of ConfigObj 5.0.1 with errors. I expect we'll start seeing this issue raised on our mailing list in the near future, as we've seen for a similar circumstance of one of our other dependencies, Babel.

In Trac we do some error-checking like was is shown here, so that an attempted installation in an unsupported Python environment will abort with a nice error message. Additionally, I don't see ConfigObj 4.7.2 being available on PyPI anymore. Is there anything additional we can do to make a Python 2.5 or earlier environment locate ConfigObj 4.7.2 and install that version?

$ easy_install ConfigObj
Searching for ConfigObj
Reading http://pypi.python.org/simple/ConfigObj/
Best match: configobj 5.0.1
Downloading https://pypi.python.org/packages/source/c/configobj/configobj-5.0.1.tar.gz#md5=c8a41fb55689e2d0e1ae53b6f1ab5e2e
Processing configobj-5.0.1.tar.gz
Writing /tmp/easy_install-3WzMHi/configobj-5.0.1/setup.cfg
Running configobj-5.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-3WzMHi/configobj-5.0.1/egg-dist-tmp-eeST9v
build/bdist.linux-x86_64/egg/configobj.py:1235: Warning: 'with' will become a reserved keyword in Python 2.6
  File "build/bdist.linux-x86_64/egg/configobj.py", line 1235
    with open(infile, 'rb') as h:
            ^
SyntaxError: invalid syntax

build/bdist.linux-x86_64/egg/validate.py:770: Warning: 'as' will become a reserved keyword in Python 2.6
  File "build/bdist.linux-x86_64/egg/validate.py", line 770
    except ValueError as e:
                       ^
SyntaxError: invalid syntax

zip_safe flag not set; analyzing archive contents...
/home/user/Workspace/t11500/teo-rjollos.git/py2.5/lib/python2.5/site-packages/configobj-5.0.1-py2.5.egg/configobj.py:1235: Warning: 'with' will become a reserved keyword in Python 2.6
  File "/home/user/Workspace/t11500/teo-rjollos.git/py2.5/lib/python2.5/site-packages/configobj-5.0.1-py2.5.egg/configobj.py", line 1235
    with open(infile, 'rb') as h:
            ^
SyntaxError: invalid syntax

/home/user/Workspace/t11500/teo-rjollos.git/py2.5/lib/python2.5/site-packages/configobj-5.0.1-py2.5.egg/validate.py:770: Warning: 'as' will become a reserved keyword in Python 2.6
  File "/home/user/Workspace/t11500/teo-rjollos.git/py2.5/lib/python2.5/site-packages/configobj-5.0.1-py2.5.egg/validate.py", line 770
    except ValueError as e:
                       ^
SyntaxError: invalid syntax

Adding configobj 5.0.1 to easy-install.pth file

Installed /home/user/Workspace/t11500/teo-rjollos.git/py2.5/lib/python2.5/site-packages/configobj-5.0.1-py2.5.egg
Processing dependencies for ConfigObj
Searching for six==1.5.2
Reading http://pypi.python.org/simple/six/
Best match: six 1.5.2
Downloading https://pypi.python.org/packages/source/s/six/six-1.5.2.tar.gz#md5=322b86d0c50a7d165c05600154cecc0a
Processing six-1.5.2.tar.gz
Writing /tmp/easy_install-StPhwj/six-1.5.2/setup.cfg
Running six-1.5.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-StPhwj/six-1.5.2/egg-dist-tmp-XyUsUa
no previously-included directories found matching 'documentation/_build'
zip_safe flag not set; analyzing archive contents...
six: module references __file__
Adding six 1.5.2 to easy-install.pth file

Installed /home/user/Workspace/t11500/teo-rjollos.git/py2.5/lib/python2.5/site-packages/six-1.5.2-py2.5.egg
Finished processing dependencies for ConfigObj

• Includes #173 so the CI can pass and to avoid merge conflicts.

• A little workaround is needed for 3.7 on Travis, see https://github.com/travis-ci/travis-ci/issues/9815.

• Also fixes this deprecation warning on Python 3.7 (with pytest):

================================================= warnings summary =================================================
/usr/local/lib/python3.7/site-packages/_pytest/config/findpaths.py:42: RemovedInPytest4Warning: [pytest] section in setup.cfg files is deprecated, use [tool:pytest] instead.
  config=config,

/private/tmp/configobj/src/configobj/__init__.py:546: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  elif isinstance(value, collections.Mapping) and not unrepr:

-- Docs: https://docs.pytest.org/en/latest/warnings.html

• And fixes this deprecation warning in pytest (with Python 2.7 and 3.7):

================================================= warnings summary =================================================
/usr/local/lib/python3.7/site-packages/_pytest/config/findpaths.py:42: RemovedInPytest4Warning: [pytest] section in setup.cfg files is deprecated, use [tool:pytest] instead.
  config=config,

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Bumps wheel from 0.29.0 to 0.38.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Currently, a member specification like the following:

keyword = float(min=0, max=1, default=0.5)

will pass validation when setting keyword = None.

I have therefore to check this afterwards in my code.

I was wondering, whether this could be implemented within ConfigObj.

More generally, there could be an option for allowing/disallowing None values, e.g.,

keyword = float(allow_none=False)

There are small typos in:

• docs/configobj.rst
• src/configobj/init.py

Fixes:

• Should read occurred rather than occured.
• Should read likelihood rather than likelyhood.

Semi-automated pull request generated by https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

Looks like latest pytest shows some warnings

============================================================================= warnings summary =============================================================================
tests/test_configobj.py:41
  /home/tkloczko/rpmbuild/BUILD/configobj-5.0.6/tests/test_configobj.py:41: DeprecationWarning: invalid escape sequence \s
    ws_chars = len(re.search('^(\s*)', first_content).group(1))

tests/test_configobj.py:51
  /home/tkloczko/rpmbuild/BUILD/configobj-5.0.6/tests/test_configobj.py:51: DeprecationWarning: invalid escape sequence \s
    return [re.sub('^\s{0,%s}' % ws_chars, '', line).encode('utf-8')

-- Docs: https://docs.pytest.org/en/stable/warnings.html
====================================================================== 71 passed, 2 warnings in 0.38s ======================================================================