Old behavior: Throw TypeError, unhelpful error message, hard to debug underlying issue.

New behavior: Throw IOError with helpful error message indicating cause of failure.

Also decided to remove the .idea folder. Let me know if that should be kept for some reason.

Checklist:

• [x] Bump minor version to 2.3.0
• [x] Add appropriate pytests

If the SECRETS_FILE is not available it'll drop an error and trace:

[email protected]:/opt/app# export SECRETS_FILE=".secrets/stage.json"
[email protected]:/opt/app# python manage.py shell
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 325, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 57, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 107, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/app/project/__init__.py", line 8, in <module>
    if settings.DOGSTATSD_ENABLED:
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 57, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 107, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/app/project/settings.py", line 27, in <module>
    region_name=AWS_REGION,
  File "/usr/local/lib/python3.6/site-packages/confidential/secrets_manager.py", line 23, in __init__
    secrets = self.parse_secrets_file(secrets_file) if secrets_file else {}
  File "/usr/local/lib/python3.6/site-packages/confidential/secrets_manager.py", line 85, in parse_secrets_file
    config[key] = json.loads(decrypted_string)
  File "/usr/local/lib/python3.6/json/__init__.py", line 348, in loads
    'not {!r}'.format(s.__class__.__name__))
TypeError: the JSON object must be str, bytes or bytearray, not 'NoneType'

what

Addresses #12, an issue with accessing secrets in nested keys.

Example:

{
"secrets": 
	{
	"key": "secret:value"
	}
}

Currently, if you have a key inside an object in <env>.json that is

{
    'service': {   
              'api_key': 'secret:env/app/my_api_key',
              'base_url': 'url'
    }
}

It will not see the secret inside the service object.

However, if it's top level, it'll properly decrypt:

{
   'another_key':  'secret:env/app/my_api_key'
}

Steps to Reproduce

Take a working key from a top level of the json and place it inside the object like the first example. You should only see the the string stored in .json from where you store your secret paths.

what

Adds push to private pypi.

why

Confidential was apparently pushed manually to pypi using dvf's account. Until we get that access we should push to our internal pypi so that we can still iterate.

Helpful error messaging for AWS permissions error. Currently unhelpful response is:

manager.py", line 100, in decrypt_string
    result = json.loads(decrypted_string)
  File "/Users/dvf/.pyenv/versions/3.7.4/lib/python3.7/json/__init__.py", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

⭐️ Added test for {"SecretString": None} secrets manager response.

What does this do?

Adds an optional parameter (-p,--profile) to allow the user to pass in an alternative profile.

Why did you do this?

The current configuration limits confidential to the default profile. For users with multiple profiles, they may wish to use confidential with something beyond the default profile.

How did you test this change?

Ran with a known working profile, with a known non-working profile, with a profile that doesn't exist, and lastly, with no profile specified at all.

etc

I had thought of catching the exception raised when a profile is specified but could not be found (botocore.exceptions.ProfileNotFound) but found the error message to be descriptive enough on its own.

ex:

botocore.exceptions.ProfileNotFound: The config profile (candidco) could not be found

We introduced a regression in v.2.1.0 where we don't check for an integer. This should fix that. Also added some more tests.

Thanks to @jsundy for finding it.

what

fixes builds

why

they dont work, specifically because of deprecated options passed to black. also because of an unavailable flag being passed to poetry ([ValueError] Setting settings.virtualenvs.create does not exist ##[error]Process completed with exit code 1.)

https://github.com/candidco/confidential/runs/428287858

Secrets files can now be deeply-merged:

# defaults.json
{
  "django": {
    "debug": true,
    "database": {
      "hostname": "123",
      "port": 8000,
    }
  }
}

# overrides.json
{
  "django": {
    "database": {
      "hostname": "456",
    }
  }
}

Result:

{
  "django": {
    "debug": true,
    "database": {
      "hostname": "456",
      "port": 8000,
    }
  }
}

Also fixed a minor typo.

Bumps certifi from 2021.10.8 to 2022.12.7.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.