client = Cloudant(serviceUsername, servicePassword, url=serviceURL) client.connect()

• Python 3 `--------------------------------------------------------------------------- UnicodeError Traceback (most recent call last) /Users/dsg281/anaconda/lib/python3.6/encodings/idna.py in encode(self, input, errors) 164 if not (0 < len(label) < 64): --> 165 raise UnicodeError("label empty or too long") 166 if len(labels[-1]) >= 64:

UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

UnicodeError Traceback (most recent call last) in () 1 # Use the Cloudant library to create a Cloudant client. 2 client = Cloudant(serviceUsername, servicePassword, url=serviceURL) ----> 3 client.connect()

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/client.py in connect(self) 140 self.r_session.headers.update(self._client_user_header) 141 --> 142 self.session_login() 143 144 # Utilize an event hook to append to the response message

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/client.py in session_login(self, user, passwd) 181 :param str auth_token: Authentication token used to connect to server. 182 """ --> 183 self.change_credentials(user=user, auth_token=passwd) 184 185 def change_credentials(self, user=None, auth_token=None):

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/client.py in change_credentials(self, user, auth_token) 191 """ 192 self.r_session.set_credentials(user, auth_token) --> 193 self.r_session.login() 194 195 def session_logout(self):

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/_client_session.py in login(self) 148 'POST', 149 self._session_url, --> 150 data={'name': self._username, 'password': self._password}, 151 ) 152 resp.raise_for_status()

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/_client_session.py in request(self, method, url, **kwargs) 63 """ 64 resp = super(ClientSession, self).request( ---> 65 method, url, timeout=self._timeout, **kwargs) 66 67 return resp

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json) 497 498 settings = self.merge_environment_settings( --> 499 prep.url, proxies, stream, verify, cert 500 ) 501

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/sessions.py in merge_environment_settings(self, url, proxies, stream, verify, cert) 670 # Set environment's proxies. 671 no_proxy = proxies.get('no_proxy') if proxies is not None else None --> 672 env_proxies = get_environ_proxies(url, no_proxy=no_proxy) 673 for (k, v) in env_proxies.items(): 674 proxies.setdefault(k, v)

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/utils.py in get_environ_proxies(url, no_proxy) 690 :rtype: dict 691 """ --> 692 if should_bypass_proxies(url, no_proxy=no_proxy): 693 return {} 694 else:

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/utils.py in should_bypass_proxies(url, no_proxy) 674 with set_environ('no_proxy', no_proxy_arg): 675 try: --> 676 bypass = proxy_bypass(netloc) 677 except (TypeError, socket.gaierror): 678 bypass = False

/Users/dsg281/anaconda/lib/python3.6/urllib/request.py in proxy_bypass(host) 2610 return proxy_bypass_environment(host, proxies) 2611 else: -> 2612 return proxy_bypass_macosx_sysconf(host) 2613 2614 def getproxies():

/Users/dsg281/anaconda/lib/python3.6/urllib/request.py in proxy_bypass_macosx_sysconf(host) 2587 def proxy_bypass_macosx_sysconf(host): 2588 proxy_settings = _get_proxy_settings() -> 2589 return _proxy_bypass_macosx_sysconf(host, proxy_settings) 2590 2591 def getproxies_macosx_sysconf():

/Users/dsg281/anaconda/lib/python3.6/urllib/request.py in _proxy_bypass_macosx_sysconf(host, proxy_settings) 2560 if hostIP is None: 2561 try: -> 2562 hostIP = socket.gethostbyname(hostonly) 2563 hostIP = ip2num(hostIP) 2564 except OSError:

UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)`

Is there a way to pass a base64 auth token to Cloudant() for creating a new client? Right now I only see passing it username and password. It calls the attribute "auth_token" but it doesn't seem to work unless I use the explicit password.

Code I'm trying:

    with Cloudant(config['cloudant_user'],config['cloudant_auth'], account=config['cloudant_user']) as client:

Reponse:

Traceback (most recent call last):
  File "./dirscan.py", line 818, in <module>
    main(sys.argv[1:])
  File "./dirscan.py", line 152, in main
    load_config(config['passed_config_file'])
  File "./dirscan.py", line 390, in load_config
    with Cloudant(config['cloudant_user'], config['cloudant_auth'], account=config['cloudant_user']) as client:
AttributeError: __exit__

Related question: I see that I can generate an API key pair, but I don't see any calls in the API to set what permissions those keys have. (Such as setting their permissions on a specific database once they're created.) Am I just missing it?

Checklist

• [x] Tick to sign-off your agreement to the Developer Certificate of Origin (DCO) 1.1
• [x] Added tests for code changes or test/build only changes
• [x] Updated the change log file (CHANGES.md|CHANGELOG.md) or test/build only changes
• [x] Completed the PR template below:

Description

Make Result's iteration faster paginating by startkey instead of skip.

Fixes #436

Approach

As per #436 , paginating by skip could be super-slow. This patch fix the problem by using startkey so every iteration run in constant time.

Schema & API Changes

• "No change"

Security and Privacy

• "No change"

Testing

• No new tests because current ones are enough

Monitoring and Logging

• "No change"

Bug Description

Document.fetch() fails when it uses its default json decoder and simplejson is in the environment.

Steps to reproduce and code sample to demonstrate the issue

To reproduce, just run a simple document fetch like this having simplejson package in the environment:

from cloudant import cloudant
from cloudant.database import CloudantDatabase
from cloudant.document import Document

def fetch_by_id(usr, pwd,url,db_name,id):
  with cloudant(user=usr, passwd=pwd, url=url) as client:
    db = CloudantDatabase(client, db_name)
    doc = Document(db, id)
    if doc.exists():
        doc.fetch()

Expected behavior

The full document should be retrieved by fetch function, instead when simplejson is reachable in the environment, it will raise the following error:

File \"/usr/local/lib/python3.6/site-packages/simplejson/__init__.py\", line 535, in loads
return cls(encoding=encoding, **kw).decode(s)
TypeError: __init__() got an unexpected keyword argument 'encoding'

What actually happened

Starting in this line: self.update(resp.json(cls=self.decoder)):

1. json() function from requests.models is called with argument cls and value json.JSONDecoder, as self.decoder was initialized by default when no decoder is passed to Document
2. a JSON decoder is imported selectively in requests.compat module: when simplejson is present, then the json decoder implementation comes from that package
3. the JSON decoder from simplejson package is used to load the contents of the response
4. now, argument cls is a json.JSONDecoder (see point 1), and it does not expect an argument called encoding, thus a TypeError is raised.

Environment details

Version affected: 2.10.0 Note that in version 2.9.0, cls=self.decoder was not passed to resp.json().

• Cloudant (python-cloudant) version 2.0.0
• Python version 2.7.10

I can POST a list of keys as JSON data to the requests library, but on the same DB using the Cloudant python library, the call fails with an unsupported media type error. Is it not specifying the content-type properly?

Below is the code. old_method() uses requests, new_method() uses python-cloudant.

        def old_method():
            # Uses requests library
            myurl = 'https://{0}.cloudant.com/{1}/_all_docs?include_docs=true'.format(
                self.config['cloudant_account'],
                self.scandb.metadata()['db_name']
            )
            my_header = {'Content-Type': 'application/json'}
            try:
                r = requests.post(
                    myurl,
                    headers = my_header,
                    auth = (config['cloudant_user'],config['cloudant_auth']),
                    data = json.dumps({ 'keys': self.file_doc_batch.keys() })
                )
                result = r.json()
            except Exception as e:
                logging.fatal("Unable to execute HTTP POST: {0}".format(e))
                sys.exit("Unable to execute HTTP POST: {0}".format(e))
            return result

        def new_method():
            # Uses Cloudant python library
            result = self.scandb.all_docs(
                include_docs = True,
                keys = self.file_doc_batch.keys()
            )
            return result

old_method() returns JSON from the _all_docs endpoint with the expected 'rows' array for each key included.
new_method() errors with HTTP 415:

  File "./dirscan.py", line 758, in new_method
    keys = self.file_doc_batch.keys()
  File "/Library/Python/2.7/site-packages/cloudant/database.py", line 372, in all_docs
    resp.raise_for_status()
  File "/Library/Python/2.7/site-packages/requests/models.py", line 844, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 415 Client Error: Unsupported Media Type for url: https://bradwbonn.cloudant.com/scandb-1464114444/_all_docs?include_docs=true

…ocument if it is available in the database

Checklist

• [v] Tick to sign-off your agreement to the Developer Certificate of Origin (DCO) 1.1
• [v] Added tests for code changes or test/build only changes
• [v] Updated the change log file (CHANGES.md|CHANGELOG.md) or test/build only changes
• [v] Completed the PR template below:

Description

This adds a simple way to retrieve a previous revision of a document.

Approach

Changed cloudant.document.Document.fetch to take keyword arguments and pass them through to the raw session object.

Changed cloudant.document.Document.exists to take keyword arguments and pass them through to the raw session object when making the HEAD request.

Added a new method cloudant.document.Document.get_revision which creates a new Document instance, checks if the data for the revision exists, and returns a dict holding the document data for that revision.

Schema & API Changes

cloudant.document.Document.exists and cloudant.document.Document.fetch now can also take keyword arguments.

Security and Privacy

No change

Testing

Modified tests.unit.document_tests.test_document_exists_raises_httperror to account for the changed API

Monitoring and Logging

No change

What

Fix the Document get_attachment method so that it now correctly creates text and binary files as exepected as well as returns text, binary, and json content appropriately.

How

• Make the attachment_type method argument optional.
• Add logic to figure out what type of content should be returned by the method as well as what kind of file should be created if a write_to argument is provided.

Testing

• Add a test to verify getting a text attachment and writing it to a file.
• Add a test to verify getting a json attachment and writing it to a file.
• Add a test to verify getting a binary attachment and writing it to a file.

Reviewers

reviewer: @emlaver reviewer: @ricellis

Issues

• #102

What:

Add Python 3 compatibility.

Why:

Python 3, now in it's fifth revision, has obtained enough critical mass in the developer community to make supporting it worthwhile. Bluemix supports Python 3 Cloud Foundry applications as a built in community runtime and makes attaching a Cloudant service an easy and desirable option. Lack of a Python 3 compatible API inhibits adoption of the Cloudant service.

How:

A new private module, _2to3.py, was added containing compatibility methods. py2 or py3 unique types and methods were replaced throughout the code base with equivalents from the compatibility module. Python 2 is treated as the special case given it's unique semantics are "deprecated" and py3 and onward is the new normal (all hail the BDFL).

Tests:

All test cases under the tests/ directory were updated to function under py2 and py3 with similar compatibility methods. All unit tests (tests/unit/*) were run and pass with py2.7 and py3.5 using a local couchdb instance. The 85 cloudant specific test cases have not yet been run to my knowledge. The Travis CI environment only executes a py2.7 test -- a py3.5 test should be added before a final merge is complete (I'm not sure how to do it).

Issues: #23

reviewer @alfinkel reviewer @evansde77

What:

Remove the enforcement of the fields as a required parameter for making a query.

Why:

Cloudant supports not passing in the fields when making a query. If no fields are provided then all the fields of each document are returned. This PR makes sure that python-cloudant mimics that behavior.

How:

• In CloudantDatabase.get_query_result() the fields parameter is checked for existence before instantiating the Query.
• In Query.__call__ I removed the check for fields being None or empty.

Tests:

• The query callable tests that test for fields being empty or None were removed.
• The tests were updated to treat fields as a **kwargs argument.
• Test that calling db.get_query_result handles a nonexistent fields parameter.

Issues:

• #86

reviewer @alfinkel reviewer @ricellis

Bug Description

We are using the cloudant-python lib to handle our connections with our database. Randomly the code raises a ConnectionResetError.

1. Steps to reproduce and the simplest code sample possible to demonstrate the issue

I could not create a external scenario to reproduce it, but I will outline the context of the error scenario.

I have an code that make iterative calls to our couchdb. Every loop make the same call to an specific view, but with different keys.

Sometimes, the request is not sent and the program return the error below.

At the first I thought that could be something related with our proxy - Haproxy. But, I have tested the script connecting directly with the database and the error continued. On the next, without the HAPROXY, I replaced the request call (from the python-cloudant) to a simple request (from request package) and the error disappeared, so I think that the error is related with this lib. In addition, my couchdb server does not log any request call, so, its is one more info that suggest that the error is on the client side.

2. What you expected to happen

The script should make the requests and finish without any error.

3. What actually happened

Eventually in some requests the script raise the error below.

2021-02-10 11:16:14 ERROR: Exception occurred on parse_candidate
Traceback (most recent call last):
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/util/retry.py", line 531, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/common/decorators/log_exception.py", line 8, in _log_exception_and_ignore
    return function(*args, **kwargs)
  File "/home/danilo/Workspace/stakeholders-extraction/sl_stakeholders_extraction/tse/parsers/candidature_parser.py", line 57, in parse_candidate
    candidate_info['CurrentParty'] = get_party(candidature['SiglaPartido'])
  File "/home/danilo/Workspace/stakeholders-extraction/sl_stakeholders_extraction/tse/parsers/__init__.py", line 5, in get_party
    return get_party_by_initials(party_initials)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/common/utils/parties.py", line 16, in get_party_by_initials
    return _get_party(initials, PARTIES_VIEW_NAME, 'by_initials')
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/common/utils/parties.py", line 21, in _get_party
    party = _get_party_from_view(ddoc_id, view_name, key)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/common/utils/parties.py", line 28, in _get_party_from_view
    view_result = PARTIES_DB.get_view_result(view_name=view_name,
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/cloudant/result.py", line 427, in all
    return self[:]
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/cloudant/result.py", line 223, in __getitem__
    data = self._ref(**self.options)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/cloudant/view.py", line 236, in __call__
    resp = get_docs(self._r_session,
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/cloudant/_common_util.py", line 261, in get_docs
    resp = r_session.get(url, headers=headers, params=f_params)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/cloudant/_client_session.py", line 130, in request
    return super(BasicSession, self).request(
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/cloudant/_client_session.py", line 65, in request
    resp = super(ClientSession, self).request(
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/home/danilo/Workspace/stakeholders-extraction/env/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

Environment details

• Cloudant

  2.14.0

• Python version

  3.9.1

• [x] Tick to sign-off your agreement to the Developer Certificate of Origin (DCO) 1.1
• [x] You have added tests for any code changes
• [x] You have updated the CHANGES.md
• [x] You have completed the PR template below:

What

I added 2 things, I needed both of them at the same time so I ended up doing a single PR instead of 2, sorry for that.

• Move create_query_index from CloudantDatabase to CouchDatabase as the JSON indexes for Mango queries are now part of CouchDB.
• Add a feature to allow checking if a document exists in a DB without having to resort to try-catch blocks, using standard dict syntax:

_id = 'julia30'
doc_exists = _id in my_database

if doc_exists:
    print('doc with _id julia30 exists in DB')

How

• create_query_index: move the method from one class to the other. Move the tests from Cloudant specific to CouchDB.
• Add a dict override for __contains__ that checks in the remote DB if the document exists.

Testing

Create a CouchDatabase object and call the create_query_index the same way you would for a Cloudant database.

Issues

I didn't create an issue before solving it.