encoding with 'idna' codec failed (UnicodeError: label empty or too long)

[Rahulvks]

client = Cloudant(serviceUsername, servicePassword, url=serviceURL) client.connect()

• Python 3
  `---------------------------------------------------------------------------
  UnicodeError Traceback (most recent call last)
  /Users/dsg281/anaconda/lib/python3.6/encodings/idna.py in encode(self, input, errors)
  164 if not (0 < len(label) < 64):
  --> 165 raise UnicodeError("label empty or too long")
  166 if len(labels[-1]) >= 64:

UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

UnicodeError Traceback (most recent call last)
in ()
1 # Use the Cloudant library to create a Cloudant client.
2 client = Cloudant(serviceUsername, servicePassword, url=serviceURL)
----> 3 client.connect()

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/client.py in connect(self)
140 self.r_session.headers.update(self._client_user_header)
141
--> 142 self.session_login()
143
144 # Utilize an event hook to append to the response message

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/client.py in session_login(self, user, passwd)
181 :param str auth_token: Authentication token used to connect to server.
182 """
--> 183 self.change_credentials(user=user, auth_token=passwd)
184
185 def change_credentials(self, user=None, auth_token=None):

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/client.py in change_credentials(self, user, auth_token)
191 """
192 self.r_session.set_credentials(user, auth_token)
--> 193 self.r_session.login()
194
195 def session_logout(self):

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/_client_session.py in login(self)
148 'POST',
149 self._session_url,
--> 150 data={'name': self._username, 'password': self._password},
151 )
152 resp.raise_for_status()

/Users/dsg281/anaconda/lib/python3.6/site-packages/cloudant/_client_session.py in request(self, method, url, **kwargs)
63 """
64 resp = super(ClientSession, self).request(
---> 65 method, url, timeout=self._timeout, **kwargs)
66
67 return resp

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
497
498 settings = self.merge_environment_settings(
--> 499 prep.url, proxies, stream, verify, cert
500 )
501

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/sessions.py in merge_environment_settings(self, url, proxies, stream, verify, cert)
670 # Set environment's proxies.
671 no_proxy = proxies.get('no_proxy') if proxies is not None else None
--> 672 env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
673 for (k, v) in env_proxies.items():
674 proxies.setdefault(k, v)

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/utils.py in get_environ_proxies(url, no_proxy)
690 :rtype: dict
691 """
--> 692 if should_bypass_proxies(url, no_proxy=no_proxy):
693 return {}
694 else:

/Users/dsg281/anaconda/lib/python3.6/site-packages/requests/utils.py in should_bypass_proxies(url, no_proxy)
674 with set_environ('no_proxy', no_proxy_arg):
675 try:
--> 676 bypass = proxy_bypass(netloc)
677 except (TypeError, socket.gaierror):
678 bypass = False

/Users/dsg281/anaconda/lib/python3.6/urllib/request.py in proxy_bypass(host)
2610 return proxy_bypass_environment(host, proxies)
2611 else:
-> 2612 return proxy_bypass_macosx_sysconf(host)
2613
2614 def getproxies():

/Users/dsg281/anaconda/lib/python3.6/urllib/request.py in proxy_bypass_macosx_sysconf(host)
2587 def proxy_bypass_macosx_sysconf(host):
2588 proxy_settings = _get_proxy_settings()
-> 2589 return _proxy_bypass_macosx_sysconf(host, proxy_settings)
2590
2591 def getproxies_macosx_sysconf():

/Users/dsg281/anaconda/lib/python3.6/urllib/request.py in _proxy_bypass_macosx_sysconf(host, proxy_settings)
2560 if hostIP is None:
2561 try:
-> 2562 hostIP = socket.gethostbyname(hostonly)
2563 hostIP = ip2num(hostIP)
2564 except OSError:

UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)`

[emlaver]

@Rahulvks Can you verify that your URL can be properly encoded as per the IDNA rules? (stackoverflow resource).

If the URL is properly encoded, can you share which python 3 version you are using? Our library has been tested with Python 2.7 and 3.5.

[Rahulvks]

@emlaver Am using Python 3.6.3 / conda 4.3.29
this is piece of code am using to connect DB to download the logs

serviceUsername = *****-bluemix"
    servicePassword = "****"
    serviceURL = "https:*******-bluemix.cloudant.com"
client = Cloudant(serviceUsername, servicePassword, url=serviceURL)
client.connect()  

[smithsz]

Our build passes for Python 3.6 (although not officially supported).

It seems your service URL cannot be properly encoded. Can you please contact Cloudant support via support@cloudant.com (this can also be done through your Cloudant dashboard if you prefer)? Be sure to include your full username and also reference this ticket. We'll pick it up from there and get back to you.

Thanks.

[Rahulvks]

@smithsz I have raised the request through Cloudant dashboard.

Thanks

[smithsz]

Thanks @Rahulvks. I can see your ticket 100174. I'll close this issue. We'll get back to you shortly via email.

[Rahulvks]

Hi @smithsz

It's working fine on Python 3.4.

I get error in Python 3.6.3

[markstur]

Get this error when the URL also contains the user/pass in py3 (works in py2). Would be nice if the client could strip the redundant stuff and just work.

[smithsz]

@markstur Can you share the client constructor please? I'm unable to reproduce this in Python 3.6.1. Does the password have any special characters etc? Thanks.

[markstur]

I had to mask user/pass and pull some code out to make a test.py, but I think this covers it. Notice that IBM Cloud generates long user/password. Also the provided URL contains user:pass@ which is very handy when it works, but w/ py3 needs to be stripped in favor of passing user/pass separately.

This output w/ py2 and py3 demonstrates that py2 correctly fails (it is not my real user/pass) but py3 hits the "...too long..." exception:

$ python test.py
401 Client Error: Unauthorized for url: https://a9a9a9a9-a9a9-a9a9-a9a9-a9a9a9a9a9a9-bluemix:a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9@d8a01891-e4d2-4102-b5f8-751fb735ce31-bluemix.cloudant.com/_session
$ python3 test.py
encoding with 'idna' codec failed (UnicodeError: label empty or too long)

Test code is here:

from cloudant.client import Cloudant

CLOUDANT_USERNAME = "a9a9a9a9-a9a9-a9a9-a9a9-a9a9a9a9a9a9-bluemix"
CLOUDANT_PASSWORD="a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9"
CLOUDANT_URL="https://a9a9a9a9-a9a9-a9a9-a9a9-a9a9a9a9a9a9-bluemix:a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9a9@d8a01891-e4d2-4102-b5f8-751fb735ce31-bluemix.cloudant.com"

try:
Cloudant(CLOUDANT_USERNAME,
CLOUDANT_PASSWORD,
url=CLOUDANT_URL,
connect=True)
except Exception as e:
print(e)

[ricellis]

https://bugs.python.org/issue32958

via

psf/requests#4168

[tomblench]

@markstur as you state, the workaround (in your case) for now is to remove the username and password from the URL.

As @ricellis pointed out, there is an underlying Python issue which is outside of our control, where long/IDNA domain names cause an error.

[DhanashreeChavan]

I see this issue in python2.7. Was this fixed ?

[ricellis]

@DhanashreeChavan as far as we know the underlying Python issue only applies to 3.6+. Are you able to provide the exact python 2.7 version, build and platform that is exhibiting the UnicodeError: label empty or too long error?

In any case the workaround until Python is fixed is to strip the user:pass@ user information from the URL e.g.

# URL with user information as provided in IBM Cloud credentials
# serviceURL = "https://example-abc123-bluemix:examplepassword@example-abc123-bluemix.cloudant.com"
# URL with user information removed to workaround Python bug
serviceURL = "https://example-abc123-bluemix.cloudant.com"
# Username and password from URL
serviceUsername = "example-abc123-bluemix"
servicePassword = "examplepassword"
client = Cloudant(serviceUsername, servicePassword, url=serviceURL)
client.connect()  

[DhanashreeChavan]

Hi Rich Ellis, I am trying to connect to AWS IoT using SigV4Utils to sign the url request, I am following the MQTT + Websocket protocol. SIgV4 utils is used to sign the url request. The url that gets generated is long with the signature, *here is the documentation:* https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html *The sample example* below shows how a signed url request will look: wss:// sample.iot.region.amazonaws.com/mqtt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=private/key/region/ioT/aws4_request&X-Amz-Date=45645509T8hh9929Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=dfjgaw7r74w9r472yt928t0u8uguherig8r29jfwoio89ujiuha *Python *considers below part of the url as one piece, since the method in idna splits url on '.' and hence the error that the label is too long. /mqtt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=private/key/region/ioT/aws4_request&X-Amz-Date=45645509T002929Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature= dfjgaw7r74w9r472yt928t0u8uguherig8r29jfwoio89ujiuha So, my question is How do I make this short ? So is there any other way to pass the the signed query parameters like using authorization headers ? my python version is* 2.7.10. * Thanks, Dhanashree Chavan

…

On Fri, May 11, 2018 at 1:26 AM, Rich Ellis ***@***.***> wrote: @DhanashreeChavan <https://github.com/DhanashreeChavan> as far as we know the underlying Python issue <https://bugs.python.org/issue32958> only applies to 3.6+. Are you able to provide the exact python 2.7 version, build and platform that is exhibiting the UnicodeError: label empty or too long error? In any case the workaround until Python is fixed is to strip the user:pass@ user information from the URL e.g. # URL with user information as provided in IBM Cloud credentials# serviceURL = ***@***.***"# URL with user information removed to workaround Python bug serviceURL = "https://example-abc123-bluemix.cloudant.com"# Username and password from URL serviceUsername = "example-abc123-bluemix" servicePassword = "examplepassword" client = Cloudant(serviceUsername, servicePassword, url=serviceURL) client.connect() — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#363 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFocpQgKUb3jHvdlmImGUf9HhRSUhvldks5txUsjgaJpZM4SK2Ca> .

[ricellis]

I'm failing to see the connection between AWS/MQTT and this python-cloudant library, which is a library for connecting to IBM Cloudant or Apache CouchDB from Python. If that's not what you're doing we can't really help.

[DhanashreeChavan]

I have nothing to do with python-cloudant library or BM Cloudant or Apache CouchDB . I am using Python2.7 library which has a ''idna.py" which is giving me the error.

…

On Fri, May 11, 2018 at 9:17 AM, Rich Ellis ***@***.***> wrote: I'm failing to see the connection between AWS/MQTT and this python-cloudant library, which is a library for connecting to IBM Cloudant or Apache CouchDB from Python. If that's not what you're doing we can't really help. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#363 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFocpYdQLHsp4iXQnUE81uJetDOwtkmGks5txbmigaJpZM4SK2Ca> .

[valdirsalustino]

Dear all,
Have you guys get any update on this issue? I am having the same issue with both python3 versions:
3.4.5, 3.5.1 and 3.6.4.

I am desperate to solve this :(

Regards,

[emlaver]

@valdirsalustino Have you tried the workaround above that strips the user/pass from the URL? #363 (comment)

We have merged a PR that removes the user/pass from the URL (#383). We'll update this issue when there's a new release.

[ricellis]

The 2.9.0 release has the code to remove user/pass from URL which will hopefully keep them short enough to workaround the bug in python.

[BeOleg]

python 3.7 with the same issue here

[ricellis]

@BeOleg there's nothing else we can do in python-cloudant to help with really long URLs, the underlying Python issue is still unresolved https://bugs.python.org/issue32958

[cjauvin]

I stumbled on this problem with an Heroku app, with which I have a very long Redis URL with auth (using Python 3.9). I solved it using urllib.parse.urlparse to decompose the elements into the redis.Redis constructor.

[mojito317]

@cjauvin thank you for sharing the workaround you have figured out!

[normenczepa0815]

yes i also can confirm, that urllib.parse is helpful, this is how it worked out for me

messageData = {'text': 'lambda with python webhook test'}
parseResult = urlparse(protocol+hook_url)
headers = {'Content-type': 'application/json'}
conn = http.client.HTTPSConnection(parseResult.netloc)
conn.request("POST", parseResult.path, json.dumps(messageData), headers)
response = conn.getresponse()
status = response.status
conn.close()
print("status:", status)

👍

[ErikBoesen]

When using Redis, I also found success by replacing my Redis(host) instantiation with Redis.from_url(host).