Need Updates for python 3.10, using collections.Iterable as an example

[ed9w2in6]

System Info:

Python 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:25:18) [GCC 9.4.0]
pyhive 0.6.4    | pyhd8ed1ab_0   | conda-forge

Logs:

~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/hive.py in execute(self, operation, parameters, **kwargs)
    442             sql = operation
    443         else:
--> 444             sql = operation % _escaper.escape_args(parameters)
    445 
    446         self._reset_state()

~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/common.py in escape_args(self, parameters)
    211             return {k: self.escape_item(v) for k, v in parameters.items()}
    212         elif isinstance(parameters, (list, tuple)):
--> 213             return tuple(self.escape_item(x) for x in parameters)
    214         else:
    215             raise exc.ProgrammingError("Unsupported param format: {}".format(parameters))

~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/common.py in <genexpr>(.0)
    211             return {k: self.escape_item(v) for k, v in parameters.items()}
    212         elif isinstance(parameters, (list, tuple)):
--> 213             return tuple(self.escape_item(x) for x in parameters)
    214         else:
    215             raise exc.ProgrammingError("Unsupported param format: {}".format(parameters))

~/anaconda3/envs/****/lib/python3.10/site-packages/pyhive/common.py in escape_item(self, item)
    246         elif isinstance(item, basestring):
    247             return self.escape_string(item)
--> 248         elif isinstance(item, collections.Iterable):
    249             return self.escape_sequence(item)
    250         elif isinstance(item, datetime.datetime):

AttributeError: module 'collections' has no attribute 'Iterable'

This is because collections.Iterable is deprecated and we should use collections.abc.Iterable instead.

In fact, there has always been an DeprecationWarning in place:

DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

Are there any plans for a code review for such updates?

[quickcoffee]

Can confirm that an environment with Python 3.10 breaks the code, which is working in Python 3.7.11

[rwarren]

As mentioned, the issue with collections.Iterable usage is that that has been moved to collections.abc.Iterable since python 3.3.

collections.Iterable is used in the default ParamEscaper here:
https://github.com/dropbox/PyHive/blob/master/pyhive/common.py#L248

I should really just PR this (not sure they are being accepted since unsupported?), but for those in need of a quick fix, you can just monkey patch this particular problem with this:

collections.Iterable = collections.abc.Iterable

I know PyHive is unsupported. I'm just using the escaping functions to supplement some pyathena usage, where the escaping is currently extremely thin.

[ed9w2in6]

[Off topic] I saw some recommended https://github.com/cloudera/impyla as an alternative. It seems that they provided a similar interface (DB API). I have not tried using it though.

[sscarduzio]

Why is this not fixed for so long?

[mdeshmu]

Addressed in #451