Table.scan finally breaks appropriate exception handling

[gorlins]

Currently the finally block inside Table.scan attempts to close the scanner, but if this fails, any error raised inside the try block is masked. Simple example is CTRL-C during a scan:

conn = Connection(**kwargs)
table = conn.table('some_table')

for r, d in table.scan():
    # Let it go a few seconds, the CTRL-C
    # Results actually vary depending on exact timing of code
    pass

---------------------------------------------------------------------------
TProtocolException                        Traceback (most recent call last)
<ipython-input-4-5c7edf571612> in <module>()
----> 1 for r, d in table.scan():
      2     pass
      3

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/happybase/table.pyc in scan(self, row_start, row_stop, row_prefix, columns, filter, timestamp, include_timestamp, batch_size, scan_batching, limit, sorted_columns)
    411                         return  # scan has finished
    412         finally:
--> 413             self.connection.client.scannerClose(scan_id)
    414             logger.debug(
    415                 "Closed scanner (id=%d) on '%s' (%d returned, %d fetched)",

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/thriftpy/thrift.pyc in _req(self, _api, *args, **kwargs)
    196         # wait result only if non-oneway
    197         if not getattr(result_cls, "oneway"):
--> 198             return self._recv(_api)
    199
    200     def _send(self, _api, **kwargs):

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/thriftpy/thrift.pyc in _recv(self, _api)
    208
    209     def _recv(self, _api):
--> 210         fname, mtype, rseqid = self._iprot.read_message_begin()
    211         if mtype == TMessageType.EXCEPTION:
    212             x = TApplicationException()

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/thriftpy/protocol/compact.pyc in read_message_begin(self)
    145             raise TProtocolException(TProtocolException.BAD_VERSION,
    146                                      'Bad protocol id in the message: %d'
--> 147                                      % proto_id)
    148
    149         ver_type = self.read_ubyte()

TProtocolException: TProtocolException(type=4)

I'm not sure if the TProtocolException is meaningful in this case, but my calling code would want to capture the KeyboardInterrupt (or whatever the actual exception is).

In practice, this may only be an issue with interrupts and degenerate code.

Here is a degenerate case which masks the fact that the connection is no longer open:

n=0
for r, d in table.scan():
    n += 1
    if n > 500:
        conn.close()  # Next loop raises the same TProtocolException

I would suggest wrapping the close inside a try block or otherwise attempting to cleanup in a safer manner

[gorlins]

mistake in last bit, the latter example raises

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-5f7f1ade8493> in <module>()
----> 1 for r, d in table.scan():
      2     n += 1
      3     if n > 500:
      4         conn.close()
      5

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/happybase/table.pyc in scan(self, row_start, row_stop, row_prefix, columns, filter, timestamp, include_timestamp, batch_size, scan_batching, limit, sorted_columns)
    411                         return  # scan has finished
    412         finally:
--> 413             self.connection.client.scannerClose(scan_id)
    414             logger.debug(
    415                 "Closed scanner (id=%d) on '%s' (%d returned, %d fetched)",

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/thriftpy/thrift.pyc in _req(self, _api, *args, **kwargs)
    193         result_cls = getattr(self._service, _api + "_result")
    194
--> 195         self._send(_api, **kwargs)
    196         # wait result only if non-oneway
    197         if not getattr(result_cls, "oneway"):

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/thriftpy/thrift.pyc in _send(self, _api, **kwargs)
    205         args.write(self._oprot)
    206         self._oprot.write_message_end()
--> 207         self._oprot.trans.flush()
    208
    209     def _recv(self, _api):

thriftpy/transport/framed/cyframed.pyx in thriftpy.transport.framed.cyframed.TCyFramedTransport.flush (thriftpy/transport/framed/cyframed.c:2319)()

thriftpy/transport/framed/cyframed.pyx in thriftpy.transport.framed.cyframed.TCyFramedTransport.c_flush (thriftpy/transport/framed/cyframed.c:2053)()

/Users/sgorlin/miniconda/envs/dna-7f686269/lib/python2.7/site-packages/thriftpy/transport/socket.pyc in write(self, buff)
    127
    128     def write(self, buff):
--> 129         self.sock.sendall(buff)
    130
    131     def flush(self):

AttributeError: 'NoneType' object has no attribute 'sendall'

[wbolster]

yeah, it's tricky. dunno what the "right" solution is unfortunately...