[WIP/RFC] allow polling on a provided fd

[bfredl]

This allows a plugin to poll for readability and/or writability on a specified fd.
here is an example integrating with a zeromq socket.

However, I'm not actually sure we want to spawn a greenlet for every poll event, because this event doesn't imply that there actually is useful data that needs to be handled. ( For instance, zmq uses the same fd also for internal communication with its worker threads and so at times there could be lots of spurious poll events) Perhaps its better to spawn a greenlet only when there is message, for instance:

     def _on_poll(self, fd, read, write):
        while self.sock.getsockopt(zmq.EVENTS) & zmq.POLLIN:
            self.vim.session.schedule(lambda: self.on_msg( self.sock.recv()))

One might even want back to yield back to an existing greenlet that is blocking on receiving (or sending) on the fd.
Or it could be that spawning new greenlets is so cheap, that its not worth bothering avoiding it, I don't know.

[tarruda]

However, I'm not actually sure we want to spawn a greenlet for every poll event, because this event doesn't imply that there actually is useful data that needs to be handled

I've been thinking that it would be best to add automatic greenlet creation as a session filter that is disabled by default and enabled with something like: greenlet_vim = vim.with_filter(GreenletWrap). Plugins running in the host would enable greenlets with per-method decorators

[bfredl]

A constraint to consider might be making it convenient for plugins to use greenlets to handle messaging to other sources. Right now in my ipython plugin I do this to wait for a reply asynchrously:

def waitfor(self, msg_id):
    gr = greenlet.getcurrent()
    self.pending_shell_msgs[msg_id] = gr
    return gr.parent.switch()

and then in an event callback:

def on_shell_msg(self, m):
    msg_id = m['parent_header']['msg_id']
    try:
        handler = self.pending_shell_msgs.pop(msg_id)
    except KeyError:
        debug('unexpected shell msg: %r', m)
        return
    handler.parent = greenlet.getcurrent()
    handler.switch(m)

Which "just happens" to work given how the python client library uses greenlets. Perhaps it would be a cleaner abstraction if there was a session.schedule_greenlet method (i e switch to this greenlet as soon as the event loop is idle)

[tarruda]

Which "just happens" to work given how the python client library uses greenlets. Perhaps it would be a cleaner abstraction if there was a session.schedule_greenlet method (i e switch to this greenlet as soon as the event loop is idle)

greenlet wrapping can be implemented as decorators, for example:

@neovim.plugin
class Plugin(object):
    def __init__(self, nvim):
        self.nvim = nvim
        self.greenlet_nvim = nvim.with_hook(GreenletWrap)

    @neovim.greenlet
    def handler1(self):
        # runs in a greenlet
        print(self.greenlet_nvim.eval('1'))

    def handler2(self):
        # doesnt run in a greenlet
        def cb(value):
            print value
         self.nvim.eval('1', cb)

I dont think the greenlet hook is even required, session.request can simply detect if it is being called inside a greenlet and act accordingly(eg: receive a callback as last argument if not inside a greenlet)

[bfredl]

Yeah, I think it would be cleaner API if one doesn't have to use two different nvim objects. A possible interface could be:

nvim.eval('1')  ==> yield from this greenlet (if running inside one)
nvim.eval('1', cb) ==> use callback
nvim.eval('1', None) ==> ignore reply

[bfredl]

Updated, now it has simpler vim.session.poll_fd(fd, on_readable=callback) api, and a greenlet is not automatically spawned.

[bfredl]

Updated. Goes back to spawn a greenlet per default, to behave like a an ordinary (async) neovim command handler or async_call callback (i e api calls can be done in the poll callback, principle of least surprise).