Use the external push gRPC protocol for external scaler => interceptor communication

[arschles]

We currently use a simple HTTP/JSON protocol for the scaler to communicate with the interceptor. Because this protocol is just request/response, it has limitations on latency that can cause very slow scale up times, which are especially noticeable when scaling up from 0 replicas.

It would be helpful to have some kind of two way protocol rather than what we have now. The external push scaler provides just this. We should take this and use it in the scaler.

Use-Case

As mentioned above, this would be useful to speed up scale-up (and down) latencies. This may help with #60 as well, but that's not the primary goal here.

Specification

[benjaminhuo]

Is there any plan to implement the external push interface?

[arschles]

@benjaminhuo we would likely implement the external push interface or something similar but simpler (because my guess is we won't need all the complexity of the current external push interface)

[benjaminhuo]

Looking forward to seeing this added!

[arschles]

Same here @benjaminhuo. @ajanth97 and I are going to come up with a design in the near future and the current plan is he's going to implement it. This will probably all start in earnest after the big #206 is merged.

Let me know if you'd like to get involved - you're certainly welcome to!

[benjaminhuo]

@arschles Yes sure, I'm happy to get involved!

[arschles]

ok. let's start by collaborating on a design. I've started a document here. feel free to leave comments etc...

Note: If you haven't used hackmd before, you have to login with your GitHub (or it supports some other SSO types I think) account before you can edit or comment.

[ajanth97]

Awesome

[benjaminhuo]

ok. let's start by collaborating on a design. I've started a document here. feel free to leave comments etc...

Note: If you haven't used hackmd before, you have to login with your GitHub (or it supports some other SSO types I think) account before you can edit or comment.

@arschles That's a great place to start!
@tpiperatgod @wanjunlei you can also take a look at this doc https://hackmd.io/ttpSY5KQQtGIqbAzyKex3g if you're interested

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[arschles]

This is ongoing. @ajanth97 is working on it

[tomkerkhove]

Any update on this? Is it still relevant?

[ajanth97]

Hi Sorry, I will work on this soon

[tomkerkhove]

Great, thanks!

[arschles]

Is it still relevant?

FWIW, I think it still is 😄