[Feature Request]: Add support to trigger serving functions via gRPC protocol

[dhruva-jasper]

Feature Type

• Adding new functionality to MLRun

• Changing existing functionality in MLRun

• Removing existing functionality in MLRun

Problem Description

I wish I could invoke the deployed serving function via gRPC instead of HTTP.

Feature Description

The feature implementation would require running a gRPC server inside the serving container similar to described in https://grpc.io/docs/what-is-grpc/introduction/

Alternative Solutions

No alternate solutions.

Additional Context

Code examples are available here:
https://grpc.io/docs/languages/python/basics/

[liranbg]

This needs to be addressed first on Nuclio's side. @dhruva-jasper - may I ask the intention here of why using grpc over http? the former must have a pre-defined and agreed schema. the latter is more flexible, easier modify and extend.
What is the gain here? I suppose you are not familiar with Nuclio's (the serving function "engine") architecture here?

[dhruva-jasper]

We use gRPC for all communications within our company's systems. gRPC has some benefits over HTTP in some situations:

1. It's faster.

2. Allows clients to specify deadlines/timeouts.

3. Assume a system with multiple server instances running behind a Load Balancer. If a sequence of inference requests is needed to hit the same server instance, a GRPC stream will hold a single connection throughout the lifetime and hence ensure the requests are delivered to the same instance.

4. If the order of requests/responses needs to be preserved over the network, a GRPC stream will ensure that the server receives the requests in the same order as they were sent from the client.

Side note, KServe has a standardized schema for inference using gRPC.