Added secure gRPC server option to kserve #3605
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: cbalbera-imaige The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: cbalbera-imaige <150292789+cbalbera-imaige@users.noreply.github.com>
| @@ -20,7 +21,7 @@ | |||
| import socket | |||
| import sys | |||
| from multiprocessing import Process | |||
| from typing import Any, Callable, Dict, List, Optional, Union | |||
| from typing import Dict, List, Optional, Union, Callable, Any, IO | |||
There was a problem hiding this comment.
Can keep in order - thanks!
| ): | ||
|
|
||
| def __init__(self, http_port: int = args.http_port, | ||
| grpc_port: int = args.grpc_port, |
There was a problem hiding this comment.
ah, good catch, I'll fix that
| self._grpc_server = GRPCServer( | ||
| grpc_port, self.dataplane, self.model_repository_extension | ||
| ) | ||
| if self.secure_grpc_server: |
There was a problem hiding this comment.
Maybe we can move these validations to a util file?
certs_utils?
There was a problem hiding this comment.
this would be fine. could put them in the regular utils file (kserve/utils/utils.py) if you would like - let me know.
There was a problem hiding this comment.
That's sounds good, but let wait to hear from others.
There was a problem hiding this comment.
did not realize there's already a creds_utils file, going to put them in there
| @@ -1,6 +1,6 @@ | |||
| [tool.poetry] | |||
| name = "kserve" | |||
| version = "0.12.0" | |||
| version = "0.13.0" | |||
There was a problem hiding this comment.
ah, good catch - have been updating this locally so the package that depends on my updates to kserve pulls my version rather than the latest (0.12.0) from pypi. will revert.
| listen_addr = f'[::]:{self._port}' | ||
|
|
||
| if self._secure_server: | ||
| server_credentials = grpc_ssl_server_credentials( |
There was a problem hiding this comment.
can we have a test for this?
There was a problem hiding this comment.
I hadn't seen any existing tests for the server start functionality (neither gRPC or REST), so was unsure what the scope of testing was supposed to be here. can write a few if that would be best, let me know
There was a problem hiding this comment.
One simple test could be done by creating the certificates and spawn a ssl protect server and try to make any request.
There was a problem hiding this comment.
Going to test this - as well as the helpers - by creating a folder with a set of dummy SSL certs for testing purposes. Just noting this for posterity (& will document as well) because they will eventually expire and require refreshing
|
You might need to sign your commit. |
What this PR does / why we need it:
This PR givs KServe the ability to use gRPC's built-in support for ssl/tls to both authenticate requests & encrypt data.
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #
Type of changes:
New feature (non-breaking change which adds functionality)
Feature/Issue validation/testing:
Please describe the tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.
Special notes for your reviewer:
Checklist:
Release note: