Add Kubernetes job runner (#347) #349
Conversation
|
Since kubernetes is dropping docker, can this allow us to use the kubernetes cluster's container runtime? (Mainly interested in containerd). |
|
@DrissiReda, yes. The Kubernetes Job abstraction is at a higher level than Kubernetes' Container Runtime Interface (CRI), which is where Docker plugs into Kubernetes if you use Docker. And the k3s cluster I used for development and testing of this pull request already doesn't use Docker. |
jaredjennings
force-pushed
the
k8s-job-runner
branch
2 times, most recently
from
6738e9b
to
9cc6cbe
Compare
|
I've separated better the changes that add support to the entrypoint for trusting CA certs. There are separate trusts for Elasticsearch and for outgoing web services. I have not made a separate pull request for those, but I can. They are possibly of more general interest than the Kubernetes job runner. |
jaredjennings
force-pushed
the
k8s-job-runner
branch
from
9cc6cbe
to
15ed398
Compare
|
OK. I went ahead and split the Elasticsearch authentication and CA cert stuff out into #362. I gave the commits here a hard look, and decided that the story of progression toward functionality they told wasn't something that needed to be preserved. So I collapsed the commits into one, for easier review. The "cruft picked up along the way," which I mentioned at the top of the thread, is gone. |
|
@jaredjennings any updates on this? I assume that at this point this feature is still not implemented? |
|
@tl-Bruno-Braga, here it sits. Works for me, though I haven't taken it to production yet. I've asked a couple of times on the Discord about it. |
This commits my current progress in re-creating the docker-compose pipeline for PeekabooAV in Kubernetes. That includes deployments, services, and hard-coded config files for each step in the pipeline, modeled after what was done in the [pipeline](/sett17/peekabooav-installer/tree/pipeline). The yamls for cortex, and the set-up job, are included, although cortex does currently not work inside of Kubernetes, due to the missing docker runner. There is an open [PR](/TheHive-Project/Cortex/pull/349) and corresponding issue. Except for above mentioned cortex, the pipeline is fully functional. Meaning one can send an email to the postfix_tx deployment, which is then sent to the postfix_rx deployment and then processed by rspamd and Peekaboo. This was tested and developed with microk8s and a single node.
This commits my current progress in re-creating the docker-compose pipeline for PeekabooAV in Kubernetes. That includes deployments, services, and hard-coded config files for each step in the pipeline, modeled after what was done in the [pipeline](/Sett17/PeekabooAV-Installer/tree/pipeline). The yamls for cortex, and the set-up job, are included, although cortex does currently not work inside of Kubernetes, due to the missing docker runner. There is an open [PR](/TheHive-Project/Cortex/pull/349) and corresponding issue. Except for above mentioned cortex, the pipeline is fully functional. Meaning one can send an email to the postfix_tx deployment, which is then sent to the postfix_rx deployment and then processed by rspamd and Peekaboo. This was tested and developed with microk8s and a single node.
jaredjennings
force-pushed
the
k8s-job-runner
branch
from
15ed398
to
009c997
Compare
|
I know it's been a while - but I just wanted to say thank you @jaredjennings for creating this PR. I'm disheartened that this hasn't been merged yet, given Kubernetes' popularity. For now, I'm having to maintain a fork just to run Cortex on a managed Kubernetes cluster. |
jaredjennings
force-pushed
the
k8s-job-runner
branch
from
25a727f
to
afb3e63
Compare
jaredjennings
force-pushed
the
k8s-job-runner
branch
from
afb3e63
to
16e795a
Compare
These changes implement a job runner that makes Kubernetes API calls to run Dockerized neurons as Kubernetes Jobs.
There's also a bit of cruft picked up along the way, like enabling Dockerized Cortex to connect to Elasticsearch using HTTPS. Please look out for rookie mistakes: this is my first Scala code.