There are many reasons that the natural inclination to look at the cloud for execution of Kubernetes data science analytics workloads may not be the best first choice for some organizations but CNCF still shows the way forward (Interactive version) towards structuring both infrastructure and applications to embrace on-premise environments that enable either eventual or simultaneous scale out to cloud based services.
Note: The Big Picture
As part of our overall concepts, we are putting together a "big picture" overview of the OSS spectrum of available projects for a Kubernetes based end-to-end Machine Learning distributed processing continuum from Ingestion to Analytics to Visualization.
- https://github.com/cognonic/awesome-AI-kubernetes covers the distributed platform for the machine learning processing elements
- (this effort) https://github.com/cognonic/on-premise-kubernetes covers the often neglected on-premise Kubernetes configurations in the range between development laptop and full cloud build-out. This will also include some basic Kubernetes and Docker container bits.
- https://github.com/cognonic/The-Cognitive-and-ML-List covers the wide variety of specific machine learning type libraries including a detailed ONNX focus
- https://github.com/cognonic/The-React-List covers the mysterious common usage in the Data Science field of JavaScript for the presentation and visualization of the hard earned results of all that distributed processing power
- https://github.com/cognonic/Augmented-Reality-Hololens covers how that hard earned visualization really deserves to move fully into this millenium as real-time Augmented Reality and Virtual Reality (i.e. Mixed Reality) walk-throughs of your data to fully embrace the potential of our modern tools.
- https://github.com/cognonic/The-Big-Tech-Players-GitHub-List covers many of the major companies on GitHub which can indicate the "backing" power of certain OSS projects, a concept more of interest to enterprise consumers of OSS.
Comments, any general feedback, and additional content are always welcome and easily acheived by just opening an issue.
See Awesome AI on Kubernetes for some specific information on data science workloads on Kubernetes.
Here, we will be focusing on Kubernetes infrastructure and architecture to support on-premise orchestration of Awesome AI on Kubernetes types of workloads.
Some Reference Info:
- Kubernetes
- Docker
- CNCF
- CNCF Projects
- awesome-kubernetes
- Awesome Helm
- Awesome Operators
- Awesome Docker
- Awesome Machine Learning Operations
- Service Mesh
- Rook
- Ambassador
- Argo
- Airflow
- Banzai Pipeline
- Pachyderm
- Seldon Core
- Brigade
- ksonnet
more to come... stay tuned... keep watching here...