Hadoop is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, providing a highly-available service on top of a cluster of computers, each of which may be prone to failures
The hadoop Operator manages hadoop clusters deployed to Kubernetes, and provides a cloud-native way to submit Hadoop Yarn jobs.
hadoop operator relies on cert-manager, you must have cert-manager installed and configured in your Kubernetes cluster. cert-manager is a Kubernetes add-on that automates the management of certificates and is a requirement for this Operator to function correctly.
Installing cert-manager:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.3/cert-manager.yaml📖 Read more about installing cert-manager using kubectl apply and static manifests.
kubectl apply -k github.com/chriskery/hadoop-operator/manifests/defaultPlease refer to the quick-start.md for more information.
We thrive to build a welcoming and open community for anyone who wants to use the operator or contribute to it.