I'm Joe Zack, software engineer who works with streaming analytics using a lot of Kafka, Elasticsearch, and Kubernetes. I have a podcast with a couple friends where we talk about all things Software Development, if you like this presentation, then you'll probably like Coding Blocks!
In this presentation we are going to build a local development environment for the Elastic Stack that will be easy to work with in Kubernetes.
- Why you should consider running Elastic Cloud on Kubernetes (ECK)
- See what you can do with it
The code for this up on Github, there is a link in the description. Of course, I tried to make it as easy as possible to get started. I tried to be really careful when I did this, because sometimes presentations or videos or blogs will boil something down to a really simple prototype. As soon as you try to make any changes, the whole thing falls apart because the prototype was designed to be demonstratable, not for production.
I want this demonstration to be different. I want to give you something you can take and make your own. In fact we'll be creating and evolving several different envronments throughout this talk.
Elastic Cloud on Kubernetes is a essentially a plugin for Kubernetes that greatly simplifies the orchestration of Elastic bits. Allgedly it's what Elastic uses as the engine under the covers for Elastic Cloud, minus the snazzy UI.
- Benefits of Kubernetes
- Software Defined Architecture
- Portable
- Self-Documenting
- Experimentation
- Elastic Cloud
- Complexity
- Performance
- Start Docker
- Settings > Kubernetes
- Settings > Resources
- k9s -n all
-
Open-Source- CLI tool for interacting w/ Kubernetes
-
Resource oriented like Kubernetes
- See all the :abilities with :alias
- Filter w/ /
-
Supports most of the kubectl commands
- AutoComplete
- Easy filtering
- Log Viewer
-
Polls resources
- Great for presentations!
- Pods
- Smallest Unit that Kubernetes cares about
- Hosts one or more containers
- Can be associated with PersistentVolumes
- StatefulSets
- Define one or more pods
- Designed to maintain state via PersistentVolumes
- Services
- Abstraction for DNS, Load Balancers, external resources etc
- Associates name/IP with one or more pods
- Set of Custom Resource Definitions that extend the kubernetes API
- Adds definitions for new types of resources: Elasticsearch, Kibana etc
- kubectl create -f https://download.elastic.co/downloads/eck/2.3.0/crds.yaml
- Now Kubernetes speaks our language!
- :elasticsearch
kubectl apply -f https://download.elastic.co/downloads/eck/2.3.0/operator.yaml- Creates an operator in :namespace elastic-system
- An operator is like a person on your team that is responsible for managing your Kubernetes cluster
- It will take your Elasticsearch and Kibana resource definitions and translate it to Kubernetes resources
- Operator documentation
kubectl apply -f k8s/3_cluster/elasticsearch.yamlkubectl apply -f k8s/3_cluster/kibana.yaml- :elasticsearch, :statefulset, :pods, :persistentvolume, :service
- Clusters are closed to the outside world by default
- We need to open up a port, k9s makes it easy to port-forward
- And it uses https
- And it generates a password for authentication
- Get the password with :secrets
- How can we use this in our apps?
kubectl apply -f k8s/3_cluster/app.yaml
curl \
--cacert /certificates/ca.crt \
--key /certificates/tls.key \
--cert /certificates/tls.crt \
-u elastic:$ELASTIC_PASSWORD \
https://quickstart-es-http:9200/*/_search?pretty- Adding nodes
- Delete nodes
- What happens to replicas?
- Rolling upgrade
- Downgrade not supported!
- Different types of services
- Kibana
- Setting up beats
- See everything we've set up today:
kubectl -n default get all
- Presentation Materials
- Helm Templates
- Certificates
- Custom applications
- Kubernetes
- Elastic Cloud on Kubernetes
- Optional Recommendations