<!doctype html><script src="eveal.js"></script>
by [traceypooh](https://twitter.com/tracey_pooh)
June 2017
ESC for slide overview
- Docker - ~15min
- Kubernetes - ~15min
- Proposing more docker - ~15min
- Open Issues / Brainstorm / Discusion / Questions - ~15min
https://www.docker.com/docker-mac
- mac: hypervisor (pthread per vCPU)
- linux: lxc
- windows: hyper-v
- AWS
- GCE / Google Compute Platform
- images readonly
- containers copy-on-write layers; writes over first relevant layer
-
4-6x times # of docker servers as KVM VMs
-
Google search, gmail, Docs, etc -- containers
-
docker just abstracts the OS kernel -- not the entire device (VMs)
-
local dev environments exactly like live server
-
test/run multiple dev environments & versions -- all on same host
-
reliable, rebuildable entire recipe for re/building
docker it!
docker it!
docker it!
docker it!
slap a docker sticker on it!
run xenial:
docker run --rm -it ubuntu:latest bashrun zesty:
docker run --rm -it ubuntu:rolling bashthis talk:
git clone https://github.com/traceypooh/docker-talker; open docker-talker/index.html-
docker best practices
-
misc
open-source system for automating deployment, scaling and management of containerized applications that was originally designed by Google and donated to the Cloud Native Computing Foundation.
It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts"
-
pods of containers
- can share resources among whole group
-
each pod runs on on physical machine
- so all up / all down on machine, by design
-
can share folders and folders w/ tcp/ip connections between them
-
BFFs:
- scheduling
- load / resource balancing / high availability
- versioning and rolling rollout
- ... with rollback
- secrets
- can "prefer" placement, when avail
- eg: derive on primary if resources avail
- saves network copying
- else derive like normal, remote, copy item down first..
- eg: derive on primary if resources avail
- https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ *
- https://coreos.com/kubernetes/docs/latest/services.html
* and Jeopardy! answer ;-)
- using kubeadm tool - 1 master; 3 nodes
- kallr
watch -dc -n 1 kubectl get deploy,rs,pods,services --all-namespaces
- kube-dev1
- kubeadm-setup-master
- kwebnode
- kwebnode-scale
- kube-dev2..4
- kubeadm-setup-node
- kproxy
- /etc/hosts localhost archive.org
- dashboard http://localhost:81/ui
- deployments
- scale
- secrets
- nodes
- conditions
- pods
- logs
- deployments
-
https://archive.org/details/camels (not located)
-
https://archive.org/details/prelinger (located recently, OK)
-
sudo docker exec -it [CONTAINER] zshsupervisorctl statusps fauxxx
- All item changes in containers
- only single dir visible / writable - same model:
- if item dir local:
- r/w shared volume
- process
- inside IA, item dir remote:
- rsync item over
- process
- copy back incremental / final updates (see Act III)
- if you're in Russia:
- item public?
- [YES] /compress/ pulldown
- [NO] ia jake tool!
- ia jake tool when done! (S3 back in)
- item public?
- workers managed by Kubernetes
- allows ramp up/down based on webnodes
- or other processing
- could derive on primary when avail
- allows ramp up/down based on webnodes
- datanode container
- for item serving - readonly
- light processing, eg:
- page/image transforms
- zipview
- ZOTF
/petabox/docker/supervisord.conf.datanode- nginx
- nginx-priv
- php-cgi
- php-cgi-priv
- locatord
- cron
- live-stats
- apache2
- nullmailer
- rsync
- nginx
| ia600600* | Container | What? | When? |
|---|---|---|---|
| datanode | items serving | always | |
| workerA | itemA modifier, eg: fixer task | on demand | |
| workerB | itemB modifier | on demand | |
| " | " | " | |
| worker1 | deriving | as resources avail | |
| worker2 | deriving | as resources avail | |
| " | " | " | |
| webnode | archive.org | lottery / as needed |
* random primary
-
faster OS upgrades
-
decouple baseline OS and apps
-
split:
- item serving (eg: S3)
- item processing (eg: EC2)
-
more efficient use of IA resources
-
industry std
- <redacted> dumping PHX bespoke datacenter for Google Compute Platform
-
we have "< MVP" working now!
-
Could allow remote (non-IA) modifiers to upload logs w/ new/changed item files
-
Container Linux / CoreOS (based on Gentoo Linux (compiled per machine, v. efficient))
- docker / rkt builtin
- super thin / small
- security paranoid; auto-updates
- auto-versions
- huge K8 contributors
- then.. [/0 .. /35] raw dirs?
-
Lambda?
-
logs / stats
- industry std is to send/collect to one big resource
- can drop log rotates / cron daily push/pull
- can be more realtime
- eg: logstash, rsyslog, et al
- industry std is to send/collect to one big resource
-
local OCR?
-
UDP locate?
- have MDAPI / redis do/handle?