kodekloud-assignment

This repo contains solutions for two tasks,

1. Deploy a docker swarm stack that would run a container with privileged mode. For more details check here.

2. Run a ubuntu based docker container which can inturn run docker inside (docker in docker mode). For more details check here.

Directory Structure

.
├── README.md
├── docker-compose.yml
├── task-one
│   ├── Dockerfile
│   ├── README.md
│   ├── arch_diagram.png
│   ├── container-handler.sh
│   └── docker-compose.yml
└── task-two
    ├── Dockerfile
    ├── README.md
    ├── apparmor-profiles
    │   ├── audit-all-writes
    │   └── deny-all-writes
    ├── arch_diagram.png
    ├── container-handler.sh
    └── docker-compose.yml

• docker-compose.yml - Used to deploy docker swarm stack with two services (task-one and task-two) in a docker swarm cluster.
• task-one
  • Dockerfile - Copies container-handler script into the filesystem of the container and execute it within the container
  • README.md - Contains instructions to build image and deploy docker swarm stack in swarm cluster
  • arch_diagram.png - Architecture diagram for task one docker swarm stack
  • container-handler.sh - Creates two sibiling containers, one with priviliged mode enabled. Also waits for docker stack rm <stack_name> signal and kills the sibiling when SIGTERM is received
  • docker-compose.yml - It has one service and uses an existing network to deploy container-handler container to bring up sibiling containers, one with priviliged mode enabled
• task-two
  • Dockerfile - Copies apparmor profiles and container-handler script into the filesystem of the container and execute the script within container
  • README.md - Contains instructions to build image and deploy docker swarm stack in swarm cluster
  • apparmor-profiles
    • audit-all-writes - This profile will audit all the writes (i.e creating dirs/files) happening within the container and logs it to the kernel log
    • deny-all-writes - This profile will deny all the writes (i.e creating dirs/files) happening within the container and logs it to the kernel log
  • arch_diagram.png - Architecture diagram for task two docker swarm stack
  • container-handler.sh - Creates one sibiling upper containers with priviliged mode, updates/installs apparmor packages, copies apparmor profiles to /etc/apparmo.d/ in the upper container, creates two child inner containers with apparmor profiles applied. Also waits for docker stack rm <stack_name> signal and kills the sibiling upper container (including child inner containers) when SIGTERM is received
  • docker-compose.yml - It has one service and uses an existing network to deploy container-handler container to bring up sibiling upper container with priviliged mode enabled and two child inner containers

Deploy task-one and task-two to Swarm Cluster

In this section, we will be deploying both the tasks task-one and task-two in swarm cluster using a single docker-compose file. We can deploy this stack to any cloud platform or even a local machine (with docker installed). For this deployment I'm going to use AWS Cloud platform.

Pre-requisites

Docker Images

• Task one service image - You can either build image using Dockerfile (if you also want to use custom image name and tag, make sure to change the same in docker-compose.yml) or you can use my docker image in pramodhayyappan/kk-task-one-container-handler in dockerhub

  # To build image with custom name and tag. By default, it uses the latest tag
  docker build -f task-one/Dockerfile task-one -t pramodhayyappan/kk-task-one-container-handler:<tag name>

• Task two service image - You can either build image using Dockerfile (if you also want to use custom image name, make sure to change the same image name in docker-compose.yml) or you can use my docker image in pramodhayyappan/kk-task-two-container-handler in dockerhub

  # To build image with custom name and tag. By default, it uses the latest tag
  docker build -f task-two/Dockerfile task-two -t pramodhayyappan/kk-task-two-container-handler:<tag name>

Cloud Infra deployment

• Assuming that you have already set up your AWS Credentials in your local machine, create a Security Group(SG) to allow communication between nodes and make note of the SG name

  aws ec2 create-security-group --group-name swarm-cluster-sg --description "swarm cluster security group" --vpc-id <vpc-id>

• Create ingress rules with the protocols and ports mentioned in the offical doc

  aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol tcp --port 22 --cidr <cidr ip>
  aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol tcp --port 2377 --cidr <cidr ip>
  aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol tcp --port 7946 --cidr <cidr ip>
  aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol udp --port 7946 --cidr <cidr ip>
  aws ec2 authorize-security-group-ingress --group-id <sg name from previous step> --protocol udp --port 4789 --cidr <cidr ip>

• Create user-data.sh script file with below content

  #!/bin/bash
  sudo apt-get update
  sudo apt-get install \
      ca-certificates \
      curl \
      gnupg \
      lsb-release -y
  sudo mkdir -p /etc/apt/keyrings
  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
  echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
  sudo apt-get update
  sudo apt-get install docker-ce docker-ce-cli containerd.io  docker-compose-plugin git -y
  sudo groupadd docker
  sudo usermod -aG docker $USER
  newgrp docker

• Create two EC2 instances for Docker Swarm cluster

  aws ec2 run-instances --image-id <ami-id> --count 1 --instance-type t2.xlarge --key-name <key-pair-name> --security-group-ids <sg name from first step> --subnet-id <subnet-id> --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=ManagerNode}]' --user-data file://user-data.sh
  
  aws ec2 run-instances --image-id <ami-id> --count 1 --instance-type t2.xlarge --key-name <key-pair-name> --security-group-ids <sg name from first step> --subnet-id <subnet-id> --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=WorkerNode1}]' --user-data file://user-data.sh

• The infrastructure for docker swarm cluster with docker installed is ready

Deploy docker stack

Stack deployment

• Create a swarm cluster by executing the below command in manager node

  docker swarm init --advertise-addr <MANAGER-IP>

• Join worker node to the cluster by using the output of the init command or use join-token to generate the join command. The command will look like

  docker swarm join-token worker
  docker swarm join --token <token> <ip>:<port>

• Clone this repo to the manager node

  git clone https://github.com/pa/kodekloud-assignment.git
  
  cd kodekloud-assignment

• Deploy docker stack

  docker stack deploy --compose-file docker-compose.yml <stack-name>

• Some useful commands to list stack, services, container and inspect container

  # To list docker stacks
  docker stack ls
  
  # To list services deployed by stack
  docker stack services <stack-name>
  
  # To list docker containers
  docker ps
  
  # to inspect docker container
  docker inspect <container id or name>
  
  # runs a new command on a running container
  docker exec -it <container id or name> <bash or shell>
  
  # to remove a docker stack
  docker stack rm <stack-name>

Demo

Demonstration and Testing of docker stack deployment