DevOps-Training-GGITS

DevOps Pipeline

Step 1- Developer -- Code -Git/ GitHub
Step 2- Build (.exe) (Tools - Jenkins/Maven)
Step 3- Testing (Selenium)
Step 4- Quality Assurance
Step 5- Deploy (Tools - Chef/Docker/Puppet)
Step 6- Maintenance + Employee Training
Step 7- Monitoring

Git and GitHub

Create New Repository on GitHub

Step 1 : - Click on + icon show in top right corner
Step 2 : - Select New Repository option

Step 3 : - Name the Repository

Step 4 : - Click on Create Repository button

Step 5 : - Copy the URL.

Your new repository has been created successfully.

Install Git on EC2 Instance

Login with ec2-user

# switch to superuser 
sudo su 

# update the packages
yum update -y

# install git
yum install git -y

# check git version
which git

Git Configuration

# Set Configuration
git config --global user.name "FirstName LastName"
git config --global user.email user@gmail.com

# Check Configuration
git config --list

Create Project Directory and File on EC2 Instance

# create directory 
mkdir Project

# move inside Project directory
cd Project

# craete new file 
cat > myFile.txt 
My name is _____.

(press Ctrl + D to save file and quit)

Manage Project Versions in Local Repository

# initialise the local directory with git
git init 

# Staging the changes
git add myFile.txt
or 
git add . 

# check all files in staging area
git status

# Save the changes on local repository
git commit -m "File-v.1.0"

# checks commit history
git log

Push Project to GitHub (Remote Repository

# Connect local repository to GitHub repository
git remote add origin GitHub_Repo_URL

# Push to GitHub
git push -u origin master

Important Terms & Definitions

Working Space - It is where the code is written & is called the workspace.

Staging Area - When code needs to be committed, it is added to the staging area. In other words we can say when the code needs to be finalized it comes in staging area. This process is called "Add".

Local Repo - When the code is final, we commit the code from staging area to local repository. Local repo is present inside the Linux machine.

Remote Repo - A remote repo is a git repository hosted on the internet or some other network.

Commit ID and Commit - Commit ID is the metadata stored with the file, it is 40 alphanumeric characters. Bringing the file from staging area to Local repository is called "Commit", it means that all the code that has been written is finally saved.

Cheeksum - It is a numeric value attached with the file, when the file is updated the checksum value changes hence we get to know that the file/data/code has been updated.

Version ID - Reference to identify each change, when the user updates the file the version ID depicts about who did so.

Tag - With the commit ID a tag can also be attached which contains a message of what has been updated lastly.

Snapshot - The changes that are made in file is snapshot, changes that are made in a particular time.

Push - What all has been committed in local repository now is pushed to Central Repository(GitHub/GitLab or any other) this is called push.

Pull - Pull operation copes the changes from a remote repository to a local machine. Pull operation is also used as synchronization between the two repositories

Branch - Code is same so only one repository is created but when we want to work in parallel, 'branch' is created in which changes are personal to that branch only and finally it is clubbed together with other branches.

Checkout - This command Is used to switch in between the branches and check for the updated files or any new commits to it.

Chef

Chef is a Configuration Management Tool. Chef Architecture has 3 Components

• Workstation (Where we write the code)
  • CookBook
  • Recipe
• Chef Server (Where we store the code)
  • Knife
• Chef Client (Node) (Where we apply the code)
  • Ohai

Installation of Chef on AWS EC2

• Goto www.chef.io using windows machine
• Click on Download button on top right corner
• Click on Chef Workstation
• Select your OS for download ( select Amazon Linux for EC2)

• Click on download and fill the information form (if there are)
• If the downloading is started, pause them
• Press Ctrl + j for go to downloads in chrome
• Right click on link and copy link address (url)

• Open your EC2 terminal using putty
• login with ec2-user

# switch to root user 
sudo su

# download chef workstation in linux
wget <Paste Chef URL> 

# check the downloaded file
ls

# install the chef package
yum install <paste the above chef-workstation path>
or 
yum install <paste the above chef-workstation path> -y

# Check chef version
which chef
chef -v

Create Cookbook

# Create a directory
mkdir cookbooks

# change directory 
cd cookbooks

# list directory 
ls

# create a new cookbook
chef generate cookbook test-cookbook1

# install tree package 
yum install tree -y

# show the tree structure of the directory
tree

Create Recipe

# change directory 
cd test-cookbook1

# create new recipe
chef generate recipe test-recipe1

# show the tree structure of the directory
tree

Write Code in Recipe

The code is written in ruby language. The file extension/format of the ruby file is rb.

To edit the code in CLI, there are 3 types of code editor.

• vi
• vim
• nano

# open recipe file in vi editor
vi ./recipes/test-recipe1.rb

• Press i for INSERT MODE

• Add the following code in Recipe file

file '/myfile' do
content 'Welcome to DevOps - Chef configuration Management Tool - GGITS'
action:create
end

• Press Esc to exit from INSERT MODE

• Type :wq and press Enter to save and exit.

# read the Recipe file using cat
cat ./recipes/test-recipe1.rb

Run the Recipe

# execute the recipe file
chef exec ruby -c ./recipes/test-recipe1.rb

# run the Recipe on all nodes
chef-client -zr "recipe[test-cookbook1::test-recipe1]"
end

# check the files in root 
ls /

If the Recipe is successfully executed then you will see the myfile is created in root directory.

# read myfile using cat 
cat /myfile

(We wrote this message in test-recipe.rb file.)

Install and Run Apache Web Server on EC2 using Chef

# list directory
ls

# change directory
cd cookbooks

# create new cookbook
chef generate cookbook apache-cookbook

# change directory
cd apache-cookbook

# create new recipe
chef generate recipe apache-recipe

# show directory 
tree

# open recipe in vi editor
vi ./recipes/apache-recipe.rb

add following code in recipe

package 'httpd' do
action :install
end

file '/var/www/html/index.html' do
content '<h1>Hello World!</h1><h3>This is My First Website</h3>'
action :create
owner 'root'
group 'root'
end

service 'httpd' do
action [:enable, :start]
end

press `Es`c then press **:wq** and press `Enter` to save and exit

# execute the recipe
chef exec ruby -c ./recipes/apache-recipe.rb

# run the recipe 
chef-client -zr "recipe[apache-cookbook::apache-recipe]"

AWS EC2 Security Setup

STEP 1 : - Select EC2 instance then select Security Column

STEP 2 : - Click on Security Group ID

STEP 3 : - Click on Edit inbound rules

STEP 4 : - Change Type from HTTPS to HTTP and save rules

Open Web Page

STEP 1 : - Copy Public IPv4 address of your EC2 instance

STEP 2 : - Paste the address on web browser

You're web-page is successfully deployed.

Update Website

# open recipe using vi
vi ./recipes/apache-recipe.rb

# execute and run recipe
chef exec ruby -c ./recipes/apache-recipe.rb
chef-client -zr "recipe[apache-cookbook::apache-recipe]"

Attributes of Chef

ohai

ohai ipaddress

ohai memory

ohai memory/total

Show the Attributes of Chef using Recipe

STEP 1:- Create a new Recipe

cd cookbooks
 
cd apache-cookbook

chef generate recipe attributes-recipe

STEP 2 :- Edit the recipe

vi ./recipes/attributes-recipe.rb

# execute and run recipe
chef exec ruby -c ./recipes/apache-recipe.rb
chef-client -zr "recipe[apache-cookbook::apache-recipe]"

STEP 3 :- Add the following code in recipe

file '/basicfile' do
content "This is a file to get and learn attributes
HOSTNAME: #{node['hostname']}
IPADDRESS: #{node['ipaddress']}
MEMORY: #{node['memory']['total']}"
owner 'root'
group 'root'
action :create
end

STEP 4 :- Execute and run the recipe

chef exec ruby -c ./recipes/attributes-recipe.rb
chef-client -zr "recipe[apache-cookbook::attributes-recipe]"

Chef Server (Attaching Chef Server to WorkStation)

STEP 1:- Open https://manage.chef.io

STEP 2:- Create your account

STEP 3:- Go to starter kit

STEP 4:- Download and Extract the file.

STEP 5:- After extracting the file we will have file named chef-repo and we have to copy this file to our Linux system to do that

STEP 6:- Download WinSCP Applicatrion in windos machine. With this we can transfer files from our local windows machine to Linux machine. WinSCP Link - https://winscp.net/eng/download.php

STEP 7:- Install it (just click next ...next while installing)

STEP 8:- Open WinSCP, this screen will come up

STEP 9:- Fill these empty fields

• Host name = PUBLIC IPv4 DNS OF LINUX MACHINE
• User name = ec2-user

STEP 10:- Go to Advance option.

• Go to Authentication
• Give the same private key as your Linux machine (that ppk private key)

STEP 11:- After selecting the private key, click and just Login

STEP 12:- Now two windows will open left one will be for windows and right one for Linux. From window of windows go to chef-repo folder drag that folder to Linux machine.

STEP 13:- Now we can go to Linux CLI and check if the chef-repo file is there by typing ls command

cd chef-repo

ls -a

cd .chef

ls 

cat config.rb 

(chef server url will come up along with some other info)

STEP 14:- Now we can connect our workstation by using knife command

knife ssl check 

ssl - secure socket layer

This means your workstation is successfully connected to Chef Server

Create New Chef Node

Important
Note the Availability Zone of your Work Station Machine (We create Node Machine at same Availability Zone)

STEP 1:- Create new AWS Linux Machine

STEP 2:- Select Availability Zone same as workstation machine

STEP 3:- Write the default executable code (It execute while instance is initialling)

STEP 4:- Add Tag

STEP 5:- Configure Security Group

STEP 6:- Create New key pair

Connect Nodes to Chef Server

STEP 1:- Login in workstation machine with ec2-user

# switch to super user
sudo su

# list directory
ls

# Change directory
cd cookbooks

# list directory
ls

STEP 2:- Move apache-cookbook to chef-repo/cookbooks

# go to perent directory
cd ..

# move apache-cookbook and test-cookbook1
mv cookbooks/apache-cookbook chef-repo/cookbooks/
mv cookbooks/test-cookbook1 chef-repo/cookbooks/

# delete old cookbooks directory
rm -rf cookbooks

# list directory
ls

# check chef-repo/cookbooks
cd chef-repo
cd cookbooks
ls

STEP 3:- Copy Node-key.pem file to Chef workstation using WinSCP software

# Check the key
cd .. 
ls

STEP 4 :- Bootstrapping (Connecting Node to Chef Server using Knife)

knife bootstrap <node private IP> --ssh-user ec2-user --sudo -i Node-key.pem -N Node1

# Show all nodes
knife node list

STEP 5:- Set Cookbook to Run List of Chef Node

# Set Cookbook 
knife node run_list set Node1 "recipe[apache-cookbook::apache-recipe]"

# Check the run_list of the Node
knife node show Node1

STEP 6:- Upload the cookbook to Node

knife cookbook upload apache-cookbook

Cherf Node Setup

STEP 1:- Convert Node-key.pem to Node-key.ppk using putty-gen

STEP 2:- Login to node machine with ec2-user

# switch to super user
sudo su

# run the pull request
chef-client 

STEP 3:- Automate the pull request process

# open crontab file using vi editor
vi /etc/crontab

Add the following line

* * * * * root chef-client 

Save and quit from editor using
:wq

Check the Web page

STEP 1:- Copy Node's Public IPv4 address

Step 2:- Paste on Web Browser

Automate Chef Node While Creating/Launching

Add the default command in Configuration Instance Details Step

#!/bin/bash
sudo su
yum update -y
echo "* * * * * root chef-client"  >>  /etc/crontab

Update the Web Page

Goto Chef Work Station Terminal

STEP 1:- Edit the Recipe

# Open apache-recipe in vi editor
vi apache-cookbook/recipes/apache-recipe.rb

STEP 2 :- Upload the cookbook to Node

knife cookbook upload apache-cookbook

STEP 3:- Check the Website

Interview Questions

• Q.1 Why DevOps? Where you used DevOps?
• Q.2 Which s/w you used in DevOps?
• Q.3 Which tool u use?
• Q.4 Difference b/w Development and Operations.
• Q.5 Git / GitLab / GitHub?
• Q.6 Which platform you use in DevOps?
• Q.7 Why infinite loop in DevOps?
• Q.8 Where you use Git Bash?
• Q.9 Configuration Management and Types of Version Control Systems ?
• Q.10 Chef workstation / Chef server/ Chef nodes/ Chef client / knife / recipe / bootstrapping,
• Q.11 Operational tool in DevOps?
• Q.12 Have you do testing in the DevOps project?
• Q.13 Agile description / Scrum Framework?
• Q.14 CI/CD (*Imp)
• Q.15 Learn more about cloud computing (*Imp)?
• Q.16 LINUX Command
  • package install command
  • file system
  • remove command
  • learn more basics command

Final Project (Connect Chef-Server and Nodes to GitHub Repository and Automate them)

STEP 1:- Create a New Folder in your Windows machine.

STEP 2 :- Create a index.html file in the folder.

STEP 3 :- Open index.html file in any text editor and write the html code.

STEP 4 :- Create New GitHub Repository

STEP 5 :- Open Git Bash in same folder.

STEP 6 :- Connect local repository to GitHub repository and push the code.

git init

git add .

git commit -m "index.html file added"

git remote add origin <github_repo_ssh_url>

git push -u origin master

STEP 7 :- Check GitHub Repository

STEP 8 :- Open EC2 instance terminal with ec2-user

# switch to super user
sudo su

# goto apache-cookbook directory
cd chef-repo/cookbooks/apache-cookbook/

# open apache recipe in vi editor
vi recipes/apache-recipe.rb

STEP 9 :- Deleting existing html file

Add the following ruby code in apache-recipe.rb file

file '/var/www/html/index.html' do 
action :delete
end

STEP 10 :- Test the recipe in chef server

# Execute the recipe
chef exec ruby -c ./recipes/apache-recipe.rb 

# deploy the recipe
chef-client -zr "recipe[apache-cookbook::apache-recipe]"

STEP 11 :- Deploy the cookbook to chef nodes

# goto parent directory 
cd ..

# Upload the apache-cookbook 
knife cookbook upload apache-cookbook

it will delete index.html file from all connected nodes

STEP 12 :- Connect Chef to GitHub

# goto apache-cookbook directory
cd chef-repo/cookbooks/apache-cookbook/

# open apache recipe in vi editor
vi recipes/apache-recipe.rb

Add the following ruby code in apache-recipe.rb file

package 'httpd' do
action :install
end

package 'git' do
action :install
end

bash 'git_pull' do
user 'root'
cwd '/var/www/html'
code <<-EOH
git init
git remote add origin <YOUR_GITHUB_REPO_URL>
git pull origin master
EOH
end

service 'httpd' do
action [:enable, :start]
end

STEP 13 :- Test the recipe in chef server

# Execute the recipe
chef exec ruby -c ./recipes/apache-recipe.rb 

# deploy the recipe
chef-client -zr "recipe[apache-cookbook::apache-recipe]"

STEP 14 :- Copy Chef-server IPv4 and Check the web-page

We successfully connected our chef server to GitHub repository.

STEP 15 : - deploy the recipe to chef node machines

# goto parent directory 
cd ..

# Upload the apache-cookbook
knife cookbook upload apache-cookbook

STEP 16 :- Copy Chef-Node IPv4 and Check the web-page

STEP 17 :- Now we want to automate the process of pull the code from GitHub and uploading the cookbook to chef nodes.

Open chef workstation terminal

# open crontab file
vi /etc/crontab

Add the following command

* * * * * root "cd /var/www/html && git pull origin master && cd /home/ec2-user/chef-repo/cookbooks knife cookbook upload apache-cookbook"

STEP 18 :- Test the automation.

Update the html file in your local machine

Commit changes and push to GitHub

Copy your node IPv4 and paste in web browser

Now our chef is successfully connected to GitHub and deployment is fully automated.

STEP 19:- Edit Html code with your creativity and push to GitHub. Your website is automatically updated within a minute.

HTML code link - https://github.com/Rajsoni03/DevOps-Training-GGITS/blob/master/index.html

Congratulations Your Project is Successfully Automated.