1 Setup Project on DigitalOcean CentOS 7

HOWTO: Setup Over Fifty Five on DigitalOcean for Production using CentOS 7

Description

The goal of this article is to provide step-by-step instructions on how to setup academicstoday-django on a CentOS 7 production machine located at DigitalOcean.

Instructions

Prerequisites

• Create a droplet to your configuration.

• The # character in # yum install -y epel-release means that we want to run this script as a root user account.

• The $ character in $ sudo reboot means that we want to run this command as a privileged administrator user account.

• From your local developer machine, connect to the newly created server with this command: $ ssh -l root SERVER_DOMAIN_NAME_OR_IP

Pre-installation

1. We want to be using the latest libraries. Read more about Extra Packages for Enterprise Linux.

   # yum install -y epel-release;
   

2. Update the library.

   # sudo yum -y update;
   

3. As root user, copy and paste the following code to install all the libraries we will be using in our project. Other libraries will have individual instructions on how to setup. We are doing this to help speedup the time of installation.

   sudo yum -y install yum-utils;
   sudo yum -y groupinstall development;
   sudo yum -y install firewalld;
   sudo yum -y install ntp;
   sudo yum -y install fail2ban;
   sudo yum -y install nginx;
   sudo yum -y install python36;
   sudo yum -y install python36-devel;
   sudo rpm -Uvh https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-centos10-10-2.noarch.rpm
   sudo rpm -ivh https://yum.postgresql.org/9.6/redhat/rhel-7.3-x86_64/pgdg-centos96-9.6-3.noarch.rpm
   sudo yum -y install postgresql10-server postgresql10
   sudo yum -y install postgis2_10;
   sudo yum -y install redis;
   sudo yum -y install yum-cron
   sudo yum -y install GraphicsMagick-c++-devel;
   sudo yum -y install boost-devel;
   

4. Update your ``root` password of your droplet. Be sure to set a secure password with the following requirements: (a) 8 characters (b) 1 uppercase character (c) one special character (d) no common words. Finally be sure to not forget this password!

   # passwd root
   

Secure Root Account and Setup Administrative Account

1. This section starts off assuming you have completed the above section.

2. Create our technical operations account which we will use. Afterwords assign a password to that account.

   # adduser techops
   # passwd techops
   

3. Grant administrative privileges to the user.

   # gpasswd -a techops wheel
   

4. (Optional) On your local developers machine, generate the ssh key pair values. When prompted about passphase, skip it. After generating the key, print the public keys to the console.

   local$ ssh-keygen
   local$ cat ~/.ssh/id_rsa.pub
   

5. Now on your server, copy and paste your ssh key pair into the new user.

   # su - techops
   $ mkdir .ssh
   $ chmod 700 .ssh
   $ vi .ssh/authorized_keys
   

6. Restrict the permissions of the authorized_keys file with this command. Do not skip this command as you will be unable to ssh into this server without setting the permissions here:

   $ chmod 600 .ssh/authorized_keys
   

7. On your local developers machine, attempt to log into the server with the techops user account to confirm it is working. If you cannot log in then please review steps 1 to 7 or search online for answers. Here is an example:

   local$ ssh -l techops SERVER_DOMAIN_NAME_OR_IP
   

8. Exit from the techops user and disable root login for the sshd app.

   $ exit
   # vi /etc/ssh/sshd_config
   

9. Find this line of code and change it to look like this:

   PermitRootLogin no
   

10. Reload sshd with our latest change.

    # systemctl reload sshd
    

11. Now on your local machine, try connecting to the server using the root account and you'll notice you cannot access it~.

12. In the future commands, you have to login to the techops user account and use the sudo command privilege elevation to use administrative commands.

Setup Firewall

2. Then enable the application.

   $ sudo systemctl enable firewalld
   $ sudo reboot
   

3. Log back in from your localmachine.

4. Start the application.

   $ sudo systemctl start firewalld
   

5. Confirm it's working.

   $ sudo firewall-cmd --state
   

6. Add rules so nginx will be able to accessible by the the internet.

   $ sudo firewall-cmd --permanent --add-service=http
   $ sudo firewall-cmd --permanent --add-service=https
   $ sudo firewall-cmd --reload
   $ sudo firewall-cmd --runtime-to-permanent
   

7. Would you like to know more? Learn more. It is encouraged to read this article to understand the next few steps made.

Adjust Timezone

1. List what available timezones there are.

   $ sudo timedatectl list-timezones
   

2. Set America/Toronto timezone for our OS build.

   $ sudo timedatectl set-timezone America/Toronto
   

3. Make the timezone change permanent.

   $ sudo timedatectl
   

4. Start ntp app, and enable it to start at boot-time.

   $ sudo systemctl start ntpd
   $ sudo systemctl enable ntpd
   

Auto-Update Cron

• https://www.techrepublic.com/article/how-to-enable-automatic-security-updates-on-centos-7-with-yum-cron/

1. Enable and start our cron.

   $ sudo systemctl start yum-cron;
   $ sudo systemctl enable yum-cron;
   

2. Confirm that our installation was a successs.

   $ systemctl status yum-cron.service
   

3. Look at our configuration:

   sudo vi /etc/yum/yum-cron.conf
   

4. Next, locate the line:

   apply_updates = no
   

5. And change to:

   apply_updates = yes
   

6. Restart the service.

   $ sudo systemctl restart yum-cron
   

7. Confirm that our cron service is running.

   $ systemctl status  yum-cron.service
   

Setup Fail2ban

1. Enable the service to start at boot-time.

   $ sudo systemctl enable fail2ban
   

2. Setup our configuration. Start by opening up our configuration.

   $ sudo vi /etc/fail2ban/jail.local
   

3. Copy and paste.

   [DEFAULT]
   # Ban hosts for one hour:
   bantime = 3600
   ignoreip = 127.0.0.1/8
   
   # Override /etc/fail2ban/jail.d/00-firewalld.conf:
   banaction = iptables-multiport
   
   [sshd]
   enabled = true
   

4. Start the service fail2ban app.

   $ sudo systemctl start fail2ban
   

5. Check to confirm that the fail2ban app has been successfully installed and is currently running in our build.

   $ sudo fail2ban-client status
   

6. Would you like to know more? Read more here.

Nginx

1. Start nginx.

   $ sudo systemctl start nginx
   

2. Enable nginx to startup on boot-time. Use the following command to do so:

   $ sudo systemctl enable nginx
   

3. Confirm our server works. Open in a web browser:

   http://SERVER_DOMAIN_NAME_OR_IP
   

4. (Optional) Here are the nginx important files and directories to take note of:

• The default server root directory (top level directory containing configuration files): /etc/nginx
• The main Nginx configuration file: /etc/nginx/nginx.conf
• Server block (virtual hosts) configurations can be added in: /etc/nginx/conf.d
• The default server document root directory (contains web files): /usr/share/nginx/html
• The default log for errors: /var/log/nginx/error.log
• The default log for access: /var/log/nginx/access.log

Python Extra

1. Confirm you installed the correct library.

   $ python3.6 -V
   

2. We will next install pip, which will manage software packages for Python:

   $ curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
   $ sudo python3.6 get-pip.py
   

3. Confirm we have the proper version.

   $ pip -V
   

4. Install some dependent libraries.

   $
   $ sudo pip install virtualenv
   

Postgres

1. Initialize our database.

   $ sudo /usr/pgsql-10/bin/postgresql-10-setup initdb
   

2. Open the HBA configuration with your favorite text editor. We will use vi:

   $ sudo vi /var/lib/pgsql/10/data/pg_hba.conf
   

3. Find the lines that looks like this, near the bottom of the file:

   host    all             all             127.0.0.1/32            ident
   host    all             all             ::1/128                 ident
   

4. Then replace "ident" with "md5", so they look like this:

   host    all             all             127.0.0.1/32            md5
   host    all             all             ::1/128                 md5
   

5. Open the configuration with your favorite text editor. We will use vi:

   $ sudo vi  /var/lib/pgsql/10/data/postgresql.conf
   

6. Then modify the following to look like this:

   $ listen_addresses = '*'
   

7. Start the service

   $ sudo systemctl start postgresql-10
   

8. Make sure it boots all the time.

   $ sudo systemctl enable postgresql-10
   

9. Begin using it...

   $ sudo -i -u postgres
   $ psql
   

10. Would you like to know more about PostGres setup? Learn more.

Redis

https://www.linode.com/docs/databases/redis/deploy-redis-on-centos-7

$ sudo systemctl start redis
$ sudo systemctl enable redis

You can edit.

$ sudo vi /etc/redis.conf

Setup the Django User

1. Create a services user for the application:

   sudo groupadd --system django;
   sudo useradd --system --gid django --shell /bin/bash --home /opt/django django;
   
   

2. Create the Django project home inside /opt:

   sudo mkdir /opt/django
   

3. Give the permissions to the django user:

   sudo chown django:django /opt/django
   

4. Go into our new user. Note: sudo is to elevate privilege and su to switch users.

   $ sudo su - django
   

Setup the project

Setup from GitHub

1. Clone the project.

   $ cd ~/
   $ git clone https://github.com/AcademicsToday/academicstoday-django.git
   $ cd ~/academicstoday-django
   

2. Setup our virtual environment

   $ virtualenv -p python3.6 env
   

3. Now lets activate virtual environment

   $ source env/bin/activate
   

4. Now lets install the libraries this project depends on.

   (env)$ pip install -r requirements.txt
   

Database Setup

1. Log into the techops user account.

2. Go to your postgres account.

   $ sudo -i -u postgres
   $ psql
   

3. Enter the following:

   drop database academicstoday_db;
   create database academicstoday_db;
   \c academicstoday_db;
   CREATE USER django WITH PASSWORD '123password';
   GRANT ALL PRIVILEGES ON DATABASE academicstoday_db to django;
   ALTER USER django CREATEDB;
   ALTER ROLE django SUPERUSER;
   CREATE EXTENSION postgis;

Environment Variables Setup

1. Populate the environment variables for our project.

   (env)$ cd ~/academicstoday-django
   (env)$ ./setup_credentials.sh
   

2. Go inside the environment variables.

   (env)$ vi ./academicstoday/academicstoday/.env
   

3. Edit the file to suite your needs.

Setup the Application

1. Run the following commands to populate the database.

   cd ~/academicstoday-django/academicstoday; \
   python manage.py makemigrations; \
   python manage.py migrate; \
   python manage.py setup_academicstoday;
   

DigtalOcean Nameserver

1. Go and log into DigitalOcean.

2. Go to Networking section.

3. Enter the domain and add it. Note: Please note that the URL used here is different.

4. Add A record where the hostname points to our droplet.

5. Add CNAME where the hostname is www and the alias is @.

6. Add CNAME where the hostname is * and the alias is @.

Confirm Redis Operational Status

1. Confirm that the redis service has no errors by running:

   $ sudo systemctl status redis
   

2. Confirm we can connect to the redis server with the command-line client:

   $ redis-cli
   

3. Confirm connectivity by typing into the console:

   ping
   

4. You should see the following output:

   PONG
   

Setup Nginx with Project

1. Ensure you are operating as the techops user before running the following.

2. Open up a new file which we will save as our Nginx config file:

   sudo vi /etc/nginx/nginx.conf;
   

3. Paste the following into that newly created file and save it:

   server {
     listen 80 default_server;
     server_name academicstoday.io, www.academicstoday.io, *.academicstoday.io;
     charset     utf-8;
     access_log off;
     gzip on;
     client_max_body_size 0;  # Unlimited Upload File Size
   
     # Optional if you want logging in Nging.
     access_log /var/log/nginx/default-access.log;
     error_log /var/log/nginx/default-error.log;
   
     location = /favicon.ico { access_log off; log_not_found off; }
     location /static/ {
         root /opt/django/academicstoday-django/academicstoday_project;
     }
     location /media/ {
         root /opt/django/academicstoday-django/academicstoday_project;
     }
     location /static/admin {
         alias /opt/django/academicstoday-django/env/lib/python3.6/site-packages/django/contrib/admin/static/admin;
     }
   
     location / {
         proxy_set_header         Host $host;  # (1)
         proxy_pass               http://127.0.0.1:8001;
         proxy_set_header         X-Forwarded-Host $server_name;
         proxy_set_header         X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_set_header         X-Real-IP $remote_addr;
         add_header               P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
   
         # Extend our timeout to handle longer processes.
         proxy_connect_timeout 75s;
         proxy_read_timeout 300s;
     }
   }

4. (Optional) The following article should be read if you are setting up Django REST Framework with your project:

   https://stackoverflow.com/a/34233258

5. Test your nginx configuration for syntax errors by typing:

   $ sudo nginx -t
   

6. If any errors occur please investigate and fix before proceeding further. Else if no errors are reported, go ahead and restart nginx by typing:

   $ sudo systemctl restart nginx
   

7. Because we are running CentOS 7 we need to run the following because of this article and do not skip this:

   $ sudo setsebool -P httpd_can_network_connect 1
   

8. Because of the security policies of the SELinux, we need to manually add the httpd_t to the list of permissive domains, run this command:

   $ sudo semanage permissive -a httpd_t
   

9. Also please run this command because of this article and do not skip this step:

   $ sudo chcon -Rt httpd_sys_content_t /opt/django/academicstoday-django/academicstoday_project/static
   

OPTION 1 of 2: Systemd Method

If you would like to get the project up and running simply by using systemd the follow the next series or steps.

Setup Gunicorn with Systemd

This section explains how to integrate our project with systemd so our operating system will handle stopping, restarting or starting.

1. (OPTIONAL) If you cannot access the server, please stop and review the steps above. If everything is working proceed forward.

2. While you are logged in as a techops user, please write the following into the console.

   $ sudo vi /etc/systemd/system/gunicorn.service
   

3. Implement

   [Unit]
   Description=gunicorn daemon
   After=network.target
   
   [Service]
   User=django
   Group=nginx
   WorkingDirectory=/opt/django/academicstoday-django
   ExecStart=/opt/django/academicstoday-django/env/bin/gunicorn -c gunicorn_config.py academicstoday_project.wsgi
   
   [Install]
   WantedBy=multi-user.target
   

4. We can now start the Gunicorn service we created and enable it so that it starts at boot:

   $ sudo systemctl start gunicorn
   $ sudo systemctl enable gunicorn
   

5. Confirm our service is running.

   $ systemctl status gunicorn.service
   

6. And verify the URL works in the browser.

   http://SERVER_DOMAIN_NAME_OR_IP/en/
   

7. Would you like to know more?

• Create a Gunicorn Systemd Service File.

• How to make Python script run as service?

Setup Redis django_rq with Systemd

1. While you are logged in as a techops user, please write the following into the console.

   $ sudo vi /etc/systemd/system/django_rq.service
   

2. Implement

   [Unit]
   Description=Redis rqworker for Django App
   After=network.target
   
   [Service]
   Type=simple
   User=django
   Group=nginx
   WorkingDirectory=/opt/django/academicstoday-django
   ExecStart=/opt/django/academicstoday-django/django_rq.sh
   
   [Install]
   WantedBy=multi-user.target
   

3. We can now start the Gunicorn service we created and enable it so that it starts at boot:

   $ sudo systemctl start django_rq
   $ sudo systemctl enable django_rq
   

4. Confirm our service is running.

   $ systemctl status django_rq.service
   

Setup Redis rq_scheduler with Systemd

1. While you are logged in as a techops user, please write the following into the console.

   $ sudo vi /etc/systemd/system/rq_scheduler.service
   

2. Implement

   [Unit]
   Description=Redis rq_scheduler for Django App
   After=network.target
   
   [Service]
   Type=simple
   User=django
   Group=nginx
   WorkingDirectory=/opt/django/academicstoday-django
   ExecStart=/opt/django/academicstoday-django/rq_scheduler.sh
   
   [Install]
   WantedBy=multi-user.target
   

3. We can now start the Gunicorn service we created and enable it so that it starts at boot:

   $ sudo systemctl start rq_scheduler
   $ sudo systemctl enable rq_scheduler
   

4. Confirm our service is running.

   $ systemctl status rq_scheduler.service
   

OPTION 2 of 2: Supervisor Method

TODO: Write something...