Skip to content

AWS deployment

Jump to bottom
Ilya Sytchev edited this page Mar 7, 2019 · 40 revisions

Quick start

Prerequisite: Terraform v0.11, git v2.17.1+ and jq v1.6+

Download Refinery code:

$ git clone git@github.com:parklab/refinery-platform.git

Create a Python 2.7 virtual environment (optional but recommended, assumes virtualenvwrapper is installed):

$ cd refinery-platform/deployment
$ mkvirtualenv -a $(pwd) refinery-deployment

Install deployment tools (assumes header files for Python are installed):

$ pip install -r deployment/requirements.txt

Configure AWS CLI with the Access Key ID, the Secret Access Key for the IAM account that you are going to use to deploy (enter us-east-1 for the region):

$ aws configure

Create deployment/terraform/live/terraform.tfvars and configure the site name, for example:

site_domain = "www.example.org"

Create S3 bucket to store Terraform config and initialize Terraform:

$ aws s3 mb s3://<terraform-bucket-name> --region us-east-1
$ cd deployment/terraform/live
$ terraform init -upgrade -backend-config="bucket=<terraform-bucket-name>" -backend-config="region=us-east-1"

Create new terraform workspace (with a meaningful name based on deployment)

$ terraform workspace new <meaningful-name>

Deploy (note that you will incur AWS charges even if you have free tier available):

$ terraform apply

Set up a CNAME or AWS Alias record for the site_domain terraform variable with the host name of the Elastic Load Balancer.

Once the web app EC2 instance is fully provisioned (35-40 minutes) open site_domain in your web browser.

Production deployment

Configure the following prerequisites in your AWS account in addition to the quick start setup:

  • EC2 key pair for SSH access to the app server instance
  • admin e-mail address (for example, refinery-admin@example.org) that is verified with SES. AWS has instructions for verifying e-mail addresses. This e-mail address will appear in the From: field of emails that Refinery sends out, and so will need to be verified with SES.
  • move your account out of the SES sandbox in order to be able to send email notification from the app server instance
  • obtain a TLS certificate for the hostname you will be using for site_domain in terraform.tfvars and add it to the AWS Certificate Manager.
  • obtain and configure Google reCAPTCHA v2

Configure Terraform variables (terraform.tfvars):

  • django_admin_email, django_default_from_email, and django_server_email entries should be an e-mail address that the AWS account is allowed to send e-mail from
  • tags:
    • owner: email from your local Github config will be used unless set here
    • terraform: true is always added to all resources that support tags
    • Name is set using the workspace name as prefix

Launching

Follow the instructions here to create the terraform-specific AWS assets needed for a cloud deployment.

Create a Route 53 record using the site_url from terraform.tfvars to point to the ELB DNS name:

  • Type: A - IPv4 address
  • Alias: Yes
  • Alias Target: <ELB load balancer name>
  • Routing Policy: Simple
  • Evaluate Target Health: No

Deleting

In deployment/terraform/live directory:

$ terraform workspace select <workspace-name>
$ terraform destroy

The last command above will show an error message about deleting module.object_storage.aws_s3_bucket.media_files. This is a safeguard against deleting the data file bucket by accident. If this is actually what you intend to do then empty the media files bucket and run:

$ terraform destroy

Optionally remove the workspace (if no longer needed):

$ terraform workspace delete <workspace-name>

See Terraform wiki page for more details.

Updating

  1. Stop the app server instance
  2. Take a snapshot of the app server EBS volume
  3. Update terraform.tfvars with this new EBS volume snapshot ID
  4. terraform apply
  5. Delete the EBS volume if it was replaced by a new one and detached from the app server
  6. Delete the EBS volume snapshot

Updating the app server instance

  1. Copy fabricrc.sample to your $HOME dir on the host where you've configured your deployment environment; set test_branch to the actual release branch (for example: release-1.3.0).
  2. Obtain Refinery EC2 test instance public IP and assign it to test_host (and so on for other hosts if necessary).
  3. Run fab test update to update the test_host (from refinery-deployment virtual environment).

Notes:

  • database migrations will be applied if necessary
  • UI code will be rebuilt
  • relevant services will be restarted
  • see fabfile.py for details
  • system configuration changes will not be applied

To update system and/or CloudFormation configuration it is necessary to delete and re-create the stack (make sure to check out the correct code commit before running the make web-stack command).

Tips and operational guidelines

  • Make sure to apply tags to resources for corresponding projects, environments, products and subproducts (don't just use the default tags)
  • Prod stack should be the only one running continuously
  • Test stack should be up during release testing only
  • Dev stacks should not exist for more than a day
  • Terminate unused Galaxy clusters and worker nodes (the most expensive resource)
  • Cleanup unused EBS and RDS snapshots

Troubleshooting

  • Output of the user-data script (Bash and Puppet) is stored in /var/log/cloud-init-output.log
  • To rerun deployment process:
sudo su -
wget http://169.254.169.254/latest/user-data
sh ./user-data
  • Check that your AWS credentials are configured properly: aws ec2 describe-regions

Home

Administration

Development

Testing

Troubleshooting

Miscellaneous

Clone this wiki locally