Developers guide to installing the GMS on an OpenStack instance

This guide is intended for a developer using the OpenStack dashboard who wishes to configure an OpenStack instance for installation of GMS.

Before Beginning This Guide

These instructions were tested on a non-default instance flavor called "m1.2xlarge". The flavor provides 32GB of RAM, 8 VCPUs, and 160GB of disk. Using fewer resources may not yield successful results. Check that you have a large enough instance flavor available in the Launch Instance dialog by navigating to Instances in the OpenStack dashboard, then clicking the Launch Instance button. Once inside the Launch Instance dialog, the Flavor Details show the resources provided by the select flavor. The details of other flavors can be displayed by changing the selected flavor. If there is not a sufficient flavor, contact the administrator of your OpenStack cluster. For more information on flavors, see the OpenStack documentation.

The OpenStack cluster used to test these instructions had 2,000 GB of block storage available. Check that you have at least this much additional storage by navigating to the Volumes menu on the OpenStack dashboard and clicking on the Create Volume button. The Volume Limits graph indicates how much storage is available. See the OpenStack block storage documentation for more information.

GMS is built on top of Ubuntu Precise. An Ubuntu Precise image is required. The default kernel supplied with Ubuntu Precise does not perform file system resizing efficiently (bug report). OpenStack performs file system resizing when initializing a new instance, so booting a Precise vm with a large root volume can be very slow. The Ubuntu Precise image used in this guide was configured with the 3.8 Linux kernel. Using the default 3.2 Linux kernel distributed with Ubuntu Precise in conjunction with the m1.2xlarge instance flavor may not boot in a desirable amount of time.

Configure OpenStack with Your SSH Key

1. From the computer that you plan to access your OpenStack instance, verify that you have already generated an ssh keypair. If you have already generated ssh keys, you probably have a file named ~/.ssh/id_rsa.pub. If you do not have this file, you may not have generated an ssh keypair and you should run ssh-keygen to create a keypair.

2. On the OpenStack dashboard, navigate to the Keypairs page by selecting Access & Security in the lefthand menu, then choosing the Keypairs tab. If your keypair is not listed, you will need to import it. If you are not sure whether your keypair is listed, you see if any of the fingerprints listed on the Keypairs page matches the fingerprint of your public key.

3. If your keypair was already imported into OpenStack, then no further steps are necessary to configure OpenStack for SSH. Otherwise, it is time to import your public key. From the computer that you plan to access your OpenStack instance, copy the contents of the ~/.ssh/id_rsa.pub file. On the Keypairs page of the OpenStack dashboard, click Import Keypair. Import your public key by typing a short, descriptive name for your keypair in the Keypair Name field, then paste your public key into the Public Key field. Click Import Keypair to finish importing your public key.

4. Check that your newly imported public key is listed on the Keypairs page. If so, then you have configured OpenStack with your SSH key.

Start a Virtual Machine Instance on OpenStack

1. On the OpenStack dashboard Instances page, click the Launch Instance button. Complete the Launch Instance form starting with the Details tab:

• Availability Zone: Check with your OpenStack administrator if you are unsure
• Instance Name: Choose a descriptive name
• Flavor: m1.2xlarge see "Before Beginning This Guide" above
• Instance Count: 1
• Instance Boot Source: Boot from Image (Creates New Volume)
• Image Name: Choose an Ubuntu Precise image with a 3.8 kernel
• Device Size: 950
• Device Name: vda
• Delete on Terminate: Place a check in this box

2. Continue completing the Launch Instance form by selecting the Access & Security tab. Complete as follows:

• Keypair: Choose the keypair that you setup during "Configure OpenStack with Your SSH Key"
• Admin Pass: Leave blank
• Confirm Admin Pass: Leave blank
• Security Groups: Select the appropriate security groups. Check with your OpenStack administrator if you are unsure

3. Continue to the Networking tab. You may need to select your network by using your mouse to move a network from the list of Available Networks to the list of Selected Networks. Check with your OpenStack administrator if you need help choosing a network.
4. Nothing needs to be configured on the Post-Creation tab. Click the Launch button to launch your new virtual machine instance as configured. A new instance entry will display on the Instances page.
5. Click the Associate Floating IP button to associate a floating IP address with the new instance. If no floating IP addresses are available in the dropdown, you may need to select an IP address from the pool of floating IP addresses by clicking on the + symbol. Make note of the floating IP address, as you'll need it later.
6. While the instance continues to boot, create a new volume. Navigate to the Volumes page by clicking the Volumes link from the menu. Click the Create Volume button to display the Create Volume form. Complete the form as follows:

• Volume Name: sgms-opt
• Description: Optionally, enter a short description
• Type: Leave Blank
• Size (GB): 1000
• Volume Source: No source, empty volume

7. Click the Create Volume button. The sgms-opt volume will appear in the listing on the Volumes page.
8. Once the sgms-opt volume has been created and the sgms instance has booted, attach the sgms-opt volume to the vm instance. Click the Edit Attachments button next the sgms-opt volume. On the form, select the sgms vm instance from the the Attach to Instance dropdown. Enter /dev/vdb as the Device Name. Click the Attach Volume button to exit the form and attach the volume to the instance.

Install GMS on the Instance

(More information available here)

1. From a terminal, log into the instance using ssh, the floating IP address that you noted in step 5 of "Start a Virtual Machine Instance on OpenStack", and the "ubuntu" user account. For example, if your floating IP address is 10.0.41.108, your ssh command would look like: ssh ubuntu@10.0.41.108. The first time you connect to the instance, ssh may ask you whether it should continuing connecting to the host. Answer 'yes' to continue connecting. If the floating IP address you have select was used previously for another host which you have connected to previously, you may need to remove the old entry from your ~/.ssh/known_hosts file. The remaining steps in this section will be performed on the vm instance.
2. Setup passwordless sudo by peforming the following steps on the vm instance:

• sudo bash
• visudo
• In the editor, add this near the top of the sudoers files: Defaults !authenticate
• Save and exit the editor
• Exit the root shell

3. Format the new volume: sudo mkfs -t ext4 /dev/vdb
4. Mount the new volume: sudo mount -t ext4 /dev/vdb /opt
5. Following the "Installation on Ubuntu 12.04" instructions, execute commands on the instance to install GMS:

• sudo apt-get -y install git ssh make
• git clone https://github.com/genome/gms.git
• cd gms
• sudo apt-get update (may be necessary sometimes)
• make (takes about 37 minutes)

6. Log out of the vm instance before continuing the next section so that new shell environment settings may take effect.

Prime GMS with Demonstration Data

(More information available here)

1. From a terminal, log into the instance using ssh as in step 1 of "Install GMS on the Instance".
2. Change directories into the gms directory: cd gms
3. Run the following command to download demonstration data and prepare the system to run demonstration genome models (takes about 1 hour):

• ./setup/prime-system.pl --data=hcc1395_1tenth_percent --sync=tarball --low_resources --memory=30gb

4. Again, log out of the vm instance before continuing the next section so that new shell environment settings may take effect.

Build the Demonstration Genome Models

(More information available here)

1. From a terminal, log into the instance using ssh as in step 1 of "Install GMS on the Instance".
2. Start the following models one at a time, waiting for each model to complete before starting the next. Each model build should complete within 1 or 2 hours, depending on the model. You can determine whether a model is still running using the genome model build list command.

• genome model build start "name='hcc1395-normal-snparray'"
• genome model build start "name='hcc1395-tumor-snparray'"
• genome model build start "name='hcc1395-normal-refalign-exome'"
• genome model build start "name='hcc1395-tumor-refalign-exome'"
• genome model build start "name='hcc1395-normal-refalign-wgs'"
• genome model build start "name='hcc1395-tumor-refalign-wgs'"
• genome model build start "name='hcc1395-normal-rnaseq'"
• genome model build start "name='hcc1395-tumor-rnaseq'"
• genome model build start "name='hcc1395-somatic-exome'"
• genome model build start "name='hcc1395-somatic-wgs'"
• genome model build start "name='hcc1395-differential-expression'"
• genome model build start "name='hcc1395-clinseq'"

3. After all of the genome models have been built successfully, you have completed the build of the demonstration data.