/
cluster_instructions.Rmd
98 lines (65 loc) · 4.99 KB
/
cluster_instructions.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
---
title: "Using a computing cluster"
output:
html_document:
theme: cosmo
---
## High performance computing
URI offers two clusters for High Performance Computing. Seawulf is the educational cluster; Bluewaves is the research cluster.
For more information on HPC at URI go to the HPC [website](https://web.uri.edu/hpc-research-computing/).
## Using HPC
* Open your Terminal (Mac) or GitBash (Windows). Alternatively, if you are using Windows download and install [PuTTY](http://www.putty.org/).
* Connect to URI's seawulf cluster using your ssh (**s**ecure **sh**ell) and your URI username. For example
```ssh rsschwartz@seawulf.uri.edu```
Your username is the first half of your email, and your password is your id number.
There are two ways to run jobs on a cluster. First, you can run them interactively just like you do on your own computer. The advantage of this is you can work easily and directly. The disadvantage is that you might want to run a job that takes a long time and you'd like to take advantage of the computing power of a the cluster (that's why you're using it in the first place).
You are now logged in on on the "head node". A computing cluster is just that - multiple computers attached together. Obviously it would be inefficient for many people to use the same computer on the cluster. It could even crash the cluster!
* Access your own "node" of the cluster to work on by submitting an interactive job.
```interactive```.
The above command launches an interactive session with 1 core for 8 hours executing bash shell.
When you submit an interactive session it will go to one of the available partitions.
Partitions offer different options.
On seawulf, the debug queue is for short jobs and is meant to be available without waiting.
The general queue may require a wait; however, you may run your job for as long as necessary.
View partitions by typing ```sinfo```.
You can now run almost all the same commands you do on your own computer. Because the cluster is running Linux (CentOS) there may be a few that are slightly different. For example, both `man ls` and `ls —-help` work on Linux. Additionally, while you have access to many installed programs you need to load them before you can use them.
* List available software on seawulf
```module avail```
* Because that list can be long limit your list to software starting with known values (e.g. P)
```module avail P```
* Load software so you can use it (for example Python)
```module load Python/3.5.2-foss-2016b```
You can use tab complete to get the full title of the module.
* List what you have loaded already
```module list```
* Run python (this is a programming language we will use later in the course) to show it has been loaded
```python```
* Use secure copy (scp) to move files between your computer and the cluster. Copy your whole data-shell folder to the cluster.
```scp -r data-shell rsschwartz@seawulf.uri.edu:```
* Make a change to a file in the data-shell folder. Rather than copying the whole folder again just update with these recent changes.
```rsync -r data-shell rsschwartz@seawulf.uri.edu:```
* To download files from the internet, the easiest tool to use is wget. The syntax is relatively straightforward: ```wget https://some/link/to/a/file.tar.gz``` We’ve actually done this before to download our example files:
``` wget https://hpc-carpentry.github.io/hpc-intro/files/bash-lesson.tar.gz```
* Now unzip the file
``` tar -xvf bash-lesson.tar.gz```
If you are working with analyses that will take some time you should not use an interactive job. Instead you will write a script to submit a job. Your script might look like
```
#!/bin/bash
#SBATCH -t 1:00:00
#SBATCH -p general # partition (queue)
#SBATCH -N 1 # number of nodes
#SBATCH -n 1 # number of cores
module load StringTie/1.3.3-foss-2016b
touch SomeRandomNumbers.txt
for i in {1..100000}; do
echo $RANDOM >> SomeRandomNumbers.txt
done
sort -n SomeRandomNumbers.txt > SortedRandomNumbers.txt
```
You are already familiar with the shebang line. All lines starting with `#SBATCH` indicate parameters related to job submission. `#SBATCH -p general` specifies the partition. `#SBATCH -t 1:00:00` allows the job to run for up to 1 hour. `#SBATCH -N 1` indicates you need 1 node. `#SBATCH -n 1` indicates you need 1 processor on that node. There are many other possible SBATCH parameters not included here.
* Exit your interactive session. Submit this batch file as a job by running `sbatch job_example.sh`
* Check the status of your job (and others) using `squeue`.
* Your output and any errors have been sent to files rather than printed on the screen. Use `ls` and `cat` to see the files and your output.
If you submit a job that you need to cancel first run `squeue` to get the job id then run `scancel <jobid>`
Note: If you have any large or important data you are working you should not store it in your home directory. Additional storage is connected to the cluster and access can be arranged by the HPC manager.
Now check how the cluster is doing with ```htop```