Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebalance and rename content in Read This First and Quick Start #19

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
22 changes: 11 additions & 11 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,36 +10,36 @@ These docs are designed to help you get the best out of the CLIMB-BIG-DATA infra

How to access CLIMB-BIG-DATA and find your way around via the Bryn web interface.

[Registration](getting-started/how-to-register.md)
[Registration](getting-started/how-to-register.md)
How to register and access CLIMB-BIG-DATA.

[Authentication](getting-started/authentication.md)
[Authentication](getting-started/authentication.md)
How to login to Bryn, and setup two-factor authentication.

### Notebook Servers

Everything you need to understand, use and get the most out of Jupyter Notebook Servers.

[Introduction](notebook-servers/read-this-first.md)
An introduction to the what and why of notebook servers.
[Why Notebook Servers?](notebook-servers/why-notebook-servers.md)
An explanation of the why CLIMB-BIG-DATA uses Notebook Servers.

[Quick start](notebook-servers/quick-start.md)
How to launch, access and get started using a notebook server.
[Notebook Server Basics]
The basics of how to launch and use Notebook Servers.

[Using the terminal](notebook-servers/using-the-terminal.md)
[Using the terminal](notebook-servers/using-the-terminal.md)
How to use the terminal inside a notebook server, with an explanation of caveats.

[Understanding storage](storage/index.md)
[Understanding storage](storage/index.md)
An explanation of the different storage options available and when to use what.

[Installing software with Conda](notebook-servers/installing-software-with-conda.md)
[Installing software with Conda](notebook-servers/installing-software-with-conda.md)
How to install software using Conda, in the context of a containerized environment.

[Using Nextflow](notebook-servers/using-nextflow.md)
[Using Nextflow](notebook-servers/using-nextflow.md)
How to use Nextflow with CLIMB-BIG-DATA.

[Metagenomics walkthrough](walkthroughs/metagenomics-tutorial.md)
A simple walk-through of some CLIMB-BIG-DATA functionality.

[QIIME 2](walkthroughs/qiime2.md)
How to install QIIME 2 on a notebook server and basic usage.
How to install QIIME 2 on a notebook server and basic usage.
121 changes: 121 additions & 0 deletions docs/notebook-servers/basics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Notebook Server Basics

Welcome to CLIMB's Jupyter Notebook Servers, your new home on CLIMB-BIG-DATA.

## How to launch and access a notebook server

1. [Log in to Bryn](../getting-started/authentication.md).
2. Using the navigation menu on the left hand side, select 'Notebook servers' under the 'Compute' subheading.
3. Click the 'Launch notebook server' green action button on the right hand side.
4. Select a profile, for example 'Standard server' or 'GPU server' (tier dependent).
5. Click 'Launch Server' and monitor the progress bar.
6. Once ready, click the URL beneath the 'User notebook server'.
7. On first login, you may be asked to authorize access to your Bryn account. Click 'Authorize'.
8. The JupyterLab interactive computing interface should open in a new tab.

## A brief look at JupyterLab

When you first open the notebook server, you'll be presented with a JupyterLab interface. This is a web-based interface that allows you to interact with the notebook server.

![jupyterhub dashboard](../img/jh-dash.png)

The JupyterLab interface is divided into a few areas. You'll see context menus at the top (File, Edit, View, Run etc.), a file browser pane on the left, and an activity area that initially displays a launcher interface with tiles. Clicking on one of these tiles will open a new tab in the activity area.

These tiles are shortcuts to launch different applications. The most common one is the `Terminal`, which will open a terminal in the activity area. You can also launch a Jupyter Notebook, RStudio, or a text editor.

You can have multiple tabs open in the activity area, and you can drag and drop them to rearrange them. You can also drag and drop files from the file browser into the activity area to open them in the appropriate application. To create a new tab, click the `+` icon in the top right of the activity area, or use `File > New Launcher`.

![jupyterhub tabs](../img/jh-tab.png)

For this reason, there is no option to launch multiple notebook servers from Bryn. You should be able to have all your work running in tandem in one notebook server.

## Organising your data storage

The notebook server has a few different storage locations available to you. We can explore those via the `terminal`.

![terminal tile](../img/terminal-tile.png)

From the terminal, you can see your home directory, which is where you'll be by default when you open the terminal. You can list the contents of your home directory by typing `ls` (or `ls -l` for a detailed listing):

```
jovyan:~$ cd ~/
jovyan:~$ ls -l
total 12
drwxr-sr-x 3 jovyan users 4096 Jul 19 17:02 R
lrwxrwxrwx 1 jovyan users 14 Jul 19 18:23 shared-public -> /shared/public
lrwxrwxrwx 1 jovyan users 12 Jul 19 18:23 shared-team -> /shared/team
-rw-r--r-- 1 jovyan users 617 Jul 20 10:53 Untitled1.ipynb
-rw-r--r-- 1 jovyan users 72 Jul 19 18:22 Untitled.ipynb
-rw-r--r-- 1 jovyan users 0 Jul 20 11:35 untitled.txt
jovyan:~$ ls -l
```

Your home directory will look similar to home directories on other UNIX-based system, as a collection of files of folders. There are two special folders, specific to notebooks:

* `shared-team` is a symbolic link (a kind of shortcut, indicated by `->`) to the team share, which is a large, fast, shared storage location for your team. You can read and write to this folder, and it is shared with all members of your team.
* `shared-public` is a symbolic link to a read-only share managed by the CLIMB-BIG-DATA team. This contains some useful data and public resources for bioinformatics workflows.

Try listing the contents of the `shared-public/db` folder and you'll see a few folders:
```
jovyan:~$ ls ~/shared-public/db -l
total 0
drwxrwxr-x+ 2 4744 users 620 Apr 15 12:17 blast
drwxrwxr-x+ 2 4744 users 1 Apr 16 20:56 gtdb
drwxrwxr-x+ 14 4744 users 12 Apr 15 11:16 kraken2
drwxrwxr-x+ 2 4744 users 9 Apr 16 17:36 taxonomy
jovyan:~$
```

These are large databases for bioinformatics tools. For example, the `kraken2` folder contains the databases for Kraken 2 and Bracken.
You can use these in your workflows, without having to download them yourself.
These locations do not count towards your storage quota.

The `shared-team` folder is a good place to store your data. It is visible to everyone in your team, making it easy to collaborate on projects.

You can list the contents of the `shared-team` folder:
```
jovyan:~$ ls ~/shared-team
```

It will likely be empty, if this is your first time using CLIMB. You have full access to this location and can create folders and files as you wish. Try this with `touch`, which will create an empty file (if it doesn't exist).
Try using `touch` on other locations, and you will see that you can only write to your home directory, the `shared-team` folder and special locations like `tmp/`.

In locations where you can write, you will see the empty file `this` appear:
```
jovyan:~$ touch ~/this
jovyan:~$ ls -l
total 12
drwxr-sr-x 3 jovyan users 4096 Jul 19 17:02 R
lrwxrwxrwx 1 jovyan users 14 Jul 19 18:23 shared-public -> /shared/public
lrwxrwxrwx 1 jovyan users 12 Jul 19 18:23 shared-team -> /shared/team
-rw-r--r-- 1 jovyan users 0 Jul 20 11:47 this
-rw-r--r-- 1 jovyan users 617 Jul 20 10:53 Untitled1.ipynb
```

As opposed to `~/shared-public` and system locations like `/usr/local/bin` where we cannot `touch this`:
```
jovyan:~$ touch ~/shared-public/this
touch: cannot touch '/home/jovyan/shared-public/this': Read-only file system
jovyan:~$ touch /usr/local/bin
touch: setting times of '/usr/local/bin': Permission denied
```

For more information about data storage, please read the [Understanding Storage](../storage/index.md) guide.

## What's next?

The notebook server is flexible, allowing you work the way you want. You may have existing data and workflows that you want to bring to the notebook server, if so please read the [Understanding Storage](../storage/index.md) guide.

You may also want to read more about the specific features of the notebook server:

* [Using the terminal](using-the-terminal.md)
* [Using Jupyter notebooks](using-jupyter.md)
* [Using RStudio](using-rstudio.md)
* [Using Nextflow](using-nextflow.md)

Or there are walkthroughs that will take you through a worked example of a bioinformatics analysis:

* [Metagenomics in Brum](../walkthroughs/metagenomics-tutorial.md)
* [Genome assembly](../walkthroughs/genome-assembly/spades.md)
* [Custom Nextflow Workflows](../walkthroughs/nextflow-custom-workflows/nextflow-custom.md)
* [QIIME 2](../walkthroughs/qiime2.md)
21 changes: 0 additions & 21 deletions docs/notebook-servers/index.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/notebook-servers/installing-software-with-conda.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ base /opt/conda
```


The base Conda environment is installed at `/opt/conda`. Since we are running inside a container, any changes made to this part of the filesystem will not be retained once the container is stopped and restarted (unlike your home dir and shares, which are persisted).
The base Conda environment is installed at `/opt/conda`. Since we are running inside a container, any changes made to this part of the filesystem will not be retained once the container is stopped and restarted (unlike your home dir and shares, which are persistent).

We've made the base environment read only to prevent any confusion.

Expand Down
22 changes: 0 additions & 22 deletions docs/notebook-servers/quick-start.md

This file was deleted.