This repo contains the source code and documentation for the https://spinalcordmri.org/ website.
The main website is a simple landing page written using the Jekyll static site generator, and is deployed and hosted using GitHub Pages. The website uses a custom domain name purchased from NameCheap, then linked with GitHub Pages. (This way, instead of https://spinalcordmri.github.io, we get to use https://spinalcordmri.org.)
Configuration settings can be found here:
- GitHub Pages: https://github.com/spinalcordmri/spinalcordmri.github.io/settings/pages
- NameCheap: https://ap.www.namecheap.com/Domains/DomainControlPanel/spinalcordmri.org/domain/
Configuration is relatively simple, and was done using instructions from the following pages:
- GitHub Pages: https://help.github.com/articles/quick-start-setting-up-a-custom-domain/
- NameCheap: https://www.namecheap.com/support/knowledgebase/article.aspx/9645/2208/how-do-i-link-my-domain-to-github-pages
The page can be built locally after installing Jekyll. See more details here.
The webforum "forum.spinalcordmri.org" is a subdomain of the spinalcordmri.org website. The forum was originally envisioned as a general web forum and community for discussing MRI processing, acquisition, etc. That being said, the spinalcordmri.org forum is often colloquially referred to as the "SCT Forum", because the most active part of the forum is the "SCT" subsection.
The forum runs using the open-source forum software Discourse, and operates within a VM provided by DigitalOcean. This means that it is an entirely separate site from the main website, and so the setup, configuration, and administration of the forum is a bit more involved than the Jekyll part of the site.
If you need to remake the forum's VM from scratch (e.g. to debug an issue without impacting the production server), then follow the instructions below.
NB: The following instructions are heavily based off of Discourse's official Cloud Installation instructions found here. If anything is unclear, it may be helpful to refer to those instructions for further guidance.
Before you begin, first choose an FQDN (i.e. a domain name). This can be forum.spinalcordmri.org if you're starting completely from scratch (i.e. there is no currently-running forum instance) or something like forum.dev.spinalcordmri.org (if you're setting up a separate test server).
NB: For the rest of these instructions, we will use the placeholder
{subdomain}.spinalcordmri.org, but make sure you substitute in whichever domain name you decided on.
First, make sure you have an account with DigitalOcean. Next, contact someone on the admin team; they will add you to the NeuroPoly DigitalOcean team. This will grant you access to the Spinal Cord MRI project.
From the Spinal Cord MRI project page, you can create a new "droplet", which is DigitalOcean's name for cloud servers.
Next, create a droplet using the following settings:
- Choose an image: Distribution -> Ubuntu -> Version: 22.04 (LTS) x64
- Choose a plan:
- Shared CPU: Basic
- CPU Options: Regular
- Price: $6/mo (Specs: 1 GB/1 CPU, 25 GB SSD Disk, 1000 GB Transfer)
- Add block storage: Skip this step
- Choose a datacenter region: Toronto 1
- VPC Network: Skip this step
- Authentication: If your key isn't present yet, press "New SSH Key" and follow the instructions to authenticate the device you're currently using.
- Select additional options: Check "Monitoring" and leave the rest blank.
- How many Droplets?: 1 Droplet
- Choose a hostname: This is a very critical step! Make sure you enter the same domain name you chose earlier (
{subdomain}.spinalcordmri.org) - Add tags: Skip this step
Once the droplet has finished creating, you should see a public IP address on the droplet's page that looks something like ipv4: 142.93.152.255. You can then use this IP address to test that you've set the hostname correctly by running the following command on any linux machine:
user@device:~$ dig +short -x 142.93.152.255
{subdomain}.spinalcordmri.org.Once you've confirmed that it returns the domain name you chose, you're all set to connect to the droplet.
Before we make any changes to the server, though, we first have to configure some DNS settings.
First, make sure you have an account with Namecheap. Again, you will need to contact someone on the admin team; they will grant you permissions for specific domains on a case-by-case basis. In this case, you will want access to the spinalcordmri.org domain, which will grant you access to the Spinalcordmri.org Dashboard.
Next, on the dashboard, navigate to the Advanced DNS tab. Then, modify the following sections:
First you will want to make note of two things:
- The subdomain label of your domain name. For
{subdomain}.spinalcordmri.org, you will use{subdomain}to identify your subdomain. - The IP address of the server (from step 1). For
{subdomain}.spinalcordmri.org, the IP address was142.93.152.255.
You can now use the red "Add New Record" button to add 3 new records in the following form:
| Type | Host | Value | TTL |
|---|---|---|---|
| A Record | {subdomain} | 142.93.152.255 | Automatic |
| TXT Record | {subdomain} | v=spf1 a mx ip4:142.93.152.255 ~all | Automatic |
| TXT Record | _dmarc.{subdomain} | v=DMARC1; p=none | Automatic |
These entries accomplish the following:
- A Record: Maps the
{subdomain}.spinalcordmri.orgsubdomain to the DigitalOcean droplet's IP address. - SPF Record: This will help later on when setting up the forum's mail server. It authorizes the DigitalOcean droplet as a valid sender of mail, which helps prevent the forum's notification emails from being caught as spam.
- DMARC Record: This will help later on when setting up the forum's mail server. It defines what should happen in case a message sent by the forum fails to be authenticated.
You can double-check the current values by running the following dig commands from an external device, not the server itself:
user@device:~$ dig +short {subdomain}.spinalcordmri.org
142.93.152.255
user@device:~$ dig +short TXT {subdomain}.spinalcordmri.org
"v=spf1 a mx ip4:159.89.119.65 include:spf.efwd.registrar-servers.com ~all"
user@device:~$ dig +short TXT _dmarc.{subdomain}.spinalcordmri.org
"v=DMARC1; p=none"NB: Please note that it may take up to a few hours for changes to these records to propagate. So, if you're running
digand see no change, that's why!
Next, scroll down to the "Mail Settings" section and find the table containing MX records. Then, add the following entry:
| Type | Host | Mail Server | Priority | TTL |
|---|---|---|---|---|
| MX Record | {subdomain} | {subdomain}.spinalcordmri.org. | 0 | Automatic |
Again, this setting will help us later when we set up our mail server. (There's not much here to talk about, since details such as 'priority' are only really relevant for setups with multiple mail servers.)
You can double-check the current value of this record by running the following command:
user@device:~$ dig +short MX {subdomain}.spinalcordmri.org
0 {subdomain}.spinalcordmri.org.Now that NameCheap is configured, we can set up the server itself, starting with its hostname.
You can connect to the server either through SSH (assuming your local device contains the SSH key you added earlier):
user@device:~$ ssh root@{subdomain}.spinalcordmri.org
Welcome to Ubuntu 23.10 (GNU/Linux 6.5.0-13-generic x86_64)
Last login: Fri Dec 9 23:06:11 2023 from 162.243.188.66
root@forum:~# Or, you can navigate to your droplet's main page, then click the "Console" button in the top-left corner:
From here, you can make changes to the server itself.
NB: If the initial DNS records haven't propagated yet, you can also SSH directly into the server using its IP address (
ssh root@142.93.152.255) while you wait.
First, connect to the server using either SSH or the built-in DigitalOcean console. Next, edit two files using the text editor of your choice:
root@forum:~# sudo nano /etc/hostname
{subdomain}.spinalcordmri.org
root@forum:~# sudo nano /etc/mailname
{subdomain}.spinalcordmri.orgAdditionally, edit the /etc/hosts file to ensure that the hostname of the server ({subdomain}.spinalcordmri.org) maps to the server's permanent IP address instead of 127.0.1.1.
root@forum:~# sudo nano /etc/hosts
# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.debian.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
# /etc/cloud/cloud.cfg or cloud-config from user-data
#
142.93.152.255 {subdomain}.spinalcordmri.org
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allroutersReboot the server by running reboot, then reconnect and ensure that the following command also returns the same domain name as above:
root@forum:~# hostname
{subdomain}.spinalcordmri.orgWe run a small mail server on the same server as Discourse for it to send notifcations and password resets. Discourse recommends using a cloud service like MailGun or Amazon SES or SendGrid, but our usage is so small that the overhead (and risk) of outsourcing is high. Mail servers are something of an arcane art now, but never fear, these instructions will make it work.
First, we must install our mail server software of choice, opensmtpd.
# apt-get install -y opensmtpd
Replace the existing contents of /etc/smtpd.conf with the following:
pki {subdomain}.spinalcordmri.org cert "/var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.cer"
pki {subdomain}.spinalcordmri.org key "/var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.key"
listen on eth0 tls-require pki {subdomain}.spinalcordmri.org
listen on eth0 tls-require pki {subdomain}.spinalcordmri.org auth port 587
table aliases file:/etc/aliases
action "local_mail" maildir "~/.mail" alias <aliases>
match for local action "local_mail"
action "relay_mail" relay helo "{subdomain}.spinalcordmri.org"
match from auth for any action "relay_mail"
Edit the following file:
sudo nano /etc/systemd/system/multi-user.target.wants/opensmtpd.service
And make the following change:
-ExecStart=/usr/sbin/smtpd
+ExecStart=/usr/sbin/smtpd -vWe need an SMTP account that Discourse can send mail via. opensmtpd simply uses the OS's users by default, so we will make an OS user for outgoing emails.
- Run a password generator and save the result somewhere secure. (You will need the result for the remainder of this tutorial.)
- If you have a password manager, see if it has a password generator built in. Otherwise there's Diceware and xkpasswd and xkcdpass and pwgen
- Note: Do NOT use symbols in your password. It can cause weird bugs in Discourse's YAML config.
- Create the user
forum@{subdomain}.spinalcordmri.orgusing the following commands:useradd -s /usr/sbin/nologin forum: Creates the user account.passwd forum: Sets the password for the account. Paste the password you generated earlier.
NB: This username is not the same as what's on the email headers, because
opensmtpdallows authenticated users to spoof their identities, and we want to send asnoreply@{subdomain}.spinalcordmri.org.
Connect to the droplet server provided by Digital Ocean, then do:
-
Install Docker:
wget -qO- https://get.docker.com/ | sh -
Clone Discourse deploy
mkdir /var/discourse git clone https://github.com/discourse/discourse_docker.git /var/discourse cd /var/discourse -
Increase timeout values
Before we can install Discourse, we must make a small tweak to the default Discourse Docker container template.
Open the file
/var/discourse/samples/standalone.ymland add the following lines to the SMTP section:## TODO: The SMTP mail server used to validate new accounts and send notifications # SMTP ADDRESS, username, and password are required # WARNING the char '#' in SMTP password can cause problems! DISCOURSE_SMTP_ADDRESS: smtp.example.com #DISCOURSE_SMTP_PORT: 587 DISCOURSE_SMTP_USER_NAME: user@example.com DISCOURSE_SMTP_PASSWORD: pa$$word + DISCOURSE_SMTP_OPEN_TIMEOUT: 60 + DISCOURSE_SMTP_READ_TIMEOUT: 60 #DISCOURSE_SMTP_ENABLE_START_TLS: true # (optional, default true) #DISCOURSE_SMTP_DOMAIN: discourse.example.com # (required by some providers) #DISCOURSE_NOTIFICATION_EMAIL: noreply@discourse.example.com # (address to send notifications from)
The default values for these options are
5seconds, which is too strict for our (slower) internal mail server. So, we must increase the timeout to avoidNet::ReadTimeouterrors. -
Install Discourse
./discourse-setup Hostname : {subdomain}.spinalcordmri.org Email : [initial administrator's email address -- will be used for 1st admin account creation] SMTP address : {subdomain}.spinalcordmri.org SMTP port : 587 SMTP username : forum SMTP password : xxxxxxxxxxxxxxxxxxxxxxx Notification : noreply@{subdomain}.spinalcordmri.org Let's Encrypt : neuropoly-admin@liste.polymtl.ca Maxmind License : [Enter]
This will download the Discourse base image, then build the image and initialize the Docker container.
Before continuing, make sure that Discourse has generated its SSL certs. (They will be generated automatically, so long as you provide an email address for Let's Encrypt.)
The files exist in /var/discourse/shared/standalone/ssl/:
root@forum:~# ls -l /var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.{cer,key}
-rw-r--r-- 1 root root 3799 Dec 5 08:33 /var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.cer
-rw------- 1 root root 3247 Dec 5 08:33 /var/discourse/shared/standalone/ssl/{subdomain}.spinalcordmri.org.key
These files must be present for opensmtpd to be able to send mail correctly, as per the previous /etc/smtpd.conf configuration we pasted in during a previous step.
Once you've confirmed that the SSL certs are present, you can start the mail server with the following command:
systemctl enable --now opensmtpd
You should be able to connect to the mail server by running:
root@forum:~# nc {subdomain}.spinalcordmri.org 587
220 {subdomain}.spinalcordmri.org ESMTP OpenSMTPDTo create the first admin account on the newly-installed Discourse forum, email delivery must be working, because Discourse will need to send out a confirmation email.
To test email delivery, rather than sending messages to specific personal addresses, we use https://www.mail-tester.com/. This page is very helpful for finding issues missed during setup, especially around spamminess. Email is very very complicated and this helps a lot.
Go to https://www.mail-tester.com/ and copy the email address it gives you. You can use this address alongside one of three testing methods:
-
Simple test (
mail)You will first need to install Mailutils using
sudo apt-get install mailutils. Then, you can run:echo "Test Message" | mail -s "This is a message" somethingsomething@mail-tester.com
-
Complex test (
swaks)You will first need to install the "Swiss Army Knife for SMTP" (
swaks) usingsudo apt-get install swaks. Then, you can run:# You will need to enter the password that you set during the previous "User Setup" section!! swaks --to me@example.com --from noreply@{subdomain}.spinalcordmri.org --server {subdomain}.spinalcordmri.org -p 587 --auth-user forum --tls-verify --tls -
Discourse-specific test
First,
cdinto/var/discourse. Then, run the following command:./discourse-doctor
It will run some automated tests, then it will prompt you for an email to send a test message to.
NB: There is a possibility that you may encounter a
Net::ReadTimeouterror. In that case, edit the/var/discourse/containers/app.ymlfile to increase the values ofDISCOURSE_SMTP_OPEN_TIMEOUTandDISCOURSE_SMTP_READ_TIMEOUT. Then, restart the docker container by runningcd /var/discourse,./launcher destroy app, and./launcher start app.
With each of these tests, your goal here is to ensure that:
- The email message gets delivered.
- Your mail-tester score is relatively high (ideally at least 9/10, but at the very minimum, 7/10).
Once you send your test message, you can click "View my results" on the mail-tester webpage. It will give you feedback on things that need to be changed. So, try to iteratively address the feedback, send a new test message, and refresh the page until you get a high score.
Navigate to the page https://{subdomain}.spinalcordmri.org/, and follow the instruction to create an admin email account. Discourse will send you a confirmation email. If you've set up mail correctly, you should receive the email and be able to finish making your account.
Once you log in for the first time, you will be presented with an onboarding flow that will ask for things like "Community name" and "Point of contact". Feel free to put some sensible defaults (e.g. "Spinalcordmri.org", "neuropoly-admin@liste.polymtl.ca", etc.). However, if you will be loading a previous backup of the forum, then these settings will be overwritten anyway, making them moot.
If there is a currently-running instance of the forum, you can download backups from the Backups page.
Next, you need to upload this backup to the forum. You can try using the Backups page on the dev forum, however sometimes uploads will fail inexplicably with (due to the large size of the backups). So, you may need to instead use FileZilla to transfer files to the Digital Ocean Droplet. Specifically, the backup will need to be uploaded to the /var/discourse/shared/standalone/backups/default folder.
Finally, you will need to enable the "allow restore" setting on the dev server's Settings page. Then, you can return to the Backups page and restore the uploaded backup.
Currently, the only mandatory plugin in use by the forum is discourse-solved. To install it, please follow the instructions from the Install Plugins in Discourse documentation page.
Note: These steps are not currently used in the production server for spinalcordmri.org
6.1 Configuring Google login for Discourse (reference)
Go to https://console.developers.google.com, click on Credentials and create a new Project.
- Project name
Forum spinalcordmri - Project id
forum-spinalcordmri
Select Credentials in the left menu, Create credentials and OAuth client ID type for the credentials.
- Application type
Web application - Name
Forum spinalcordmri.org - Authorized JavaScript origins
http://{subdomain}.spinalcordmri.org - Authorized redirect URIs
http://{subdomain}.spinalcordmri.org/auth/google_oauth2/callback
Configure your OAuth Consent Screen
- Product name shown to users
Forum spinalcordmri.org - Homepage URL
http://www.spinalcordmri.org/ - Privacy policy URL
http://www.spinalcordmri.org/
Click Library in the left menu and you’ll see a huge list of Google API’s. Find Google+ API and enable them.
The API will create google_client_id and google_client_secret which you can add under http://{subdomain}.spinalcordmri.org/admin/site_settings/category/login, after checking enable google oauth2 logins
6.2 Configure GitHub login for Discourse (reference)
Under github.com/spinalcordmri, click Settings (the gear icon), then look for OAuth Applications in the left menu. Select Register new application.
- Application name
Forum spinalcordmri
- Homepage URL
http://{subdomain}.spinalcordmri.org/
- Application description
Forum spinalcordmri
- Authorization callback URL
http://{subdomain}.spinalcordmri.org//auth/github/callback
The app will create github_client_id and github_client_secretwhich you can add under http://{subdomain}.spinalcordmri.org/admin/site_settings/category/login, after checking enable github logins