Automatic RoamResearch backup

This script helps you backup your RoamResearch graphs!

This script automatically

• Downloads a markdown archive of your RoamResearch workspace
• Downloads a json archive of your RoamResearch workspace
• Unzips them to your git directory
• Commits and pushes the difference to Github

Demo

See it in action!. This repo is updated using roam-to-git.

Why to use it

• You have a backup if RoamResearch loses some of your data.
• You have a history of your notes.
• You can browse your Github repository easily with a mobile device

Use it with Github Actions (recommended)

Create a (private) Github repository for all your notes

With gh: gh repo create notes (yes, it's private)

Or manually

Configure Github secrets

• Go to github.com/your/repository/settings/secrets

Add 3 (separate) secrets where the names are

ROAMRESEARCH_USER

ROAMRESEARCH_PASSWORD

ROAMRESEARCH_DATABASE

• Refer to env.template for more information

• when inserting the information, there is no need for quotations or assignments

Add GitHub action

cd notes
mkdir -p .github/workflows/
curl https://raw.githubusercontent.com/MatthieuBizien/roam-to-git-demo/master/.github/workflows/main.yml > \
    .github/workflows/main.yml
git add .github/workflows/main.yml
git commit -m "Add github/workflows/main.yml"
git push --set-upstream origin master

Check that the Github Action works

• Go to github.com/your/repository/actions
• Your CI job should start in a few seconds

Note:

If the backup does not automatically start, try pushing to the repository again

Use it locally

Note: if your file system is not case-sensitive, you will not backup notes that have the same name in different cases

Install Roam-To-Git

With pipx (if you don't know pipx, you should look at it, it's wonderful!)

pipx install git+https://github.com/MatthieuBizien/roam-to-git.git

Create a (private) Github repository for all your notes

With gh: gh repo create notes (yes, it's private)

Or manually

Then run git push --set-upstream origin master

Configure environment variables

• curl https://raw.githubusercontent.com/MatthieuBizien/roam-to-git/master/env.template > notes/.env
• Fill the .env file: vi .env
• Ignore it: echo .env > notes/.gitignore; cd notes; git add .gitignore; git commit -m "Initial commit"

Manual backup

• Run the script: roam-to-git notes/
• Check your Github repository, it should be filled with your notes :)

Automatic backup

One-liner to run it with a cron every hours: echo "0 * * * * '$(which roam-to-git)' '$(pwd)/notes'" | crontab -

NB: there are issues on Mac with a crontab.

Debug

Making roam-to-git foolproof is hard, as it depends on Roam, on Github Action or the local environment, on software not very stable (pyppeteer we still love you 😉 ) and on the correct user configuration.

For debugging, please try the following:

• Check that the environment variables ROAMRESEARCH_USER, ROAMRESEARCH_PASSWORD, ROAMRESEARCH_DATABASE are correctly setup
• Login into Roam using the username and the password. You may want to ask a new password if you have enabled Google Login, as it solved some user problems.
• Run roam-to-git --debug to check the authentification and download work
• Look at the traceback
• Look for similar issues
• If nothing else work, create a new issue with as many details as possible. I will try my best to understand and help you, no SLA promised 😇

Task list

Backup all RoamResearch data

• Download automatically from RoamResearch
• Create Cron
• Write detailed README
• Publish the repository on Github
• Download images (they currently visible in Github, but not in the archive so not saved in the repository 😕)

Format the backup to have a good UI

Link formatting to be compatible with Github markdown

• Format [[links]]
• Format #links
• Format attribute::
• Format [[ [[link 1]] [[link 2]] ]]
• Format ((link))

Backlink formatting

• Add backlinks reference to the notes files
• Integrate the context into the backlink
• Manage / in file names

Other formatting

• Format {{TODO}} to be compatible with Github markdown
• Format `{{query}}``

Make it for others

• Push it to Github
• Add example repository
• Make the backup directory configurable
• Publicize it
  • RoamResearch Slack thread
  • RoamResearch Reddit
  • Twitter

Some ideas, I don't need it, but PR welcome 😀

• Test it/make it work on Windows
• Pre-configure a CI server so it can run every hour without a computer Thanks @Stvad for #4!