Migrate documentation from wiki to Read The Docs

[cirulls]

Summary

I'm raising this issue here to collect the conversation that happened in the past in various xspec repos, namely #1053 (comment), xspec/xspec.github.io#2 and xspec/xspec.github.io#3

Using the GitHub wiki for storing xspec documentation has the following shortcomings:

1. We cannot create PRs to fix documentation errors as discussed in Example in https://github.com/xspec/xspec/wiki/Writing-Scenarios#function-scenarios uses true() as parameter value but accompanying text explains false() is being passed in #1053 (comment). The wiki works on a single branch and any change to the docs needs to be applied directly to that branch, no review mechanism is currently possible on the wiki.
2. There is no search functionality for the content (the only search functionality provided by the wiki is search by title page).
3. When a new feature is developed, we have to wait for the feature to be merged into the master branch to create the relevant documentation.
4. There is no concept of documentation versioning in the wiki (e.g. a feature may have a difference behaviour from - say - version 1 to version 2 but the wiki documentation can't reflect this).

My initial idea was to migrate the documentation to the xspec website as it allows the usual PR review workflow to update documentation. However, I found a neater solution in Read the Docs which is a popular framework for storing and sharing documentation.

I set up a proof of concept with a couple of documentation pages at https://xspec-docs.readthedocs.io

This is currently linked to my personal GitHub repository where the docs files are stored at https://github.com/cirulls/xspec-docs/tree/main/docs/source

I believe setting up a Read The Docs site fixes the following shortcomings of the wiki:

1. We can use the usual workflow PR -> review -> merge -> publish new documentation.
2. There is a full search functionality for content. You can see it in action in this screenshot:
   
   More details on how search is implemented in Read The Docs are in https://docs.readthedocs.io/en/stable/development/search.html and https://docs.readthedocs.io/en/stable/guides/advanced-search.html
3. We can prepare the documentation in a PR while the feature is being developed and update the docs when the feature is merged.
4. Read The Docs supports versioning.

But I can also see some issues in using Read The Docs:

1. The documentation is in a separate repository so there is a separation between the code and the documentation. This could be a good or a bad thing but I want to raise it.
2. As far as I could read, using the hosted version of Read The Docs requires to write the documentation in reStructuredText which is a documentation format popular in Python. Markdown is supported only on the deploy-yourself Read the Docs (I need to do more digging here but this is what I understood so far). It is fairly easy to pick up and there are tools like pandoc to automatically convert from Markdown to reStructuredText (in fact, I have been using pandoc for the proof of concept with very little manual editing). Again, I want to raise this as it is yet another documentation format that we don't currently use in XSpec.

Next steps

I would like to get some feedback on the proof of concept at https://xspec-docs.readthedocs.io and see if it is worth for me to continue with the migration. In particular, I'd like feedback to the following questions:

1. Do you find the proof of concept interesting? Could Read The Docs be a good tool for migrating our documentation? Can you see any other advantages/disadvantages?
2. Shall I continue with migrating XSpec docs to Read the Docs? If so, I'll like to create a new repo in the xspec organisation (instead of using my personal repo) or move the ownership to the xspec organisation and continue migrating the docs from there.
3. If 2 is yes, is anyone interested in helping out with the docs migration? This would involve migrating the current docs, reviewing PRs, testing the Read the Docs website, contributing with ideas, etc.

Thanks!

[galtm]

Hi, @cirulls .

Thanks for exploring this idea! It looks interesting. I'm not very familiar with the xspec website, so I wondered what makes Read The Docs better.

Given that the Read The Docs approach would put the doc in a separate GitHub repo compared to the XSpec code, is there a way to make it easy for XSpec contributors or XSpec users to update the doc and code together if that's what they want? Is that what submodules do?

The Home page of the XSpec wiki says 43 pages. Is that the number of pages we would need to migrate, if we go ahead? I should be able to pitch in, though it might be spread out over a period of time.

[cmarchand]

I missed this one !

Read the docs seems to be a good solution, but :

• there will be now 3 sites on xspec, github, xspec.io, and xspec-docs.read...), which
  is not the best thing for a centralized access
• would we be able to automate documentation publication to this new site ?
• do we still need xspec.io site ?