Added Read the Docs support

[jordiferrero]

Hi all,

I have wanted to do this for very long...
I have now added support for the read the docs. This means that we can have explanations on the functions we write/methodologies in a nice formatted way.

As far as I am aware, such guides can be added to user_guide/file.rst. Feel free to add content there.
I have a list of pages I would like to add for the analysis of PL data (coming soon).
@jlaehne Maybe you could take care on the Jacobian transform documentation if you feel like explaining in a bit more detail how LumiSpy does this transform. That'd be a great opportunity to test it out too.

I believe having this will help external users to get started, as a nicely explained guide (like the EDX analysis page in HyperSpy docs) can really become "the place to check" on how to do data analysis of a particular data signal. I think the demo notebooks are great, but they can also get a bit messy.

Happy to hear suggestions!

PS: The Read the docs should trigger a build every time there is a merge. It will loop though all the docstrings and update the docs automatically.

[jordiferrero]

This issue can be closed anytime, as it has already been resolved. It was more of an announcement.

[jlaehne]

Great, thanks @jordiferrero for starting out on this. I have been wanting to suggest this for a while as well. Now that the basic setup is done, it should be straightforward to contribute.

I would propose that I redirect the domain https://lumispy.org to the documentation then.

Given our logo, we should maybe use a dark theme as standard. For example that is possible with the theme used by https://kikuchipy.org (though the background behind the logo is not completely black there).

[jlaehne]

Given our logo, we should maybe use a dark theme as standard. For example that is possible with the theme used by https://kikuchipy.org (though the background behind the logo is not completely black there).

See #103

[jlaehne]

ToDo:

• Decide on whether to use .rst or .md format,
• Adapt color behind logo, provide dark theme (see add dark theme support #103),
• Add badges to index page (see add dark theme support #103),
• Write introductory page that gives an overview of the signal types under development and summarizes functionality present and planned (See Documentation introduction #111),
• Decide whether tutorials (lumispy-demos) should be integrated and how -- add links where appropriate,
• Write documentation about energy/wavenumber conversion and Jacobian, mention variance (see Documentation: axis conversion #113),
• Write documentation on fitting,
• Add documentation on variance to fitting chapter,
• Document metadata structure (see Metadata structure #109),
• Give overview of other utility functions (see Write utility documentation #128),
• Merge Installation guide in the main folder into the new one in the documentation and correct links,
• Mention license somewhere in the documentation (see fix license statements, add to user guide #108),
• Check that all docstrings are properly parsed in the api documentation (see Fix inconsistent docstring formatting #107),
• Link realeasing_guide (convert to .rst) from the contributor's instructions (see add dark theme support #103),
• Color of math in dark mode (see Documentation: axis conversion #113),
• Redirect https://lumispy.org to the documentation (needs to be set up in Read the Docs settings).

[jordiferrero]

How did you get a .org domain? First news to me!

All ToDo's I completely agree and we are getting started in Cambridge to cover some of those initial documentation files.

One last comment I had was whether to use .md or .rst files for the documentation. As far as I am aware, both can be compiled by Readthedocs. I used .rst because that's default, but I find .md cleaner and easier to write. Any thoughts @jlaehne?

[jlaehne]

How did you get a .org domain? First news to me!

Reserved it about a year ago on private account (costs only 18 Euro a year). So far, it points to the GitHub page.

All ToDo's I completely agree and we are getting started in Cambridge to cover some of those initial documentation files.

Feel free to add points to the list! Just thought it would be good to document the process/progress. If we add the associated PRs to the end of the points, we can also document who is working on what.

One last comment I had was whether to use .md or .rst files for the documentation. As far as I am aware, both can be compiled by Readthedocs. I used .rst because that's default, but I find .md cleaner and easier to write. Any thoughts @jlaehne?

I can live with both. But converting some file types yesterday, I agree that .md looks cleaner - and we should decide now, before there is more in the docs. @ericpre @hakonanes is there any good reason not to opt for .md? I guess the choice for HyperSpy and kikuchipy is historic?

[ericpre]

I don't know, but I would expect rst format to support more features? For example, can you use all features provided by these extensions with the markdown fromat?

[jlaehne]

Here is some reading on MarkDown vs ReStructuredText:
https://www.zverovich.net/2016/06/16/rst-vs-markdown.html

Seems that markdown is definitely more limited. In particular when it comes to e.g. math in the documentation (which we will have). Note that also the docstrings are in ReST and using different languages there makes little sense.

As ReST is the standard in python documentation and used in related projects such as HyperSpy, I would stick to it.

What we could do, is to use the markdown extension for sphinx (https://www.sphinx-doc.org/en/master/usage/markdown.html) and then individual pages, such as the ones residing in the main folder can remain in MD.

[hakonanes]

Given our logo, we should maybe use a dark theme as standard. For example that is possible with the theme used by https://kikuchipy.org (though the background behind the logo is not completely black there).

The Furo theme is light (default) or dark depending on the request by the user's browser. We use default Furo without customization in kikuchipy. Note that some care must be taken with Matplotlib figures as backgrounds should be explicitly set to white so that axes labels and such are readable in the dark theme. HyperSpy's axes_manager should also be explicitly printed with print(s.axes_manager), otherwise the "fancy" display isn't readable in the dark theme.

Redirect https://lumispy.org/ to the documentation.

If I remember correctly, it was as simple as pointing to it via the Read the Docs admin pages.

I guess the choice for HyperSpy and kikuchipy is historic?

As with most development choices in kikuchipy (testing, documentation, CI, and so on), including the choice of .rst for non-user-guide pages (user guides are Jupyter Notebooks run and converted with nbsphinx), kikuchipy follows in HyperSpy's footsteps. I've no experience with converting other than from ReST to MD for the changelog using pandoc.

[jlaehne]

Redirect https://lumispy.org/ to the documentation.

If I remember correctly, it was as simple as pointing to it via the Read the Docs admin pages.

@jordiferrero can you add the domain on readthedocs as described in this link? I would then set up the pointer on the provider side.

[jlaehne]

Given our logo, we should maybe use a dark theme as standard. For example that is possible with the theme used by https://kikuchipy.org (though the background behind the logo is not completely black there).

The Furo theme is light (default) or dark depending on the request by the user's browser. We use default Furo without customization in kikuchipy. Note that some care must be taken with Matplotlib figures as backgrounds should be explicitly set to white so that axes labels and such are readable in the dark theme. HyperSpy's axes_manager should also be explicitly printed with print(s.axes_manager), otherwise the "fancy" display isn't readable in the dark theme.

Thanks @hakonanes for the feedback and advice. I really like the Furo theme as well, but given that our logo's background is black (and I really prefer that version over the white one), the best solution to me was to adapt the theme from Pillow to have a black background behind the logo both in the dark and light version. But thanks for the comments for possible caveats of the documentation in dark theme.

[jlaehne]

@jordiferrero, we should discuss whether we should integrate the demos into the user guide.

I basically see three options,

• actually write the main parts of the userguide directly as tutorials using jupyter-notebooks, as done on https://kikuchipy.org
• keep them separate, but add an extra section 'Tutorials' to the menu that integrates the notebooks into the documentation
• just integrate a link to the lumispy-demos repository in the menu

[jordiferrero]

Two comments:

1. Thank you @jlaehne for the amazing additions you have done in the last month to the Read the Docs! It is really coming together!
2. For the tutorials any option works. In my opinion, I see documentation more of a "text book" file where the functions are explained with some physical explanation (best example is the ML section in the Hyperspy docs). The demos are "plug-and-play" files for people to actually use (without really understanding what's happening behind the scenes). So I would "keep them separate", knowing that this option implies x2 more things to keep updating over time.

Happy to discuss further!

[ericpre]

2. For the tutorials any option works. In my opinion, I see documentation more of a "text book" file where the functions are explained with some physical explanation (best example is the ML section in the Hyperspy docs). The demos are "plug-and-play" files for people to actually use (without really understanding what's happening behind the scenes). So I would "keep them separate", knowing that this option implies x2 more things to keep updating over time.

I strongly agree with this. The risk with using notebook is that it is very easy to have too much code and not enough explanation, which means that the section is more a demo than a tutorial or a user guide.
Using notebook is great to make sure the documentation is up to date and works fine but it requires the discipline of focusing on the explanation!

[jlaehne]

Redirect https://lumispy.org/ to the documentation.

If I remember correctly, it was as simple as pointing to it via the Read the Docs admin pages.

@jordiferrero can you add the domain on readthedocs as described in this link? I would then set up the pointer on the provider side.

At the moment lumispy.org points to the lumispy.readthedocs.org - but with the settings described here, we should be able to make it work so that it looks as if you stay on lumispy.org

[jordiferrero]

Sorry for the delay.
I have now set this up but I still need @jlaehne to set up the following in the lumispy.org settings provider:

• I have done steps 1-5.
• What is left to do is to setup the DNS provider accordingly. I believe you need to point lumispy.org to readthedocs.io. Since we want to point it to the root domain of lumispy.org, you should use an ANAME or ALIAS record (not 100% sure what that means).

I do not have access to the domain website, so I need you help :-)

[jlaehne]

• What is left to do is to setup the DNS provider accordingly. I believe you need to point lumispy.org to readthedocs.io. Since we want to point it to the root domain of lumispy.org, you should use an ANAME or ALIAS record (not 100% sure what that means).

I'm checking with the provider how to set that up, seems that they may have to edit the entry.

[jlaehne]

• What is left to do is to setup the DNS provider accordingly. I believe you need to point lumispy.org to readthedocs.io. Since we want to point it to the root domain of lumispy.org, you should use an ANAME or ALIAS record (not 100% sure what that means).

I'm checking with the provider how to set that up, seems that they may have to edit the entry.

Aargh, the provider does not support ANAME or ALIAS (apparently a proprietary, non-standard add-on to DNS). Only possibility apart from migrating the domain seems to be setting a CNAME record for docs.lumispy.org (lumispy.org could then redirect there). For the moment, could you please change the readthedocs settings to the domain docs.lumispy.org, @jordiferrero .

[ericpre]

Would it be possible to get lumispy.org fixed as it make the check links build failed in the rosettasciio repository?

[jordiferrero]

It is all working now with docs.lumispy.org as the CNAME.
readthedocs and lumispy.org all redirect to docs.lumispy.org

[jlaehne]

The following points were still remaining on the checklist:

• Write documentation on fitting,
• Add documentation on variance to fitting chapter,
• Merge Installation guide in the main folder into the new one in the documentation and correct links,

[jlaehne]

As you had been at it already with #141, and both installation pages are from you -- could you have a look at the installation guides again @jordiferrero?

For the fitting and variance documentation, would that be something for you @LMSC-NTappy?

[LMSC-NTappy]

Thanks for the nice work @jlaehne !

For me it looks good, I don't think much more than this is needed in the fitting and variance documentation.

I know it will sound like an attempt at self promotion (which it is), but do you think it could be beneficial to add a reference to my article covering the topic as well? Feel free to refuse :)

https://aip.scitation.org/doi/10.1063/5.0080486

[jlaehne]

Thanks for the nice work @jlaehne !

For me it looks good, I don't think much more than this is needed in the fitting and variance documentation.

I know it will sound like an attempt at self promotion (which it is), but do you think it could be beneficial to add a reference to my article covering the topic as well? Feel free to refuse :)

https://aip.scitation.org/doi/10.1063/5.0080486

There is the additional page on fitting in general, which is still a stub: https://docs.lumispy.org/en/latest/user_guide/fitting_luminescence.html
Motivated by your paper, I thought you could maybe help extending that one, and yes a reference could be included (even though unfortunately the paper and associated jupyter-notebooks do not include a reference to LumiSpy, but only to HyperSpy).

[LMSC-NTappy]

Fair enough, I will expend the doc on fitting when I find time and cite my paper (Indeed, I did not make use of lumispy then). Sorry it might take a few month before I am able to find time to do it, as I am currently writing my dissertation =].

Cheers

Nicolas

[jlaehne]

Fair enough, I will expend the doc on fitting when I find time and cite my paper (Indeed, I did not make use of lumispy then). Sorry it might take a few month before I am able to find time to do it, as I am currently writing my dissertation =].

@Attolight-NTappy, could you have another look at https://docs.lumispy.org/en/latest/user_guide/fitting_luminescence.html#signal-variance-noise and maybe add a small paragraph on the variance. Last thing on the list of this issue.