Quarto and Obsidian - a great combination

[beyond2060]

Description

I like keeping the number of tools I use to a minimum, and try to really get to know them. I use Quarto for my own website, as well as closed websites for clients. In the latter I include dynamically generated graphs with Python, and I use VS code to edit those sites. However for my own website I don't really use that functionality. I've discovered that Obsidian is a really great tool for editing a Quarto site if you're not using the dynamic content functionality. The markdown editor in the Obsidian app is much nicer to use than VS Code for editing content. Obsidian only requires a few config changes to make it ideal to use as a Quarto site editor. I've added a page on my (Quarto generated) website with a list of the config changes I make to Obsidian to make it suitable to use as an editor for Quarto. Here's a link: Using Obsidian

[neeldhara]

This is really nice to know! Once I started using things like callouts a lot, transitioning seamlessly between both tools became hard, so I am now sticking mostly to VS Code and Quarto, and I love Quarto so much that I moved my notes over from Obsidian to Quarto (my "PKM" is now a local quarto site :) ).

But if you can pull off working with both Quarto and Obsidian and leverage the strengths of each --- and this is something I did for as long as I could --- that's awesome!

[beyond2060]

Thanks. I am still working on my setup. Obsidian is great because of all the plugins it has and the editing experience is nicer. I tried using Quarto as a PKM but didn't like that I had to compile it to view the content properly. But I'm still working on my setup. Once I have used the two together for a while I'll do a writeup.

[beyond2060]

Just FYI, I'm mainly using Quarto now as my PKM, but I'm also using Obsidian as a note inbox. Obsidian is really good for certain things - there are loads of great plugins - so for example extracting the contents of a web page as Markdown.

[neeldhara]

Interesting. Yeah, I absolutely love Obsidian, and am especially fascinated by their latest releases (properties, table editing, etc).

However, Quarto + VS Code has grown on me now and it feels reasonably stable, so except occasionally I am not switching around on apps (I do have to go back to Notion and Google Docs for collaborative projects, and Obsidian for some of my older notes).

[fbob]

Too bad the formats aren't unified at the moment. Seamless communication between Obsidian and Quarto's .qmd files would be a huge benefit in terms of potential usage and community development.

[beyond2060]

Agreed absolutely! There are some amazing plug-ins with Obsidian, it would be wonderful to be able to use them with Quarto. It would be the perfect combination. But at the moment there are too many minor frustrations to work with both I think.

[Gewerd-Strauss]

I know this is super absurdly niche, but if you are persistent (or dumb) enough like me, it actually works.

Granted, I have a rather odd workflow, in that I use obsidian for manuscript-writing and super-rudimentary knowledge management. But I mostly work on a per-project basis, and as such vaults tend to be rather specifically purpose-built.

In my experience, there are a few things that obsidian excels at compared to quarto:

1. embedding note-sections
   Obsidian's embed ![[file#section]] is imo much more sensible than quarto's {{< include _basics.qmd >}}, particularly cuz we get preview on obsidian's embeds - which combines nicely with plugins such as the "Hover Editor".

2. General level of ease for using hotkeys on absolutely everything

3. A markdown-linter which works better in my experience than compared plugins for VS Code. For RStudio, to my knowledge no linter counterpart exists.

---

Now, with that said stuff like computational source-code should still be written externally. Obsidian should really not be forced to behave as an coding interface.

---

Frustrations and incompatibilities

Keep in mind that I mostly keep my stuff in markdown, and if I render to "normal-people-outputs", it is almost exclusively done to html, docx and pdf

Obsidian & Quarto has a couple key incompatibilities:

1. embeds. As mentioned, I prefer Obsidian's embeds, particularly as I can embed sections of a note instead of the entire note itself. As far as I know, quarto does not allow such things.
2. callouts. For me this is no issue because I just write quarto-compliant ones most of the time. I write manuscripts mostly, so I don't really care about how it looks in markdown - I am interested in the rendered output. Quarto's callout-format is reasonably simple to remember, and if all fails obsidian's 3rd-party templater-plugin just does the rest for me
3. latex equations. They are incompatible. There is no easy fix, but a fix does exist.
4. certain frontmatter conventions are incompatible, and a pain in the ass to deal with. Again this is fixable, just not that simple

---

My weird-as-shit workflow

If I write manuscript requiring quarto-compliance (e.g. requiring computations via code-blocks, using correct cross-refs), I write the manuscript in obsidian's format, while using quarto-flavouring on certain things (eg. callouts, I don't look at them in obsidian anyways).
The following pipeline is then issued (simplified description, there are a few things to note):

1. Execute the python-package obsidian-html with the manuscript as destination. This converts the obsidian-formatted file to standard-compliant markdown. This means that ![[note#section]] get replaced with their respective sections of note. Particularly, my own fork is used to allow embedding sections while optionally trimming its title. This is not well-documented however, as I only use this personally. Also note that the source-project from which I forked is currently not under active development, and the author has not made it clear when/if they will return.

2. So, now we have a standard-markdown compliant single manuscript.md-file. This file is then sent through a bit of post-processing for various reasons, to circumvent issues arising from not fulfilling compliance to quarto's format later on.
   Those include, among others:

   • fixing latex-equation cross-refs
   • fixing spacing-issues in certain yaml-fields
   • fixing certain crossrefs (for legacy, this utility of mine was initially written for bookdown)
   • fixing mermaid- and dot-diagram code-block headers from obsidians mermaid- to quarto's {mermaid}-format
   • fixing quarto-breaking invalid frontmatter-fields containing null or invalid arrays
   • converting image source-tags <img src=...> to knitr::include_graphics(...)-calls for embedded image files.
   • fixing various output artifacts of obsidian-html's output (tags and others)
     These things are fixed before writing the converted file-string to a .qmd-file

3. Quarto's cli is called to render the .qmd-file to its outputs. Configuration options can be configured for most config-options which quarto allows for. They are written to a format-specific .yml-file in the .qmd-file's directory, and used when calling quarto render XXX. Alternatively and for legacy reasons, R's quarto-package can be used. In that case, an .R-scriptfile is built which calls the quarto-rpackage to trigger quarto-execution.

If I work on a project, I setup a folder as a quarto-project. The script I outlined above can also convert and keep the output file in the same location as the input-file. along with a couple of commandline flags, this allows me to only convert an obsidian-flavoured md-file to a quarto-flavoured qmd-file, without rendering it to its output format directly. As a result, I can implement this in the quarto-project normal rendering-workflow.

Markdown-files that don't require qmd-features are not converted and rendered as raw markdown.
If a file requires qmd-markup but does not contain obsidian-like formatting (e.g. obsidian-like embeds), a direct .qmd-copy is created and used instead.
If the file requires qmd-markup and does contain obsidian-like formatting or requires codeblock-execution, the processing steps outlined in 2. are performed for that file. With proper setup, this process can be fairly performant. It obviously takes a certain fix time-cost to do this, although this can be drastically reduced with certain steps.

The major Caveat - handling manuscripts with obsidian-like embeds and code-execution requirements

The primary conversion of obsidian-flavoured md to standard-flavoured md via obsidian-html uses a python3-package, which to my knowledge is platform-agnostic. Everything outlined in step 2. is written in Autohotkey v1.1. This introduces several issues:

1. ahk is windows-only - both when executed as a script, and when compiled and executed as a standalone .exe-file. The entire intermediary processing that is performed before executing quarto is thus platform-locked. The steps are not complicated and could just as well be implemented in another language - I just don't have the need or time to do so.
2. ahk v.1.1 is deprecated in favour of the newer v.2.x. People really shouldn't learn v1.1, but I started on this util long before v2 became the official branch and have no time to port it.
3. On a more personal side, the public version of this ahk-util currently does not contain all required features, as the commandline-behaviour is still under development. And of course, I cannot guarantee it to not fuck up your shit. It obviously shouldn't, but yea. It also does not contain proper documentation on its options yet, which is probably the biggest issue for people that aren't me.

---

Closing thoughts

Why do I write this rather long wall of text?
This is not meant as an advertisement. I honestly don't think this exact workflow should be used by others, and the code is probably not documented sufficiently for any one person to just use it. I am merely writing this to give an example for an (admittedly somewhat exotic) implementation that solves most of the issues outlined in this thread.
These general steps allow one to use obsidian and all its goodies with overall minimal concessions for writing, while using quarto for rendering.

The post-processing steps can just as well be implemented in python, same as the automation of obsidian-html and quarto and its optional GUI-frontend. Heck, you could even write this intermediary processing in R - it's just string- and regex-replace on a lot of aspects in the end.

[dkapitan]

@fbob Happy to have bumped into this 'superniche' discussion. Was going through the same process, over the Xmas break (see https://kapitan.net/blog/obsidian-setup.html). Will have a look at your workflow and see what works for me and pass it on.