Support Antora

[Skeeve]

Yesterday, 2021-08-22 I learned about Antora at FrOSCon 2021. It was presented by the developer of the IntelliJ Asciidoctor Plugin.

While I'm not yet using Asciidoctor, I only learned about it a day ago ;), It sounds quite promising and useful, what he presented. Unfortunately there is no recording available yet.

Features

workspace

• support non-git workspace when Antora support is enabled #695

antora.yml

• detect antora.yml files in the workspace and ask user if they want to enable Antora support

site-manifest.json

• allow to configure a path to a site-manifest.json file
• read and parse the site-manifest.json file to provide autocompletion on pages (xref, include, link...)

Preview

• include AsciiDoc attributes defined in the antora.yml in the preview #691
• resolve image resource IDs in the preview
• resolve include resource IDs in the preview
• resolve xref resource IDs in the preview

Navigate

• support "Go to" (cmd+click) on image resource IDs (for instance, image:2.0@cli:commands:seaswell.png[]) #692
• support "Go to" (cmd+click) on xref resource IDs (for instance, xref:module:filename.adoc[])
• support "Go to" (cmd+click) on include resource IDs (for instance, include::include::ROOT:partial$treeline-warning.adoc[])

Auto-completion

• add images auto-completion when Antora is enabled #693
  • images within the same module
  • images from other modules
  • images from other components
• add includes auto-completion in this order
  • attachments within the same module
  • attachments from other modules
  • attachments from other components
• add xref auto-completion in this order
  • pages within the same module
  • pages from other modules
  • pages from other components

Suggestions/actions

• transform a resource ID to a fully qualified ID (for instance, from image:seaswell.png[] to image:2.0@cli:commands:seaswell.png[])
• suggest to simplify a resource ID (for instance, from image:2.0@cli:commands:seaswell.png to image:seaswell.png[] if the image is referenced in the same component/module/version)

[aisbergde]

Antora support in the IntelliJ Asciidoctor Plugin is excellent and was the only reason for me to install and use IntelliJ.

But my main editor is VSC and of course I would like to do also much more in VSC when working with asciidoc files.

Another point is: It is very easy to install VSC also without admin rights and the asciidoctor-vscode plugin even works in Azure Data Studio. And it is always allowed to install Azure Data Studio in my projects, because it is installed together with SSMS (SQL Server Management Studio). So normally I find ways to install and use this VSC Plugin, but it is much harder to be allowed to install and use IntelliJ.

[Skeeve]

The recording: https://media.ccc.de/v/froscon2021-2656-online-dokumentation_fur_nutzer_mit_asciidoc_und_antora

German only :(

[danyill]

It would be good to have a list of capabilities that mean "supporting Antora" so that we can progress them. I'm beginning to use Antora on a project and would like some of these capabilities also.

We have quite a lot of issues requesting support now (#230, #358, #380 and #409). Perhaps we can use this issue to scope out some requirements.

What are the most important things? From the current issues it seems to be:

• Processing a resource id in images, pages (xrefs and links)
• Being able to link to a production site?
• Do we need to be able to reference a playbook file and have vs-code open other folders/workspaces for different components?

How do achieve this concretely?

• Do we re-implement Antora functions (resource id processing might be something we can just "pull in"?) or do we effectively bundle Antora into this extension?
• Do we use our own UI or wait until Antora 3 and use extensions to gain the content and perhaps reduce the level of processing used (we probably don't need to build a full UI -- or do we ).

I don't speak German... I also haven't used the IntelliJ tool -- however there is a good feature list on the extension docs which is probably where we should start:

https://intellij-asciidoc-plugin.ahus1.de/docs/users-guide/features/advanced/antora.html

[anb0s]

I'm investigating the AsciiDoc with Antora as alternative solution to Markdown with MkDocs for middle and big architecture documentation projects (many products, components, branches etc.). I'm also investigating the issues with relative image rendering (PlantUML, Vega etc.), completion support for xrefs and others in VSCode.

@Skeeve the video above now encouraged me to continue investigating the issues and help with reporting, testing and may be resolving them...

@danyill in the Q&A session from the video the maintainer (Alexander Schwartz) of IntelliJ AsciiDoc plugin is talking about sharing knowledge and support (chat, online meeting etc.) if similar features are needed in VSCode and somebody wants to implement them :)

Lets start with the feature list and known issues...

[ggrossetie]

It would be good to have a list of capabilities that mean "supporting Antora" so that we can progress them. I'm beginning to use Antora on a project and would like some of these capabilities also.

💯
That would be extremely useful!
Maybe @ahus1 can help us since the IntelliJ plugin has an excellent support for Antora.

I've created a similar issue on the browser extension: asciidoctor/asciidoctor-browser-extension#299.

I guess the first would be to detect that we are in an Antora module (probably locating the antora.yml file). If that's the case, then we should use a specific logic to resolve external resources (images, partials, examples...).

[ahus1]

Hi @Mogztter - thanks for summoning me.

I am happy to support in answering questions like "How did you..." and "Why did you...", also "What did you do before/after...". Pick my brain :-)

Regarding the video: I pre-recorded the same talk in English for DevConf.US and also FOSDEM, see https://www.youtube.com/watch?v=MG-sajf2PP4 for details. As it was pre-recorded, there is no Q&A at the end. DevConf.US published a recording that included the Q&A as well, see https://www.youtube.com/watch?v=LT0a--DNJhI (but the quality of the pre-recorded part is not as good there).

Regarding features: I learned when implementing it for the IntelliJ plugin that the functionality can be grown incrementally. Users will provide feedback on early functionality. Ideally there is fast feedback (think earth-sky as described by the ZertoMQ community). For the IntelliJ plugin, @danhaywood picked "sky", shared his needs, did a lot of tests and provided valuable input that steered development.

Adding to @Mogztter's thoughts: recognizing that an AsciiDoc file is part of a component by locating an antora.yml file is definitely the first step. What could be next is to set some attributes for the preview like :icons: font (for nicer admonitions) as Antora does this by default, and setting the imagesdir to the module's folder so that images show up in the preview.

Functionality could then be added bit by bit: Picking up attributes from the component descriptor, settings more Antora attributes, picking up page names for xrefs, etc.

For the IntelliJ plugin I decided to imitate Antora's behavior by re-implementing some of the behavior, for example resolving includes and xrefs. I can show where I hooked into Asciidoctor to inject the necessary functionality.

When growing functionality, some features targeted the editor for auto-completion and inspections, others targeted the preview.
For a start, it was supporting only references within the same component. Only later I added functionality to resolve cross-component references.

This week and next I'll have only time to comment on issues. From Sep 27 onwards I'm happy to join a synchronous communication (video call, etc.)

[LukeCz]

Hi, it would be great to include 2 features from Antora to this plugin: extended syntax for include and image macros, basically ability to specify module.

[MLNW]

Native support for Antora would be a great addition to this plugin!

[apupier]

some Antora support has been provided with this PR #580
it is not part of a released version (yet).
I do not know the features that are covered.
@marieflorescontact Could you provide some information on it?

[marieflorescontact]

For now the extension detects the antora.yml file and offers the user to activate the Antora support (#580).

I am currently working on the following features:

• Resoving Antora Ids for the following macros: image, video, xref (include seems a little bit more complicated 🤔)
• Drop into editor using this api to create a macro depending on the file type. (Use the drop into editor API (VS Code 1.70) #624)

[aisbergde]

When is a preview version planned? I would like to test this Antora feature.

[omfgnuts]

Any progress has been done on this matter?

Would definitely love to come back to VS Code, Antora support in WebStorm is the only thing holding me on this java monstrosity

[ggrossetie]

Yes, some work has been done in 3.x (pre-release) but we are still missing a lot of features.
I think we should create dedicated issues for each feature otherwise it's too daunting.

We can use this issue as a "meta issue"

[eddennison]

I am not seeing image resource ID resolution working in the current plug-in, even in the simplest case where the image is stored in the standard images directory in the current module:

Antora build output:

Is this supposed to be working? References to other modules in the current component or modules in other components also don't seem to work. (I was hoping to use this code as a model for getting include directives working.)

[ggrossetie]

Did you enable Antora support? What version are you using?

[eddennison]

Did you enable Antora support? What version are you using?

3.1.5 with Antora support enabled. I also have allow-uri-read set in asciidoctorAttributes.

[ggrossetie]

Could you please share a repository where I can reproduce this issue? It's working fine on my projects.

[eddennison]

Could you please share a repository where I can reproduce this issue? It's working fine on my projects.

It's a private repo in a corporate environment. Do you have a test repo that you can share? That would let me identify if there were an issue with my system or environment. The demo Antora demo repo has no images.

I need also to do an experiment -- we use LFS for binary data, and that may be the problem.

[ggrossetie]

Are you using Windows? Could you please toggle the Developer Tools and copy/paste what you see in the "Console" tab when you open your AsciiDoc file?

[eddennison]

Are you using Windows? Could you please toggle the Developer Tools and copy/paste what you see in the "Console" tab when you open your AsciiDoc file?

[Extension Host] Unable to find an applicable Antora configuration file in [file:///c%3A/Users/ed/antora/sw_is.ad/isa/antora.yml, file:///c%3A/Users/ed/antora/common.ad/doc/antora.yml, file:///c%3A/Users/ed/antora/hw_pnp.ad/x80/antora.yml, file:///c%3A/Users/ed/antora/common.ad/snippets/antora.yml] for the AsciiDoc document file:///c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/goodbye.adoc
console.ts:137 [Extension Host] Unable to find an applicable Antora configuration file in [file:///c%3A/Users/ed/antora/sw_is.ad/isa/antora.yml, file:///c%3A/Users/ed/antora/hw_pnp.ad/x80/antora.yml, file:///c%3A/Users/ed/antora/common.ad/snippets/antora.yml, file:///c%3A/Users/ed/antora/common.ad/doc/antora.yml] for the AsciiDoc document file:///c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/goodbye.adoc
resourceLoading.ts:82 Error: Unable to read file 'c:\Users\ed\antora\common.ad\doc\modules\ROOT\pages\same-in-english.jpg' (Error: Unable to resolve nonexistent file 'c:\Users\ed\antora\common.ad\doc\modules\ROOT\pages\same-in-english.jpg')
    at r.w (fileService.ts:607:10)
    at r.v (fileService.ts:592:15)
    at async v (resourceLoading.ts:68:18)
    at async s.rb (webviewElement.ts:733:19)
index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:999     GET https://file+.vscode-resource.vscode-cdn.net/c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/same-in-english.jpg 404
(anonymous) @ index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:999
setTimeout (async)
onFrameLoaded @ index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:997
(anonymous) @ index.html?id=f67e5496-d36b-4c40-8dc1-9b25cfd6b174&origin=3a73e17a-8f8d-450c-8c89-b71d3bbdebf2&swVersion=4&extensionId=asciidoctor.asciidoctor-vscode&platform=electron&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&parentOrigin=vscode-file%3A%2F%2Fvscode-app:1028

[ggrossetie]

It might be an issue with the directory name common.ad. In theory, the antora.yml file (file:///c%3A/Users/ed/antora/common.ad/doc/antora.yml) should be assigned to the AsciiDoc file file:///c%3A/Users/ed/antora/common.ad/doc/modules/ROOT/pages/goodbye.adoc.

Something must be wrong in:

asciidoctor-vscode/src/features/antora/antoraSupport.ts

Lines 172 to 189 in 15b4562

	export async function findAntoraConfigFile (textDocumentUri: Uri): Promise<Uri | undefined> {
	const pathToAsciidocFile = textDocumentUri.toString()
	const cancellationToken = new CancellationTokenSource()
	cancellationToken.token.onCancellationRequested((e) => {
	console.log('Cancellation requested, cause: ' + e)
	})
	const antoraConfigs = await vscode.workspace.findFiles('**/antora.yml', '/node_modules/', 100, cancellationToken.token)
	// check for Antora configuration
	for (const antoraConfig of antoraConfigs) {
	const modulesUri = antoraConfig.with({ path: path.join(path.dirname(antoraConfig.path), 'modules') })
	if (pathToAsciidocFile.startsWith(modulesUri.toString()) && pathToAsciidocFile.slice(modulesUri.toString().length).match(/^\/[^/]+\/pages\/.*/)) {
	console.log(`Found an Antora configuration file at ${antoraConfig.toString()} for the AsciiDoc document ${pathToAsciidocFile}`)
	return antoraConfig
	}
	}
	console.log(`Unable to find an applicable Antora configuration file in [${antoraConfigs.join(', ')}] for the AsciiDoc document ${pathToAsciidocFile}`)
	return undefined
	}

[eddennison]

I changed the directory names to not include ".ad" but the test on line 182 is failing. Here's what I see in the debugger (and after adding a debug log message):

There is an extra slash (file:////c:/ vs file:///c:/) in the path from antoraConfig when compared with textDocumentUri.

[eddennison]

Also, I have a bunch of other questions about the code:

In order to support include directives for preview in the plug in, here's what I think is required:

• At document load time, need to search for and enumerate all Antora modules within all content source roots within the workspace directory. This code would be along the lines of findAntoraConfigFile() in antoraSupport.ts.
• This enumeration would need to generate a correspondence between the module name and a file system path to the module root directory (the directory with antora.yml).
• When include directives are encountered in the file being prepared for preview, the content must be loaded from either the current module or the specified module. The fs code loading the file must map from a module name to a specific fs path, then attempt to open the requested file.
• I believe that in order to actually open and read the file, the fs path needs to be sanitized to work with VS Code, specifically by passing it through asWebviewUri().

In the "normal" Antora execution path, the module name to fs correspondences are presumably generated when the playbook file is processed. Specifically, any module references are going to be limited to modules found in sources declared in the playbook file. And of course they're not really file system references but git references, since they include branch information.

Since the plugin code doesn't process playbooks, I assuming that we need specific code to search for modules in the workspace directory. I think this is what the IntelliJ plugin is doing. Can you give me any insights into where in the plugin code I should be looking?

[m1027]

Thanks for the fast response.

I've been trying with fresh VS Code and VS Codium installations on several machines (linux/windows) and the simple directory/file tree given above.
With no success.

Well, what exact description would you need additionally...?

[ggrossetie]

Are you using the latest version? Can you take a screenshot of your settings?

Do you have a .vscode/settings.json? How do you open your AsciiDoc file? Do you start VS Code then "Open Folder..."? Did you try to "File > New Window" then "File > Open Folder..."? Did you make sure that Antora support is still enabled? Did you try to restart VS Code and/or close/reopen the preview?

[m1027]

Thanks for the details. I can answer all questions with yes.

Here some screenshots:

Some more hints:

• I tried antora support enabled/disabled generally (while keeping antora enabled in .vscode/settings.json).
• I had been asked whether to enable antora support by a popup which created the settings file automatically.
• I am currently testing under Windows.
• Files are here c:\tmp\asciidoc-test, with no spaces anywhere.
• No git init in that directory yet.
• The preview shows the test image when it is moved into the folder where the adoc file is, see this screenshot:

Hm... I believe it is something trivial (while not obvious...).

[ggrossetie]

It says "Also Modified in Workspace", is it enabled on the workspace?

[m1027]

Right. AFAIK "Also modified in Workspace" means that we have a global configuration as well as a configuration in the workspace, via .vscode/sessings.json.

I am adding again a screenshot of the console (developer tools):

It says two things:

• An antora configuration has been detected.
• But also, the image file is still looked for at the wrong, non-antora, location (and obviously not found).

So, it seems that despite of the extension's global configuration as well as the workspace's local json config, the extension still does not switch into antora mode.

Does that make sense?

[ggrossetie]

Thanks for providing additional information.

It seems that the extension correctly identifies that the AsciiDoc file is part of an Antora documentation component. For some unknown reason, I believe that the content catalog, which is built from the files in the workspace, does not contain the image, or it can't be found.

I will see if it's possible to add trace or debug logs to facilitate analysis and troubleshooting. Are you using symbolic links or anything unusual on your file system?

[ggrossetie]

Also, could you try to use a different name for your documentation component (in antora.yml)?

If it still does not work, could you try to use a qualified Antora resource ID?

[mojavelinux]

I believe that the content catalog, which is built from the files in the workspace, does not contain the image, or it can't be found.

If possible, I believe that it should render a broken image in that case instead of allowing the preview to find the adjacent one...because Antora won't find it there. If that's possible.

[m1027]

@ggrossetie :

Are you using symbolic links or anything unusual on your file system?

No. The above tests were under Windows. The direcotory is just plain c:\tmp\asciidoc-test\.

Also, could you try to use a different name for your documentation component (in antora.yml)?

Of course. Name changed from ROOT to test1, in antora.yml, file saved. Still no image in the preview, though.

Let me know whenever you need further information, logs etc.

[m1027]

One more note on workspaces (I am not a VS Code user):

The setup here was:

• Copying the antora directory tree and content into c:\tmp\asciidoc-test.
• Starting vscode
• Opening the folder c:\tmp\asciidoc-test.

So, acutally I am not using workspaces really. If I explictly save everything as a workspace (file) it and load that, it doesn't fix the preview though either.

[m1027]

@ggrossetie :

If it still does not work, could you try to use a qualified Antora resource ID?

See the new screenshot:

The relative path works, the rest doesn't.

[ggrossetie]

Could you please disable the CSP using the command palette "Manage preview security settings"?

[ggrossetie]

If possible, I believe that it should render a broken image in that case instead of allowing the preview to find the adjacent one...because Antora won't find it there. If that's possible.

👍

[m1027]

Could you please disable the CSP using the command palette "Manage preview security settings"?

Done. Switched to "Disabled" and triggered preview refresh by some edits. The images are not shown though.

[ggrossetie]

Would you be able to share your workspace on GitHub (i.e, asciidoc-test directory). Also could you please copy/paste what you get using "Help > About". Thanks !

[m1027]

Of course!

Repo asciidoc-test is now here: https://github.com/m1027/asciidoc-test

Output of help / about:

Version: 1.84.2 (user setup)
Commit: 1a5daa3a0231a0fbba4f14db7ec463cf99d7768e
Date: 2023-11-09T10:51:52.184Z
Electron: 25.9.2
ElectronBuildId: 24603566
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.18363

[ggrossetie]

Yay! I can finally reproduce it! 🎉

It does not work because modules is at the root of your "workspace". If you create a docs directory and move everything into it, it will work.
This is unexpected and it needs to be fixed.

[MLNW]

The power of a proper MWE

[ggrossetie]

That's true, Minimal Non Working Example 😄
I just released version 3.1.10, could you please give it a try?

[m1027]

Sure...: I am happy to confirm that 3.1.10. fixes this issue! Now I am getting all images displayed in the preview. Thank you so much!

This is really good news as a working image preview is crucial here for non-dev users.

[m1027]

After a short hiatus concerning image insertion :-) I permit myself to ask for further tweaks when in antora mode:

(1) As asked by @mojavelinux above:

If possible, I believe that it should render a broken image in that case instead of allowing the preview to find the adjacent one...because Antora won't find it there. If that's possible.

In 3.1.10, images are still detected and previewed from the pages directory, although antora wouldn't accept this later on. So, users in antora mode may get misled by the (working) preview.

Especially because...

(2) ... images pasted from the clipboard will be inserted into the pages directory and not into the images directory. So, users in antora mode need to remember moving them manually afterwards. It would be fantastic to honor the antora location for the paste action.

[eddennison]

@ggrossetie Thank you so much for this update. It also properly supports this functionality:

• Including shared content from the partials directory in the current module:

include::partial$snippet.adoc[]

• Including partials and images from other components:

image::shared:ROOT:dial.jpg[]
include::shared:ROOT:partial$snippet.adoc[]

These were issues that I reported back in September. Antora's support for content reuse/single-sourcing through the include directive is a key feature, and supporting it in VS Code is a big win.

Thanks again!

[m1027]

The image preview does not work here when version: true is configured in antora.yml. It works though when version: ~ is used.

[ggrossetie]

@m1027 Thanks for the follow-up, could you please open a specific issue for version: true?

[ggrossetie]

... images pasted from the clipboard will be inserted into the pages directory and not into the images directory. So, users in antora mode need to remember moving them manually afterwards. It would be fantastic to honor the antora location for the paste action.

Could you please open a dedicated issue for that?

[ggrossetie]

In 3.1.10, images are still detected and previewed from the pages directory, although antora wouldn't accept this later on. So, users in antora mode may get misled by the (working) preview.

I guess we should also open a new issue for this one 😉

[m1027]

Alright, I have created some new issues.

[man-chi]

as a workaround before antora is fully supported, I managed to preview images in vscode-asciidoctor web preview by doing the following setting in vscode web preview:

• vscode-asciidoc extension > settings > asciidoc >preview : asciidoctor attributes
• Edit in settings.json
  "asciidoc.preview.asciidoctorAttributes": {
  "imagesdir":"../images/"
  }