Index quarkiverse guides

[marko-bekhta]

fixes #143

I've played with this a bit, and from what it seems, the simplest way is to just parse the https://docs.quarkiverse.io/index/explore/index.html ...

I haven't found any repository where the HTMLs are pre-built so we could read from them rather than over the HTTP...

since there can be multiple pages under one "extension-guide" I've prefixed all of them with the name of the extension itself.

This will only include the Quarkiverse guides in the search on https://quarkus.io/guides/

Do we want to do anything on the https://docs.quarkiverse.io/index/explore/index.html as well ?

[yrodiere]

I haven't found any repository where the HTMLs are pre-built so we could read from them rather than over the HTTP...

Assuming HTML files are generated, you could adjust the Quarkiverse build to generate a YAML file, too. @gastaldi any pointers to the Quarkiverse website build?

Do we want to do anything on the https://docs.quarkiverse.io/index/explore/index.html as well ?

Most likely we'll want to integrate search.quarkus.io there, yes, but that can be a second step.

[yrodiere]

Sounds like a problem, no?

Though I'm not sure why there was a translation in the first place...

[marko-bekhta]

It's mainly about the Japanese version. There are those few Quarkiverse guides that are mentioned on the quarkus.io/guides, and their titles have been translated. But now, when we index this from the Quarkiverse side, there are no translation files. We probably could try to use the translation files for the main version and hope to find some matches, but I wasn't sure it was worth it. In particular, because the titles are concatenated here + the summary isn't the one from the yaml, so there probably won't be a lot of matches.

[marko-bekhta]

With these two:

quarkiverse/quarkiverse-docs#159
quarkiverse/antora-ui-quarkiverse#70

It works like this:

the way I've been testing it locally

• start the search.quarkus.io locally
• update the .env file in [antora-ui-quarkiverse](https://github.com/quarkiverse/antora-ui-quarkiverse) and set _SEARCH_SERVICE_HOST="HTTP://localhost:8080"
• follow the theme build instructions and run gulp bundle
• update the ui.bundle.url in the quarkiverse-docs to point to the generated local theme from the previous step
• build the quarkiverse docs using npm run build
• then serve the generated HTMLs with npx http-server build/site -c-1

[yrodiere]

Well that's quite a bit of CXF :]

LGTM, though I'd personally prefer the Quarkiverse build to generate some YAML for you to parse. HTML scraping isn't exactly the most reliable way of going about that sort of things :)

So:

I haven't found any repository where the HTMLs are pre-built so we could read from them rather than over the HTTP...

Assuming HTML files are generated, you could adjust the Quarkiverse build to generate a YAML file, too. @gastaldi any pointers to the Quarkiverse website build?

[gastaldi]

Assuming HTML files are generated, you could adjust the Quarkiverse build to generate a YAML file, too. @gastaldi any pointers to the Quarkiverse website build?

The website is built here: https://github.com/quarkiverse/quarkiverse-docs/blob/main/.github/workflows/publish.yml#L36. Adding a new step there or changing the package.json as @marko-bekhta already did in the draft PR works too

[yrodiere]

Assuming HTML files are generated, you could adjust the Quarkiverse build to generate a YAML file, too. @gastaldi any pointers to the Quarkiverse website build?

The website is built here: https://github.com/quarkiverse/quarkiverse-docs/blob/main/.github/workflows/publish.yml#L36. Adding a new step there or changing the package.json as @marko-bekhta already did in the draft PR works too

I don't think the package.json change is related, but okay, so the changes need to go there somehow.

@marko-bekhta you'll probably want to update CORS rules as well, to allow *.quarkiverse.io to use search.quarkus.io.

[marko-bekhta]

I thought about this a bit.. If we'd go with adding additional steps to the Quarkiverse build, for it to make sense we'd need:

• add an additional yml file to each Quarkiverse extension docs that would describe the structure of the docs pages (title, summary)
• when running the build/deploy
  • combine these ymls into a single file,
  • push the generated HTMLs and combined yml file to a branch of quarkiverse-docs so that we could pull it and work with the htmls/yml in search.quarkus.io

antora.yml files in extension repositories (e.g. https://github.com/quarkiverse/quarkus-cxf/blob/main/docs/antora.yml) do not help us much as they don't have the docs-tree structure, and working with https://github.com/quarkiverse/quarkus-cxf/tree/main/docs/modules/ROOT/pages would mean that we'd need to parse adoc files ... If that's so I'd rather deal with parsing HTMLs in Java 😃 then doing all of the above 🙈. The HTML layout should be stable and can only change if the Quarkiverse theme layout (https://github.com/quarkiverse/antora-ui-quarkiverse) is modified

[yrodiere]

add an additional yml file to each Quarkiverse extension docs that would describe the structure of the docs pages (title, summary)

That... seems like a blocker, considering the amount of Quarkiverse extensions.

If that's so I'd rather deal with parsing HTMLs in Java 😃 then doing all of the above 🙈. The HTML layout should be stable and can only change if the Quarkiverse theme layout (https://github.com/quarkiverse/antora-ui-quarkiverse) is modified

Parsing nav.adoc would be rather straightforward though, no? I wouldn't expect this file to be complex in any Quarkiverse extension.

[yrodiere]

If that's so I'd rather deal with parsing HTMLs in Java 😃 then doing all of the above 🙈. The HTML layout should be stable and can only change if the Quarkiverse theme layout (https://github.com/quarkiverse/antora-ui-quarkiverse) is modified

Parsing nav.adoc would be rather straightforward though, no? I wouldn't expect this file to be complex in any Quarkiverse extension.

I suspect we already parse this file, by the way, judging from the comments near the end of this one: https://raw.githubusercontent.com/quarkiverse/quarkus-cxf/main/docs/modules/ROOT/nav.adoc (extensions: START/extensions: END).