PlantUML Build / Release Process

[matthew16550]

The PlantUML build / release process has evolved over many years. Several automatic & manual systems are involved.

This is a place to discuss simplifying & automating some more.

(There are earlier discussions in #139, #486, #557)

[matthew16550]

As a start, here is my attempt to list all the main pieces of the puzzle. (Will update as more are found)

Source Repositories

• plantuml (Sourceforge)
  • Subversion
  • This is the main development repo
• plantuml (GitHub)
  • Manually synced with Sourceforge
• plantuml-epl
• plantuml-mit
• plantuml-server
• plantuml-stdlib

Continuous Integration

GitHub

• Builds & tests every PR & every commit on master branch
• Creates a GitHub Release for every "v" tag (this is new functionality and hasn't been used yet)

Travis

• Has not run any builds since June 2021
• Assume we would need to apply for OSS credits to continue using it

Artifact Repositories

Docker Hub

• plantuml-server container

GitHub plantuml releases

• jar, javadoc, sources as built by Maven

GitHub plantuml-server releases

• plantuml-server war

Maven Central

• jar, javadoc, sources as built by Maven

Sourceforge

• jar
• nodot jar
• war
• jars in .zip with various licenses
• sources in .tar.gz with various licenses

[matthew16550]

New public beta of GitHub Releases

Currently, users will need to enable it in Feature Preview.

I think the big thing for us is that tags no longer show in the Release list view.

Also, Release Notes can now be auto-generated. Not clear if that will help us, or if it can be part of the CI workflow.

EDIT: There is an API for auto-generated release notes, the docs are still being written but a draft is available. So I assume it could be part of our CI.

[The-Lum]

Hello @matthew16550, and all,

Linked to:

• Make jar available from Github? #557

Is it possible to have directly plantuml.jar (on artefact list) or another artefact with only the plantuml.jar, even for intermediate CI and not only for Release?
Especially to test intermediate version, without download the full artefact and extract only the minimal plantuml.jar.

And same question as mine on #557:

• How to retrieve the URL of the latest artefact?
  [Hints on stackoverflow or comments on github.community]

Regards.

[The-Lum]

It is possible to add a timestamp file on the snapshot.
I can make a PR...

Regards.

[The-Lum]

FYI @matthew16550, I just post a PR... see:

• mod: for 'release snapshot', add timestamp.lock + link to commits [no ci] #799

[matthew16550]

Hello @The-Lum,

I would like to rename the timestamp file to plantuml-SNAPSHOT.timestamp.lock or maybe just plantuml-SNAPSHOT.timestamp?

The reason is for #769 I am adding .pom & .asc signature files to the snapshot release so there will be quite a lot of files and if we change the filename to ... dot timestamp then the list of files in the release will alpha sort more nicely, see this example in my forked repo.

We add the files with the gh release create command, I tried listing them there in a particular order but that does not change how they are shown in the web UI. It seems they are always sorted alphabetically.

p.s. What are you using the timestamp for, I don't see it in the gallery repo yet?

[The-Lum]

Hello @matthew16550,

Currently the timestamp file name is: plantuml-SNAPSHOT-timestamp.lock.

But You can rename the file, as your convenience:

• plantuml-SNAPSHOT.timestamp.lock
• plantuml-SNAPSHOT.timestamp
• plantuml-SNAPSHOT.timestamp.txt

I will plan to use this file to control the download or not of the plantuml-SNAPSHOT.jar... WIP....

Regards.

[matthew16550]

Thanks @The-Lum,

I wonder if it might be cleaner to get the timestamp from the REST api?

This is the timestamp of the commit that is linked to the snapshot:

curl -s https://api.github.com/repos/plantuml/plantuml/releases/tags/snapshot | jq -r .created_at

And jq -r .published_at is when the snapshot was published.

[The-Lum]

FYI @matthew16550, @arnaudroques,

The last Snapshot is not the last Snapshot...
When several PR are merged, it seems that occurs this issue.

See last Snapshot: it is from 7f29798 (#776 / action 328) and not from 2098d24 of the last action (#775 / action 329).

[The-Lum]

Ah! I see, is that action 328 lasted longer than action 329...
Then the Snapshot of 328 has overwritten the Snapshot of 329.
QED.

• How to counter that?

[arnaudroques]

Well, I think I have to wait for an action to finish before merging another PR :-)
That's the easiest solution.

[matthew16550]

I looked into this and have not found a simple automated solution. Manually waiting before merging the next PR does seem the best approach for now 😢.

GH actions does not currently support sequential running of workflows in branch commit order. The concurrency option would probably solve this except it cancels other pending runs (community/community#5435). For CI on the main branch we really should build each commit so do not want pending runs cancelled.

Tried some ideas around a job after the build that has concurrency set and searches runs in commit order for which jars should become the snapshot. Or a separate workflow that decides which commit to build next. But they are all complicated and probably have strange edge cases so we would be in a messier situation than now.

Hopefully GH will add something that solves it for us.

[The-Lum]

Hello @matthew16550 and all,

Here is an open question about CI:

• What is the reason to make the CI on create?
  

  plantuml/.github/workflows/ci.yml

  Line 4 in 29e652a

  create:

• It is necessary or not?
  Especially on fork repo., each type I create a new branch the CI runs on my local repo.... and that make time...
  (is it to be sure that it compiles before a first modification..., But in any case, the CI occurs also when you create a #PR)

Regards.

[matthew16550]

The current release process is triggered by creating a tag. It was not obvious how to trigger specifically on tags because create: seems to not accept any arguments.

Those extra builds had been bugging me too and you prompted more googling. Using on.push.tags might solve it.

But I think #769 will remove the need to trigger on tags, I will start looking at that today.

EDIT: Relevant forum topic https://github.community/t/how-to-run-github-actions-workflow-only-for-new-tags/16075

[The-Lum]

Thank you very much @matthew16550, it's perfectly clear...
I also discover GitHub and the associated doc. of GitHub actions...