PMD build tools

Artifact containing configuration data and scripts to build and release pmd/pmd from source.

Note: This project does not use semantic versioning.

build-env
scripts
- Overview
- Usage
files
Testing
Miscellaneous

build-env

Ubuntu Linux based, same as github actions runner, see Runner Images. It can be used to test the scripts and perform the builds without github actions.

Once build the docker container:

$ docker build \
    --tag pmd-build-env \
    build-env

This is only needed once. This builds the image, from which new containers can be started. A new image needs to be created, if e.g. ubuntu is to be updated or any other program.

Then run the container, mounting in the pmd-build-tools repo as a volume:

$ docker run \
    --interactive \
    --tty \
    --name pmd-build-env \
    --mount type=bind,source="$(pwd)",target=/workspaces/pmd/build-tools \
    pmd-build-env:latest

You're now in a shell inside the container. You can start a second shell in the same container:

$ docker exec \
    --interactive \
    --tty pmd-build-env \
    /bin/bash --login

The container is stopped, if the first shell is exited. To start the same container again:

$ docker start \
    --interactive \
    pmd-build-env

To list the running and stopped containers:

$ docker ps \
    --all \
    --filter name=pmd-build-env

If not needed anymore, you can destroy the container:

$ docker rm pmd-build-env

scripts

Overview

Scripts are stored in scripts subfolder. There are two types:

Shell scripts to be executed as programs. The extension is ".sh".
Library functions to be included by those scripts. The extension is ".bash" and they are located in scripts/inc.

All scripts are bash scripts.

The shell scripts might depend on one or more library scripts. They need to fetch their dependencies before doing any work. This is always done in the function "fetch_ci_scripts()". The global variable PMD_CI_SCRIPTS_URL is used as the base url to fetch the scripts.

Library functions may depend on other library functions as well.

Namespaces: Exported global variables use the prefix PMD_CI_. Functions of a library use the same common prefix starting with pmd_ci_ followed by the library name, followed by the actual function name.

Use shellcheck to verify the scripts.

Usage

inc/fetch_ci_scripts.bash

Little helper script to download dependencies.

The only function is fetch_ci_scripts.

Use it in other scripts like this:

MODULE="my-library"
SCRIPT_INCLUDES="log.bash"
# shellcheck source=inc/fetch_ci_scripts.bash
source "$(dirname "$0")/inc/fetch_ci_scripts.bash" && fetch_ci_scripts

# other parts of your script

That's the only script, that needs to be copied and existing before. Only with this script, the other scripts can be fetched as needed.

Used global vars:

PMD_CI_SCRIPTS_URL - defaults to https://raw.githubusercontent.com/pmd/build-tools/master/scripts

inc/log.bash

Namespace: pmd_ci_log

Functions:

pmd_ci_log_error
pmd_ci_log_info
pmd_ci_log_success
pmd_ci_log_debug

Vars:

PMD_CI_LOG_COL_GREEN
PMD_CI_LOG_COL_RED
PMD_CI_LOG_COL_RESET
PMD_CI_LOG_COL_YELLOW

Used global vars:

PMD_CI_DEBUG: true|false.

inc/utils.bash

Namespace: pmd_ci_utils

Functions:

pmd_ci_utils_get_os: returns one of "linux", "macos", "windows"
pmd_ci_utils_determine_build_env. Sets many variables, e.g. GITHUB_BASE_URL, PMD_CI_IS_FORK, ...
pmd_ci_utils_is_fork_or_pull_request
pmd_ci_utils_fetch_ci_file

Used global vars:

PMD_CI_SCRIPTS_URL: This is the base url from where to fetch additional files. For setting up secrets, the file private-env.asc is fetched from there. Defaults to https://raw.githubusercontent.com/pmd/build-tools/master/scripts The files are fetched from the sub directory "files".

Test with: bash -c "source inc/utils.bash; pmd_ci_utils_get_os" $(pwd)/test.sh

inc/openjdk.bash

Namespace: pmd_ci_openjdk

Functions:

pmd_ci_openjdk_install_adoptium. Usage e.g. pmd_ci_openjdk_install_adoptium 11 Supports also EA builds, e.g. pmd_ci_openjdk_install_adoptium 16-ea
pmd_ci_openjdk_install_zuluopenjdk. Usage e.g. pmd_ci_openjdk_install_zuluopenjdk 7
pmd_ci_openjdk_setdefault. Usage e.g. pmd_ci_openjdk_setdefault 11

Test with: bash -c "source inc/openjdk.bash; pmd_ci_openjdk_install_adoptium 11" $(pwd)/test.sh

inc/github-releases-api.bash

Namespace: pmd_ci_gh_releases

Functions:

pmd_ci_gh_releases_createDraftRelease
pmd_ci_gh_releases_getLatestDraftRelease
pmd_ci_gh_releases_deleteRelease
pmd_ci_gh_releases_getIdFromData
pmd_ci_gh_releases_getTagNameFromData
pmd_ci_gh_releases_uploadAsset
pmd_ci_gh_releases_updateRelease
pmd_ci_gh_releases_publishRelease

Used global vars:

GITHUB_TOKEN - this is the default github actions token
GITHUB_BASE_URL

Test with:

bash -c 'set -x ; \
         export GITHUB_TOKEN=.... ; \
         export GITHUB_BASE_URL=https://api.github.com/repos/pmd/pmd ; \
         export PMD_CI_DEBUG=false ; \
         source inc/github-releases-api.bash ; \
         pmd_ci_gh_releases_createDraftRelease "pmd_releases/6.30.0" "d2e4fb4ca370e7d5612dcc96fb74c29767a7671e" ; \
         sleep 1; \
         pmd_ci_gh_releases_getLatestDraftRelease ; \
         export therelease="$RESULT" ; \
         pmd_ci_gh_releases_uploadAsset "$therelease" "inc/github-releases-api.bash"
         export body='\''the body \
         line2'\'' ; \
         pmd_ci_gh_releases_updateRelease "$therelease" "test release" "$body" ; \
         #pmd_ci_gh_releases_deleteRelease "$therelease" ; \
         #pmd_ci_gh_releases_publishRelease "$therelease" ; \
         ' $(pwd)/test.sh

inc/setup-secrets.bash

Namespace: pmd_ci_setup_secrets

Functions:

pmd_ci_setup_secrets_private_env
pmd_ci_setup_secrets_gpg_key
pmd_ci_setup_secrets_ssh

Used global vars:

PMD_CI_SECRET_PASSPHRASE: This is provided as a github secret (PMD_CI_SECRET_PASSPHRASE: ${{ secrets.PMD_CI_SECRET_PASSPHRASE }}) in github actions workflow. It is used to decrypt further secrets used by other scripts (github releases api, ...)

Test with:

bash -c 'set -e; \
         export PMD_CI_SECRET_PASSPHRASE=.... ; \
         export PMD_CI_DEBUG=false ; \
         source inc/setup-secrets.bash ; \
         pmd_ci_setup_secrets_private_env ; \
         pmd_ci_setup_secrets_gpg_key ; \
         pmd_ci_setup_secrets_ssh ; \
         # env # warning: prints out the passwords in clear! ; \
         ' $(pwd)/test.sh

inc/sourceforge-api.bash

Namespace: pmd_ci_sourceforge

Functions:

pmd_ci_sourceforge_uploadReleaseNotes
pmd_ci_sourceforge_uploadFile
pmd_ci_sourceforge_selectDefault
pmd_ci_sourceforge_rsyncSnapshotDocumentation
pmd_ci_sourceforge_createDraftBlogPost
pmd_ci_sourceforge_publishBlogPost

Used global vars:

PMD_SF_USER
PMD_SF_APIKEY
PMD_SF_BEARER_TOKEN

Test with:

bash -c 'set -e; \
         export PMD_CI_SECRET_PASSPHRASE=.... ; \
         export PMD_CI_DEBUG=false ; \
         source inc/setup-secrets.bash ; \
         source inc/sourceforge-api.bash ; \
         pmd_ci_setup_secrets_private_env ; \
         #pmd_ci_setup_secrets_gpg_key ; \
         pmd_ci_setup_secrets_ssh ; \
         pmd_ci_sourceforge_uploadReleaseNotes "pmd/Release-Script-Test" "Testing release notes" ; \
         echo "test file" > "release-test-file.txt" ; \
         pmd_ci_sourceforge_uploadFile "pmd/Release-Script-Test" "release-test-file.txt" ; \
         rm "release-test-file.txt" ; \
         pmd_ci_sourceforge_selectDefault "Release-Script-Test" ; \
         mkdir -p "docs/pmd-doc-Release-Script-Test/" ; \
         echo "test-file" > "docs/pmd-doc-Release-Script-Test/release-test.txt" ; \
         pmd_ci_sourceforge_rsyncSnapshotDocumentation "Release-Script-Test" "test-Release-Script-Test" ; \
         rm "docs/pmd-doc-Release-Script-Test/release-test.txt"; rmdir "docs/pmd-doc-Release-Script-Test"; rmdir "docs" ; \
         pmd_ci_sourceforge_createDraftBlogPost "draft post 1" "text with labels" "label1,label2" ; \
         blog="${RESULT}" ; \
         echo "URL: ${blog}" ; \
         pmd_ci_sourceforge_createDraftBlogPost "draft post 2" "text without labels" ; \
         blog="${RESULT}" ; \
         echo "URL: ${blog}" ; \
         #pmd_ci_sourceforge_publishBlogPost "${blog}" ; \
         ' $(pwd)/test.sh

Note that "pmd_ci_sourceforge_selectDefault" won't be successful, because the file to be selected as default doesn't exist.

Don't forget to delete https://sourceforge.net/projects/pmd/files/pmd/Release-Script-Test and https://pmd.sourceforge.io/test-Release-Script-Test after the test.

And also the created blog posts under https://sourceforge.net/p/pmd/news/.

inc/maven.bash

Namespace: pmd_ci_maven

Functions:

pmd_ci_maven_setup_settings
pmd_ci_maven_get_project_version: exports PMD_CI_MAVEN_PROJECT_VERSION
pmd_ci_maven_get_project_name
pmd_ci_maven_verify_version
pmd_ci_maven_display_info_banner
pmd_ci_maven_isSnapshotBuild
pmd_ci_maven_isReleaseBuild

Used global vars:

PMD_CI_BRANCH
PMD_CI_TAG

Exported global vars:

PMD_CI_MAVEN_PROJECT_VERSION

Test with:

bash -c 'set -e; \
         export PMD_CI_SECRET_PASSPHRASE=.... ; \
         export PMD_CI_DEBUG=true ; \
         source inc/maven.bash ; \
         pmd_ci_maven_setup_settings ; \
         cd .. ; \
         pmd_ci_maven_get_project_version ; \
         echo "version: $RESULT" ; \
         pmd_ci_maven_get_project_name ; \
         echo "name: $RESULT" ; \
         PMD_CI_MAVEN_PROJECT_VERSION="1.2.3-SNAPSHOT" ; \
         PMD_CI_BRANCH="test-branch" ; \
         pmd_ci_maven_verify_version ; \
         unset PMD_CI_BRANCH ; \
         PMD_CI_TAG="test-tag" ; \
         PMD_CI_MAVEN_PROJECT_VERSION="1.2.3" ; \
         pmd_ci_maven_verify_version ; \
         pmd_ci_maven_display_info_banner ; \
         pmd_ci_maven_isReleaseBuild && echo "release build" ; \
         PMD_CI_MAVEN_PROJECT_VERSION="1.2.3-SNAPSHOT" ; \
         unset PMD_CI_TAG ; \
         PMD_CI_BRANCH="test-branch" ; \
         pmd_ci_maven_isSnapshotBuild && echo "snapshot build" ; \
         ' $(pwd)/test.sh

inc/pmd-code-api.bash

Namespace: pmd_ci_pmd_code

Functions:

pmd_ci_pmd_code_uploadFile
pmd_ci_pmd_code_uploadZipAndExtract
pmd_ci_pmd_code_removeFolder
pmd_ci_pmd_code_createSymlink

Used global vars:

Test with:

bash -c 'set -e; \
         export PMD_CI_SECRET_PASSPHRASE=.... ; \
         export PMD_CI_DEBUG=true ; \
         source inc/setup-secrets.bash ; \
         source inc/pmd-code-api.bash ; \
         pmd_ci_setup_secrets_private_env ; \
         pmd_ci_setup_secrets_ssh ; \
         echo "test file" > "test-file-for-upload.txt" ; \
         zip "test-zip.zip" "test-file-for-upload.txt" ; \
         pmd_ci_pmd_code_uploadFile "/httpdocs/test-folder" "test-file-for-upload.txt" ; \
         echo "test file" > "test-file-for-upload.txt" ; \
         pmd_ci_pmd_code_uploadZipAndExtract "/httpdocs/test-folder2" "test-zip.zip" ; \
         rm "test-zip.zip" "test-file-for-upload.txt" ; \
         pmd_ci_pmd_code_createSymlink "/httpdocs/test-folder" "/httpdocs/test-folder3" ; \
         pmd_ci_pmd_code_removeFolder "/httpdocs/test-folder" ; \
         pmd_ci_pmd_code_removeFolder "/httpdocs/test-folder2" ; \
         pmd_ci_pmd_code_removeFolder "/httpdocs/test-folder3" ; \
         ' $(pwd)/test.sh

check-environment.sh

Usage in github actions step:

- name: Setup Environment
  shell: bash
  run: |
    echo "LANG=en_US.UTF-8" >> $GITHUB_ENV
    echo "MAVEN_OPTS=-Dmaven.wagon.httpconnectionManager.ttlSeconds=180 -Dmaven.wagon.http.retryHandler.count=3" >> $GITHUB_ENV
    echo "PMD_CI_SCRIPTS_URL=https://raw.githubusercontent.com/pmd/build-tools/master/scripts" >> $GITHUB_ENV
- name: Check Environment
  shell: bash
  run: |
    f=check-environment.sh; \
    mkdir -p .ci && \
    ( [ -e .ci/$f ] || curl -sSL "${PMD_CI_SCRIPTS_URL}/$f" > ".ci/$f" ) && \
    chmod 755 .ci/$f && \
    .ci/$f

The script exits with code 0, if everything is fine and with 1, if one or more problems have been detected. Thus it can fail the build.

files

private-env.asc

This file contains the encrypted secrets used during the build, e.g. github tokens, passwords for sonatype, ...

It is encrypted with the password in PMD_CI_SECRET_PASSPHRASE.

Here's a template for the file:

#
# private-env
#
# encrypt:
# printenv PMD_CI_SECRET_PASSPHRASE | gpg --symmetric --cipher-algo AES256 --batch --armor --passphrase-fd 0 private-env
#
# decrypt:
# printenv PMD_CI_SECRET_PASSPHRASE | gpg --batch --decrypt --passphrase-fd 0 --output private-env private-env.asc
#

export PMD_CI_SECRET_PASSPHRASE=...

# CI_DEPLOY_USERNAME - the user which can upload net.sourceforge.pmd:* to https://oss.sonatype.org/
# CI_DEPLOY_PASSWORD
export CI_DEPLOY_USERNAME=...
export CI_DEPLOY_PASSWORD=...

# CI_SIGN_KEYNAME - GPG key used to sign the release jars before uploading to maven central
# CI_SIGN_PASSPHRASE
# the passphrase is also used for the keystore password in pmd-eclipse-plugin for JAR signing.
export CI_SIGN_KEYNAME=...
export CI_SIGN_PASSPHRASE=...

export PMD_SF_USER=...
# https://sourceforge.net/p/forge/documentation/Using%20the%20Release%20API/
export PMD_SF_APIKEY=...
# https://sourceforge.net/p/forge/documentation/Allura%20API/ (blog, wiki, ...)
# https://sourceforge.net/auth/oauth/
export PMD_SF_BEARER_TOKEN=...

# https://sonarcloud.io/dashboard?id=net.sourceforge.pmd%3Apmd
# The token can be configured here: https://sonarcloud.io/account/security/
export SONAR_TOKEN=...

# https://coveralls.io/github/pmd/pmd
# when logged in, the token is display on that page
export COVERALLS_REPO_TOKEN=...

# for pmd-regression-tester
# https://rubygems.org/settings/edit
export GEM_HOST_API_KEY=...

# These are also in public-env:
export PMD_CI_CHUNK_TOKEN=...

release-signing-key-D0BF1D737C9A1C22.asc

Export the private key as "release-signing-key-D0BF1D737C9A1C22" and encrypt it with PMD_CI_SECRET_PASSPHRASE:

printenv PMD_CI_SECRET_PASSPHRASE | gpg --symmetric --cipher-algo AES256 --batch --armor \
  --passphrase-fd 0 \
  release-signing-key-D0BF1D737C9A1C22

The public key is available here:

And in the file release-signing-key-D0BF1D737C9A1C22-public.asc.

Updating the key:

From time to time the key needs to be renewed. Import the private and public key into your local gpg keystore and renew it. Make sure to renew all subkeys. Then export it again.

Note: The private key is additionally secured by a passphrase - see "CI_SIGN_PASSPHRASE".

You can verify the expiration date with gpg --fingerprint --list-sigs D0BF1D737C9A1C22:

pub   rsa4096 2019-12-16 [SC] [expires: 2024-12-31]
      EBB2 41A5 45CB 17C8 7FAC  B2EB D0BF 1D73 7C9A 1C22
uid           [ultimate] PMD Release Signing Key <releases@pmd-code.org>
sig 3        D0BF1D737C9A1C22 2023-11-23  PMD Release Signing Key <releases@pmd-code.org>
sig 3        93450DF2DF9A3FA3 2019-12-16  Andreas Dangel <andreas.dangel@adangel.org>
sig 3        D0BF1D737C9A1C22 2019-12-16  PMD Release Signing Key <releases@pmd-code.org>
sig 3        D0BF1D737C9A1C22 2020-11-02  PMD Release Signing Key <releases@pmd-code.org>
sub   rsa4096 2019-12-16 [E] [expires: 2024-12-31]
sig          D0BF1D737C9A1C22 2023-11-23  PMD Release Signing Key <releases@pmd-code.org>

Upload the exported public key to

Verify the uploaded key expiration date:

gpg --show-keys release-signing-key-D0BF1D737C9A1C22-public.asc curl 'https://keys.openpgp.org/vks/v1/by-fingerprint/EBB241A545CB17C87FACB2EBD0BF1D737C9A1C22' | gpg --show-keys curl 'https://keyserver.ubuntu.com/pks/lookup?search=0xEBB241A545CB17C87FACB2EBD0BF1D737C9A1C22&fingerprint=on&exact=on&options=mr&op=get' | gpg --show-keys curl 'http://pgp.mit.edu/pks/lookup?op=get&search=0xD0BF1D737C9A1C22' | gpg --show-keys

pmd.github.io_deploy_key.asc

Created with ssh-keygen -t ed25519 -C "ssh key for pmd. used for github actions to push to pmd.github.io" -f pmd.github.io_deploy_key.

Encrypt it with PMD_CI_SECRET_PASSPHRASE:

printenv PMD_CI_SECRET_PASSPHRASE | gpg --symmetric --cipher-algo AES256 --batch --armor \
  --passphrase-fd 0 \
  pmd.github.io_deploy_key

The corresponding public key pmd.github.io_deploy_key.pub is here for convenience. It is configured as a deploy key for the repository pmd.github.io with write access.

In order to use this key to push, you need to clone the repo with this url: git@github.com-pmd.github.io:pmd/pmd.github.io.git.

pmd-eclipse-plugin-p2-site_deploy_key.asc

Created with ssh-keygen -t ed25519 -C "ssh key for pmd. used for github actions to push to pmd-eclipse-plugin-p2-site" -f pmd-eclipse-plugin-p2-site_deploy_key.

Encrypt it with PMD_CI_SECRET_PASSPHRASE:

printenv PMD_CI_SECRET_PASSPHRASE | gpg --symmetric --cipher-algo AES256 --batch --armor \
  --passphrase-fd 0 \
  pmd-eclipse-plugin-p2-site_deploy_key

The corresponding public key pmd-eclipse-plugin-p2-site_deploy_key.pub is here for convenience. It is configured as a deploy key for the repository pmd-eclipse-plugin-p2-site with write access.

In order to use this key to push, you need to clone the repo with this url: git@github.com-pmd-eclipse-plugin-p2-site:pmd/pmd-eclipse-plugin-p2-site.git.

pmd-code.org_deploy_key.asc

Created with ssh-keygen -t ed25519 -C "ssh key for pmd. used for github actions push to pmd-code.org" -f pmd-code.org_deploy_key.

Encrypt it with PMD_CI_SECRET_PASSPHRASE:

printenv PMD_CI_SECRET_PASSPHRASE | gpg --symmetric --cipher-algo AES256 --batch --armor \
  --passphrase-fd 0 \
  pmd-code.org_deploy_key

The corresponding public key pmd-code.org_deploy_key.pub is here for convenience. It is configured in ~/.ssh/authorized_keys on pmd@pmd-code.org.

web.sourceforge.net_deploy_key.asc

Created with ssh-keygen -t ed25519 -C "ssh key for pmd. used for github actions push to web.sourceforge.net" -f web.sourceforge.net_deploy_key.

Encrypt it with PMD_CI_SECRET_PASSPHRASE:

printenv PMD_CI_SECRET_PASSPHRASE | gpg --symmetric --cipher-algo AES256 --batch --armor \
  --passphrase-fd 0 \
  web.sourceforge.net_deploy_key

The corresponding public key web.sourceforge.net_deploy_key.pub is here for convenience. It is configured in for user "PMD_SF_USER" (see private-env) on sourceforge: https://sourceforge.net/auth/shell_services.

Note: The same key is used to push to "git.code.sf.net" as user "PMD_SF_USER".

maven-settings.xml

The maven-settings.xml file contains a profile sign, which brings in the configuration for maven-gpg-plugin to sign the artifacts prior to uploading it to maven central. The configuration comes in via environment variables: CI_SIGN_KEYNAME and CI_SIGN_PASSPHRASE.

It also contains the credentials for uploading the artifacts to maven-central for the server ossrh. Again, the actual configuration comes in via environment variables: CI_DEPLOY_USERNAME and CI_DEPLOY_PASSWORD.

Testing

To test a complete build (or run it manually), you can use the docker build-env. The script create-gh-actions-env.sh can simulate a Github Actions environment by setting up some specific environment variables. With these variables set, utils.bash/pmd_ci_utils_determine_build_env can figure out the needed information and utils.bash/pmd_ci_utils_is_fork_or_pull_request works.

Example session for a pull request:

pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ unset PMD_CI_SECRET_PASSPHRASE
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ export PMD_CI_SCRIPTS_URL=https://raw.githubusercontent.com/adangel/build-tools/gh-action-scripts/scripts
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ eval $(~/create-gh-actions-env.sh pull_request adangel/build-tools gh-actions-scripts)
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ .ci/build.sh
...

Example session for a forked build (a build executing on a forked repository):

pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ unset PMD_CI_SECRET_PASSPHRASE
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ export PMD_CI_SCRIPTS_URL=https://raw.githubusercontent.com/adangel/build-tools/gh-action-scripts/scripts
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ eval $(~/create-gh-actions-env.sh push adangel/build-tools gh-actions-scripts)
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ .ci/build.sh
...

Example session for a push build on the main repository:

pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ export PMD_CI_SECRET_PASSPHRASE=...
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ export PMD_CI_SCRIPTS_URL=https://raw.githubusercontent.com/adangel/build-tools/gh-action-scripts/scripts
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ eval $(~/create-gh-actions-env.sh push pmd/build-tools gh-actions-scripts)
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ .ci/build.sh
...

Example session for a release build on the main repository from tag "v1.0.0":

pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ export PMD_CI_SECRET_PASSPHRASE=...
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ export PMD_CI_SCRIPTS_URL=https://raw.githubusercontent.com/adangel/build-tools/gh-action-scripts/scripts
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ eval $(~/create-gh-actions-env.sh push pmd/build-tools refs/tags/v1.0.0)
pmd-ci@6cc27446ef02:~/workspaces/pmd/build-tools$ .ci/build.sh
...

Note, that create-gh-actions-env.sh sets up MAVEN_OPTS with -DskipRemoteStaging=true, so that no maven artifacts are deployed automatically. You need to remove this, if you really want to perform a release. Also note, that the property autoReleaseAfterClose is not configured and the default is false, so that you would need to manually publish the staging repo. See also the section below about "Nexus Staging Maven Plugin".

Miscellaneous

Nexus Staging Maven Plugin

See https://github.com/sonatype/nexus-maven-plugins/tree/master/staging/maven-plugin.

This plugin is used, to upload maven artifacts to https://oss.sonatype.org/ and eventually to maven central using the open source workflow by sonatype, see OSSRH Guide.

The plugin can be configured, see https://github.com/sonatype/nexus-maven-plugins/tree/master/staging/maven-plugin#configuring-the-plugin for some options.

Most important here are these:

skipRemoteStaging=true: Used during test runs of releases. This makes sure, the artifacts are only staged locally and never uploaded to https://oss.sonatype.org/.

Property: skipRemoteStaging
autoReleaseAfterClose=true: After all modules have been uploaded to the staging repository it is automatically closed (this can be controlled through skipStagingRepositoryClose but is the default behavior). And with autoReleaseAfterClose, the closed staging repository will be automatically released and published to maven central. This allows for fully automated releases.

This property is set via MAVEN_OPTS in the workflow (build.yml). It is not set in the pom.xml as a plugin configuration directly in order to allow to override this setting from command line if needed (e.g. during release tests).

Property: autoReleaseAfterClose
stagingProgressTimeoutMinutes=30: This increases the default timeout of 5 minutes to 30 minutes for interaction with oss.sonatype.org. The main PMD repo has a lot of modules and depending on the load of oss.sonatype.org, the release of the staging repo might take a while.

Property: stagingProgressTimeoutMinutes

After the staging repository has been released, it is eventually synced to maven central. The release won't appear here immediately but usually within 2 hours. You can check the current publish latency at https://status.maven.org/.

Remote debugging

Debugging remotely is possible with https://github.com/mxschmitt/action-tmate.

Just add the following step into the job:

      - name: Setup tmate session
        uses: mxschmitt/action-tmate@v3

The workflow troubleshooting in PMD can be started manually, which already contains the tmate action.

Note: This is dangerous for push/pull builds on repositories of pmd itself, because these have access to the secrets and the SSH session is not protected. Builds triggered by pull requests from forked repositories don't have access to the secrets.

See also https://docs.github.com/en/actions/reference/encrypted-secrets.

Intermittent connection resets or timeouts while downloading dependencies from maven central

Root issue seems to be SNAT Configs in Azure, which closes long running idle TCP connections after 4 minutes.

The workaround is described in actions/virtual-environments#1499 and WAGON-545 and WAGON-486:

The setting -Dmaven.wagon.httpconnectionManager.ttlSeconds=180 -Dmaven.wagon.http.retryHandler.count=3 makes sure, that Maven doesn't try to use pooled connections that have been unused for more than 180 seconds. These settings are placed as environment variable MAVEN_OPTS in the workflow, so that they are active for all Maven executions (including builds done by regression tester).

Alternatively, pooling could be disabled completely via -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false. This has the consequence, that for each dependency, that is being downloaded, a new https connection is established.

More information about configuring this can be found at wagon-http.

Update: Since Maven 3.9.0, the native transport instead of wagon is used:

The Maven Resolver transport has changed from Wagon to “native HTTP”, see Resolver Transport guide.

Therefore, the property to configure the timeouts changed to -Daether.connector.http.connectionMaxTtl=180. Retry count is by default 3 and can be omitted. See https://maven.apache.org/resolver/configuration.html for all available properties.

Note: This system property only works with Maven 3.9.2 or later!

Name		Name	Last commit message	Last commit date
Latest commit History 376 Commits
.ci		.ci
.github/workflows		.github/workflows
.mvn/wrapper		.mvn/wrapper
build-env		build-env
eclipse		eclipse
intellij-idea		intellij-idea
scripts		scripts
src		src
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
mvnw		mvnw
mvnw.cmd		mvnw.cmd
pom.xml		pom.xml
release-howto.md		release-howto.md

License

pmd/build-tools

Folders and files

Latest commit

History

Repository files navigation

PMD build tools

build-env

scripts

Overview

Usage

inc/fetch_ci_scripts.bash

inc/log.bash

inc/utils.bash

inc/openjdk.bash

inc/github-releases-api.bash

inc/setup-secrets.bash

inc/sourceforge-api.bash

inc/maven.bash

inc/pmd-code-api.bash

check-environment.sh

files

private-env.asc

release-signing-key-D0BF1D737C9A1C22.asc

pmd.github.io_deploy_key.asc

pmd-eclipse-plugin-p2-site_deploy_key.asc

pmd-code.org_deploy_key.asc

web.sourceforge.net_deploy_key.asc

maven-settings.xml

Testing

Miscellaneous

Nexus Staging Maven Plugin

Remote debugging

Intermittent connection resets or timeouts while downloading dependencies from maven central

About

Resources

License

Stars

Watchers

Forks

Languages