diff --git a/.gitignore b/.gitignore index 46882055d..37c58ce83 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,2 @@ -openjdk/ -target .idea/ -alsa-lib* -freetype* +workspace diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..7ed393513 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,115 @@ +# Changelog for openjdk-build scripts + +## Version 1.0.0 (14 May 2018) + +See [Commit History](https://github.com/AdoptOpenJDK/openjdk-build/commits/master) +up until May the 14th 2018. + +## Version 2.0.0 (TBA) + +A major overhaul to split out Docker and Native builds, fix a host of small +issues and place build jobs into Groovy Pipeline scripts. + +### Core Build Changes + +1. _configureBuild.sh_ added. The pre-build configuration logic now resides in +this script. +1. _native-build.sh_ added. This script is invoked for building (Adopt) OpenJDK +binaries natively. +1. _docker-build.sh_ added. This script is invoked for building (Adopt) OpenJDK +binaries in a Docker container. +1. _sbin/prepareWorkspace.sh_ added +1. _sbin/colour-codes.sh_ removed to simplify code +1. _makejdk.sh_ removed - please use _makejdk-any-platform.sh_ or (rarely) +_sbin/build.sh_ instead. +1. _sbin/common-functions.sh_ removed and its logic split +1. _sbin/common/common.sh_ added +1. _sbin/common/config_init.sh_ added +1. _sbin/common/constants.sh_ added +1. _sbin/build.template_ added for saving off the configure configuration. +1. _sbin/signalhandler.sh_ moved to _signalhandler.sh_ +1. _sbin/build.sh_ enhanced, now requires a 'saved' build configuration to run. +This 'saved' build configuration is created by _makejdk-any-platform.sh_ but +can be generated manually as well. +1. _sign.sh_ added for code signing functionality. + + +#### _makejdk-any-platform.sh_, _build.sh_, _makejdk.sh_ usage changes + +1. More versions added, `jdk8u | jdk9 | jdk10 | jfx | amber` are now all supported +1. `-B` is now used for specifying the build number (long form `--build-number`). +1. `-bv` is removed, (long form `--variant` changes to `--build-variant`). +1. `-c` (long form `--clean-docker-build`) added to build from a clean docker container. +1. `-ca` changes to `-C`, (long form `--configure-args` stays the same). +1. `-D` (long form `--docker`) added for building in a docker container. +1. `-dsgc` is removed, (long form `--disable-shallow-git-clone` stays the same). +1. `-ftd` changes to `-f`, (long form `--freetype-dir` stays the same). +1. `-h` (long form `--help`) added. +1. `-i` (long form `--ignore-container`) added to ignore existing docker container. +1. `-j, --jtreg` and `-js, --jtreg-subsets` are removed as tests should be run +via the openjdk-tests repo / project. +1. `-J` (long form `--jdk-boot-dir` added to set JDK boot dir. +1. `-nc` changes to `-n`, (long form `--no-colour` stays the same). +1. `-p` (long form `--processors`) added to set number of processors in docker build. +1. `-sf` changes to `-F`, (long form `--skip-freetype` stays the same). +1. `--sudo` added to run the docker container as root. +1. `--tmp-space-build` (set a temporary build space if regular workspace is unavailable). +1. `-T` (long form `--target-file-name` added to specify the final name of the binary. +1. `-u` (long form `--update-version`) added to specify the update version. +1. `-V` (long form `--jvm-variant` specify the JVM variant (server or client). + +Please see _makejdk-any-platform.1_ man page for full details. + +### Test Changes + +1. _sbin/jtreg.sh_ removed (superseded by the openjdk-tests project). +1. _sbin/jtreg_prep.sh_ removed (superseded by the openjdk-tests project). + +### Docker Support + +1. `-D` (long form `--docker`) has been added for building in a docker container. +1. `-c` (long form `--clean-docker-build`) has been added to build from a clean +docker container. +1. `-i` (long form `--ignore-container`) has been added to ignore existing docker +container. +1. `-p` (long form `--processors`) added to set number of processors in docker build. +1. `--sudo` added to run the docker container as root. +1. _docker-build.sh_ added. This script is invoked for building (Adopt) OpenJDK +binaries in a Docker container. +1. _docker/jdk/x86_64/ubuntu/Dockerfile_ updated for various bug fixes. +1. _docker/jdk/x86_64/ubuntu/dockerConfiguration.sh_ files added. These +contain Docker specific environment variables that the build scripts need (as +opposed to falsely picking up the underlying native env). + +### Build Farm Support + +1. New _build-farm/make-adopt-build-farm.sh_ added for the new AdoptOpenJDK +Build Farm jenkins pipeline to build Adopt OpenJDK binaries. Sets the default +environment variables that are currently set in individual jobs. This allows +us to now track and version these variables. +1. New _build-farm/set-platform-specific-configurations.sh_ added for the new +AdoptOpenJDK Build Farm jenkins pipeline to build Adopt OpenJDK binaries. Sets +the default environment variables that are currently set in individual jobs. +This allows us to now track and version these variables. +1. New _build-farm/set-platform-specific-configurations/.sh added for +the new AdoptOpenJDK Build Farm jenkins pipeline to build Adopt OpenJDK binaries. +Sets the default environment variables for specific platforms that are currently +set in individual jobs. This allows us to now track and version these variables. +1. New _build-farm/sign-releases.sh added for the new AdoptOpenJDK Build Farm +jenkins pipeline to code sign Adopt OpenJDK binaries (Mac and Windows for now). +1. _pipelines/build/build_base_file.groovy_ added. This co-ordinates the various + pipeline builds. +1. _pipelines/build/openjdk\_build\_pipeline.groovy_ added. This forms the base +pipeline code for each build. +1. _pipelines/build/openjdk\\_\\_\\_pipeline.groovy_ +files added. These will eventually replace the existing individual jobs with a +Pipeline for each version and variant. + +### Documentation and Misc + +1. _README.md_ updated to reflect new scripts +1. _images/AdoptOpenJDK_Build_Script_Relationships.png_ added to show script +relationship. +1. _.gitignore_ changed to reflect new `workspace` base directory, please check +your local .gitignore for the diff. +1. _makejdk-any-platform.1_ man page updated to reflect new script usage diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 823823454..7e6e24347 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -28,7 +28,7 @@ We ask that you include a line similar to the following as part of your pull req ``` DCO 1.1 Signed-off-by: Random J Developer ``` -“DCO” stands for “Developer Certificate of Origin,” and refers to [the same text used in the Linux Kernel community](http://elinux.org/Developer_Certificate_Of_Origin). Of course, you should replace "Random J Developer" by your own real name. +“DCO” stands for “Developer Certificate of Origin,” and refers to [the same text used in the Linux Kernel community](https://elinux.org/Developer_Certificate_Of_Origin). Of course, you should replace "Random J Developer" by your own real name. By adding this simple comment, you are telling the community that you wrote the code you are contributing, or you have the right to pass on the code that you are contributing. @@ -39,7 +39,7 @@ All the project's source files must start with a comment, as near to the top of We don't place explicit copyright statements in the project source files. The project comprises many distinct pieces of code, spread across numerous source files, and authored by a variety of individuals. Managing copyright statements is unproductive and [can lead to confusion and contention around the edge cases](https://opensource.com/law/14/n2/copyright-statements-source-files). Rather we utilize [the NOTICE file](https://github.com/AdoptOpenJDK/openjdk-build/blob/master/NOTICE) mechanism as a way to acknowledge copyright broadly where there is a valid reason to do so. -Finally, for similar reasons to avoiding individual copyright statements, we don't maintain `@author` tags in source files. There are good arguments to suggest that [author tags discourage open contribution](http://producingoss.com/en/managing-volunteers.html#territoriality), and we depend upon Git to maintain that information for the project. +Finally, for similar reasons to avoiding individual copyright statements, we don't maintain `@author` tags in source files. There are good arguments to suggest that [author tags discourage open contribution](https://producingoss.com/en/managing-volunteers.html#territoriality), and we depend upon Git to maintain that information for the project. ### Ensuring high quality After we receive your pull request the [Travis pull request builder](https://travis-ci.org/AdoptOpenJDK/openjdk-build) will test your changes and ensure they meet the coding style guidelines. Watch for the results posted as a comment to the PR, and investigate and fix any failures. diff --git a/LICENSE b/LICENSE index 8f71f43fe..d5dd862b1 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ Apache License Version 2.0, January 2004 - http://www.apache.org/licenses/ + https://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION @@ -192,7 +192,7 @@ you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/README.md b/README.md index 66720fc60..d5a66a326 100644 --- a/README.md +++ b/README.md @@ -1,137 +1,309 @@ -# This Branch is a legacy branch!! - -Jobs are currently being migrated to the -[new_build_scripts](https://github.com/AdoptOpenJDK/openjdk-build/tree/new_build_scripts) branch. - -If you're looking to make fixes or additions to the build scripts please do so -on the [new_build_scripts](https://github.com/AdoptOpenJDK/openjdk-build/tree/new_build_scripts) branch -first and only backport to this legacy branch if required. - -**11th of Sept 2018** - -At time of writing the new_build_scripts branch are building most jdk10 jobs - -

-

-

-

-

-

-

-

-

-

-

---- - - # Repository for code and instructions for building OpenJDK [![Build Status](https://travis-ci.org/AdoptOpenJDK/openjdk-build.svg?branch=master)](https://travis-ci.org/AdoptOpenJDK/openjdk-build) [![Slack](https://slackin-jmnmplfpdu.now.sh/badge.svg)](https://slackin-jmnmplfpdu.now.sh/) -AdoptOpenJDK makes use of these scripts to build binaries on the build farm at http://ci.adoptopenjdk.net which produces OpenJDK binaries for consumption via -https://www.adoptopenjdk.net and https://api.adoptopenjdk.net +AdoptOpenJDK makes use of these scripts to build binaries on the build farm at +https://ci.adoptopenjdk.net, which produces OpenJDK binaries for consumption via +https://www.adoptopenjdk.net and https://api.adoptopenjdk.net. + +## TLDR I want to build a JDK NOW! + +##### Build jdk natively on your system + +``` +./makejdk-any-platform.sh +i.e: +./makejdk-any-platform.sh jdk8u +``` + +##### Build jdk inside a docker container +``` +./makejdk-any-platform.sh --docker jdk8u +``` +If you need sudo to run docker on your system. +``` +./makejdk-any-platform.sh --sudo --docker jdk8u +``` + +## Build +The build has 2 modes, native and docker. + +### Native +Native builds run on whatever platform the script is invoked on, i.e +if you invoke a native build on MacOS it will build a JDK for MacOS. + +### Docker +This runs a build inside a docker container. Currently this will always +build a linux JDK. ## Repository contents -This repository contains several useful scripts in order to build OpenJDK personally or at build farm scale. +This repository contains several useful scripts in order to build OpenJDK +personally or at build farm scale. + +1. The `build-farm` folder contains shell scripts for multi configuration Jenkins +build jobs used for building Adopt OpenJDK binaries. TODO This may get removed. +2. The `docker` folder contains DockerFiles which can be used as part of building +OpenJDK inside a Docker container. +3. The `git-hg` folder contains scripts to clone an OpenJDK mercurial forest into +a GitHub repo ()and regularly update it). +4. The `images` folder contains diagrams to aid understanding. +5. The `mercurial-tags/java-tool` folder contains scripts for TODO. +6. The `pipelines` folder contains the Groovy pipeline scripts for Jenkins +(e.g. build | test | checksum | release). +7. The `sbin` folder contains the scripts that actually build (AdoptOpenJDK). +`build.sh` is the entry point which can be used stand alone but is typically +called by the `native-build.sh` or `docker-build.sh` scripts (which themselves +are typically called by `makejdk-any-platform.sh`). +8. The `security` folder contains a script and `cacerts` file that is bundled +with the JDK and used when building OpenJDK: the `cacerts` file is an important +file that's used to enable SSL connections. + +## The makejdk-any-platform.sh script + +`makejdk-any-platform.sh` is the entry point for building (Adopt) OpenJDK binaries. +Building natively or in a docker container are both supported. This script (and +its supporting scripts) have defaults, but you can override these as needed. +The scripts will auto detect the platform and architecture it is running on and +configure the OpenJDK build accordingly. The supporting scripts will also +download and locally install any required dependencies for the OpenJDK build, +e.g. The ALSA sound and Freetype font libraries. + +Many of the configuration options are passed through to the `configure` and +`make` commands that OpenJDK uses to build binaries. Please see the appropriate +_README-builds.html_ file for the OpenJDK source repository that you are building. + +**NOTE:** Usage can be found via `makejdk-any-platform.sh --help`. Here is the +man page re-formatted for convenience. + +``` +USAGE + +./makejdk-any-platform [options] version + +Please visit https://www.adoptopenjdk.net for further support. + +VERSIONS + +jdk8u - Build Java 8, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk8u +jdk9u - Build Java 9, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk9u +jdk10u - Build Java 10, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk10u +jdk11 - Build Java 10, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk11 +jfx - Build OpenJFX, defaults to https://github.com/AdoptOpenJDK/openjdk-jfx +amber - Build Project Amber, defaults to https://github.com/AdoptOpenJDK/openjdk-amber + +OPTIONS + +-b, --branch +specify a custom branch to build from, e.g. dev. +For reference, AdoptOpenJDK GitHub source repos default to the dev +branch which may contain a very small diff set to the master branch +(which is a clone from the OpenJDK mercurial forest). + +-B, --build-number +specify the OpenJDK build number to build from, e.g. b12. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the build number being the suffix at the end. + +-c, --clean-docker-build +removes the existing docker container and persistent volume before starting +a new docker based build. + +-C, --configure-args +specify any custom user configuration arguments. + +-d, --destination +specify the location for the built binary, e.g. /path/. +This is typically used in conjunction with -T to create a custom path +/ file name for the resulting binary. + +-D, --docker +build OpenJDK in a docker container. + +--disable-shallow-git-clone +disable the default fB--depth=1 shallow cloning of git repo(s). + +-f, --freetype-dir +specify the location of an existing FreeType library. +This is typically used in conjunction with -F. + +-F, --skip-freetype +skip building Freetype automatically. +This is typically used in conjunction with -f. -1. The `docker` folder contains a Docker file which can be used to create a Docker container for building OpenJDK -2. The `git-hg` folder contains scripts to clone an OpenJDK mercurial forest into a GitHub repo and regularly update it -3. The `mercurial-tags/java-tool` folder contains scripts for TODO -4. The `pipelines` folder contains the Groovy pipeline scripts for Jenkins (e.g. build | test | checksum |release) -5. The `sbin` folder contains the scripts called by the main script. -6. The `security` folder contains a script and `cacerts` file that is bundled with the JDK and used when building OpenJDK: the `cacerts` file is an important -file that's used to enable SSL connections +-i, --ignore-container +ignore the existing docker container if you have one already. -The main script to build OpenJDK is `makejdk-any-platform.sh` + -J, --jdk-boot-dir +specify the JDK boot dir. +For reference, OpenJDK needs the previous version of a JDK in order to build +itself. You should select the path to a JDK install that is N-1 versions below +the one you are trying to build. +-k, --keep +if using docker, keep the container after the build. + +-n, --no-colour +disable colour output. + +-p, --processors +specify the number of processors to use for the docker build. + +-r, --repository +specify the repository to clone OpenJDK source from, +e.g. https://github.com/karianna/openjdk-jdk8u. + +-s, --source +specify the location to clone the OpenJDK source (and dependencies) to. + +-S, --ssh +use ssh when cloning git. + +--sign +sign the OpenJDK binary that you build. + +--sudo +run the docker container as root. + +-t, --tag +specify the repository tag that you want to build OpenJDK from. + +-T, --target-file-name +specify the final name of the OpenJDK binary. +This is typically used in conjunction with -D to create a custom file +name for the resulting binary. + +-u, --update-version +specify the update version to build OpenJDK from, e.g. 162. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the update number being the number after the '_' +(162) or the 3rd position in the semVer version string (4). +This is typically used in conjunction with -b. + +--use-jep319-certs +Use certs defined in JEP319 in Java 8/9. This will increase the volume of traffic downloaded, however will +provide an upto date ca cert list. + +-v, --build-variant +specify a OpenJDK build variant, e.g. openj9. +For reference, the default variant is hotspot and does not need to be specified. + +-V, --jvm-variant +specify the JVM variant (server or client), defaults to server. + +Example usage: + +./makejdk-any-platform --docker jdk8u +./makejdk-any-platform -T MyOpenJDK10.tar.gz jdk10 + +``` + +### Script Relationships + +![Build Variant Workflow](docs/images/AdoptOpenJDK_Build_Script_Relationships.png) + +The main script to build OpenJDK is `makejdk-any-platform.sh`, which itself uses +and/or calls `configureBuild.sh`, `docker-build.sh` and/or `native-build.sh`. + +The structure of a build is: + + 1. Configuration phase determines what the configuration of the build is based on your current +platform and and optional arguments provided + 1. Configuration is written out to `built_config.cfg` + 1. Build is kicked off by either creating a docker container or running the native build script + 1. Build reads in configuration from `built_config.cfg` + 1. Downloads source, dependencies and prepares build workspace + 1. Configure and invoke OpenJDK build via `make` + 1. Package up built artifacts + +- Configuration phase is primarily performed by `configureBuild.sh` and `makejdk-any-platform.sh`. +- If a docker container is required it is built by `docker-build.sh`. +- In the build phase `sbin/build.sh` is invoked either natively or inside the docker container. +`sbin/build.sh` invokes `sbin/prepareWorkspace.sh` to download dependencies, source and perform +general preparation. +- Rest of the build and packaging is then handled from `sbin/build.sh` + ## Building OpenJDK ### Building on the Build Farm In order to build an OpenJDK variant on the build farm you need to follow the -[Adding-a-new-build-variant](https://github.com/AdoptOpenJDK/TSC/wiki/Adding-a-new-build-variant) instructions. +[Adding-a-new-build-variant](https://github.com/AdoptOpenJDK/TSC/wiki/Adding-a-new-build-variant) +instructions. The configuration options are often set in the Jenkins job and +passed into `makejdk-any-platform.sh` script. -### Building locally via Docker +Note that the build nodes (list of hosts on the LH side) also have configuration +where things like the BOOT_JDK environment variable is set. -**WARN: As of 23rd March 2018 these instructions do not work, there are several issues that need resolving ([see this issue as starting point](https://github.com/AdoptOpenJDK/openjdk-build/issues/194))** +### Building via Docker in your local environment -Make sure you have started your Docker Daemon first! For help with getting docker follow the instructions [here](https://docs.docker.com/engine/installation/). -Once you have Docker started you can then use the script below to build OpenJDK. - -``` -Usage: ./makejdk-any-platform.sh --version [version] [options] - -Versions: - jdk8u - https://github.com/AdoptOpenJDK/openjdk-jdk8u - jdk9u - https://github.com/AdoptOpenJDK/openjdk-jdk9u - jdk10u - https://github.com/AdoptOpenJDK/openjdk-jdk10u - jdk11 - https://github.com/AdoptOpenJDK/openjdk-jdk11 - -Options: - -s, --source specify the location for the source and dependencies to be cloned, defaults to ./openjdk. If it is specified, docker is not used - -d, --destination specify the location for the tarball (eg. /path/ or /path/here.tar.gz) - -r, --repository specify a custom repository (eg. username/openjdk-jdk8u) - -b, --branch specify a custom branch (eg. dev) - -k, --keep reuse docker container (prevents deleting) - -j, --jtreg run jtreg after building - -js, --jtreg-subsets select one or more jtreg tests to run - -S, --ssh use ssh when cloning git - -sf --skip-freetype skip building freetype - -nc --no-colour disable colour output - -ftd --freetype-dir specify the location of an existing FreeType library that can be used for the OpenJDK build process - -dsgc --disable-shallow-git-clone disable shallow cloning of git repo(s) using the --depth=1 CLI option - -bv --variant specify a build variant name, e.g. openj9 - -c --clean-docker-build clean docker data volume - -t --tag specify a custom tag - --sign specify the location for the windows p12 certificate. Used only for windows builds to sign DLL - -ca --configure-args specify a custom configuration arguments -``` +The simplest way to build OpenJDK using these scripts is to run `makejdk-any-platform.sh` +and have your user be in the Docker group on the machine (or use the `--sudo` +option to prefix all of your Docker commands with `sudo`). This script will create +a Docker container that will be configured with all of the required dependencies +and a base operating system in order to build OpenJDK. -The simplest way to build OpenJDK using our scripts is to run `makejdk-any-platform.sh` and have your user be in the Docker group on the machine -(or prefix all of your Docker commands with `sudo`). This script will create a Docker container that will be configured with all of the required -dependencies and a base operating system in order to build OpenJDK. For example: +Make sure you have started your Docker Daemon first! For help with getting +docker follow the instructions [here](https://docs.docker.com/engine/installation/). +Once you have Docker started you can then use the script below to build OpenJDK. -`./makejdk-any-platform.sh -c --ssh --version jdk8u` +Example Usage (TODO Add example of openj9): -* **NOTE:** If you don't use SSH keys (if you do then pass `-ssh`) to connect to GitHub then the script will challenge you for your GitHub username and password. (You can find more information about SSH keys for GitHub [here](https://help.github.com/articles/connecting-to-github-with-ssh/)) -* **NOTE:** The script will clone source code into the `--source` directory (defaults to `openjdk`). -* **NOTE:** By default the docker container is removed each time and your build will be copied from the container to the host. -To override this behaviour, specify the `-k` or `--keep` option. -* **NOTE:** The entire process will take some time, especially if you have not saved the Docker image from a previous run. -* **NOTE:** If you set the `-d` option it will pass that through to `makejdk.sh`, the resulting zipped tarball will be copied to the value for -d, for example: -`makejdk.sh /target/directory` will result in the JDK being built inside of your Docker container and then copied to `/target/directory` on the host +`./makejdk-any-platform.sh --docker --sudo jdk8u` #### Configuring Docker for non sudo use -To use the Docker commands without using the sudo prefix, you will need to be in the Docker group which can be achieved with the following three commands +To use the Docker commands without using the `--sudo` option, you will need to be +in the Docker group which can be achieved with the following three commands (performed as `root`) 1. `sudo groupadd docker`: creates the Docker group if it doesn't already exist 2. `sudo gpasswd -a yourusernamehere docker`: adds a user to the Docker group 3. `sudo service docker restart`: restarts the Docker service so the above changes can take effect -### Building in your local environment +### Building natively in your local environment + +Please note that your build host will need to have certain pre-requisites met. +We provide Ansible scripts in the +[openjdk-infrastructure](https://www.github.com/AdoptOpenJDK/openjdk-infrastructure) +project for setting these pre-requisites. + +Example Usage (TODO Add example of openj9): -Please note that your build host will need to have certain pre-requisites met. We provide Ansible scripts in the -[openjdk-infrastructure](https://www.github.com/AdoptOpenJDK/openjdk-infrastructure) project for setting these pre-requisites. +`./makejdk-any-platform.sh -s /home/openjdk10/src -d /home/openjdk/target -T MyOpenJDK10.tar.gz jdk10` -You can use the `makejdk-any-platform.sh` script by providing two parameters: +This would clone OpenJDK source from _https://github.com/AdoptOpenJDK/openjdk-jdk10_ +to `/home/openjdk10/src`, configure the build with sensible defaults according +to your local platform and then build (Adopt) OpenJDK and place the result in +`/home/openjdk/target/MyOpenJDK10.tar.gz`. -1. The _working directory_ (which is where files will be downloaded to: this includes a number of libraries used with OpenJDK itself such as FreeType and ALSA) -1. The _target directory_ which will be used to store the final _.tar.gz_ file containing the _j2sdk-image_ +### Building OpenJDK from a non Adopt source -e.g `./makejdk-any-platform.sh -s /path/to/workspace -d /target/directory` +These scripts default to using AdoptOpenJDK as the OpenJDK source repo to build +from, but you can override this with the `-r` flag. -**NOTE:** Usage can be found via `makejdk-any-platform.sh --help`, the exact usage is available for this script as well. -### None of the above? +# Build farm -You can use the `makejdk.sh` script by providing two parameters: +The documentation for files that define the build farm builds can be found at [Build Farm Definition](/docs/build.md). -1. The _working directory_ (which is where files will be downloaded to: this includes a number of libraries used with OpenJDK itself such as FreeType and ALSA) -1. The _target directory_ which will be used to store the final _.tar.gz_ file containing the _j2sdk-image_ +# Build status -e.g `./makejdk.sh -s /path/to/workspace -d /target/directory` +Table generated with `generateBuildMatrix.sh` +| Platform | Java 8 | Java 9 | Java 10 | Java 11 | +| ------------------------- | ------ | ------ | ------- | ------- | +| aix-ppc64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-hotspot) | +| aix-ppc64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-openj9) | +| linux-aarch64-hotspot | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-aarch64-hotspot) | +| linux-arm-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-arm-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-arm-hotspot) | +| linux-ppc64le-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-hotspot) | +| linux-ppc64le-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-openj9) | +| linux-s390x-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-hotspot) | +| linux-s390x-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-openj9) | +| linux-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-hotspot) | +| linux-x64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9) | +| linux-x64-openj9-linuxXL | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9-linuxXL) | +| mac-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-mac-x64-hotspot) | +| windows-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-windows-x64-hotspot) | +| windows-x64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-windows-x64-openj9) | diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh new file mode 100755 index 000000000..c485ec0d4 --- /dev/null +++ b/build-farm/make-adopt-build-farm.sh @@ -0,0 +1,106 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +set -e + +PLATFORM_SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +## Very very build farm specific configuration + +TIMESTAMP="$(date +'%Y-%m-%d-%H-%M')" + +export OPERATING_SYSTEM +OPERATING_SYSTEM=$(echo "${TARGET_OS}" | tr '[:upper:]' '[:lower:]') + + +echo "BUILD TYPE: " +echo "VERSION: ${JAVA_TO_BUILD}" +echo "ARCHITECTURE ${ARCHITECTURE}" +echo "VARIANT: ${VARIANT}" +echo "OS: ${OPERATING_SYSTEM}" +echo "BRANCH: ${BRANCH}" + +OPTIONS="" +EXTENSION="" +# shellcheck disable=SC2034 +CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS:-""} +BUILD_ARGS=${BUILD_ARGS:-""} +VARIANT_ARG="${JAVA_TO_BUILD}-" + +if [ -z "${JDK_BOOT_VERSION}" ] +then + echo "Detecting boot jdk for: ${JAVA_TO_BUILD}" + currentBuildNumber=$(echo "${JAVA_TO_BUILD}" | tr -d "[:alpha:]") + echo "Found build version: ${currentBuildNumber}" + JDK_BOOT_VERSION=$((currentBuildNumber-1)) + echo "Boot jdk version: ${JDK_BOOT_VERSION}" +fi + +case "${JDK_BOOT_VERSION}" in + "7") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK7_BOOT_DIR}";; + "8") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK8_BOOT_DIR}";; + "9") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK9_BOOT_DIR}";; + "10") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK10_BOOT_DIR}";; + "11") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; + *) export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; +esac + + +if [ ! -d "${JDK_BOOT_DIR}" ] +then + export JDK_BOOT_DIR="${JAVA_HOME}" +fi + +echo "Boot jdk: ${JDK_BOOT_DIR}" + + +if [ "${OPERATING_SYSTEM}" == "linux" ] ; then + EXTENSION="tar.gz" + + if [ ! -z "${TAG}" ]; then + OPTIONS="${OPTIONS} --tag $TAG" + fi +elif [ "${OPERATING_SYSTEM}" == "aix" ] ; then + EXTENSION="tar.gz" +elif [ "${OPERATING_SYSTEM}" == "mac" ] ; then + EXTENSION="tar.gz" +elif [ "${OPERATING_SYSTEM}" == "windows" ] ; then + EXTENSION=zip +fi + +if [ ! -z "${BRANCH}" ] +then + OPTIONS="${OPTIONS} -b ${BRANCH}" +fi + +# shellcheck source=build-farm/set-platform-specific-configurations.sh +source "${PLATFORM_SCRIPT_DIR}/set-platform-specific-configurations.sh" + +# Set the file name +JAVA_TO_BUILD_UPPERCASE=$(echo "${JAVA_TO_BUILD}" | tr '[:lower:]' '[:upper:]') + +if [ ! -z "${ADDITIONAL_FILE_NAME_TAG}" ]; then + FILENAME="Open${JAVA_TO_BUILD_UPPERCASE}_${ARCHITECTURE}_${OPERATING_SYSTEM}_${VARIANT}_${ADDITIONAL_FILE_NAME_TAG}_${TIMESTAMP}.${EXTENSION}" +else + FILENAME="Open${JAVA_TO_BUILD_UPPERCASE}_${ARCHITECTURE}_${OPERATING_SYSTEM}_${VARIANT}_${TIMESTAMP}.${EXTENSION}" +fi + +echo "Filename will be: $FILENAME" + +export BUILD_ARGS="${BUILD_ARGS} --use-jep319-certs" + +# shellcheck disable=SC2086 +bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" diff --git a/build-farm/platform-specific-configurations/aix.sh b/build-farm/platform-specific-configurations/aix.sh new file mode 100755 index 000000000..61a88a01f --- /dev/null +++ b/build-farm/platform-specific-configurations/aix.sh @@ -0,0 +1,79 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +export PATH="/opt/freeware/bin:/usr/local/bin:/opt/IBM/xlC/13.1.3/bin:/opt/IBM/xlc/13.1.3/bin:$PATH" +export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-memory-size=18000 --with-cups-include=/opt/freeware/include" + +if [ "${JAVA_TO_BUILD}" != "${JDK11_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-extra-ldflags=-lpthread --with-extra-cflags=-lpthread --with-extra-cxxflags=-lpthread" +fi + +export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" + +if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ]; +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} DF=/usr/sysv/bin/df" + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk8_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk9_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk10_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk10_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar DF=/usr/sysv/bin/df" + fi +fi + +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ]; +then + export JDK10_BOOT_DIR="$PWD/jdk-10" + if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then + mkdir -p "$JDK10_BOOT_DIR" + wget -q -O - "https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=aix&release=latest&arch=${ARCHITECTURE}" | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" + fi + export JDK_BOOT_DIR=$JDK10_BOOT_DIR + + + if [ "${VARIANT}" == "hotspot" ]; then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} DF=/usr/sysv/bin/df" + fi + + if [ "${VARIANT}" == "openj9" ]; then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-warnings-as-errors" + if [ -r /usr/local/gcc/bin/gcc-7.3 ]; then + # Following line ensures the latest ccache is picked up too + export PATH=/usr/local/gcc/bin:$PATH + export CC=gcc-7.3 + export CXX=g++-7.3 + export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib + fi + fi + + export LANG=C + export PATH=/opt/freeware/bin:$JAVA_HOME/bin:/usr/local/bin:/opt/IBM/xlC/13.1.3/bin:/opt/IBM/xlc/13.1.3/bin:$PATH + +fi diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh new file mode 100755 index 000000000..f310d233c --- /dev/null +++ b/build-farm/platform-specific-configurations/linux.sh @@ -0,0 +1,99 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +if [ "${ARCHITECTURE}" == "x64" ] +then + export PATH=/opt/rh/devtoolset-2/root/usr/bin:$PATH +fi + +if [ "${ARCHITECTURE}" == "s390x" ] +then + export LANG=C + + if [ "${VARIANT}" == "openj9" ] + then + export PATH="/usr/bin:$PATH" + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + if which g++-4.8; then + export CC=gcc-4.8 + export CXX=g++-4.8 + fi + fi + fi +fi + +if [ "${ARCHITECTURE}" == "ppc64le" ] +then + export LANG=C +fi + +if [ "${ARCHITECTURE}" == "arm" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="--with-jobs=4 --with-memory-size=2000" +fi + +# Skip Building Freetype for certain platforms +if [ "${ARCHITECTURE}" == "aarch64" ] || [ "${ARCHITECTURE}" == "ppc64le" ] +then + export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" +fi + +if [ "${ARCHITECTURE}" == "s390x" ] || [ "${ARCHITECTURE}" == "ppc64le" ] +then + if [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] && [ "${VARIANT}" == "openj9" ] + then + if [ -z "$JDK9_BOOT_DIR" ]; then + export JDK9_BOOT_DIR="$PWD/jdk-9+181" + if [ ! -r "$JDK9_BOOT_DIR" ]; then + wget -O - https://github.com/AdoptOpenJDK/openjdk9-releases/releases/download/jdk-9%2B181/OpenJDK9_s390x_Linux_jdk-9.181.tar.gz | tar xpfz - + fi + fi + + export JDK_BOOT_DIR=$JDK9_BOOT_DIR + fi +fi + +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] +then + export JDK10_BOOT_DIR="$PWD/jdk-10" + if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then + downloadArch="${ARCHITECTURE}" + [ "$downloadArch" == "arm" ] && downloadArch="arm32" + + mkdir -p "$JDK10_BOOT_DIR" + wget -q -O - "https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=linux&release=latest&arch=${downloadArch}" | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" + fi + export JDK_BOOT_DIR=$JDK10_BOOT_DIR +fi +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] || [ "${VARIANT}" == "openj9" ]; then + # If we have the RedHat devtoolset 7 installed, use gcc 7 from there, else /usr/local/gcc/bin + if [ -r /opt/rh/devtoolset-7/root/usr/bin ]; then + export PATH=/opt/rh/devtoolset-7/root/usr/bin:$PATH + [ -r /opt/rh/devtoolset-7/root/usr/bin/gcc ] && export CC=/opt/rh/devtoolset-7/root/usr/bin/gcc + [ -r /opt/rh/devtoolset-7/root/usr/bin/g++ ] && export CXX=/opt/rh/devtoolset-7/root/usr/bin/g++ + elif [ -r /usr/local/gcc/bin ]; then + export PATH=/usr/local/gcc/bin:$PATH + [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 + [ -r /usr/local/gcc/bin/g++-7.3 ] && export CXX=/usr/local/gcc/bin/g++-7.3 + export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib + fi +fi diff --git a/build-farm/platform-specific-configurations/mac.sh b/build-farm/platform-specific-configurations/mac.sh new file mode 100755 index 000000000..808e951e1 --- /dev/null +++ b/build-farm/platform-specific-configurations/mac.sh @@ -0,0 +1,49 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +export MACOSX_DEPLOYMENT_TARGET=10.8 +export BUILD_ARGS="${BUILD_ARGS}" + +XCODE_SWITCH_PATH="/"; + +if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] +then + XCODE_SWITCH_PATH="/Applications/Xcode.app" +fi +sudo xcode-select --switch "${XCODE_SWITCH_PATH}" + + +if [ "${JAVA_TO_BUILD}" != "${JDK8_VERSION}" ] +then + export PATH="/Users/jenkins/ccache-3.2.4:$PATH" +fi + + +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-extra-cxxflags=-mmacosx-version-min=10.8" + + export JDK10_BOOT_DIR="$PWD/jdk-10" + if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then + mkdir -p "$JDK10_BOOT_DIR" + wget -q -O - 'https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=mac&release=latest' | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" + fi + export JDK_BOOT_DIR=$JDK10_BOOT_DIR +fi \ No newline at end of file diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh new file mode 100755 index 000000000..0524afe60 --- /dev/null +++ b/build-farm/platform-specific-configurations/windows.sh @@ -0,0 +1,110 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +export ANT_HOME=/cygdrive/C/Projects/OpenJDK/apache-ant-1.10.1 +export ALLOW_DOWNLOADS=true +export LANG=C +export JAVA_HOME=$JDK_BOOT_DIR +export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" + +if [ "${ARCHITECTURE}" == "x86-32" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache --with-target-bits=32 --target=x86" + + if [ "${VARIANT}" == "hotspot" ] + then + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2010 --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" + fi + fi + + if [ "${VARIANT}" == "openj9" ] + then + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + + # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" + export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + + # Next line a potentially tactical fix for https://github.com/AdoptOpenJDK/openjdk-build/issues/267 + export PATH="/usr/bin:$PATH" + fi + fi +fi + + +if [ "${ARCHITECTURE}" == "x64" ] +then + if [ "${VARIANT}" == "hotspot" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + fi + fi + + if [ "${VARIANT}" == "openj9" ] + then + export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" + export HAS_AUTOCONF=1 + export BUILD_ARGS="${BUILD_ARGS} --freetype-version 2.5.3" + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" + export PATH="$PATH:/c/cygwin64/bin" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --with-toolchain-version=2017" + fi + fi +fi diff --git a/build-farm/set-platform-specific-configurations.sh b/build-farm/set-platform-specific-configurations.sh new file mode 100755 index 000000000..beb62164d --- /dev/null +++ b/build-farm/set-platform-specific-configurations.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../sbin/common/constants.sh" + +if [ "${JAVA_TO_BUILD}" != "${JDK8_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-warnings-as-errors" +fi + +if [ "${VARIANT}" != "hotspot" ] +then + export VARIANT_ARG="--build-variant ${VARIANT}" +fi + +# shellcheck disable=SC1091,SC1090 +source "$SCRIPT_DIR/platform-specific-configurations/${OPERATING_SYSTEM}.sh" \ No newline at end of file diff --git a/build-farm/sign-releases.sh b/build-farm/sign-releases.sh new file mode 100755 index 000000000..8ea2ab5e8 --- /dev/null +++ b/build-farm/sign-releases.sh @@ -0,0 +1,43 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +BUILD_ARGS=${BUILD_ARGS:-""} +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +export OPERATING_SYSTEM + +if [ "${OPERATING_SYSTEM}" == "mac" ] ; then + EXTENSION="tar.gz" +elif [ "${OPERATING_SYSTEM}" == "windows" ] ; then + EXTENSION="zip" +else + echo "OS does not need signing ${OPERATING_SYSTEM}" + exit 0 +fi + +echo "files:" +ls -alh workspace/target/ + +echo "OpenJDK*.${EXTENSION}" + +find workspace/target/ -name "OpenJDK*.${EXTENSION}" | while read -r file; +do + echo "signing ${file}" + + # shellcheck disable=SC2086 + bash "${SCRIPT_DIR}/../sign.sh" ${CERTIFICATE} "${file}" +done \ No newline at end of file diff --git a/configureBuild.sh b/configureBuild.sh new file mode 100755 index 000000000..84f492c0c --- /dev/null +++ b/configureBuild.sh @@ -0,0 +1,248 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +################################################################################ + +################################################################################ +# +# This script sets up the initial configuration for an (Adopt) OpenJDK Build. +# See the configure_build function and its child functions for details. +# It's sourced by the makejdk-any-platform.sh script. +# +################################################################################ + +set -eu + +# i.e. Where we are +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/sbin/common/constants.sh" + +# shellcheck source=sbin/common/common.sh +source "$SCRIPT_DIR/sbin/common/common.sh" + +# Bring in the source signal handler +sourceSignalHandler() +{ + #shellcheck source=signalhandler.sh + source "$SCRIPT_DIR/signalhandler.sh" +} + +# Parse the command line arguments +parseCommandLineArgs() +{ + # Defer most of the work to the shared function in common-functions.sh + parseConfigurationArguments "$@" + + # this check is to maintain backwards compatibility and allow user to use + # -v rather than the mandatory argument + if [[ "${BUILD_CONFIG[OPENJDK_FOREST_NAME]}" == "" ]] + then + if [[ $# -eq 0 ]] + then + echo "Please provide a java version to build as an argument" + exit 1 + fi + + while [[ $# -gt 1 ]] ; do + shift; + done + + # Now that we've processed the flags, grab the mandatory argument(s) + setOpenJdkVersion "$1" + fi +} + +# Extra config for OpenJDK variants such as OpenJ9, SAP et al +# shellcheck disable=SC2153 +doAnyBuildVariantOverrides() +{ + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]]; then + # current location of Extensions for OpenJDK9 for OpenJ9 project + local repository="ibmruntimes/openj9-openjdk-${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" + fi + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "SapMachine" ]]; then + # current location of SAP variant + local repository="SAP/SapMachine" + # sapmachine10 is the current branch for OpenJDK10 mainline + # (equivalent to jdk/jdk10 on hotspot) + local branch="sapmachine10" + fi + + BUILD_CONFIG[REPOSITORY]=${repository:-${BUILD_CONFIG[REPOSITORY]}}; + BUILD_CONFIG[BRANCH]=${branch:-${BUILD_CONFIG[BRANCH]}}; +} + +# Set the working directory for this build +setWorkingDirectory() +{ + if [ -z "${BUILD_CONFIG[WORKSPACE_DIR]}" ] ; then + if [[ "${BUILD_CONFIG[USE_DOCKER]}" == "true" ]]; + then + BUILD_CONFIG[WORKSPACE_DIR]="/openjdk/"; + else + BUILD_CONFIG[WORKSPACE_DIR]="$PWD/workspace"; + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}" || exit + fi + else + echo "Workspace dir is ${BUILD_CONFIG[WORKSPACE_DIR]}" + fi + + echo "Working dir is ${BUILD_CONFIG[WORKING_DIR]}" +} + +# shellcheck disable=SC2153 +determineBuildProperties() { + local build_type=normal + local default_build_full_name=${BUILD_CONFIG[OS_KERNEL_NAME]}-${BUILD_CONFIG[OS_ARCHITECTURE]}-${build_type}-${BUILD_CONFIG[JVM_VARIANT]}-release + + BUILD_CONFIG[BUILD_FULL_NAME]=${BUILD_CONFIG[BUILD_FULL_NAME]:-"$default_build_full_name"} +} + +# Set variables that the `configure` command (which builds OpenJDK) will need +# shellcheck disable=SC2153 +setVariablesForConfigure() { + + local openjdk_core_version=${BUILD_CONFIG[OPENJDK_CORE_VERSION]} + + # TODO Regex this in the if or use cut to grab out the number and see if >= 9 + # TODO 9 should become 9u as will 10 shortly.... + if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDKHEAD_CORE_VERSION}" ]; then + local jdk_path="jdk" + local jre_path="jre" + #BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-"--disable-warnings-as-errors"} + elif [ "$openjdk_core_version" == "${JDK8_CORE_VERSION}" ]; then + local jdk_path="j2sdk-image" + local jre_path="j2re-image" + else + echo "Please specify a version, either jdk8u, jdk9, jdk10, amber etc, with or without a 'u' suffix. e.g. $0 [options] jdk8u" + exit 1 + fi + + BUILD_CONFIG[JDK_PATH]=$jdk_path + BUILD_CONFIG[JRE_PATH]=$jre_path +} + +# Set the repository to build from +setRepository() { + local repository="${BUILD_CONFIG[REPOSITORY]:-adoptopenjdk/openjdk-${BUILD_CONFIG[OPENJDK_FOREST_NAME]}}"; + repository="$(echo "${repository}" | awk '{print tolower($0)}')"; + + BUILD_CONFIG[REPOSITORY]=$repository; +} + +# Specific platforms need to have special build settings +processArgumentsforSpecificPlatforms() { + + case "${BUILD_CONFIG[OS_KERNEL_NAME]}" in + "darwin") + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] ; then + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="false" + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="true" + fi + ;; + esac + +} + +# Specific architectures need to have special build settings +# shellcheck disable=SC2153 +processArgumentsforSpecificArchitectures() { + local jvm_variant=server + local build_full_name="" + local make_args_for_any_platform="" + local configure_args_for_any_platform="" + + case "${BUILD_CONFIG[OS_ARCHITECTURE]}" in + "s390x") + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] && [ "${BUILD_CONFIG[BUILD_VARIANT]}" != "openj9" ]; then + jvm_variant=zero + else + jvm_variant=server + fi + + build_full_name=linux-s390x-normal-${jvm_variant}-release + make_args_for_any_platform="CONF=${build_full_name} DEBUG_BINARIES=true images" + ;; + + "ppc64le") + jvm_variant=server + build_full_name=linux-ppc64-normal-${jvm_variant}-release + + if [ "$(command -v rpm)" ]; then + # shellcheck disable=SC1083 + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]=${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]:="--build=$(rpm --eval %{_host})"} + fi + + ;; + + "armv7l") + jvm_variant=zero + make_args_for_any_platform="DEBUG_BINARIES=true images" + configure_args_for_any_platform="--with-jobs=${NUM_PROCESSORS}" + ;; + + "aarch64") + BUILD_CONFIG[FREETYPE_FONT_VERSION]="2.5.2" + ;; + esac + + BUILD_CONFIG[JVM_VARIANT]=${BUILD_CONFIG[JVM_VARIANT]:-$jvm_variant} + BUILD_CONFIG[BUILD_FULL_NAME]=${BUILD_CONFIG[BUILD_FULL_NAME]:-$build_full_name} + BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]:-$make_args_for_any_platform} + BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-$configure_args_for_any_platform} +} + +# Different platforms have different default make commands +# shellcheck disable=SC2153 +setMakeCommandForOS() { + local make_command_name + case "$OS_KERNEL_NAME" in + "aix") + make_command_name="gmake" + ;; + "SunOS") + make_command_name="gmake" + ;; + esac + + BUILD_CONFIG[MAKE_COMMAND_NAME]=${BUILD_CONFIG[MAKE_COMMAND_NAME]:-$make_command_name} +} + +################################################################################ + +configure_build() { + configDefaults + + # Parse the CL Args, see ${SCRIPT_DIR}/configureBuild.sh for details + parseCommandLineArgs "$@" + + # Update the configuration with the arguments passed in, the platform etc + setVariablesForConfigure + setRepository + processArgumentsforSpecificPlatforms + processArgumentsforSpecificArchitectures + setMakeCommandForOS + + determineBuildProperties + sourceSignalHandler + doAnyBuildVariantOverrides + setWorkingDirectory +} diff --git a/docker-build.sh b/docker-build.sh new file mode 100755 index 000000000..6612c323a --- /dev/null +++ b/docker-build.sh @@ -0,0 +1,139 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# This script deals with the configuration to build (Adopt) OpenJDK in a docker +# container. +# It's sourced by the makejdk-any-platform.sh script. +# +################################################################################ + +set -eu + +# Create a data volume called ${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}, +# this gets mounted at /openjdk/build inside the container and is persistent +# between builds/tests unless -c is passed to this script, in which case it is +# recreated using the source in the current ./openjdk directory on the host +# machine (outside the container) +createPersistentDockerDataVolume() +{ + set +e + ${BUILD_CONFIG[DOCKER]} volume inspect "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}" > /dev/null 2>&1 + local data_volume_exists=$? + set -e + + if [[ "${BUILD_CONFIG[CLEAN_DOCKER_BUILD]}" == "true" || "$data_volume_exists" != "0" ]]; then + + # shellcheck disable=SC2154 + echo "Removing old volumes and containers" + ${BUILD_CONFIG[DOCKER]} rm -f "$(${BUILD_CONFIG[DOCKER]} ps -a --no-trunc | grep "${BUILD_CONFIG[CONTAINER_NAME]}" | cut -d' ' -f1)" || true + ${BUILD_CONFIG[DOCKER]} volume rm -f "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}" || true + + # shellcheck disable=SC2154 + echo "Creating tmp container" + ${BUILD_CONFIG[DOCKER]} volume create --name "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}" + fi +} + +# Build the docker container +buildDockerContainer() +{ + echo "Building docker container" + + local dockerFile="${BUILD_CONFIG[DOCKER_FILE_PATH]}/Dockerfile" + + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" != "" && -f "${BUILD_CONFIG[DOCKER_FILE_PATH]}/Dockerfile-${BUILD_CONFIG[BUILD_VARIANT]}" ]]; then + # TODO dont modify config in build + BUILD_CONFIG[CONTAINER_NAME]="${BUILD_CONFIG[CONTAINER_NAME]}-${BUILD_CONFIG[BUILD_VARIANT]}" + echo "Building DockerFile variant ${BUILD_CONFIG[BUILD_VARIANT]}" + dockerFile="${BUILD_CONFIG[DOCKER_FILE_PATH]}/Dockerfile-${BUILD_CONFIG[BUILD_VARIANT]}" + fi + + writeConfigToFile + + ${BUILD_CONFIG[DOCKER]} build -t "${BUILD_CONFIG[CONTAINER_NAME]}" -f "${dockerFile}" . --build-arg "OPENJDK_CORE_VERSION=${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" +} + +# Execute the (Adopt) OpenJDK build inside the Docker Container +buildOpenJDKViaDocker() +{ + + # TODO This could be extracted overridden by the user if we support more + # architectures going forwards + local container_architecture="x86_64/ubuntu" + + BUILD_CONFIG[DOCKER_FILE_PATH]="docker/${BUILD_CONFIG[OPENJDK_CORE_VERSION]}/$container_architecture" + + # shellcheck disable=SC1090 + source "${BUILD_CONFIG[DOCKER_FILE_PATH]}/dockerConfiguration.sh" + + if [ -z "$(command -v docker)" ]; then + # shellcheck disable=SC2154 + echo "Error, please install docker and ensure that it is in your path and running!" + exit + fi + + echo "Using Docker to build the JDK" + + createPersistentDockerDataVolume + + # If keep is true then use the existing container (or build a new one if we + # can't find it) + if [[ "${BUILD_CONFIG[REUSE_CONTAINER]}" == "true" ]] ; then + # shellcheck disable=SC2086 + # If we can't find the previous Docker container then build a new one + if [ "$(${BUILD_CONFIG[DOCKER]} ps -a | grep -c \"${BUILD_CONFIG[CONTAINER_NAME]}\")" == 0 ]; then + echo "No docker container for reuse was found, so creating '${BUILD_CONFIG[CONTAINER_NAME]}' " + buildDockerContainer + fi + else + # shellcheck disable=SC2154 + echo "Since you specified --ignore-container, we are removing the existing container (if it exists) and building you a new one{$good}" + # Find the previous Docker container and remove it (if it exists) + ${BUILD_CONFIG[DOCKER]} ps -a | awk '{ print $1,$2 }' | grep "${BUILD_CONFIG[CONTAINER_NAME]}" | awk '{print $1 }' | xargs -I {} "${BUILD_CONFIG[DOCKER]}" rm -f {} + + # Build a new container + buildDockerContainer + fi + + # Show the user all of the config before we build + displayParams + + local hostDir; + hostDir="$(pwd)"; + + echo "Target binary directory on host machine: ${hostDir}/target" + mkdir -p "${hostDir}/workspace/target" + + local cpuSet; + cpuSet="0-$((BUILD_CONFIG[NUM_PROCESSORS] - 1))" + + # shellcheck disable=SC2140 + # Pass in the last important variables into the Docker container and call + # the /openjdk/sbin/build.sh script inside + ${BUILD_CONFIG[DOCKER]} run -lst \ + --cpuset-cpus="${cpuSet}" \ + -v "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}:/openjdk/build" \ + -v "${hostDir}/workspace/target":"/${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}" \ + -e BUILD_VARIANT="${BUILD_CONFIG[BUILD_VARIANT]}" \ + --entrypoint /openjdk/sbin/build.sh "${BUILD_CONFIG[CONTAINER_NAME]}" + + # If we didn't specify to keep the container then remove it + if [[ -z ${BUILD_CONFIG[KEEP_CONTAINER]} ]] ; then + ${BUILD_CONFIG[DOCKER]} ps -a | awk '{ print $1,$2 }' | grep "${BUILD_CONFIG[CONTAINER_NAME]}" | awk '{print $1 }' | xargs -I {} "${BUILD_CONFIG[DOCKER]}" rm {} + fi +} \ No newline at end of file diff --git a/docker/README.md b/docker/README.md index 15ec7e29f..dba7a0770 100644 --- a/docker/README.md +++ b/docker/README.md @@ -5,7 +5,7 @@ Dockerfiles and build scripts for generating various Docker Images building Open # License The Dockerfiles and associated scripts found in this project are licensed under -the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0.html). +the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html). # Steps to build diff --git a/docker/jdk10/x86_64/ubuntu/.gitignore b/docker/jdk10/x86_64/ubuntu/.gitignore deleted file mode 100644 index c97f963b3..000000000 --- a/docker/jdk10/x86_64/ubuntu/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.sh diff --git a/docker/jdk10/x86_64/ubuntu/Dockerfile b/docker/jdk10/x86_64/ubuntu/Dockerfile index bb3b41850..d9c7f72e0 100644 --- a/docker/jdk10/x86_64/ubuntu/Dockerfile +++ b/docker/jdk10/x86_64/ubuntu/Dockerfile @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -23,7 +23,9 @@ RUN apt-get update \ && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0x219BD9C9 \ && add-apt-repository 'deb http://repos.azulsystems.com/ubuntu stable main' \ && apt-get update \ + && apt-get -y upgrade \ && apt-get install -qq -y --no-install-recommends \ + git \ cpio \ make \ gcc \ @@ -51,18 +53,20 @@ RUN apt-get update \ # Pick up build instructions RUN mkdir -p /openjdk/target -ADD sbin /openjdk/sbin +COPY sbin /openjdk/sbin +COPY workspace/config /openjdk/config +RUN mkdir -p /openjdk/build +RUN useradd -ms /bin/bash build +RUN chown -R build: /openjdk/ +USER build WORKDIR /openjdk/build/ -ENV JDK_BOOT_DIR=/usr/lib/jvm/zulu-9-amd64/ - # Default actions ENTRYPOINT ["/openjdk/sbin/build.sh"] CMD ["images"] - ARG OPENJDK_VERSION ENV OPENJDK_VERSION=$OPENJDK_VERSION ENV JDK_PATH=jdk diff --git a/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 b/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 index 28f42da50..1e4b7997b 100644 --- a/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 +++ b/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh b/docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh new file mode 100644 index 000000000..947c956b2 --- /dev/null +++ b/docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh @@ -0,0 +1,8 @@ +#!/bin/bash +# shellcheck disable=SC2034 +# Disable for whole file + +# This config is read in by configureBuild +BUILD_CONFIG[OS_KERNEL_NAME]="linux" +BUILD_CONFIG[OS_ARCHITECTURE]="x86_64" +BUILD_CONFIG[BUILD_FULL_NAME]="linux-x86_64-normal-server-release" diff --git a/docker/jdk8/x86_64/ubuntu/.gitignore b/docker/jdk8/x86_64/ubuntu/.gitignore deleted file mode 100644 index c97f963b3..000000000 --- a/docker/jdk8/x86_64/ubuntu/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.sh diff --git a/docker/jdk8/x86_64/ubuntu/Dockerfile b/docker/jdk8/x86_64/ubuntu/Dockerfile index 8e29c929c..484721439 100644 --- a/docker/jdk8/x86_64/ubuntu/Dockerfile +++ b/docker/jdk8/x86_64/ubuntu/Dockerfile @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -12,17 +12,27 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM ubuntu:14.04 +FROM ubuntu:16.04 MAINTAINER AdoptOpenJDK -# Install required OS tools +# Install required OS tools. Yes we have to apt-get update first in order to +# get to software-properties-common, which includes the apt-add-repository +# package, so we can add the Azul repo and then apt-get update that so we cam +# get zulu-7. This is why we can't have nice things. RUN apt-get update \ + && apt-get install -y software-properties-common \ + && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0x219BD9C9 \ + && apt-add-repository 'deb http://repos.azulsystems.com/ubuntu stable main' \ + && apt-get update \ + && apt-get -y upgrade \ && apt-get install -qq -y --no-install-recommends \ + git \ cpio \ make \ gcc \ g++ \ + file \ libx11-dev \ libxext-dev \ libxrender-dev \ @@ -31,24 +41,26 @@ RUN apt-get update \ libcups2-dev \ libfreetype6-dev \ libasound2-dev \ - openjdk-7-jdk \ + zulu-7 \ ccache \ zip \ wget \ git \ unzip \ - realpath \ && rm -rf /var/lib/apt/lists/* # Pick up build instructions RUN mkdir -p /openjdk/target -ADD sbin /openjdk/sbin +COPY sbin /openjdk/sbin +COPY workspace/config /openjdk/config +RUN mkdir -p /openjdk/build +RUN useradd -ms /bin/bash build +RUN chown -R build: /openjdk/ +USER build WORKDIR /openjdk/build/ -ENV JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-amd64 - # Default actions ENTRYPOINT ["/openjdk/sbin/build.sh"] diff --git a/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 b/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 index 831ea7367..013562822 100644 --- a/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 +++ b/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh b/docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh new file mode 100644 index 000000000..947c956b2 --- /dev/null +++ b/docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh @@ -0,0 +1,8 @@ +#!/bin/bash +# shellcheck disable=SC2034 +# Disable for whole file + +# This config is read in by configureBuild +BUILD_CONFIG[OS_KERNEL_NAME]="linux" +BUILD_CONFIG[OS_ARCHITECTURE]="x86_64" +BUILD_CONFIG[BUILD_FULL_NAME]="linux-x86_64-normal-server-release" diff --git a/docker/jdk9/x86_64/ubuntu/.gitignore b/docker/jdk9/x86_64/ubuntu/.gitignore deleted file mode 100644 index c97f963b3..000000000 --- a/docker/jdk9/x86_64/ubuntu/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.sh diff --git a/docker/jdk9/x86_64/ubuntu/Dockerfile b/docker/jdk9/x86_64/ubuntu/Dockerfile index e8ba11a8e..5789a1dea 100644 --- a/docker/jdk9/x86_64/ubuntu/Dockerfile +++ b/docker/jdk9/x86_64/ubuntu/Dockerfile @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -17,7 +17,9 @@ MAINTAINER AdoptOpenJDK # Install required OS tools RUN apt-get update \ + && apt-get -y upgrade \ && apt-get install -qq -y --no-install-recommends \ + git \ cpio \ make \ gcc \ @@ -31,6 +33,7 @@ RUN apt-get update \ libelf-dev \ libcups2-dev \ libfreetype6-dev \ + libfontconfig1-dev \ libasound2-dev \ openjdk-8-jdk \ ccache \ @@ -44,18 +47,20 @@ RUN apt-get update \ # Pick up build instructions RUN mkdir -p /openjdk/target -ADD sbin /openjdk/sbin +COPY sbin /openjdk/sbin +COPY workspace/config /openjdk/config +RUN mkdir -p /openjdk/build +RUN useradd -ms /bin/bash build +RUN chown -R build: /openjdk/ +USER build WORKDIR /openjdk/build/ -ENV JDK_BOOT_DIR=/usr/lib/jvm/java-1.8.0-openjdk-amd64 - # Default actions ENTRYPOINT ["/openjdk/sbin/build.sh"] CMD ["images"] - ARG OPENJDK_CORE_VERSION ENV OPENJDK_CORE_VERSION=$OPENJDK_CORE_VERSION ENV JDK_PATH=jdk diff --git a/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 b/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 index 831ea7367..013562822 100644 --- a/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 +++ b/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh b/docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh new file mode 100644 index 000000000..1d346759e --- /dev/null +++ b/docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh @@ -0,0 +1,9 @@ +#!/bin/bash +# shellcheck disable=SC2034 +# Disable for whole file + +# This config is read in by configureBuild +BUILD_CONFIG[OS_KERNEL_NAME]="linux" +BUILD_CONFIG[OS_ARCHITECTURE]="x86_64" +BUILD_CONFIG[BUILD_FULL_NAME]="linux-x86_64-normal-server-release" + diff --git a/docs/build.md b/docs/build.md new file mode 100644 index 000000000..53036ebea --- /dev/null +++ b/docs/build.md @@ -0,0 +1,63 @@ + + +# Build + + +The build is organised as follows: + +## Top level pipeline jobs + +These live at https://ci.adoptopenjdk.net/job/build-scripts/. Currently these jobs are: + +* [openjdk8-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk8-pipeline/) +* [openjdk9-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk9-pipeline/) +* [openjdk10-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk10-pipeline/) +* [openjdk11-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/) + +** From here on in this documentation describes the 10 pipeline however it is the same for all Java versions. ** + +The high level sequence of each of these pipelines is shown in: + +![build sequence](images/sequence.svg "Build Sequence") + +This shows a build of `jdk10-mac-x64-hotspot` and `jdk10-linux-x64-hotspot`. The `jdk10-mac-x64-hotspot` build is expanded as +an example of how each build pipeline works, however a similar pipeline will be generated for every `jdk10---` +that was requested to be built. + +### Top level pipeline code + +The actions of the pipeline are defined by [openjdk10_pipeline.groovy](/pipelines/build/openjdk10_pipeline.groovy), which +calls [build_base_file.groovy](/pipelines/build/build_base_file.groovy). + +The actions of this script are: + +1. Read in build configuration that determines what configurations have been requested to be built. +1. For each requested configuration generate a configuration that defines such things as which nodes to run the build on, optional configuration added to [openjdk10_pipeline.groovy](/pipelines/build/openjdk10_pipeline.groovy) etc. +1. Generate a build job for each requested configuration, these live at [jdk10u](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/) +1. Execute all build jobs in parallel +1. Publish binaries to GitHub + + +#### Generated build jobs + +These are generated using the template at [create_job_from_template.groovy](/pipelines/build/create_job_from_template.groovy) +This defines a pipeline job that calls [openjdk_build_pipeline.groovy](/pipelines/build/openjdk_build_pipeline.groovy), this script defines the work of what a Build Pipeline does. + +## Build Pipelines + +Generated by [create_job_from_template.groovy](/pipelines/build/create_job_from_template.groovy), and its work defined in [openjdk_build_pipeline.groovy](/pipelines/build/openjdk_build_pipeline.groovy). + +The actions of this are: + +1. Execute build +1. Spawn requested test pipelines, these are defined in [openjdk10_pipeline.groovy](/pipelines/build/openjdk10_pipeline.groovy) +1. Call sign binaries job if needed +1. Archive file + +### Execute build + +This is kicked off by the Build Pipeline, this involves calling [make-adopt-build-farm.sh](/build-farm/make-adopt-build-farm.sh). +This script defines some platform specific configuration such as operating system and architecture specific config defined in [platform-specific-configurations](/build-farm/platform-specific-configurations). +Then it calls [make-adopt-build-farm.sh](/build-farm/make-adopt-build-farm.sh) to finally invoke the build. + + diff --git a/docs/generateBuildMatrix.sh b/docs/generateBuildMatrix.sh new file mode 100755 index 000000000..53109d41a --- /dev/null +++ b/docs/generateBuildMatrix.sh @@ -0,0 +1,30 @@ +#!/bin/bash + +# Generates markdown table of build job status + +echo "| Platform | Java 8 | Java 9 | Java 10 | Java 11 |" +echo "| ------------------------- | ------ | ------ | ------- | ------- |" + +rm "/tmp/build.txt" +for i in "8u" "9u" "10u" "11"; +do + curl -s "https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk${i}/" | egrep -o "job/jdk${i}u?-[^\/]+" >> "/tmp/build.txt" +done + +cat "/tmp/build.txt" | cut -d'/' -f2 | sed -r 's/jdk[0-9]+u?\-//g' | sort | uniq | while read buildName; +do + # buildName should be of the form: aix-ppc64-hotspot + echo -n "| ${buildName} | " + for i in "8u" "9u" "10u" "11"; + do + code=$(curl -s -o /dev/null -w "%{http_code}" "https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk${i}/jdk${i}-${buildName}") + if [ $code = 200 ]; then + echo -n "[![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk${i}/jdk${i}-${buildName})](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk${i}/job/jdk${i}-${buildName})" + else + echo -n "N/A" + fi + + echo -n " | " + done + echo "" +done \ No newline at end of file diff --git a/docs/images/AdoptOpenJDK_Build_Script_Relationships.png b/docs/images/AdoptOpenJDK_Build_Script_Relationships.png new file mode 100644 index 000000000..1c84bbe8b Binary files /dev/null and b/docs/images/AdoptOpenJDK_Build_Script_Relationships.png differ diff --git a/docs/images/sequence.dia b/docs/images/sequence.dia new file mode 100644 index 000000000..d3be5bd56 --- /dev/null +++ b/docs/images/sequence.dia @@ -0,0 +1,2170 @@ + + + + + + + + + + + + + #A4# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #OpenJDK10u +Build Job# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #Start Build# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #jdk10u-mac-x64-hotspot +Build Job# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + #Generate Job# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + #Execute Build Pipeline# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #Build# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #openjdktest# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #systemtest# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #sign# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #Publish +to github# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + #jdk10-mac-x64-hotspot +Pipeline Job# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #linux-x64-hotspot +Pipeline Job# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + #Generate and Execute# + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/sequence.svg b/docs/images/sequence.svg new file mode 100644 index 000000000..1890b7423 --- /dev/null +++ b/docs/images/sequence.svg @@ -0,0 +1,256 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + OpenJDK10u + Build Job + + + + + + Start Build + + + + + + jdk10u-mac-x64-hotspot + Build Job + + + + + + + + + + + Generate Job + + + + + + + + + + + + + + Execute Build Pipeline + + + + + + Build + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + openjdktest + + + + + + + + + + + + + + + + + + + + + + + + + + + systemtest + + + + + + + + + + + + + + + + + + + + + + + + + + + sign + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Publish + to github + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + jdk10-mac-x64-hotspot + Pipeline Job + + + linux-x64-hotspot + Pipeline Job + + + + + + + + + + + + + + + + + + + Generate and Execute + + diff --git a/git-hg/add-branch-without-modules.sh b/git-hg/add-branch-without-modules.sh index 2841b4e56..b8fe3340b 100755 --- a/git-hg/add-branch-without-modules.sh +++ b/git-hg/add-branch-without-modules.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ echo "Create $bpath" mkdir -p "$bpath" || exit 1 echo "git hg clone $bpath (root)" -git hg clone "http://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 +git hg clone "https://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 echo "checkout the $branch" if [ "$branch" != "" ]; then diff --git a/git-hg/add-branch.sh b/git-hg/add-branch.sh index 053efd82f..1e3a39cfc 100755 --- a/git-hg/add-branch.sh +++ b/git-hg/add-branch.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -31,13 +31,13 @@ echo "Create $bpath" mkdir -p "$bpath" || exit 1 echo "Clone $bpath (root)" -git hg clone "http://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 +git hg clone "https://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 # shellcheck disable=SC2154 for module in "${modules[@]}" do echo "Clone $bpath -> $module" - git hg clone "http://hg.openjdk.java.net/$bpath/$module" "$bpath/$module" || exit 1 + git hg clone "https://hg.openjdk.java.net/$bpath/$module" "$bpath/$module" || exit 1 done echo "Exit hg" diff --git a/git-hg/diff-without-getsource.sh b/git-hg/diff-without-getsource.sh index 882b1fdde..bdbb03ed7 100755 --- a/git-hg/diff-without-getsource.sh +++ b/git-hg/diff-without-getsource.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ echo "git repo version: ${git_repo_version}" echo "hg repo version: ${hg_root_forest}/${hg_repo_version}" git clone -b master "https://github.com/AdoptOpenJDK/openjdk-${git_repo_version}.git" openjdk-git || exit 1 -hg clone "http://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 +hg clone "https://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 diffNum=$(diff -rq openjdk-git openjdk-hg -x '.git' -x '.hg' -x '.hgtags' | wc -l) diff --git a/git-hg/diff.sh b/git-hg/diff.sh index af198691c..85be9c2bf 100755 --- a/git-hg/diff.sh +++ b/git-hg/diff.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ echo "git repo version: ${git_repo_version}" echo "hg repo version: ${hg_root_forest}/${hg_repo_version}" git clone -b master "https://github.com/AdoptOpenJDK/openjdk-${git_repo_version}.git" openjdk-git || exit 1 -hg clone "http://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 +hg clone "https://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 cd openjdk-hg || exit 1 bash get_source.sh diff --git a/git-hg/import-common.sh b/git-hg/import-common.sh index bb05f0611..0cf3c26d6 100755 --- a/git-hg/import-common.sh +++ b/git-hg/import-common.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/mirrorMercurialIntoGit.sh b/git-hg/mirrorMercurialIntoGit.sh index 91220990c..cafa03bb4 100755 --- a/git-hg/mirrorMercurialIntoGit.sh +++ b/git-hg/mirrorMercurialIntoGit.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ echo "You are in $WORKSPACE" function checkArgs() { if [ "$1" -lt 1 ]; then echo Usage: "$0" '[jdk-updates/jdk10u|jdk/jdk] (branch)' - echo "Hg Repo supplied should match a repository in http://hg.openjdk.java.net/" + echo "Hg Repo supplied should match a repository in https://hg.openjdk.java.net/" echo "For example, to get the latest jdk development repo:" echo "$0 jdk/jdk" echo "For example, to get the raw-string-literals branch from the amber repo:" @@ -72,8 +72,8 @@ function addMercurialUpstream() { # shellcheck disable=SC2143 if [ -z "$(git remote -v | grep 'hg')" ] ; then - echo "Initial setup of hg::http://hg.openjdk.java.net/$HG_REPO" - git remote add hg hg::http://hg.openjdk.java.net/"$HG_REPO" + echo "Initial setup of hg::https://hg.openjdk.java.net/$HG_REPO" + git remote add hg hg::https://hg.openjdk.java.net/"$HG_REPO" fi } diff --git a/git-hg/setup.sh b/git-hg/setup.sh index c133aa725..9fa578118 100755 --- a/git-hg/setup.sh +++ b/git-hg/setup.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/setup8.sh b/git-hg/setup8.sh index 7d5f08e80..c024cb5ec 100755 --- a/git-hg/setup8.sh +++ b/git-hg/setup8.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/setup9.sh b/git-hg/setup9.sh index 068a7e96e..25ea0c406 100755 --- a/git-hg/setup9.sh +++ b/git-hg/setup9.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/setupAmber.sh b/git-hg/setupAmber.sh index 50207133b..88270e20a 100644 --- a/git-hg/setupAmber.sh +++ b/git-hg/setupAmber.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/update-without-modules.sh b/git-hg/update-without-modules.sh index d335540e1..805e90da2 100755 --- a/git-hg/update-without-modules.sh +++ b/git-hg/update-without-modules.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -32,8 +32,8 @@ do pushd "$hg_root_forest/$hg_repo/root" echo "Update $hg_root_forest/$hg_repo -> (root)" - git hg fetch "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" - git hg pull "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg fetch "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg pull "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" popd echo "Exit hg" diff --git a/git-hg/update.sh b/git-hg/update.sh index a3c24b282..8e0000bb3 100755 --- a/git-hg/update.sh +++ b/git-hg/update.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -33,8 +33,8 @@ do pushd "$hg_root_forest/$hg_repo/root" echo "Update $hg_root_forest/$hg_repo -> (root)" - git hg fetch "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" - git hg pull "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg fetch "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg pull "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" popd # shellcheck disable=SC2154 @@ -42,8 +42,8 @@ do do pushd "$hg_root_forest/$hg_repo/$module" echo "Update $hg_root_forest/$hg_repo -> $module" - git hg fetch "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" - git hg pull "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" + git hg fetch "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" + git hg pull "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" popd done diff --git a/git-hg/updateforesttags.sh b/git-hg/updateforesttags.sh index 1e4daded4..1523144c7 100755 --- a/git-hg/updateforesttags.sh +++ b/git-hg/updateforesttags.sh @@ -15,7 +15,7 @@ # accompanied this code). # # You should have received a copy of the GNU General Public License version -# 2 along with this work; if not, see . +# 2 along with this work; if not, see . # # =========================================================================== @@ -36,9 +36,9 @@ GITHUB_PROJECT=git@github.com:AdoptOpenJDK GITHUB_REPO="openjdk-$OPENJDK_VERSION" case "$OPENJDK_VERSION" in - jdk8*) HG_REPO=http://hg.openjdk.java.net/jdk8u/jdk8u + jdk8*) HG_REPO=https://hg.openjdk.java.net/jdk8u/jdk8u [ -z "$TAGS" ] && TAGS="jdk8u144-b34 jdk8u151-b12 jdk8u152-b16 jdk8u161-b12 jdk8u162-b12 jdk8u172-b03 jdk8u172-b11";; - jdk9*) HG_REPO=http://hg.openjdk.java.net/jdk-updates/jdk9u + jdk9*) HG_REPO=https://hg.openjdk.java.net/jdk-updates/jdk9u [ -z "$TAGS" ] && TAGS="jdk-9+181 jdk-9.0.1+11 jdk-9.0.3+9 jdk-9.0.4+11";; *) Unknown JDK version - only jdk8u and jdk9 are supported; exit 1;; esac @@ -56,7 +56,7 @@ GIT_MINOR_VERSION=$(echo "$GIT_VERSION" | cut -d. -f2) if ! which git-remote-hg 2>/dev/null; then echo "I need git-remote-hg and could not find it" - echo "Get it from http://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" + echo "Get it from https://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" exit 1 fi diff --git a/makejdk-any-platform.1 b/makejdk-any-platform.1 index 26825680c..04675c211 100755 --- a/makejdk-any-platform.1 +++ b/makejdk-any-platform.1 @@ -1,81 +1,160 @@ .\" Manpage for makejdk-any-platform.sh -.TH "0.1.0" "Apache 2" +.TH "2.0.0" "Apache 2" .SH NAME -makejdk.sh +makejdk-any-platform.sh .SH SYNOPSIS -"./makejdk-any-platform.sh --version [version] [options]" +"./makejdk-any-platform.sh [options] version" .SH DESCRIPTION -The simplest way to build OpenJDK using our scripts is to run makejdk-any-platform.sh -c --version and have your user be in the Docker group on the machine (or prefix all of your Docker commands with sudo. This script can be used to create a Docker container that will be configured with all of the required dependencies and a base operating system in order to build OpenJDK. +This script is the entry point for building (Adopt) OpenJDK binaries. Building +natively or in a docker container are both supported. This script (and its +supporting scripts) have defaults, but you can override these as needed. + +makejdk-any-platform.sh and it's supporting scripts will auto detect the +platform and architecture it is running on and configures the OpenJDK build +accordingly. The supporting scripts will also download and locally install any +required dependencies for the OpenJDK build, e.g. The ALSA sound and Freetype +font libraries. + +Many of the configuration options are passed through to the \fBconfigure\fR and +\fBmake\fR commands that OpenJDK uses to build binaries. Please see the +appropriate \fIREADME-builds.html\fR file for the OpenJDK source repository +that you are building for further details. + +\fBExample:\fR The simplest use case to run is: + +"./makejdk-any-platform.sh --docker jdk8u" + +This will start a Docker container and build you the latest Java 8 AdoptOpenJDK +binary from the source at https://github.com/AdoptOpenJDK/openjdk-jdk8u + +Please visit https://www.adoptopenjdk.net for further support -.B WARNING: Currently the Docker based build is not working (see https://github.com/AdoptOpenJDK/openjdk-build/issues/194) .SH VERSIONS .TP -.BR \jdk8u -https://github.com/AdoptOpenJDK/openjdk-jdk8u +.BR jdk8u +Build Java 8, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk8u +.TP +.BR jdk9 +Build Java 9, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk9 .TP -.BR \jdk9 -https://github.com/AdoptOpenJDK/openjdk-jdk9u +.BR jdk10 +Build Java 10, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk10 .TP -.BR \jdk10 -https://github.com/AdoptOpenJDK/openjdk-jdk10u +.BR jfx +Build OpenJFX, defaults to https://github.com/AdoptOpenJDK/openjdk-jfx .TP .BR \jdk11 https://github.com/AdoptOpenJDK/openjdk-jdk11 .TP +.BR amber +Build Project Amber, defaults to https://github.com/AdoptOpenJDK/openjdk-amber .SH OPTIONS .TP -.BR \-s ", " \-\-source " " \fI\fR -specify the location for the source and dependencies to be cloned, defaults to ./openjdk. If it is specified, docker is not used +.BR \-b ", " \-\-branch " " \fI\fR +specify a custom branch to build from, e.g. dev. +For reference, AdoptOpenJDK GitHub source repos default to the \fI\fR +branch which may contain a very small diff set to the \fI\fR branch +(which is a clone from the OpenJDK mercurial forest). +.TP +.BR \-B ", " \-\-build-number " " \fI\fR +specify the OpenJDK build number to build from, e.g. b12. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the build number being the suffix at the end. +.TP +.BR \-\-build-variant " " \fI\fR +specify a OpenJDK build variant, e.g. openj9. +For reference, the default variant is hotspot and does not need to be specified. +.TP +.BR \-c ", " \-\-clean-docker-build +removes the existing docker container and persistent volume before starting +a new docker based build. +.TP +.BR \-C ", " \-\-configure-args " " \fI\fR +specify any custom user configuration arguments. .TP .BR \-d ", " \-\-destination " " \fI\fR -specify the location for the tarball (eg. /path/ or /path/here.tar.gz) +specify the location for the built binary, e.g. /path/. +This is typically used in conjunction with \fB<-T>\fR to create a custom path +/ file name for the resulting binary. .TP -.BR \-r ", " \-\-repository " " \fI\fR -specify a custom repository (eg. username/openjdk-jdk8u) +.BR \-D ", " \-\-docker +build OpenJDK in a docker container. .TP -.BR \-b ", " \-\-branch " " \fI\fR -specify a custom branch (eg. dev) +.BR \-\-disable-shallow-git-clone +disable the default \fB--depth=1\fR shallow cloning of git repo(s). .TP -.BR \-k ", " \-\-keep -reuse docker container (prevents deleting) +.BR \-f ", " \-\-freetype-dir +specify the location of an existing FreeType library. +This is typically used in conjunction with \fB<-F>\fR. .TP -.BR \-j ", " \-\-jtreg -run jtreg after building +.BR \-F ", " \-\-skip-freetype +skip building Freetype automatically. +This is typically used in conjunction with \fB<-f>\fR. .TP -.BR \-js ", " \-\-jtreg-subsets -select one or more jtreg tests to run +.BR \-h ", " \-\-help +print this help. .TP -.BR \-sf ", " \-\-skip-freetype -skip building freetype -.BR \-nc ", " \-\-no-colour -disable colour output +.BR \-i ", " \-\-ignore-container +ignore the existing docker container if you have one already. .TP -.BR \-ftd ", " \-\-freetype-dir -specify the location of an existing FreeType library that can be used for the OpenJDK build process +.BR \-J ", " \-\-jdk-boot-dir " " \fI\fR +specify the JDK boot dir. +For reference, OpenJDK needs the previous version of a JDK in order to build +itself. You should select the path to a JDK install that is N-1 versions below +the one you are trying to build. .TP -.BR \-S ", " \-\-ssh -use ssh when cloning git +.BR \-k ", " \-\-keep +if using docker, keep the container after the build. .TP -.BR \-dsgc ", " \-\-disable-shallow-git-clone -disable shallow cloning of git repo(s) using the --depth=1 CLI option +.BR \-n ", " \-\-no-colour +disable colour output. .TP -.BR \-bv ", " \-\-variant " " \fI\fR -specify a custom build variant (eg. openj9) +.BR \-p ", " \-\-processors " " \fI\fR +specify the number of processors to use for the docker build. .TP +.BR \-r ", " \-\-repository " " \fI\fR +specify the repository to clone OpenJDK source from, +e.g. https://github.com/karianna/openjdk-jdk8u. .TP -.BR \-c ", " \-\-clean-docker-build -clean docker data volume +.BR \-s ", " \-\-source " " \fI\fR +specify the location to clone the OpenJDK source (and dependencies) to. +.TP +.BR \-S ", " \-\-ssh +use ssh when cloning git. .TP +.BR \-\-sign \fI\fR +specify the location for the certificate. For windows this is the p12 +certificate to sign the DLL. .TP -.BR \-t ", " \-\-tag " " \fI\fR -specify a custom tag +.BR \-\-sudo +run the docker container as root. .TP +.BR \-t ", " \-\-tag " " \fI\fR +specify the repository tag that you want to build OpenJDK from. .TP -.BR \-\-sign " " \fI\fR -specify the location for the windows p12 certificate. Used only for windows builds to sign DLL +.BR \-\-tmp-space-build +use a temporary workspace if the main one is unavailable. .TP +.BR \-T ", " \-\-target-file-name " " \fI\fR +specify the final name of the OpenJDK binary. +This is typically used in conjunction with \fB<-D>\fR to create a custom file +name for the resulting binary. .TP -.BR \-ca ", " \-\-configure-args " " \fI\fR -specify a custom configuration arguments +.BR \-u ", " \-\-update-version " " \fI\fR +specify the update version to build OpenJDK from, e.g. 162. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the update number being the number after the '_' +(162) or the 3rd position in the semVer version string (4). +This is typically used in conjunction with \fB<-b>\fR. .TP +.BR \-V ", " \-\-jvm-variant " " \fI\fR +specify the JVM variant (server or client), defaults to server. + +.SH EXAMPLE USAGE + +Some common example usages: + +"./makejdk-any-platform --docker jdk8u" + +"./makejdk-any-platform -s /home/openjdk10/src -d /home/openjdk/target -T MyOpenJDK10.tar.gz jdk10" diff --git a/makejdk-any-platform.sh b/makejdk-any-platform.sh index 854c7d729..6361c323c 100755 --- a/makejdk-any-platform.sh +++ b/makejdk-any-platform.sh @@ -1,10 +1,12 @@ #!/bin/bash + +################################################################################ # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -12,119 +14,49 @@ # See the License for the specific language governing permissions and # limitations under the License. # +################################################################################ -# Script to clone the OpenJDK source then build it +################################################################################ +# +# Entry point to build (Adopt) OpenJDK binaries for any platform. +# +# 1. Source scripts to support configuration, docker builds and native builds. +# 2. Parse the Command Line Args +# 3. Set a host of configuration options based on args, platform etc +# 4. Display and then persist those configuration options +# 5. Build the binary in Docker or natively +# +################################################################################ -# Optionally uses Docker, otherwise you can provide two arguments: -# the area to build the JDK e.g. $HOME/mybuilddir as -s or --source -# and the target destination for the tar.gz e.g. -d or --destination $HOME/mytargetdir -# Both must be absolute paths! You can use $PWD/mytargetdir +# TODO Comment out the 'x' once script is stable. +set -eu -# A simple way to install dependencies persistently is to use our Ansible playbooks +# i.e. Where we are +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# You can set the JDK boot directory with the JDK_BOOT_DIR environment variable +# Pull in configuration and build support +# shellcheck source=sbin/common/config_init.sh +source "${SCRIPT_DIR}/sbin/common/config_init.sh" -export OS_KERNEL_NAME="" -OS_KERNEL_NAME=$(uname | awk '{print tolower($0)}') -export OS_MACHINE_NAME="" -OS_MACHINE_NAME=$(uname -m) +# shellcheck source=docker-build.sh +source "${SCRIPT_DIR}/docker-build.sh" -# The full forest name, e.g. jdk8, jdk8u, jdk9, jdk9u, etc. -export OPENJDK_FOREST_NAME="" +# shellcheck source=native-build.sh +source "${SCRIPT_DIR}/native-build.sh" -# The abridged, core version name, e.g. jdk8, jdk9, etc. No "u"s. -export OPENJDK_CORE_VERSION="" +# shellcheck source=configureBuild.sh +source "${SCRIPT_DIR}/configureBuild.sh" -export BUILD_VARIANT="" -export REPOSITORY="" +echo "Starting $0 to configure, build (Adopt)OpenJDK binary" -counter=0 -for i in "$@"; do - (( counter++ )) - case "$i" in - "--version" | "-v") - (( counter++ )) - string="\$$counter" - OPENJDK_FOREST_NAME=$(echo "$@" | awk "{print $string}") - export OPENJDK_CORE_VERSION=${OPENJDK_FOREST_NAME} - if [[ $OPENJDK_FOREST_NAME == *u ]]; then - export OPENJDK_CORE_VERSION=${OPENJDK_FOREST_NAME%?} - fi - # Switch it back to stop it being out of sync with i - (( counter-- )) - ;; - "--variant" | "-bv") - (( counter++ )) - string="\$$counter" - BUILD_VARIANT=$(echo "$@" | awk "{print $string}") - # Switch it back to stop it being out of sync with i - (( counter-- )) - ;; - esac -done +# Configure the build, display the parameters and write the config to disk +# see ${SCRIPT_DIR}/sbin/common/config_init.sh for details +configure_build "$@" +writeConfigToFile -export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="false" -export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="false" -if [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "jdk11" ] || [ "$OPENJDK_CORE_VERSION" == "amber" ]; then - export JDK_PATH="jdk" - export JRE_PATH="jre" - if [ "$OPENJDK_CORE_VERSION" == "jdk9" ]; then - export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="true" - export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="true" - fi - export CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS_FOR_ANY_PLATFORM:-"--disable-warnings-as-errors"} -elif [ "$OPENJDK_CORE_VERSION" == "jdk8" ]; then - export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="false" - export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="true" - export JDK_PATH="j2sdk-image" - export JRE_PATH="j2re-image" +# Let's build and test the (Adopt) OpenJDK binary in Docker or natively +if [ "${BUILD_CONFIG[USE_DOCKER]}" == "true" ] ; then + buildOpenJDKViaDocker else - echo "Please specify a version with --version or -v , either jdk9, jdk10 or jdk8, with or without a \\'u\\' suffix." - man ./makejdk-any-platform.1 - exit 1 + buildOpenJDKInNativeEnvironment fi - -REPOSITORY="${REPOSITORY:-adoptopenjdk/openjdk-$OPENJDK_FOREST_NAME}"; -REPOSITORY="$(echo "${REPOSITORY}" | awk '{print tolower($0)}')"; - -case "$OS_MACHINE_NAME" in -"s390x") - if [ "$OPENJDK_CORE_VERSION" == "jdk8" ] && [ "$BUILD_VARIANT" != "openj9" ]; then - export JVM_VARIANT=${JVM_VARIANT:-zero} - else - export JVM_VARIANT=${JVM_VARIANT:-server} - fi - - export BUILD_FULL_NAME=${BUILD_FULL_NAME:-linux-s390x-normal-${JVM_VARIANT}-release} - S390X_MAKE_ARGS="CONF=${BUILD_FULL_NAME} DEBUG_BINARIES=true images" - export MAKE_ARGS_FOR_ANY_PLATFORM=${MAKE_ARGS_FOR_ANY_PLATFORM:-$S390X_MAKE_ARGS} -;; - -"ppc64le") - export JVM_VARIANT=${JVM_VARIANT:-server} - export BUILD_FULL_NAME=${BUILD_FULL_NAME:-linux-ppc64-normal-${JVM_VARIANT}-release} - # shellcheck disable=SC1083 - export FREETYPE_FONT_BUILD_TYPE_PARAM=${FREETYPE_FONT_BUILD_TYPE_PARAM:="--build=$(rpm --eval %{_host})"} -;; - -"armv7l") - export JVM_VARIANT=${JVM_VARIANT:-zero} - export MAKE_ARGS_FOR_ANY_PLATFORM=${MAKE_ARGS_FOR_ANY_PLATFORM:-"DEBUG_BINARIES=true images"} - export CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS_FOR_ANY_PLATFORM:-"--with-jobs=${NUM_PROCESSORS}"} -;; - -"aarch64") - export FREETYPE_FONT_VERSION="2.5.2" -;; -esac - -case "$OS_KERNEL_NAME" in -"aix") - export MAKE_COMMAND_NAME=${MAKE_COMMAND_NAME:-"gmake"} -;; -"SunOS") - export MAKE_COMMAND_NAME=${MAKE_COMMAND_NAME:-"gmake"} -;; - -esac -./makejdk.sh "$@" diff --git a/makejdk.sh b/makejdk.sh deleted file mode 100755 index f3f67ef84..000000000 --- a/makejdk.sh +++ /dev/null @@ -1,475 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Script to clone the OpenJDK source then build it - -# Optionally uses Docker, otherwise you can provide two arguments: -# the area to build the JDK e.g. $HOME/mybuilddir as -s or --source -# and the target destination for the tar.gz e.g. -d or --destination $HOME/mytargetdir -# Both must be absolute paths! You can use $PWD/mytargetdir - -# A simple way to install dependencies persistently is to use our Ansible playbooks - -# You can set the JDK boot directory with the JDK_BOOT_DIR environment variable - -SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck disable=SC1090 -source "$SCRIPT_DIR/sbin/common-functions.sh" - -OPENJDK_REPO_NAME=${OPENJDK_REPO_NAME:-openjdk} -SHALLOW_CLONE_OPTION="--depth=1" - -DOCKER_SOURCE_VOLUME_NAME="openjdk-source-volume" -CONTAINER=openjdk_container -TMP_CONTAINER_NAME=openjdk-copy-src -CLEAN_DOCKER_BUILD=false - -export COPY_TO_HOST=false -export USE_DOCKER=false - -TARGET_DIR_IN_THE_CONTAINER="/openjdk/target/" - -WORKING_DIR="" -USE_SSH=false -TARGET_DIR="" -BRANCH="" -TAG="" -KEEP=false -JTREG=false -BUILD_VARIANT=${BUILD_VARIANT-:""} -USER_SUPPLIED_CONFIGURE_ARGS="" -VERSION="" - -JVM_VARIANT=${JVM_VARIANT:-server} - -OPENJDK_UPDATE_VERSION="" -OPENJDK_BUILD_NUMBER="" - -PATH_BUILD="" - -determineBuildProperties - -sourceFileWithColourCodes() -{ - # shellcheck disable=SC1091 - source ./sbin/colour-codes.sh -} - -sourceSignalHandler() -{ - # shellcheck disable=SC1091 - source sbin/signalhandler.sh -} - -parseCommandLineArgs() -{ - while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do - opt="$1"; - shift; - case "$opt" in - "--" ) break 2;; - - "--source" | "-s" ) - WORKING_DIR="$1"; shift;; - - "--ssh" | "-S" ) - USE_SSH=true;; - - "--destination" | "-d" ) - TARGET_DIR="$1"; shift;; - - "--repository" | "-r" ) - REPOSITORY="$1"; shift;; - - "--branch" | "-b" ) - BRANCH="$1"; shift;; - - "--tag" | "-t" ) - TAG="$1"; SHALLOW_CLONE_OPTION=""; shift;; - - "--keep" | "-k" ) - KEEP=true;; - - "--clean-docker-build" | "-c" ) - CLEAN_DOCKER_BUILD=true;; - - "--jtreg" | "-j" ) - JTREG=true;; - - "--jtreg-subsets" | "-js" ) - JTREG=true; JTREG_TEST_SUBSETS="$1"; shift;; - - "--no-colour" | "-nc" ) - COLOUR=false;; - - "--sign" ) - export SIGN=true; export CERTIFICATE="$1"; shift;; - - "--disable-shallow-git-clone" | "-dsgc" ) - SHALLOW_CLONE_OPTION=""; shift;; - - "--skip-freetype" | "-sf" ) - export FREETYPE=false;; - - "--version" | "-v" ) - VERSION="$1"; shift;; - - "--freetype-dir" | "-ftd" ) - export FREETYPE_DIRECTORY="$1"; shift;; - - "--variant" | "-bv" ) - export BUILD_VARIANT="$1"; shift;; - - "--configure-args" | "-ca" ) - export USER_SUPPLIED_CONFIGURE_ARGS="$1"; shift;; - - *) echo >&2 "Invalid option: ${opt}"; man ./makejdk-any-platform.1; exit 1;; - esac - done -} - -doAnyBuildVariantOverrides() -{ - if [[ "${BUILD_VARIANT}" == "openj9" ]]; then - # current location of Extensions for OpenJDK9 for OpenJ9 project - REPOSITORY="ibmruntimes/openj9-openjdk-${OPENJDK_CORE_VERSION}" - [ -z "$BRANCH" ] && BRANCH="openj9" - fi - if [[ "${BUILD_VARIANT}" == "SapMachine" ]]; then - # current location of SAP variant - REPOSITORY="SAP/SapMachine" - [ -z "$BRANCH" ] && BRANCH="sapmachine10" # sapmachine10 is the current branch for OpenJDK10 mainline (equivalent to jdk/jdk10) - fi -} - -checkIfDockerIsUsedForBuildingOrNot() -{ - # Both a working directory and a target directory provided - if [ ! -z "$WORKING_DIR" ] && [ ! -z "$TARGET_DIR" ] ; then - # This uses sbin/build.sh directly - echo "Not using Docker, working area will be ${WORKING_DIR}, target for the JDK will be ${TARGET_DIR}" - fi - - # No working directory and no target directory provided - if [ -z "${WORKING_DIR}" ] && [ -z "${TARGET_DIR}" ] ; then - echo "No parameters provided, using Docker." - USE_DOCKER=true - elif [ ! -z "$TARGET_DIR" ] && [ -z "$WORKING_DIR" ] ; then - # Target directory is defined but the working directory isn't - # Calls sbin/build.sh inside of Docker followed by a docker cp command - echo "Using Docker, target directory for the tgz on the host: ${TARGET_DIR}" - USE_DOCKER=true - fi -} - -checkIfDockerIsUsedShouldTheContainerBePreserved() -{ - if [ "${KEEP}" == "true" ] ; then - echo "We'll keep the built Docker container if you're using Docker." - else - echo "We'll remove the built Docker container if you're using Docker." - fi -} - -setDefaultIfBranchIsNotProvided() -{ - if [ -z "$BRANCH" ] ; then - echo "BRANCH is undefined so checking out dev" - BRANCH="dev" - fi -} - -setWorkingDirectoryIfProvided() -{ - if [ -z "${WORKING_DIR}" ] ; then - echo "WORKING_DIR is undefined so setting to ${PWD}." - WORKING_DIR=$PWD - else - echo "Working dir is ${WORKING_DIR}." - fi -} - -setTargetDirectoryIfProvided() -{ - if [ -z "${TARGET_DIR}" ] ; then - echo "TARGET_DIR is undefined so setting to $PWD" - TARGET_DIR=$PWD - # Only makes a difference if we're in Docker - echo "If you're using Docker the build artifact will not be copied to the host." - else - echo "Target directory is ${TARGET_DIR}" - COPY_TO_HOST=true - echo "If you're using Docker we'll copy the build artifact to the host." - fi -} - -checkOpenJDKGitRepo() -{ - if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] && { [ "$OPENJDK_CORE_VERSION" == "jdk8" ] || [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "jdk11" ] ; } ; then - GIT_VERSION=$(git --git-dir "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" remote -v | grep "${OPENJDK_CORE_VERSION}") - echo "${GIT_VERSION}" - if [ "$GIT_VERSION" ]; then - # The repo is the correct JDK Version - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - echo "Will reset the repository at $PWD in 10 seconds..." - sleep 10 - echo "Pulling latest changes from git repo" - - showShallowCloningMessage "fetch" - git fetch --all ${SHALLOW_CLONE_OPTION} - git reset --hard origin/$BRANCH - if [ ! -z "$TAG" ]; then - git checkout "$TAG" - fi - git clean -ffdx - else - # The repo is not for the correct JDK Version - echo "Incorrect Source Code for ${OPENJDK_FOREST_NAME}. Will re-clone" - rm -rf "${WORKING_DIR:?}/${OPENJDK_REPO_NAME:?}" - cloneOpenJDKGitRepo - fi - cd "${WORKING_DIR}" || return - elif [ ! -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] ; then - # If it doesn't exist, clone it - echo "Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" - cloneOpenJDKGitRepo - fi -} - -cloneOpenJDKGitRepo() -{ - if [[ "$USE_SSH" == "true" ]] ; then - GIT_REMOTE_REPO_ADDRESS="git@github.com:${REPOSITORY}.git" - else - GIT_REMOTE_REPO_ADDRESS="https://github.com/${REPOSITORY}.git" - fi - - showShallowCloningMessage "cloning" - # I want to ensure SHALLOW_CLONE_OPTION isn't explcitly wrapped in quotes PR#455 - # shellcheck disable=SC2206 - GIT_CLONE_ARGUMENTS=($SHALLOW_CLONE_OPTION "-b" "$BRANCH" "$GIT_REMOTE_REPO_ADDRESS" "${WORKING_DIR}/${OPENJDK_REPO_NAME}") - - echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" - git clone "${GIT_CLONE_ARGUMENTS[@]}" - if [ ! -z "$TAG" ]; then - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || exit 1 - git checkout "$TAG" - fi - - # Building OpenJDK with OpenJ9 must run get_source.sh to clone openj9 and openj9-omr repositories - if [ "$BUILD_VARIANT" == "openj9" ]; then - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - bash get_source.sh - fi - -} - -# TODO This only works for jdk8u based releases. Will require refactoring when jdk9 enters an update cycle -getOpenJDKUpdateAndBuildVersion() -{ - - if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ]; then - # It does exist and it's a repo other than the AdoptOpenJDK one - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - echo "Pulling latest tags and getting the latest update version using git fetch -q --tags ${SHALLOW_CLONE_OPTION}" - git fetch -q --tags ${SHALLOW_CLONE_OPTION} - OPENJDK_REPO_TAG=${TAG:-$(getFirstTagFromOpenJDKGitRepo)} - if [[ "${OPENJDK_REPO_TAG}" == "" ]] ; then - echo "Unable to detect git tag" - exit 1 - else - echo "OpenJDK repo tag is $OPENJDK_REPO_TAG" - fi - - OPENJDK_UPDATE_VERSION=$(echo "${OPENJDK_REPO_TAG}" | cut -d'u' -f 2 | cut -d'-' -f 1) - OPENJDK_BUILD_NUMBER=$(echo "${OPENJDK_REPO_TAG}" | cut -d'b' -f 2 | cut -d'-' -f 1) - echo "Version: ${OPENJDK_UPDATE_VERSION} ${OPENJDK_BUILD_NUMBER}" - cd "${WORKING_DIR}" || return - - fi - -} - -showShallowCloningMessage() -{ - mode=$1 - if [[ "$SHALLOW_CLONE_OPTION" == "" ]]; then - echo "Git repo ${mode} mode: deep (preserves commit history)" - else - echo "Git repo ${mode} mode: shallow (DOES NOT contain commit history)" - fi -} - -testOpenJDKViaDocker() -{ - if [[ "$JTREG" == "true" ]]; then - mkdir -p "${WORKING_DIR}/target" - docker run \ - -v "${DOCKER_SOURCE_VOLUME_NAME}:/openjdk/build" \ - -v "${WORKING_DIR}/target:${TARGET_DIR_IN_THE_CONTAINER}" \ - --entrypoint /openjdk/sbin/jtreg.sh "${CONTAINER}" - fi -} - -createPersistentDockerDataVolume() -{ - #Create a data volume called $DOCKER_SOURCE_VOLUME_NAME, - #this gets mounted at /openjdk/build inside the container and is persistent between builds/tests - #unless -c is passed to this script, in which case it is recreated using the source - #in the current ./openjdk directory on the host machine (outside the container) - docker volume inspect $DOCKER_SOURCE_VOLUME_NAME > /dev/null 2>&1 - DATA_VOLUME_EXISTS=$? - - if [[ "$CLEAN_DOCKER_BUILD" == "true" || "$DATA_VOLUME_EXISTS" != "0" ]]; then - - echo "Removing old volumes and containers" - docker rm -f "$(docker ps -a --no-trunc | grep $CONTAINER | cut -d' ' -f1)" || true - docker volume rm "${DOCKER_SOURCE_VOLUME_NAME}" || true - - echo "Creating tmp container and copying src" - docker volume create --name "${DOCKER_SOURCE_VOLUME_NAME}" - docker run -v "${DOCKER_SOURCE_VOLUME_NAME}":/openjdk/build --name "$TMP_CONTAINER_NAME" ubuntu:14.04 /bin/bash - docker cp openjdk "$TMP_CONTAINER_NAME":/openjdk/build/ - - echo "Removing tmp container" - docker rm -f "$TMP_CONTAINER_NAME" - fi -} - -buildDockerContainer() -{ - echo Building docker container - docker build -t "${CONTAINER}" "${PATH_BUILD}" --build-arg "OPENJDK_CORE_VERSION=${OPENJDK_CORE_VERSION}" - if [[ "${BUILD_VARIANT}" != "" && -f "${PATH_BUILD}/Dockerfile-${BUILD_VARIANT}" ]]; then - CONTAINER="${CONTAINER}-${BUILD_VARIANT}" - echo Building dockerfile variant "${BUILD_VARIANT}" - docker build -t "${CONTAINER}" -f "${PATH_BUILD}/Dockerfile-${BUILD_VARIANT}" "${PATH_BUILD}" --build-arg "OPENJDK_CORE_VERSION=${OPENJDK_CORE_VERSION}" - fi -} - -buildAndTestOpenJDKViaDocker() -{ - - - PATH_BUILD="docker/${OPENJDK_CORE_VERSION}/x86_64/ubuntu" - - if [ -z "$(command -v docker)" ]; then - echo "Error, please install docker and ensure that it is in your path and running!" - exit - fi - - echo "Using Docker to build the JDK" - - createPersistentDockerDataVolume - - - # Copy our scripts for usage inside of the container - rm -r "${PATH_BUILD}/sbin" - cp -r "${SCRIPT_DIR}/sbin" "${PATH_BUILD}/sbin" 2>/dev/null - - - # Keep is undefined so we'll kill the docker image - - if [[ "$KEEP" == "true" ]] ; then - # shellcheck disable=SC2086 - if [ "$(docker ps -a | grep -c \"$CONTAINER\")" == 0 ]; then - echo "No docker container found so creating '$CONTAINER' " - buildDockerContainer - fi - else - echo "Building as you've not specified -k or --keep" - docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER" | awk '{print $1 }' | xargs -I {} docker rm -f {} - buildDockerContainer - fi - - mkdir -p "${WORKING_DIR}/target" - - docker run -t \ - -e BUILD_VARIANT="$BUILD_VARIANT" \ - -v "${DOCKER_SOURCE_VOLUME_NAME}:/openjdk/build" \ - -v "${WORKING_DIR}/target":/${TARGET_DIR_IN_THE_CONTAINER} \ - --entrypoint /openjdk/sbin/build.sh "${CONTAINER}" - - testOpenJDKViaDocker - - # Didn't specify to keep - if [[ -z ${KEEP} ]] ; then - docker ps -a | awk '{ print $1,$2 }' | grep "${CONTAINER}" | awk '{print $1 }' | xargs -I {} docker rm {} - fi -} - -testOpenJDKInNativeEnvironmentIfExpected() -{ - if [[ "$JTREG" == "true" ]]; - then - "${SCRIPT_DIR}"/sbin/jtreg.sh "${WORKING_DIR}" "${OPENJDK_REPO_NAME}" "${BUILD_FULL_NAME}" "${VERSION}" "${JTREG_TEST_SUBSETS}" - fi -} - -buildAndTestOpenJDKInNativeEnvironment() -{ - BUILD_ARGUMENTS="" - declare -a BUILD_ARGUMENT_NAMES=("--source" "--destination" "--repository" "--variant" "--update-version" "--build-number" "--repository-tag" "--configure-args") - declare -a BUILD_ARGUMENT_VALUES=("${WORKING_DIR}" "${TARGET_DIR}" "${OPENJDK_REPO_NAME}" "${JVM_VARIANT}" "${OPENJDK_UPDATE_VERSION}" "${OPENJDK_BUILD_NUMBER}" "${TAG}" "${USER_SUPPLIED_CONFIGURE_ARGS}") - - BUILD_ARGS_ARRAY_INDEX=0 - while [[ ${BUILD_ARGS_ARRAY_INDEX} < ${#BUILD_ARGUMENT_NAMES[@]} ]]; do - if [[ ${BUILD_ARGUMENT_VALUES[${BUILD_ARGS_ARRAY_INDEX}]} != "" ]]; - then - BUILD_ARGUMENTS="${BUILD_ARGUMENTS}${BUILD_ARGUMENT_NAMES[${BUILD_ARGS_ARRAY_INDEX}]} ${BUILD_ARGUMENT_VALUES[${BUILD_ARGS_ARRAY_INDEX}]} " - fi - ((BUILD_ARGS_ARRAY_INDEX++)) - done - - echo "Calling ${SCRIPT_DIR}/sbin/build.sh ${BUILD_ARGUMENTS}" - # shellcheck disable=SC2086 - "${SCRIPT_DIR}"/sbin/build.sh ${BUILD_ARGUMENTS} - - testOpenJDKInNativeEnvironmentIfExpected -} - -buildAndTestOpenJDK() -{ - if [ "$USE_DOCKER" == "true" ] ; then - buildAndTestOpenJDKViaDocker - else - buildAndTestOpenJDKInNativeEnvironment - fi -} - -################################################################## - -sourceSignalHandler -parseCommandLineArgs "$@" -doAnyBuildVariantOverrides -if [[ -z "${COLOUR}" ]] ; then - sourceFileWithColourCodes -fi -checkIfDockerIsUsedForBuildingOrNot -checkIfDockerIsUsedShouldTheContainerBePreserved -setDefaultIfBranchIsNotProvided -setWorkingDirectoryIfProvided -setTargetDirectoryIfProvided -time ( - echo "Cloning OpenJDK git repo" - checkOpenJDKGitRepo -) - -time ( - echo "Updating OpenJDK git repo" - getOpenJDKUpdateAndBuildVersion -) - -buildAndTestOpenJDK diff --git a/mercurial-tags/java-tool/MercurialTracker.java b/mercurial-tags/java-tool/MercurialTracker.java index 72d78c877..ad6f07757 100644 --- a/mercurial-tags/java-tool/MercurialTracker.java +++ b/mercurial-tags/java-tool/MercurialTracker.java @@ -84,7 +84,7 @@ public static void main(String[] args) throws IOException, InterruptedException, System.out.println(" * " + switch_buffer); System.out.println(" This one expects value. eg " + switch_buffer + "=30"); System.out.println(" it is setting the max depth of investigated path."); - System.out.println(" Note that teh compelxity is exponential in this case, sostay at 10-30. With eg 100, you can keep ti unlimited anyway."); + System.out.println(" Note that the complexity is exponential in this case, so stay at 10-30. With e.g. 100, you can keep it unlimited anyway."); System.out.println(" Without it,some high-development forests like jdk10 or jdk10-shenandoah"); System.out.println(" will run for many hours, and have unpredictable results"); System.out.println(" If yuou need to use this switch, you probably do not need this program at all."); diff --git a/native-build.sh b/native-build.sh new file mode 100755 index 000000000..cac32550b --- /dev/null +++ b/native-build.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# This script deals with the configuration to build (Adopt) OpenJDK natively. +# It's sourced by the makejdk-any-platform.sh script. +# +################################################################################ + +set -eu + +# i.e. Where we are +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +buildOpenJDKInNativeEnvironment() +{ + displayParams + bash "${SCRIPT_DIR}"/sbin/build.sh +} diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy new file mode 100644 index 000000000..aae1d2110 --- /dev/null +++ b/pipelines/build/build_base_file.groovy @@ -0,0 +1,294 @@ +import groovy.json.JsonOutput +import groovy.json.JsonSlurper + +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +/** + * This file starts a high level job, it is called from openjdk8_pipeline.groovy, openjdk9_pipeline.groovy, openjdk10_pipeline.groovy. + * + * This: + * + * 1. Generate job for each configuration based on create_job_from_template.groovy + * 2. Execute job + * 3. Push generated artifacts to github + */ + +def toBuildParams(enableTests, params) { + + List buildParams = [] + + buildParams += [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: params.get("NODE_LABEL")] + buildParams += string(name: "ENABLE_TESTS", value: "${enableTests}") + + params + .findAll { it.key != 'NODE_LABEL' } + .each { name, value -> buildParams += string(name: name, value: value) } + + return buildParams +} + +static def buildConfiguration(javaToBuild, variant, configuration, releaseTag, branch, additionalConfigureArgs) { + + def additionalNodeLabels = formAdditionalNodeLabels(configuration, variant) + + def buildParams = [ + JAVA_TO_BUILD: javaToBuild, + NODE_LABEL : "${additionalNodeLabels}&&${configuration.os}&&${configuration.arch}", + VARIANT : variant, + ARCHITECTURE : configuration.arch, + TARGET_OS : configuration.os + ] + + if (configuration.containsKey('bootJDK')) buildParams.put("JDK_BOOT_VERSION", configuration.bootJDK) + if (configuration.containsKey('buildArgs')) buildParams.put("BUILD_ARGS", configuration.buildArgs) + if (configuration.containsKey('additionalFileNameTag')) buildParams.put("ADDITIONAL_FILE_NAME_TAG", configuration.additionalFileNameTag) + + buildParams.putAll(getConfigureArgs(configuration, additionalConfigureArgs)) + + if (branch != null && branch.length() > 0) { + buildParams.put("BRANCH", branch) + } + + def isRelease = false + if (releaseTag != null && releaseTag.length() > 0) { + isRelease = true + buildParams.put("TAG", releaseTag) + } + + def testList = getTestList(configuration, isRelease) + + return [ + javaVersion: javaToBuild, + arch : configuration.arch, + os : configuration.os, + variant : variant, + parameters : buildParams, + test : testList, + ] +} + +static def getTestList(configuration, isRelease) { + if (configuration.containsKey("test")) { + def testJobType = isRelease ? "release" : "nightly" + + // hack as jenkins sandbox wont allow instanceof + if ("java.util.LinkedHashMap" == configuration.test.getClass().getName()) { + return configuration.test.get(testJobType) + } else { + return configuration.test + } + } + return [] +} + +static def formAdditionalNodeLabels(configuration, variant) { + def buildTag = "build" + + if (configuration.os == "windows" && variant == "openj9") { + buildTag = "buildj9" + } else if (configuration.arch == "s390x" && variant == "openj9") { + buildTag = "(buildj9||build)&&openj9" + } + + def labels = "${buildTag}" + + if (configuration.containsKey("additionalNodeLabels")) { + def additionalNodeLabels = null + + // hack as jenkins sandbox wont allow instanceof + if ("java.util.LinkedHashMap" == configuration.additionalNodeLabels.getClass().getName()) { + additionalNodeLabels = configuration.additionalNodeLabels.get(variant) + } else { + additionalNodeLabels = configuration.additionalNodeLabels + } + labels = "${additionalNodeLabels}&&${labels}" + } + + return labels +} + +static def getConfigureArgs(configuration, additionalConfigureArgs) { + def buildParams = [:] + def configureArgs = ""; + + if (configuration.containsKey('configureArgs')) configureArgs += configuration.configureArgs; + if (additionalConfigureArgs != null && additionalConfigureArgs.length() > 0) { + configureArgs += " " + additionalConfigureArgs + } + + if (configureArgs.length() > 0) { + buildParams.put("CONFIGURE_ARGS", configureArgs) + } + return buildParams +} + +def getJobConfigurations(javaVersionToBuild, availableConfigurations, String targetConfigurations, String releaseTag, String branch, String additionalConfigureArgs) { + def jobConfigurations = [:] + + //Parse config passed to jenkins job + new JsonSlurper() + .parseText(targetConfigurations) + .each { target -> + + //For each requested build type, generate a configuration + if (availableConfigurations.containsKey(target.key)) { + def configuration = availableConfigurations.get(target.key) + target.value.each { variant -> + GString name = "${configuration.os}-${configuration.arch}-${variant}" + if (configuration.containsKey('additionalFileNameTag')) { + name += "-${configuration.additionalFileNameTag}" + } + jobConfigurations[name] = buildConfiguration(javaVersionToBuild, variant, configuration, releaseTag, branch, additionalConfigureArgs) + } + } + } + + return jobConfigurations +} + +static Integer getJavaVersionNumber(version) { + // version should be something like "jdk8u" + def matcher = (version =~ /(\d+)/) + return Integer.parseInt(matcher[0][1]) +} + + +static def determineReleaseRepoVersion(javaToBuild) { + def number = getJavaVersionNumber(javaToBuild) + + return "jdk${number}" +} + +static def getJobName(displayName, config) { + return "${config.javaVersion}-${displayName}" +} + +static def getJobFolder(config) { + return "build-scripts/jobs/${config.javaVersion}" +} + +// Generate a job from template at `create_job_from_template.groovy` +def createJob(jobName, jobFolder, config, enableTests, scmVars) { + + def params = config.parameters.clone() + params.put("JOB_NAME", jobName) + params.put("JOB_FOLDER", jobFolder) + params.put("TEST_CONFIG", JsonOutput.prettyPrint(JsonOutput.toJson(config))) + + params.put("GIT_URI", scmVars["GIT_URL"]) + params.put("GIT_BRANCH", scmVars["GIT_BRANCH"]) + + create = jobDsl targets: "pipelines/build/create_job_from_template.groovy", ignoreExisting: false, additionalParameters: params + + return create +} + +// Call job to push artifacts to github +def publishRelease(javaToBuild, releaseTag) { + def release = false + def tag = javaToBuild + if (releaseTag != null && releaseTag.length() > 0) { + release = true + tag = releaseTag + } + + node("master") { + stage("publish") { + build job: 'build-scripts/release/refactor_openjdk_release_tool', + parameters: [string(name: 'RELEASE', value: "${release}"), + string(name: 'TAG', value: tag), + string(name: 'UPSTREAM_JOB_NAME', value: env.JOB_NAME), + string(name: 'UPSTREAM_JOB_NUMBER', value: "${currentBuild.getNumber()}"), + string(name: 'VERSION', value: determineReleaseRepoVersion(javaToBuild))] + } + } +} + +def doBuild(String javaVersionToBuild, availableConfigurations, String targetConfigurations, String enableTestsArg, String publishArg, String releaseTag, String branch, String additionalConfigureArgs, scmVars) { + + if (releaseTag == null || releaseTag == "false") { + releaseTag = "" + } + + def jobConfigurations = getJobConfigurations(javaVersionToBuild, availableConfigurations, targetConfigurations, releaseTag, branch, additionalConfigureArgs) + def jobs = [:] + + def enableTests = enableTestsArg == "true" + def publish = publishArg == "true" + + + echo "Java: ${javaVersionToBuild}" + echo "OS: ${targetConfigurations}" + echo "Enable tests: ${enableTests}" + echo "Publish: ${publish}" + echo "ReleaseTag: ${releaseTag}" + + + jobConfigurations.each { configuration -> + jobs[configuration.key] = { + def config = configuration.value + + // jdk10u-linux-x64-hotspot + def jobTopName = getJobName(configuration.key, config) + def jobFolder = getJobFolder(config) + + // i.e jdk10u/job/jdk10u-linux-x64-hotspot + def downstreamJobName = "${jobFolder}/${jobTopName}" + + catchError { + // Execute build job for configuration i.e jdk10u/job/jdk10u-linux-x64-hotspot + stage(configuration.key) { + // generate job + createJob(jobTopName, jobFolder, config, enableTests, scmVars) + + // execute build + def downstreamJob = build job: downstreamJobName, propagate: false, parameters: toBuildParams(enableTests, config.parameters) + + if (downstreamJob.getResult() == 'SUCCESS') { + // copy artifacts from build + node("master") { + catchError { + sh "rm target/${config.os}/${config.arch}/${config.variant}/* || true" + + copyArtifacts( + projectName: downstreamJobName, + selector: specific("${downstreamJob.getNumber()}"), + filter: 'workspace/target/*', + fingerprintArtifacts: true, + target: "target/${config.os}/${config.arch}/${config.variant}/", + flatten: true) + + // Checksum + sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + + // Archive in Jenkins + archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" + } + } + } + } + } + } + } + + parallel jobs + + // publish to github if needed + if (publish) { + publishRelease(javaVersionToBuild, releaseTag) + } +} + +return this \ No newline at end of file diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy new file mode 100644 index 000000000..d4a7dedaf --- /dev/null +++ b/pipelines/build/create_job_from_template.groovy @@ -0,0 +1,83 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +/** + * A template that defines a build job. + * + * This mostly is just a wrapper to call the openjdk_build_pipeline.groovy script that defines the majority of + * what a pipeline job does + */ + +String buildFolder = "$JOB_FOLDER" + +if (!binding.hasVariable('JDK_BOOT_VERSION')) JDK_BOOT_VERSION = "" +if (!binding.hasVariable('CONFIGURE_ARGS')) CONFIGURE_ARGS = "" +if (!binding.hasVariable('BUILD_ARGS')) BUILD_ARGS = "" +if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = "" +if (!binding.hasVariable('TEST_CONFIG')) TEST_CONFIG = "" +if (!binding.hasVariable('ENABLE_TESTS')) ENABLE_TESTS = "false" +if (!binding.hasVariable('BRANCH')) BRANCH = "" + + +if (!binding.hasVariable('GIT_URI')) GIT_URI = "https://github.com/AdoptOpenJDK/openjdk-build.git" +if (!binding.hasVariable('GIT_BRANCH')) GIT_BRANCH = "new_build_scripts" + + +folder(buildFolder) { + description 'Automatically generated build jobs.' +} + +pipelineJob("$buildFolder/$JOB_NAME") { + description('

THIS IS AN AUTOMATICALLY GENERATED JOB DO NOT MODIFY, IT WILL BE OVERWRITTEN.

This job is defined in create_job_from_template.groovy in the openjdk-build repo, if you wish to change it modify that

') + definition { + cpsScm { + scm { + git { + remote { + url(GIT_URI) + } + branch("${GIT_BRANCH}") + extensions { + cleanBeforeCheckout() + } + } + } + scriptPath('pipelines/build/openjdk_build_pipeline.groovy') + lightweight(true) + } + } + properties { + copyArtifactPermissionProperty { + projectNames('*') + } + } + logRotator { + numToKeep(5) + } + parameters { + stringParam('TAG', null, "git tag/branch/commit to bulid if not HEAD") + stringParam('NODE_LABEL', "$NODE_LABEL") + stringParam('JAVA_TO_BUILD', "$JAVA_TO_BUILD") + stringParam('JDK_BOOT_VERSION', "${JDK_BOOT_VERSION}") + stringParam('CONFIGURE_ARGS', "$CONFIGURE_ARGS", "Additional arguments to pass to ./configure") + stringParam('BUILD_ARGS', "$BUILD_ARGS", "additional args to call makejdk-any-platform.sh with") + stringParam('ARCHITECTURE', "$ARCHITECTURE") + stringParam('VARIANT', "$VARIANT") + stringParam('TARGET_OS', "$TARGET_OS") + stringParam('ADDITIONAL_FILE_NAME_TAG', "$ADDITIONAL_FILE_NAME_TAG") + stringParam('ENABLE_TESTS', "$ENABLE_TESTS") + stringParam('BRANCH', "$BRANCH") + textParam('TEST_CONFIG', "$TEST_CONFIG") + } +} \ No newline at end of file diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy new file mode 100644 index 000000000..d0e99a6c3 --- /dev/null +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -0,0 +1,101 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + bootJDK : "/Users/jenkins/tools/hudson.model.JDK/JDK9.0.1", + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + additionalNodeLabels: [ + hotspot: 'rhel7' + ], + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: [ + // Pinned as at time of writing build-osuosl-centos74-ppc64le-2 does not have a valid boot jdk + hotspot: 'centos7&&build-osuosl-centos74-ppc64le-1', + openj9: 'ubuntu' + ], + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + /* + "x86-32Windows" : [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012&&x86-32', + test : false + ], + */ + "linuxXL" : [ + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk10u" + +node ("master") { + def scmVars = checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) +} diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy new file mode 100644 index 000000000..c1cc1f21c --- /dev/null +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -0,0 +1,105 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : [ + nightly: ['openjdktest', 'systemtest', 'perftest', 'externaltest', 'externaltest_extended'], + release: ['openjdktest', 'systemtest', 'perftest', 'externaltest'] + ] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + x32Windows: [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + additionalNodeLabels: 'build-marist-rhel74-s390x-2', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'ubuntu', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + /* + "x86-32Windows" : [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012&&x86-32', + test : false + ], + */ + linuxXL : [ + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk11" + +node ("master") { + def scmVars = checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) +} diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy new file mode 100644 index 000000000..6db7fe706 --- /dev/null +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -0,0 +1,103 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: [ + hotspot: 'centos6', + openj9: 'build-joyent-centos69-x64-1' + ], + test : ['openjdktest', 'systemtest', 'perftest', 'externaltest', 'externaltest_extended'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: [ + hotspot: 'win2008', + //Pin to build-softlayer-win2012r2-x64-2 as build-softlayer-win2012r2-x64-1 may have issues + openj9: 'win2012&&build-softlayer-win2012r2-x64-2' + ], + test : ['openjdktest'] + ], + + x32Windows: [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: [ + hotspot: 'win2008', + openj9: 'win2012' + ], + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + linuxXL : [ + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk8u" + +node ("master") { + def scmVars = checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) +} + diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy new file mode 100644 index 000000000..7eec6b8b4 --- /dev/null +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -0,0 +1,87 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + additionalNodeLabels: 'rhel7', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'centos7', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + linuxXL : [ + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk9u" + +node ("master") { + def scmVars = checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) +} diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy new file mode 100644 index 000000000..62cfb4144 --- /dev/null +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -0,0 +1,184 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +import groovy.json.JsonSlurper + +@Library('openjdk-jenkins-helper@master') +import JobHelper +import NodeHelper + + +/** + * This file is a template for running a build for a given configuration + * A configuration is for example jdk10u-mac-x64-hotspot. + * + * This file is referenced by the pipeline template create_job_from_template.groovy + * + * A pipeline looks like: + * 1. Check out and build JDK by calling build-farm/make-adopt-build-farm.sh + * 2. Archive artifacts created by build + * 3. Run all tests defined in the configuration + * 4. Sign artifacts if needed and re-archive + * + */ + +def getJavaVersionNumber(version) { + // version should be something like "jdk8u" + def matcher = (version =~ /(\d+)/) + return Integer.parseInt(matcher[0][1]) +} + +def determineTestJobName(config, testType) { + + def variant + def number = getJavaVersionNumber(config.javaVersion) + + if (config.variant == "openj9") { + variant = "j9" + } else { + variant = "hs" + } + + def arch = config.arch + if (arch == "x64") { + arch = "x86-64" + } + + def os = config.os + if (os == "mac") { + os = "macos" + } + + return "openjdk${number}_${variant}_${testType}_${arch}_${os}" +} + +def runTests(config) { + def testStages = [:] + + config.test.each { testType -> + // For each requested test, i.e 'openjdktest', 'systemtest', 'perftest', 'externaltest', call test job + try { + println "Running test: ${testType}}" + testStages["${testType}"] = { + stage("${testType}") { + + // example jobName: openjdk10_hs_externaltest_x86-64_linux + def jobName = determineTestJobName(config, testType) + + if (JobHelper.jobIsRunnable(jobName)) { + catchError { + build job: jobName, + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), + string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}")] + } + } else { + println "Requested test job that does not exist or is disabled: ${jobName}" + } + } + } + } catch (Exception e) { + println "Failed execute test: ${e.getLocalizedMessage()}" + } + } + return testStages +} + +def sign(config) { + // Sign and archive jobs if needed + if (config.os == "windows" || config.os == "mac") { + node('master') { + stage("sign") { + def filter = "" + def certificate = "" + + if (config.os == "windows") { + filter = "**/OpenJDK*_windows_*.zip" + certificate = "C:\\Users\\jenkins\\windows.p12" + + } else if (config.os == "mac") { + filter = "**/OpenJDK*_mac_*.tar.gz" + certificate = "\"Developer ID Application: London Jamocha Community CIC\"" + } + + def signJob = build job: "build-scripts/release/sign_build", + propagate: true, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), + string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}"), + string(name: 'OPERATING_SYSTEM', value: "${config.os}"), + string(name: 'FILTER', value: "${filter}"), + string(name: 'CERTIFICATE', value: "${certificate}"), + [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: "${config.os}&&build"], + ] + + //Copy signed artifact back and rearchive + sh "rm workspace/target/* || true" + + copyArtifacts( + projectName: "build-scripts/release/sign_build", + selector: specific("${signJob.getNumber()}"), + filter: 'workspace/target/*', + fingerprintArtifacts: true, + target: "workspace/target/", + flatten: true) + + sh 'for file in $(ls workspace/target/*.tar.gz workspace/target/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + archiveArtifacts artifacts: "workspace/target/*" + } + } + } +} + +try { + def config = new JsonSlurper().parseText("${TEST_CONFIG}") + println "Executing tests: ${config}" + println "Build num: ${env.BUILD_NUMBER}" + + def enableTests = ENABLE_TESTS == "true" + + stage("build") { + if(NodeHelper.nodeIsOnline(NODE_LABEL)) { + node(NODE_LABEL) { + checkout scm + try { + sh "./build-farm/make-adopt-build-farm.sh" + archiveArtifacts artifacts: "workspace/target/*" + } finally { + if (config.os == "aix") { + cleanWs notFailBuild: true + } + } + } + } else { + error("No node of this type exists: ${NODE_LABEL}") + return + } + } + + if (enableTests && config.test != false) { + try { + testStages = runTests(config) + parallel testStages + } catch (Exception e) { + println "Failed test: ${e}" + } + } + + // Sign and archive jobs if needed + sign(config) + +} catch (Exception e) { + currentBuild.result = 'FAILURE' +} + diff --git a/pipelines/build/openjdk_pipeline.groovy b/pipelines/build/openjdk_pipeline.groovy new file mode 100644 index 000000000..babd65d11 --- /dev/null +++ b/pipelines/build/openjdk_pipeline.groovy @@ -0,0 +1,94 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'centos7', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + /* + "x86-32Windows" : [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012&&x86-32', + test : false + ], + */ + "linuxXL" : [ + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk" + +node ("master") { + def scmVars = checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + //Tests off for head builds + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, false, publish, releaseTag, branch, additionalConfigureArgs, scmVars) +} diff --git a/pipelines/openjdk10_nightly_pipeline.groovy b/pipelines/openjdk10_nightly_pipeline.groovy deleted file mode 100644 index 60d0ac9c7..000000000 --- a/pipelines/openjdk10_nightly_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -//def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm32','aarch64'] -def buildPlatforms = ['ppc64le', 'arm32'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -//buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -//buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -//buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -//buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -//buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] -//buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk10_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk10_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk10_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/pipelines/openjdk10_openj9_nightly_pipeline.groovy b/pipelines/openjdk10_openj9_nightly_pipeline.groovy deleted file mode 100644 index b47d8cb62..000000000 --- a/pipelines/openjdk10_openj9_nightly_pipeline.groovy +++ /dev/null @@ -1,55 +0,0 @@ -println "building ${JDK_VERSION}" - -//def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildPlatforms = ['ppc64le'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -//buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -//buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -//buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -//buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -def typeTests = ['openjdktest', 'systemtest'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk10_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk10_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk10_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs \ No newline at end of file diff --git a/pipelines/openjdk10_openj9_release_pipeline.groovy b/pipelines/openjdk10_openj9_release_pipeline.groovy deleted file mode 100644 index dad8d9394..000000000 --- a/pipelines/openjdk10_openj9_release_pipeline.groovy +++ /dev/null @@ -1,57 +0,0 @@ -println "building ${JDK_TAG}" - -//def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildPlatforms = ['ppc64le'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -//buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -//buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -//buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -//buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -def typeTests = ['openjdktest', 'systemtest'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk10_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk10_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}")] - } - } - } - } -} -parallel jobs - -stage('checksums') { - checksumJob = build job: 'openjdk10_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}"), - string(name: 'PRODUCT', value: 'releases')] -} -stage('publish releases') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk10_release_pipeline.groovy b/pipelines/openjdk10_release_pipeline.groovy deleted file mode 100644 index b702e1a61..000000000 --- a/pipelines/openjdk10_release_pipeline.groovy +++ /dev/null @@ -1,58 +0,0 @@ -println "building ${JDK_VERSION}" - -//def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm32', 'aarch64'] -def buildPlatforms = ['ppc64le', 'AIX', 'arm32'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -//buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -//buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -//buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] -//buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -//buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] -//buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - stage('build') { - buildJob = build job: "openjdk10_build_${archOS}", - parameters: [string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - } - if (buildMaps[platform].test) { - buildMaps[platform].test.each { - typeTests.each { - build job:"openjdk10_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] - } - } - } - } -} -parallel jobs - -def checksumJob -stage('checksums') { - checksumJob = build job: 'openjdk10_build_checksum', - parameters: [string(name: 'PRODUCT', value: 'releases')] -} -stage('installers') { - build job: 'openjdk10_build_installer', parameters: [string(name: 'VERSION', value: "${JDK_VERSION}")] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk8_nightly_pipeline.groovy b/pipelines/openjdk8_nightly_pipeline.groovy deleted file mode 100644 index b66301573..000000000 --- a/pipelines/openjdk8_nightly_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Windows', 'Windows32', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest','externaltest_extended'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk8_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk8_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk8_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk8u181-b13'), - string(name: 'VERSION', value: 'jdk8'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs - diff --git a/pipelines/openjdk8_openj9_nightly_pipeline.groovy b/pipelines/openjdk8_openj9_nightly_pipeline.groovy deleted file mode 100644 index 2ec235286..000000000 --- a/pipelines/openjdk8_openj9_nightly_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows', 'Windows32', 'LinuxXL'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest','externaltest_extended'], ArchOSs:'x86-64_linux'] -buildMaps['LinuxXL'] = [test:['openjdktest'], ArchOSs:'x86-64_linux_largeHeap'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk8_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk8_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_openj9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk8_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_openj9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk8u181-b13'), - string(name: 'VERSION', value: 'jdk8-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs - diff --git a/pipelines/openjdk8_openj9_release_pipeline.groovy b/pipelines/openjdk8_openj9_release_pipeline.groovy deleted file mode 100644 index d0e75ba78..000000000 --- a/pipelines/openjdk8_openj9_release_pipeline.groovy +++ /dev/null @@ -1,53 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows', 'Windows32', 'LinuxXL'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] -buildMaps['LinuxXL'] = [test:['openjdktest'], ArchOSs:'x86-64_linux_largeHeap'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - stage('build') { - buildJob = build job: "openjdk8_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk8_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_openj9_build_${archOS}")] - } - } - } - } -} -parallel jobs - -def checksumJob -stage('checksums') { - checksumJob = build job: 'openjdk8_openj9_build_checksum', - parameters: [string(name: 'PRODUCT', value: 'releases')] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk8-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk8_release_pipeline.groovy b/pipelines/openjdk8_release_pipeline.groovy deleted file mode 100644 index 19cf91317..000000000 --- a/pipelines/openjdk8_release_pipeline.groovy +++ /dev/null @@ -1,57 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Windows', 'Windows32', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - stage('build') { - buildJob = build job: "openjdk8_build_${archOS}", - parameters: [string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - } - if (buildMaps[platform].test) { - buildMaps[platform].test.each { - typeTests.each { - build job:"openjdk8_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_build_${archOS}")] - } - } - } - } -} -parallel jobs - -def checksumJob -stage('checksums') { - checksumJob = build job: 'openjdk8_build_checksum', - parameters: [string(name: 'PRODUCT', value: 'releases')] -} -stage('installers') { - build job: 'openjdk8_build_installer', parameters: [string(name: 'VERSION', value: "${JDK_VERSION}")] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk8'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk9_nightly_pipeline.groovy b/pipelines/openjdk9_nightly_pipeline.groovy deleted file mode 100644 index bdc81ff34..000000000 --- a/pipelines/openjdk9_nightly_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'arm32', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk9_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk9_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk-9+181'), - string(name: 'VERSION', value: 'jdk9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/pipelines/openjdk9_openj9_nightly_pipeline.groovy b/pipelines/openjdk9_openj9_nightly_pipeline.groovy deleted file mode 100644 index 00a8f9bcc..000000000 --- a/pipelines/openjdk9_openj9_nightly_pipeline.groovy +++ /dev/null @@ -1,53 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk9_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk9_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk9_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk-9.0.4+12'), - string(name: 'VERSION', value: 'jdk9-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/pipelines/openjdk9_openj9_release_pipeline.groovy b/pipelines/openjdk9_openj9_release_pipeline.groovy deleted file mode 100644 index 838264e23..000000000 --- a/pipelines/openjdk9_openj9_release_pipeline.groovy +++ /dev/null @@ -1,55 +0,0 @@ -println "building ${JDK_TAG}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk9_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk9_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}")] - } - } - } - } -} -parallel jobs - -stage('checksums') { - checksumJob = build job: 'openjdk9_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}"), - string(name: 'PRODUCT', value: 'releases')] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: '${JDK_TAG}'), - string(name: 'VERSION', value: 'jdk9-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk_amber_nightly_pipeline.groovy b/pipelines/openjdk_amber_nightly_pipeline.groovy deleted file mode 100644 index baaaf4170..000000000 --- a/pipelines/openjdk_amber_nightly_pipeline.groovy +++ /dev/null @@ -1,50 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Linux', 'Windows'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest'], ArchOSs:'x86-64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk_amber_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"amber-rsl_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk_amber_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk_amber_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk_amber_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk-amber'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk_amber_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/sbin/build.sh b/sbin/build.sh index 738a6503e..0f80b2845 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -1,101 +1,90 @@ #!/bin/bash -# + +################################################################################ # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -# +################################################################################ -# Script to download any additional packages for building OpenJDK -# before calling ./configure (using JDK 7 as the base) +################################################################################ +# +# Build OpenJDK - can be called directly but is typically called by +# docker-build.sh or native-build.sh. +# +# See bottom of the script for the call order and each function for further +# details. +# +# Calls 'configure' then 'make' in order to build OpenJDK +# +################################################################################ +set -eu SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck source=sbin/common-functions.sh -source "$SCRIPT_DIR/common-functions.sh" - -WORKING_DIR="" -TARGET_DIR="" -OPENJDK_REPO_NAME="" -JVM_VARIANT="${JVM_VARIANT:-server}" -OPENJDK_UPDATE_VERSION="" -OPENJDK_BUILD_NUMBER="" -OPENJDK_REPO_TAG="" -JRE_TARGET_PATH="" -TRIMMED_TAG="" -USER_SUPPLIED_CONFIGURE_ARGS="" -while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do - opt="$1"; - shift; - case "$opt" in - "--" ) break 2;; +# shellcheck source=sbin/prepareWorkspace.sh +source "$SCRIPT_DIR/prepareWorkspace.sh" - "--source" | "-s" ) - WORKING_DIR="$1"; shift;; +# shellcheck source=sbin/common/config_init.sh +source "$SCRIPT_DIR/common/config_init.sh" - "--destination" | "-d" ) - TARGET_DIR="$1"; shift;; +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/common/constants.sh" - "--repository" | "-r" ) - OPENJDK_REPO_NAME="$1"; shift;; +# shellcheck source=sbin/common/common.sh +source "$SCRIPT_DIR/common/common.sh" - "--variant" | "-jv" ) - JVM_VARIANT="$1"; shift;; +export OPENJDK_REPO_TAG +export OPENJDK_DIR +export JRE_TARGET_PATH +export CONFIGURE_ARGS="" +export MAKE_TEST_IMAGE="" +export GIT_CLONE_ARGUMENTS=() - "--update-version" | "-uv" ) - OPENJDK_UPDATE_VERSION="$1"; shift;; +# Parse the CL arguments, defers to the shared function in common-functions.sh +function parseArguments() { + parseConfigurationArguments "$@" - "--build-number" | "-bn" ) - OPENJDK_BUILD_NUMBER="$1"; shift;; + OPENJDK_DIR="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" - "--repository-tag" | "-rt" ) - OPENJDK_REPO_TAG="$1"; shift;; - - "--configure-args" | "-ca" ) - USER_SUPPLIED_CONFIGURE_ARGS="$1"; shift;; - - *) echo >&2 "${error}Invalid build.sh option: ${opt}${normal}"; exit 1;; - esac -done - -OPENJDK_DIR=$WORKING_DIR/$OPENJDK_REPO_NAME - - -RUN_JTREG_TESTS_ONLY="" + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_VERSION}" ]; then + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="true"; + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="true"; + fi + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_VERSION}" ]; then + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="false"; + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="true"; + fi -if [ "$JVM_VARIANT" == "--run-jtreg-tests-only" ]; then - RUN_JTREG_TESTS_ONLY="--run-jtreg-tests-only" - JVM_VARIANT="server" -fi + echo "JDK Image folder name: ${BUILD_CONFIG[JDK_PATH]}" + echo "JRE Image folder name: ${BUILD_CONFIG[JRE_PATH]}" + echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG=${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]}" + echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG=${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]}" -echo "JDK Image folder name: ${JDK_PATH}" -echo "JRE Image folder name: ${JRE_PATH}" -echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG=${COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG}" -echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG=${COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG}" + BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]:-"images"} -MAKE_COMMAND_NAME=${MAKE_COMMAND_NAME:-"make"} -MAKE_ARGS_FOR_ANY_PLATFORM=${MAKE_ARGS_FOR_ANY_PLATFORM:-"images"} -# Defaults to not building this target, for Java 9+ we set this to test-image in order to build the native test libraries -MAKE_TEST_IMAGE="" -CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS_FOR_ANY_PLATFORM:-""} + BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-""} +} +# Add an argument to the configure call addConfigureArg() { - #Only add an arg if it is not overridden by a user-specified arg. - if [[ ${CONFIGURE_ARGS_FOR_ANY_PLATFORM} != *"$1"* ]] && [[ ${USER_SUPPLIED_CONFIGURE_ARGS} != *"$1"* ]]; then + # Only add an arg if it is not overridden by a user-specified arg. + if [[ ${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]} != *"$1"* ]] && [[ ${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]} != *"$1"* ]]; then CONFIGURE_ARGS="${CONFIGURE_ARGS} ${1}${2}" fi } +# Add an argument to the configure call (if it's not empty) addConfigureArgIfValueIsNotEmpty() { #Only try to add an arg if the second argument is not empty. @@ -104,62 +93,69 @@ addConfigureArgIfValueIsNotEmpty() fi } -sourceFileWithColourCodes() +# Configure the boot JDK +configuringBootJDKConfigureParameter() { - # shellcheck disable=SC1090 - # shellcheck disable=SC1091 - source "$SCRIPT_DIR"/colour-codes.sh -} -checkIfDockerIsUsedForBuildingOrNot() -{ - # If on docker - - if [[ -f /.dockerenv ]] ; then - echo "Detected we're in docker" - WORKING_DIR=/openjdk/build - TARGET_DIR=/openjdk/target/ - OPENJDK_REPO_NAME=/openjdk - OPENJDK_DIR="$WORKING_DIR/$OPENJDK_REPO_NAME" - USE_DOCKER=true - fi + if [ -z "${BUILD_CONFIG[JDK_BOOT_DIR]}" ] ; then + echo "Searching for JDK_BOOT_DIR" - # E.g. /openjdk/build if you're building in a Docker container - # otherwise ensure it's a writable area e.g. /home/youruser/myopenjdkarea - - if [ -z "$WORKING_DIR" ] || [ -z "$TARGET_DIR" ] ; then - echo "build.sh is called by makejdk.sh and requires two parameters" - echo "Are you sure you want to call it directly?" - echo "Usage: bash ./${0} " - echo "Note that you must have the OpenJDK source before using this script!" - echo "This script will try to move ./openjdk to the source directory for you, " - echo "and this will be your working area where all required files will be downloaded to." - echo "You can override the JDK boot directory by setting the environment variable JDK_BOOT_DIR" - exit; + # shellcheck disable=SC2046 + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "darwin" ]]; then + BUILD_CONFIG[JDK_BOOT_DIR]=$(dirname $(dirname $(readlink $(which javac)))) + else + BUILD_CONFIG[JDK_BOOT_DIR]=$(dirname $(dirname $(readlink -f $(which javac)))) + fi + + echo "Guessing JDK_BOOT_DIR: ${BUILD_CONFIG[JDK_BOOT_DIR]}" + echo "If this is incorrect explicitly configure JDK_BOOT_DIR" + else + echo "Overriding JDK_BOOT_DIR, set to ${BUILD_CONFIG[JDK_BOOT_DIR]}" fi + + echo "Boot dir set to ${BUILD_CONFIG[JDK_BOOT_DIR]}" + + addConfigureArgIfValueIsNotEmpty "--with-boot-jdk=" "${BUILD_CONFIG[JDK_BOOT_DIR]}" } -createWorkingDirectory() +# Get the OpenJDK update version and build version +getOpenJDKUpdateAndBuildVersion() { - echo "Making the working directory to store source files and extensions: ${WORKING_DIR}" + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" - mkdir -p $WORKING_DIR + if [ -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ]; then - cd $WORKING_DIR || exit -} + # It does exist and it's a repo other than the AdoptOpenJDK one + cd "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return -configuringBootJDKConfigureParameter() -{ - if [ -z "$JDK_BOOT_DIR" ] ; then - echo "JDK_BOOT_DIR is ${JDK_BOOT_DIR}" - JDK_BOOT_DIR=/usr/lib/java-1.7.0 - else - echo "Overriding JDK_BOOT_DIR, set to ${JDK_BOOT_DIR}" - fi + if [ -f ".git/shallow.lock" ] + then + echo "Detected lock file, assuming this is an error, removing" + rm ".git/shallow.lock" + fi + + # shellcheck disable=SC2154 + echo "Pulling latest tags and getting the latest update version using git fetch -q --tags ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]}" + # shellcheck disable=SC2154 + echo "NOTE: This can take quite some time! Please be patient" + git fetch -q --tags ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + OPENJDK_REPO_TAG=${BUILD_CONFIG[TAG]:-$(getFirstTagFromOpenJDKGitRepo)} + if [[ "${OPENJDK_REPO_TAG}" == "" ]] ; then + # shellcheck disable=SC2154 + echo "Unable to detect git tag, exiting..." + exit 1 + else + echo "OpenJDK repo tag is $OPENJDK_REPO_TAG" + fi + + local openjdk_update_version; + openjdk_update_version=$(echo "${OPENJDK_REPO_TAG}" | cut -d'u' -f 2 | cut -d'-' -f 1) - echo "Boot dir set to ${JDK_BOOT_DIR}" + # TODO dont modify config in build script + echo "Version: ${openjdk_update_version} ${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]}" + fi - addConfigureArgIfValueIsNotEmpty "--with-boot-jdk=" "${JDK_BOOT_DIR}" + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" } # Ensure that we produce builds with versions strings something like: @@ -169,229 +165,295 @@ configuringBootJDKConfigureParameter() # OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode) configuringVersionStringParameter() { - if [ "$OPENJDK_CORE_VERSION" == "jdk8" ]; then - # Replace the default 'internal' with our own milestone string - addConfigureArg "--with-milestone=" "adoptopenjdk" + stepIntoTheWorkingDirectory + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) + echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}" + fi + + addConfigureArg "--with-milestone=" "fcs" + local dateSuffix=$(date -u +%Y%m%d%H%M) + + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + addConfigureArg "--with-user-release-suffix=" "${dateSuffix}" + fi + + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "hotspot" ]; then + addConfigureArg "--with-company-name=" "AdoptOpenJDK" + fi # Set the update version (e.g. 131), this gets passed in from the calling script - addConfigureArgIfValueIsNotEmpty "--with-update-version=" "${OPENJDK_UPDATE_VERSION}" + local updateNumber=${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]} + if [ -z "${updateNumber}" ]; then + updateNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f1 -d"-" | cut -f2 -d"u") + fi + addConfigureArgIfValueIsNotEmpty "--with-update-version=" "${updateNumber}" # Set the build number (e.g. b04), this gets passed in from the calling script - addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${OPENJDK_BUILD_NUMBER}" + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "$OPENJDK_REPO_TAG" | cut -f2 -d"-") + fi + addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${buildNumber}" + elif [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"+") + fi + + TRIMMED_TAG=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"-" ) + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + addConfigureArg "--with-version-opt=" "${dateSuffix}" + else + addConfigureArg "--without-version-opt" "" + fi + + addConfigureArg "--without-version-pre" "" + addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" else - if [ -z "$OPENJDK_REPO_TAG" ]; then - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || echo Cannot change to "${WORKING_DIR}/${OPENJDK_REPO_NAME}" - OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) - echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}" + # > JDK 9 + + # Set the build number (e.g. b04), this gets passed in from the calling script + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"+") fi - # > JDK 8 - addConfigureArg "--with-version-pre=" "adoptopenjdk" - TRIMMED_TAG=$(echo "$OPENJDK_REPO_TAG" | cut -f2 -d"-") - addConfigureArg "--with-version-string=" "${TRIMMED_TAG}" + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + addConfigureArg "--with-version-opt=" "${dateSuffix}" + else + addConfigureArg "--without-version-opt" "" + fi + + addConfigureArg "--without-version-pre" "" + addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" + addConfigureArg "--with-vendor-version-string=" "AdoptOpenJDK" fi echo "Completed configuring the version string parameter, config args are now: ${CONFIGURE_ARGS}" } +# Construct all of the 'configure' parameters buildingTheRestOfTheConfigParameters() { if [ ! -z "$(which ccache)" ]; then addConfigureArg "--enable-ccache" "" fi - addConfigureArgIfValueIsNotEmpty "--with-jvm-variants=" "${JVM_VARIANT}" - addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${WORKING_DIR}/cacerts_area/security/cacerts" - addConfigureArg "--with-alsa=" "${WORKING_DIR}/alsa-lib-${ALSA_LIB_VERSION}" + addConfigureArgIfValueIsNotEmpty "--with-jvm-variants=" "${BUILD_CONFIG[JVM_VARIANT]}" - # Point-in-time dependency for openj9 only - if [[ "${BUILD_VARIANT}" == "openj9" ]] ; then - addConfigureArg "--with-freemarker-jar=" "${WORKING_DIR}/freemarker-${FREEMARKER_LIB_VERSION}/lib/freemarker.jar" + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then + addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/cacerts_area/security/cacerts" fi - if [[ -z "${FREETYPE}" ]] ; then - case $OPENJDK_CORE_VERSION in - jdk8*|jdk9*|jdk10*) FREETYPE_DIRECTORY=${FREETYPE_DIRECTORY:-"${WORKING_DIR}/${OPENJDK_REPO_NAME}/installedfreetype"} ;; - *) FREETYPE_DIRECTORY=${FREETYPE_DIRECTORY:-bundled} ;; - esac - addConfigureArg "--with-freetype=" "$FREETYPE_DIRECTORY" + addConfigureArg "--with-alsa=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa" + + # Point-in-time dependency for openj9 only + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]] ; then + addConfigureArg "--with-freemarker-jar=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/freemarker.jar" fi - # These will have been installed by the package manager (see our Dockerfile) addConfigureArg "--with-x=" "/usr/include/X11" - # We don't want any extra debug symbols - ensure it's set to release, - # other options include fastdebug and slowdebug - addConfigureArg "--with-debug-level=" "release" + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] ; then + # We don't want any extra debug symbols - ensure it's set to release, + # other options include fastdebug and slowdebug + addConfigureArg "--with-debug-level=" "release" + addConfigureArg "--disable-zip-debug-info" "" + addConfigureArg "--disable-debug-symbols" "" + else + addConfigureArg "--with-debug-level=" "release" + addConfigureArg "--with-native-debug-symbols=" "none" + fi +} + +configureFreetypeLocation() { + if [[ ! "${CONFIGURE_ARGS}" =~ "--with-freetype" ]]; then + if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then + addConfigureArg "--with-freetype-src=" "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/freetype" + else + local freetypeDir=BUILD_CONFIG[FREETYPE_DIRECTORY] + case "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" in + jdk8*|jdk9*|jdk10*) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-"${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedfreetype"} ;; + *) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-bundled} ;; + esac + + echo "setting freetype dir to ${freetypeDir}" + addConfigureArg "--with-freetype=" "${freetypeDir}" + fi + fi + fi } +# Configure the command parameters configureCommandParameters() { configuringVersionStringParameter + configuringBootJDKConfigureParameter + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then echo "Windows or Windows-like environment detected, skipping configuring environment for custom Boot JDK and other 'configure' settings." - else echo "Building up the configure command..." - configuringBootJDKConfigureParameter buildingTheRestOfTheConfigParameters fi - #Now we add any configure arguments the user has specified on the command line. - CONFIGURE_ARGS="${CONFIGURE_ARGS} ${USER_SUPPLIED_CONFIGURE_ARGS}" + # Now we add any configure arguments the user has specified on the command line. + CONFIGURE_ARGS="${CONFIGURE_ARGS} ${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]}" + + configureFreetypeLocation echo "Completed configuring the version string parameter, config args are now: ${CONFIGURE_ARGS}" } -stepIntoTheWorkingDirectory() -{ - # Make sure we're in the source directory for OpenJDK now - cd "$WORKING_DIR/$OPENJDK_REPO_NAME" || exit +# Make sure we're in the source directory for OpenJDK now +stepIntoTheWorkingDirectory() { + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || exit echo "Should have the source, I'm at $PWD" } -runTheOpenJDKConfigureCommandAndUseThePrebuiltConfigParams() -{ +buildTemplatedFile() { echo "Configuring command and using the pre-built config params..." - cd "$OPENJDK_DIR" || exit + stepIntoTheWorkingDirectory echo "Currently at '${PWD}'" - CONFIGURED_OPENJDK_ALREADY=$(find . -name "config.status") - - if [[ ! -z "$CONFIGURED_OPENJDK_ALREADY" ]] ; then - echo "Not reconfiguring due to the presence of config.status in ${WORKING_DIR}" - else - CONFIGURE_ARGS="${CONFIGURE_ARGS} ${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" - - echo "Running ./configure with arguments '${CONFIGURE_ARGS}'" - # Depends upon the configure command being split for multiple args. Don't quote it. - # shellcheck disable=SC2086 - bash ./configure ${CONFIGURE_ARGS} - - # shellcheck disable=SC2181 - if [ $? -ne 0 ]; then - echo "${error}" - echo "Failed to configure the JDK, exiting" - echo "Did you set the JDK boot directory correctly? Override by exporting JDK_BOOT_DIR" - echo "For example, on RHEL you would do export JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el7_3.x86_64" - echo "Current JDK_BOOT_DIR value: ${JDK_BOOT_DIR}" - exit; - else - echo "${good}Configured the JDK" - fi - echo "${normal}" - fi -} - -buildOpenJDK() -{ - cd "$OPENJDK_DIR" || exit - - #If the user has specified nobuild, we do everything short of building the JDK, and then we stop. - if [ "${RUN_JTREG_TESTS_ONLY}" == "--run-jtreg-tests-only" ]; then - rm -rf cacerts_area - echo "Nobuild option was set. Prep complete. Java not built." - exit 0 - fi + FULL_CONFIGURE="bash ./configure ${CONFIGURE_ARGS} ${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]}" + echo "Running ./configure with arguments '${FULL_CONFIGURE}'" # If it's Java 9+ then we also make test-image to build the native test libraries JDK_PREFIX="jdk" - JDK_VERSION_NUMBER="${OPENJDK_CORE_VERSION#$JDK_PREFIX}" + JDK_VERSION_NUMBER="${BUILD_CONFIG[OPENJDK_CORE_VERSION]#$JDK_PREFIX}" if [ "$JDK_VERSION_NUMBER" -gt 8 ]; then MAKE_TEST_IMAGE=" test-image" # the added white space is deliberate as it's the last arg fi - FULL_MAKE_COMMAND="${MAKE_COMMAND_NAME} ${MAKE_ARGS_FOR_ANY_PLATFORM} ${MAKE_TEST_IMAGE}" - echo "Building the JDK: calling '${FULL_MAKE_COMMAND}'" + FULL_MAKE_COMMAND="${BUILD_CONFIG[MAKE_COMMAND_NAME]} ${BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]} ${MAKE_TEST_IMAGE}" + + # shellcheck disable=SC2002 + cat "$SCRIPT_DIR/build.template" | \ + sed -e "s|{configureArg}|${FULL_CONFIGURE}|" \ + -e "s|{makeCommandArg}|${FULL_MAKE_COMMAND}|" > "${BUILD_CONFIG[WORKSPACE_DIR]}/config/configure-and-build.sh" +} - if ! ${FULL_MAKE_COMMAND}; then - echo "${error}Failed to make the JDK, exiting" +executeTemplatedFile() { + stepIntoTheWorkingDirectory + + echo "Currently at '${PWD}'" + bash "${BUILD_CONFIG[WORKSPACE_DIR]}/config/configure-and-build.sh" + exitCode=$? + + if [ "${exitCode}" -eq 1 ]; then + echo "Failed to make the JDK, exiting" exit 1; - else - echo "${good}Built the JDK!" + elif [ "${exitCode}" -eq 2 ]; then + echo "Failed to configure the JDK, exiting" + echo "Did you set the JDK boot directory correctly? Override by exporting JDK_BOOT_DIR" + echo "For example, on RHEL you would do export JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el7_3.x86_64" + echo "Current JDK_BOOT_DIR value: ${BUILD_CONFIG[JDK_BOOT_DIR]}" + exit 2; fi - echo "${normal}" + } +# Print the version string so we know what we've produced printJavaVersionString() { # shellcheck disable=SC2086 - PRODUCT_HOME=$(ls -d $OPENJDK_DIR/build/*/images/${JDK_PATH}) + PRODUCT_HOME=$(ls -d ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/build/*/images/${BUILD_CONFIG[JDK_PATH]}) if [[ -d "$PRODUCT_HOME" ]]; then - echo "${good}'$PRODUCT_HOME' found${normal}" - # shellcheck disable=SC2154 - echo "${info}" + echo "'$PRODUCT_HOME' found" if ! "$PRODUCT_HOME"/bin/java -version; then - echo "${error} Error executing 'java' does not exist in '$PRODUCT_HOME'.${normal}" + echo " Error executing 'java' does not exist in '$PRODUCT_HOME'." exit -1 fi - echo "${normal}" - echo "" else - echo "${error}'$PRODUCT_HOME' does not exist, build might have not been successful or not produced the expected JDK image at this location.${normal}" + echo "'$PRODUCT_HOME' does not exist, build might have not been successful or not produced the expected JDK image at this location." exit -1 fi } +# Clean up removingUnnecessaryFiles() { echo "Removing unnecessary files now..." - echo "Fetching the first tag from the OpenJDK git repo..." if [ -z "$OPENJDK_REPO_TAG" ]; then + echo "Fetching the first tag from the OpenJDK git repo..." + echo "Dir=${PWD}" OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) fi - if [ "$USE_DOCKER" != "true" ] ; then - rm -rf cacerts_area - fi - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return cd build/*/images || return echo "Currently at '${PWD}'" - echo "moving ${JDK_PATH} to ${OPENJDK_REPO_TAG}" + echo "moving ${BUILD_CONFIG[JDK_PATH]} to ${OPENJDK_REPO_TAG}" rm -rf "${OPENJDK_REPO_TAG}" || true - mv "$JDK_PATH" "${OPENJDK_REPO_TAG}" + mv "${BUILD_CONFIG[JDK_PATH]}" "${OPENJDK_REPO_TAG}" + + if [ -d "${BUILD_CONFIG[JRE_PATH]}" ] + then + JRE_TARGET_PATH="${OPENJDK_REPO_TAG}-jre" + [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" + echo "moving ${BUILD_CONFIG[JRE_PATH]} to ${JRE_TARGET_PATH}" + rm -rf "${JRE_TARGET_PATH}" || true + mv "${BUILD_CONFIG[JRE_PATH]}" "${JRE_TARGET_PATH}" + + rm -rf "${JRE_TARGET_PATH}"/demo/applets || true + rm -rf "${JRE_TARGET_PATH}"/demo/jfc/Font2DTest || true + rm -rf "${JRE_TARGET_PATH}"/demo/jfc/SwingApplet || true + fi - JRE_TARGET_PATH="${OPENJDK_REPO_TAG/jdk/jre}" - [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" - echo "moving ${JRE_PATH} to ${JRE_TARGET_PATH}" - rm -rf "${JRE_TARGET_PATH}" || true - mv "$JRE_PATH" "${JRE_TARGET_PATH}" # Remove files we don't need rm -rf "${OPENJDK_REPO_TAG}"/demo/applets || true rm -rf "${OPENJDK_REPO_TAG}"/demo/jfc/Font2DTest || true rm -rf "${OPENJDK_REPO_TAG}"/demo/jfc/SwingApplet || true - rm -rf "${JRE_TARGET_PATH}"/demo/applets || true - rm -rf "${JRE_TARGET_PATH}"/demo/jfc/Font2DTest || true - rm -rf "${JRE_TARGET_PATH}"/demo/jfc/SwingApplet || true + find . -name "*.diz" -type f -delete || true - echo "Finished removing unnecessary files from ${OPENJDK_REPO_TAG} and ${JRE_TARGET_PATH}" + echo "Finished removing unnecessary files from ${OPENJDK_REPO_TAG}" } +# If on a Mac, mac a copy of the font lib as required makeACopyOfLibFreeFontForMacOSX() { IMAGE_DIRECTORY=$1 PERFORM_COPYING=$2 - if [[ "$OS_KERNEL_NAME" == "darwin" ]]; then + if [ ! -d "${IMAGE_DIRECTORY}" ]; then + echo "Could not find dir: ${IMAGE_DIRECTORY}" + return + fi + + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "darwin" ]]; then echo "PERFORM_COPYING=${PERFORM_COPYING}" if [ "${PERFORM_COPYING}" == "false" ]; then - echo "${info} Skipping copying of the free font library to ${IMAGE_DIRECTORY}, does not apply for this version of the JDK. ${normal}" + echo " Skipping copying of the free font library to ${IMAGE_DIRECTORY}, does not apply for this version of the JDK. " return fi - echo "${info} Performing copying of the free font library to ${IMAGE_DIRECTORY}, applicable for this version of the JDK. ${normal}" + echo " Performing copying of the free font library to ${IMAGE_DIRECTORY}, applicable for this version of the JDK. " + SOURCE_LIB_NAME="${IMAGE_DIRECTORY}/lib/libfreetype.dylib.6" + + if [ ! -f "${SOURCE_LIB_NAME}" ]; then + SOURCE_LIB_NAME="${IMAGE_DIRECTORY}/lib/libfreetype.dylib" + fi + if [ ! -f "${SOURCE_LIB_NAME}" ]; then - echo "${error}[Error] ${SOURCE_LIB_NAME} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, aborting copy process...${normal}" + echo "[Error] ${SOURCE_LIB_NAME} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, aborting copy process..." exit -1 fi + TARGET_LIB_NAME="${IMAGE_DIRECTORY}/lib/libfreetype.6.dylib" INVOKED_BY_FONT_MANAGER="${IMAGE_DIRECTORY}/lib/libfontmanager.dylib" @@ -400,122 +462,97 @@ makeACopyOfLibFreeFontForMacOSX() { echo "Copying ${SOURCE_LIB_NAME} to ${TARGET_LIB_NAME}" echo " *** Workaround to fix the MacOSX issue where invocation to ${INVOKED_BY_FONT_MANAGER} fails to find ${TARGET_LIB_NAME} ***" - set -x cp "${SOURCE_LIB_NAME}" "${TARGET_LIB_NAME}" if [ -f "${INVOKED_BY_FONT_MANAGER}" ]; then otool -L "${INVOKED_BY_FONT_MANAGER}" else # shellcheck disable=SC2154 - echo "${warning}[Warning] ${INVOKED_BY_FONT_MANAGER} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, this may cause runtime issues, please beware...${normal}" + echo "[Warning] ${INVOKED_BY_FONT_MANAGER} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, this may cause runtime issues, please beware..." fi otool -L "${TARGET_LIB_NAME}" - set +x echo "Finished copying ${SOURCE_LIB_NAME} to ${TARGET_LIB_NAME}" fi } -signRelease() + +# Get the first tag from the git repo +getFirstTagFromOpenJDKGitRepo() { - if [ "$SIGN" ]; then - case "$OSTYPE" in - "cygwin") - echo "Signing Windows release" - signToolPath=${signToolPath:-"/cygdrive/c/Program Files/Microsoft SDKs/Windows/v7.1/Bin/signtool.exe"} - # Sign .exe files - FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" -type f -name '*.exe') - echo "$FILES" | while read -r f; do "$signToolPath" sign /f "$CERTIFICATE" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; done - # Sign .dll files - FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" -type f -name '*.dll') - echo "$FILES" | while read -r f; do "$signToolPath" sign /f "$CERTIFICATE" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; done - ;; - "darwin"*) - echo "Signing OSX release" - # Login to KeyChain - # shellcheck disable=SC2046 - # shellcheck disable=SC2006 - security unlock-keychain -p `cat ~/.password` - # Sign all files with the executable permission bit set. - FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" ! -perm +111 -type f || find "${OPENJDK_REPO_TAG}" -perm /111 -type f) - echo "$FILES" | while read -r f; do codesign -s "$CERTIFICATE" "$f"; done - ;; - *) - echo "Skipping code signing as it's not supported on $OSTYPE" - ;; - esac - fi + git fetch --tags "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + justOneFromTheRevList=$(git rev-list --tags --max-count=1) + tagNameFromRepo=$(git describe --tags "$justOneFromTheRevList") + echo "$tagNameFromRepo" } +createArchive() { + repoLocation=$1 + targetName=$2 + + archiveExtension=$(getArchiveExtension) + + createOpenJDKArchive "${repoLocation}" "OpenJDK" + archive="${PWD}/OpenJDK${archiveExtension}" + + echo "Your final archive was created at ${archive}" + + echo "Moving the artifact to ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}" + mv "${archive}" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}/${targetName}" +} + +# Create a Tar ball createOpenJDKTarArchive() { COMPRESS=gzip if which pigz >/dev/null 2>&1; then COMPRESS=pigz; fi echo "Archiving the build OpenJDK image and compressing with $COMPRESS" - if [ -z "$OPENJDK_REPO_TAG" ]; then + if [ -z "${OPENJDK_REPO_TAG+x}" ] || [ -z "${OPENJDK_REPO_TAG}" ]; then OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) fi - if [ -z "$JRE_TARGET_PATH" ]; then - JRE_TARGET_PATH="${OPENJDK_REPO_TAG/jdk/jre}" - [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" + if [ -z "${JRE_TARGET_PATH+x}" ] || [ -z "${JRE_TARGET_PATH}" ]; then + JRE_TARGET_PATH="${OPENJDK_REPO_TAG}-jre" fi - + echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}. JRE path will be ${JRE_TARGET_PATH}" - if [ "$USE_DOCKER" == "true" ] ; then - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJRE.tar.gz ./"${JRE_TARGET_PATH}" - EXT=".tar.gz" + ## clean out old builds + rm -r "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[TARGET_DIR]}" || true + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}" || exit - echo "${good}Moving the artifact to ${TARGET_DIR}${normal}" - mv "OpenJDK${EXT}" "${TARGET_DIR}" - mv "OpenJRE${EXT}" "${TARGET_DIR}" - else - case "${OS_KERNEL_NAME}" in - *cygwin*) - zip -r -q OpenJDK.zip ./"${OPENJDK_REPO_TAG}" - zip -r -q OpenJRE.zip ./"${JRE_TARGET_PATH}" - EXT=".zip" ;; - aix) - GZIP=-9 tar -cf - ./"${OPENJDK_REPO_TAG}"/ | $COMPRESS -c > OpenJDK.tar.gz - GZIP=-9 tar -cf - ./"${JRE_TARGET_PATH}"/ | $COMPRESS -c > OpenJRE.tar.gz - EXT=".tar.gz" ;; - *) - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJRE.tar.gz ./"${JRE_TARGET_PATH}" - EXT=".tar.gz" ;; - esac - echo "${good}Your final ${EXT} was created at ${PWD}${normal}" - ls -l - echo "${good}Moving the artifacts to ${TARGET_DIR}${normal}" - # This really ought to be a separate parameter to makejdk_any_platform or - # TARGET_DIR should be a dir name as the name suggests, not a full filename - # This is currnently assuming TARGET_DIT has JDK in the filename, otherwise - # it wiill get overridden - mv "OpenJRE${EXT}" "${TARGET_DIR/JDK/JRE}" - mv "OpenJDK${EXT}" "${TARGET_DIR}" + if [ -d "${JRE_TARGET_PATH}" ]; then + createArchive "${JRE_TARGET_PATH}" "${BUILD_CONFIG[TARGET_FILE_NAME]/_/-jre_}" fi - + createArchive "${OPENJDK_REPO_TAG}" "${BUILD_CONFIG[TARGET_FILE_NAME]}" } +# Echo success showCompletionMessage() { echo "All done!" } -sourceFileWithColourCodes -checkIfDockerIsUsedForBuildingOrNot -createWorkingDirectory -downloadingRequiredDependencies # This function is in common-functions.sh +################################################################################ + +loadConfigFromFile +cd "${BUILD_CONFIG[WORKSPACE_DIR]}" + +parseArguments "$@" +configureWorkspace + +getOpenJDKUpdateAndBuildVersion configureCommandParameters -stepIntoTheWorkingDirectory -runTheOpenJDKConfigureCommandAndUseThePrebuiltConfigParams -buildOpenJDK +buildTemplatedFile +executeTemplatedFile + printJavaVersionString removingUnnecessaryFiles -makeACopyOfLibFreeFontForMacOSX "${OPENJDK_REPO_TAG}" "${COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG}" -makeACopyOfLibFreeFontForMacOSX "${JRE_PATH}" "${COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG}" -signRelease +makeACopyOfLibFreeFontForMacOSX "${OPENJDK_REPO_TAG}" "${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]}" +makeACopyOfLibFreeFontForMacOSX "${OPENJDK_REPO_TAG}-jre" "${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]}" createOpenJDKTarArchive showCompletionMessage + +# ccache is not detected properly TODO +# change grep to something like $GREP -e '^1.*' -e '^2.*' -e '^3\.0.*' -e '^3\.1\.[0123]$'`] +# See https://github.com/AdoptOpenJDK/openjdk-jdk8u/blob/dev/common/autoconf/build-performance.m4 diff --git a/sbin/build.template b/sbin/build.template new file mode 100644 index 000000000..7afdb62b2 --- /dev/null +++ b/sbin/build.template @@ -0,0 +1,49 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# build.template +# +# Writes the configure configuration to disk (config.status) for reuse +# +################################################################################ + +set +e + +alreadyConfigured=$(/usr/bin/find . -name "config.status") + +if [[ ! -z "$alreadyConfigured" ]] ; then + echo "Not reconfiguring due to the presence of config.status" +else + #Templated var that, gets replaced by build.sh + # shellcheck disable=SC1073,SC1054,SC1083 + {configureArg} + + exitCode=$? + if [ "${exitCode}" -ne 0 ]; then + exit 2; + fi +fi + +#Templated var that, gets replaced by build.sh +{makeCommandArg} + +exitCode=$? +# shellcheck disable=SC2181 +if [ "${exitCode}" -ne 0 ]; then + exit 3; +fi diff --git a/sbin/colour-codes.sh b/sbin/colour-codes.sh deleted file mode 100644 index e9a2196fc..000000000 --- a/sbin/colour-codes.sh +++ /dev/null @@ -1,33 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -# Purpose: This script was contains colour codes that will be commonly used across multiple scripts - -# Escape code -esc=$(echo -en "\033") - -# Set colors -# shellcheck disable=SC2034 -error="${esc}[0;31m" -# shellcheck disable=SC2034 -warning="${esc}[0;33m" #yellow -# shellcheck disable=SC2034 -good="${esc}[0;32m" -# shellcheck disable=SC2034 -info="${esc}[0;33m" -# shellcheck disable=SC2034 -git="${esc}[0;34m" -# shellcheck disable=SC2034 -normal=$(echo -en "${esc}[m\017") - diff --git a/sbin/common-functions.sh b/sbin/common-functions.sh deleted file mode 100644 index af5eb7efe..000000000 --- a/sbin/common-functions.sh +++ /dev/null @@ -1,196 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Common functions to be used in scripts - -ALSA_LIB_VERSION=${ALSA_LIB_VERSION:-1.0.27.2} -FREETYPE_FONT_SHARED_OBJECT_FILENAME=libfreetype.so.6.5.0 -FREETYPE_FONT_VERSION=${FREETYPE_FONT_VERSION:-2.4.0} -FREEMARKER_LIB_VERSION=${FREEMARKER_LIB_VERSION:-2.3.8} - -determineBuildProperties() { - JVM_VARIANT=${JVM_VARIANT:-server} - - BUILD_TYPE=normal - DEFAULT_BUILD_FULL_NAME=${OS_KERNEL_NAME}-${OS_MACHINE_NAME}-${BUILD_TYPE}-${JVM_VARIANT}-release - export BUILD_FULL_NAME=${BUILD_FULL_NAME:-"$DEFAULT_BUILD_FULL_NAME"} -} - -# ALSA first for sound -checkingAndDownloadingAlsa() -{ - echo "Checking for ALSA" - - FOUND_ALSA=$(find "${WORKING_DIR}" -name "alsa-lib-${ALSA_LIB_VERSION}") - - if [[ ! -z "$FOUND_ALSA" ]] ; then - echo "Skipping ALSA download" - else - wget -nc ftp://ftp.alsa-project.org/pub/lib/alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - if [[ ${OS_KERNEL_NAME} == "aix" ]] ; then - bzip2 -d alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - tar xf alsa-lib-"${ALSA_LIB_VERSION}".tar - rm alsa-lib-"${ALSA_LIB_VERSION}".tar - else - tar xf alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - rm alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - fi - fi -} - -# Freemarker for OpenJ9 -checkingAndDownloadingFreemarker() -{ - echo "Checking for FREEMARKER" - - FOUND_FREEMARKER=$(find "${WORKING_DIR}" -type d -name "freemarker-${FREEMARKER_LIB_VERSION}") - - if [[ ! -z "$FOUND_FREEMARKER" ]] ; then - echo "Skipping FREEMARKER download" - else - # wget --no-check-certificate "https://sourceforge.net/projects/freemarker/files/freemarker/${FREEMARKER_LIB_VERSION}/freemarker-${FREEMARKER_LIB_VERSION}.tar.gz/download" -O "freemarker-${FREEMARKER_LIB_VERSION}.tar.gz" - # Temp fix as sourceforge is broken - wget --no-check-certificate https://ci.adoptopenjdk.net/userContent/freemarker-2.3.8.tar.gz - tar -xzf "freemarker-${FREEMARKER_LIB_VERSION}.tar.gz" - rm "freemarker-${FREEMARKER_LIB_VERSION}.tar.gz" - fi -} - -checkingAndDownloadingFreeType() -{ - echo "Checking for freetype $WORKING_DIR $OPENJDK_REPO_NAME " - - FOUND_FREETYPE=$(find "$WORKING_DIR/$OPENJDK_REPO_NAME/installedfreetype/lib" -name "${FREETYPE_FONT_SHARED_OBJECT_FILENAME}") - - if [[ ! -z "$FOUND_FREETYPE" ]] ; then - echo "Skipping FreeType download" - else - # Then FreeType for fonts: make it and use - wget -nc http://ftp.acc.umu.se/mirror/gnu.org/savannah/freetype/freetype-"$FREETYPE_FONT_VERSION".tar.gz - if [[ ${OS_KERNEL_NAME} == "aix" ]] ; then - gunzip xf freetype-"$FREETYPE_FONT_VERSION".tar.gz - tar xf freetype-"$FREETYPE_FONT_VERSION".tar - rm freetype-"$FREETYPE_FONT_VERSION".tar - MAKE=gmake - else - tar xf freetype-"$FREETYPE_FONT_VERSION".tar.gz - rm freetype-"$FREETYPE_FONT_VERSION".tar.gz - MAKE=make - fi - - cd freetype-"$FREETYPE_FONT_VERSION" || exit - - # We get the files we need at $WORKING_DIR/installedfreetype - # shellcheck disable=SC2046 - if ! (bash ./configure --prefix="${WORKING_DIR}"/"${OPENJDK_REPO_NAME}"/installedfreetype "${FREETYPE_FONT_BUILD_TYPE_PARAM}" && $MAKE all && $MAKE install); then - # shellcheck disable=SC2154 - echo "${error}Failed to configure and build libfreetype, exiting" - exit; - else - # shellcheck disable=SC2154 - echo "${good}Successfully configured OpenJDK with the FreeType library (libfreetype)!" - - if [[ ${OS_KERNEL_NAME} == "darwin" ]] ; then - TARGET_DYNAMIC_LIB_DIR="${WORKING_DIR}"/"${OPENJDK_REPO_NAME}"/installedfreetype/lib/ - TARGET_DYNAMIC_LIB="${TARGET_DYNAMIC_LIB_DIR}"/libfreetype.6.dylib - echo "" - echo "Listing the contents of ${TARGET_DYNAMIC_LIB_DIR} to see if the dynamic library 'libfreetype.6.dylib' has been created..." - ls "${TARGET_DYNAMIC_LIB_DIR}" - - echo "" - echo "Releasing the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" - set -x - install_name_tool -id @rpath/libfreetype.6.dylib "${TARGET_DYNAMIC_LIB}" - set +x - - # shellcheck disable=SC2181 - if [[ $? == 0 ]]; then - echo "Successfully released the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" - else - echo "Failed to release the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" - fi - fi - fi - # shellcheck disable=SC2154 - echo "${normal}" - fi -} - -checkingAndDownloadCaCerts() -{ - cd "$WORKING_DIR" || exit - - echo "Retrieving cacerts file" - - # Ensure it's the latest we pull in - rm -rf "${WORKING_DIR}/cacerts_area" - - git clone https://github.com/AdoptOpenJDK/openjdk-build.git cacerts_area - echo "cacerts should be here..." - - # shellcheck disable=SC2046 - if ! [ -r "${WORKING_DIR}/cacerts_area/security/cacerts" ]; then - echo "Failed to retrieve the cacerts file, exiting..." - exit; - else - echo "${good}Successfully retrieved the cacerts file!" - fi -} - -downloadingRequiredDependencies() -{ - if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then - echo "Windows or Windows-like environment detected, skipping downloading of dependencies...: Alsa, Freetype, and CaCerts." - else - echo "Downloading required dependencies...: Alsa, Freetype, Freemarker, and CaCerts." - time ( - echo "Checking and download Alsa dependency" - checkingAndDownloadingAlsa - ) - - if [[ -z "${FREETYPE}" ]] ; then - if [[ -z "$FREETYPE_DIRECTORY" ]]; then - time ( - echo "Checking and download FreeType Font dependency" - checkingAndDownloadingFreeType - ) - else - echo "" - echo "---> Skipping the process of checking and downloading the FreeType Font dependency, a pre-built version provided at $FREETYPE_DIRECTORY <---" - echo "" - fi - else - echo "Skipping Freetype" - fi - if [[ "$BUILD_VARIANT" == "openj9" ]]; then - time ( - echo "Checking and download Freemarker dependency" - checkingAndDownloadingFreemarker - ) - fi - time ( - echo "Checking and download CaCerts dependency" - checkingAndDownloadCaCerts - ) - fi -} - -getFirstTagFromOpenJDKGitRepo() -{ - git fetch --tags "${GIT_CLONE_ARGUMENTS[@]}" - justOneFromTheRevList=$(git rev-list --tags --max-count=1) - tagNameFromRepo=$(git describe --tags "$justOneFromTheRevList") - echo "$tagNameFromRepo" -} diff --git a/sbin/common/common.sh b/sbin/common/common.sh new file mode 100755 index 000000000..eee4ababd --- /dev/null +++ b/sbin/common/common.sh @@ -0,0 +1,68 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +# shellcheck disable=SC2153 +function setOpenJdkVersion() { + local forest_name=$1 + + # Derive the openjdk_core_version from the forest name. + local openjdk_core_version=${forest_name} + if [[ ${forest_name} == *u ]]; then + openjdk_core_version=${forest_name%?} + fi + + BUILD_CONFIG[OPENJDK_CORE_VERSION]=$openjdk_core_version; + BUILD_CONFIG[OPENJDK_FOREST_NAME]=$forest_name; + + # 'u' means it's an update repo, e.g. jdk8u + if [[ ${BUILD_CONFIG[OPENJDK_FOREST_NAME]} == *u ]]; then + BUILD_CONFIG[OPENJDK_CORE_VERSION]=${BUILD_CONFIG[OPENJDK_FOREST_NAME]%?} + fi +} +# Create a Tar ball +getArchiveExtension() +{ + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" = *"cygwin"* ]]; then + EXT=".zip" + else + EXT=".tar.gz" + fi + + echo "${EXT}" +} + +# Create a Tar ball +createOpenJDKArchive() +{ + local repoDir="$1" + local fileName="$2" + + COMPRESS=gzip + if which pigz; then + COMPRESS=pigz; + fi + echo "Archiving the build OpenJDK image and compressing with $COMPRESS" + + EXT=$(getArchiveExtension) + + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" = *"cygwin"* ]]; then + zip -r -q "${fileName}.zip" ./"${repoDir}" + elif [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]]; then + GZIP=-9 tar -cf - ./"${repoDir}"/ | $COMPRESS -c > $fileName.tar.gz + else + GZIP=-9 tar --use-compress-program=$COMPRESS -cf "${fileName}.tar.gz" ./"${repoDir}" + fi +} \ No newline at end of file diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh new file mode 100755 index 000000000..d3ac96ba9 --- /dev/null +++ b/sbin/common/config_init.sh @@ -0,0 +1,379 @@ +#!/bin/bash + +################################################################################ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +################################################################################ + +################################################################################ +# +# This shell script deals with writing the AdoptOpenJDK build configuration to +# the file system so it can be picked up by further build steps, Docker +# containers etc +# +# We are deliberately writing to shell because this needs to work on some truly +# esoteric platforms to fulfil the Java Write Once Run Anywhere (WORA) promise. +# +################################################################################ + +# We can't use Bash 4.x+ associative arrays as as Apple won't support bash 4.0 +# (because of GPL3), we therefore have to name the indexes of the CONFIG_PARAMS +# map. This is why we can't have nice things. +CONFIG_PARAMS=( +BRANCH +BUILD_FULL_NAME +BUILD_VARIANT +CERTIFICATE +CLEAN_DOCKER_BUILD +CLEAN_GIT_REPO +CONFIGURE_ARGS_FOR_ANY_PLATFORM +CONTAINER_NAME +COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG +COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG +COPY_TO_HOST +DOCKER +DOCKER_FILE_PATH +DOCKER_SOURCE_VOLUME_NAME +FREETYPE +FREETYPE_DIRECTORY +FREETYPE_FONT_BUILD_TYPE_PARAM +FREETYPE_FONT_VERSION +KEEP_CONTAINER +JDK_BOOT_DIR +JDK_PATH +JRE_PATH +JVM_VARIANT +MAKE_ARGS_FOR_ANY_PLATFORM +MAKE_COMMAND_NAME +NUM_PROCESSORS +OPENJDK_BUILD_NUMBER +OPENJDK_CORE_VERSION +OPENJDK_FOREST_NAME +OPENJDK_SOURCE_DIR +OPENJDK_UPDATE_VERSION +OS_KERNEL_NAME +OS_ARCHITECTURE +REPOSITORY +REUSE_CONTAINER +SHALLOW_CLONE_OPTION +SIGN +TAG +TARGET_DIR +TARGET_FILE_NAME +TMP_CONTAINER_NAME +TMP_SPACE_BUILD +USE_DOCKER +USE_JEP319_CERTS +USE_SSH +USER_SUPPLIED_CONFIGURE_ARGS +WORKING_DIR +WORKSPACE_DIR +) + +# Directory structure of build environment: +########################################################################################################################################### +# Dir Purpose Docker Default Native default +########################################################################################################################################### +# Root /openjdk/ $(pwd)/workspace/ +# /config Configuration /openjdk/config $(pwd)/workspace/config +# / Build area /openjdk/build $(pwd)/workspace/build/ +# // Source code /openjdk/build/src $(pwd)/workspace/build/src +# /target Destination of built artifacts /openjdk/target $(pwd)/workspace/target + +# Helper code to perform index lookups by name +declare -a -x PARAM_LOOKUP +numParams=$((${#CONFIG_PARAMS[@]})) + +# seq not available on aix +index=0 +# shellcheck disable=SC2086 +while [ $index -lt $numParams ]; do + paramName=${CONFIG_PARAMS[$index]}; + eval declare -r -x "$paramName=$index" + PARAM_LOOKUP[$index]=$paramName + + let index=index+1 +done + +function displayParams() { + echo "# ============================" + echo "# OPENJDK BUILD CONFIGURATION:" + echo "# ============================" + for K in "${!BUILD_CONFIG[@]}"; + do + echo "BUILD_CONFIG[${PARAM_LOOKUP[$K]}]=\"${BUILD_CONFIG[$K]}\"" + done | sort +} + +function writeConfigToFile() { + if [ ! -d "workspace/config" ] + then + mkdir -p "workspace/config" + fi + displayParams | sed 's/\r$//' > ./workspace/config/built_config.cfg +} + +function loadConfigFromFile() { + if [ -f "$SCRIPT_DIR/../config/built_config.cfg" ] + then + # shellcheck disable=SC1091,SC1090 + source "$SCRIPT_DIR/../config/built_config.cfg" + elif [ -f "config/built_config.cfg" ] + then + # shellcheck disable=SC1091 + source config/built_config.cfg + elif [ -f "workspace/config/built_config.cfg" ] + then + # shellcheck disable=SC1091 + source workspace/config/built_config.cfg + elif [ -f "built_config.cfg" ] + then + # shellcheck disable=SC1091 + source built_config.cfg + else + echo "Failed to find configuration" + exit + fi +} + +# Parse the configuration args from the CL, please keep this in alpha order +function parseConfigurationArguments() { + + while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do + opt="$1"; + shift; + case "$opt" in + "--" ) break 2;; + + "--build-variant" ) + BUILD_CONFIG[BUILD_VARIANT]="$1"; shift;; + + "--branch" | "-b" ) + BUILD_CONFIG[BRANCH]="$1"; shift;; + + "--build-number" | "-B" ) + BUILD_CONFIG[OPENJDK_BUILD_NUMBER]="$1"; shift;; + + "--configure-args" | "-C" ) + BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]="$1"; shift;; + + "--clean-docker-build" | "-c" ) + BUILD_CONFIG[CLEAN_DOCKER_BUILD]=true;; + + "--clean-git-repo" ) + BUILD_CONFIG[CLEAN_GIT_REPO]=true;; + + "--destination" | "-d" ) + BUILD_CONFIG[TARGET_DIR]="$1"; shift;; + + "--docker" | "-D" ) + BUILD_CONFIG[USE_DOCKER]="true";; + + "--disable-shallow-git-clone" ) + BUILD_CONFIG[SHALLOW_CLONE_OPTION]=""; shift;; + + "--freetype-dir" | "-f" ) + BUILD_CONFIG[FREETYPE_DIRECTORY]="$1"; shift;; + + "--freetype-build-param" ) + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]="$1"; shift;; + + "--freetype-version" ) + BUILD_CONFIG[FREETYPE_FONT_VERSION]="$1"; shift;; + + "--skip-freetype" | "-F" ) + BUILD_CONFIG[FREETYPE]=false;; + + "--help" | "-h" ) + man ./makejdk-any-platform.1;; + + "--ignore-container" | "-i" ) + BUILD_CONFIG[REUSE_CONTAINER]=false;; + + "--jdk-boot-dir" | "-J" ) + BUILD_CONFIG[JDK_BOOT_DIR]="$1";shift;; + + "--keep" | "-k" ) + BUILD_CONFIG[KEEP_CONTAINER]=true;; + + "--processors" | "-p" ) + BUILD_CONFIG[NUM_PROCESSORS]="$1"; shift;; + + "--repository" | "-r" ) + BUILD_CONFIG[REPOSITORY]="$1"; shift;; + + "--source" | "-s" ) + BUILD_CONFIG[WORKING_DIR]="$1"; shift;; + + "--ssh" | "-S" ) + BUILD_CONFIG[USE_SSH]=true;; + + "--sign" ) + BUILD_CONFIG[SIGN]=true; BUILD_CONFIG[CERTIFICATE]="$1"; shift;; + + "--sudo" ) + BUILD_CONFIG[DOCKER]="sudo docker";; + + "--tag" | "-t" ) + BUILD_CONFIG[TAG]="$1"; BUILD_CONFIG[SHALLOW_CLONE_OPTION]=""; shift;; + + "--target-file-name" | "-T" ) + BUILD_CONFIG[TARGET_FILE_NAME]="$1"; shift;; + + "--tmp-space-build") + BUILD_CONFIG[TMP_SPACE_BUILD]=true;; + + "--update-version" | "-u" ) + BUILD_CONFIG[OPENJDK_UPDATE_VERSION]="$1"; shift;; + + "--use-jep319-certs" ) + BUILD_CONFIG[USE_JEP319_CERTS]=true;; + + "--version" | "-v" ) + setOpenJdkVersion "$1"; shift;; + + "--jvm-variant" | "-V" ) + BUILD_CONFIG[JVM_VARIANT]="$1"; shift;; + + *) echo >&2 "Invalid build.sh option: ${opt}"; exit 1;; + esac + done + + setBranch +} + +function setBranch() { + + # Which repo branch to build, e.g. dev by default for hotspot, "openj9" for openj9 + local branch="dev" + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]; then + branch="openj9"; + fi + + BUILD_CONFIG[BRANCH]=${BUILD_CONFIG[BRANCH]:-$branch} +} + +# Set the config defaults +function configDefaults() { + # The OS kernel name, e.g. 'darwin' for Mac OS X + BUILD_CONFIG[OS_KERNEL_NAME]=$(uname | awk '{print tolower($0)}') + + local arch=$(uname -m) + + if [ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]; then + arch=$(uname -p | sed 's/powerpc/ppc/') + fi + + # The O/S architecture, e.g. x86_64 for a modern intel / Mac OS X + BUILD_CONFIG[OS_ARCHITECTURE]=${arch} + + # The full forest name, e.g. jdk8, jdk8u, jdk9, jdk9u, etc. + BUILD_CONFIG[OPENJDK_FOREST_NAME]="" + + # The abridged openjdk core version name, e.g. jdk8, jdk9, etc. + BUILD_CONFIG[OPENJDK_CORE_VERSION]="" + + # The build variant, e.g. openj9 + BUILD_CONFIG[BUILD_VARIANT]="" + + # The OpenJDK source code repository to build from, e.g. an AdoptOpenJDK repo + BUILD_CONFIG[REPOSITORY]="" + + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="false" + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="false" + BUILD_CONFIG[FREETYPE]=true + BUILD_CONFIG[FREETYPE_DIRECTORY]="" + BUILD_CONFIG[FREETYPE_FONT_VERSION]="2.4.0" + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]="" + + BUILD_CONFIG[MAKE_COMMAND_NAME]="make" + BUILD_CONFIG[SIGN]="false" + BUILD_CONFIG[JDK_BOOT_DIR]="" + + BUILD_CONFIG[NUM_PROCESSORS]="1" + BUILD_CONFIG[TARGET_FILE_NAME]="OpenJDK" + + # Dir where we clone the OpenJDK source code for building, defaults to 'src' + BUILD_CONFIG[OPENJDK_SOURCE_DIR]="src" + + # By default only git clone the HEAD commit + BUILD_CONFIG[SHALLOW_CLONE_OPTION]=${BUILD_CONFIG[SHALLOW_CLONE_OPTION]:-"--depth=1"} + + # Set Docker Container names and defaults + BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]=${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]:-"openjdk-source-volume"} + + BUILD_CONFIG[CONTAINER_NAME]=${BUILD_CONFIG[CONTAINER_NAME]:-openjdk_container} + + BUILD_CONFIG[TMP_CONTAINER_NAME]=${BUILD_CONFIG[TMP_CONTAINER_NAME]:-openjdk-copy-src} + BUILD_CONFIG[CLEAN_DOCKER_BUILD]=${BUILD_CONFIG[CLEAN_DOCKER_BUILD]:-false} + + # Use Docker to build (defaults to false) + BUILD_CONFIG[USE_DOCKER]=${BUILD_CONFIG[USE_DOCKER]:-false} + + # Location of DockerFile and where scripts get copied to inside the container + BUILD_CONFIG[DOCKER_FILE_PATH]=${BUILD_CONFIG[DOCKER_FILE_PATH]:-""} + + # Whether we keep the Docker container after we build it + # TODO Please note that the persistent volume is managed separately + BUILD_CONFIG[KEEP_CONTAINER]=${BUILD_CONFIG[KEEP_CONTAINER]:-false} + + # Whether we use an existing container + # TODO Please note that the persistent volume is managed separately + BUILD_CONFIG[REUSE_CONTAINER]=${BUILD_CONFIG[REUSE_CONTAINER]:-true} + + # The current working directory + BUILD_CONFIG[WORKING_DIR]=${BUILD_CONFIG[WORKING_DIR]:-"./build/"} + + # Root of the workspace + BUILD_CONFIG[WORKSPACE_DIR]=${BUILD_CONFIG[WORKSPACE_DIR]:-""} + + # Use SSH for the GitHub connection (defaults to false) + BUILD_CONFIG[USE_SSH]=${BUILD_CONFIG[USE_SSH]:-false} + + # Director where OpenJDK binary gets built to + BUILD_CONFIG[TARGET_DIR]=${BUILD_CONFIG[TARGET_DIR]:-"target/"} + + + # Which repo tag to build, e.g. jdk8u172-b03 + BUILD_CONFIG[TAG]=${BUILD_CONFIG[TAG]:-""} + + # Update version e.g. 172 + BUILD_CONFIG[OPENJDK_UPDATE_VERSION]=${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]:-""} + + # build number e.g. b03 + BUILD_CONFIG[OPENJDK_BUILD_NUMBER]=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]:-""} + + # Build variant, e.g. openj9, defaults to "" which means hotspot + BUILD_CONFIG[BUILD_VARIANT]=${BUILD_CONFIG[BUILD_VARIANT]:-""} + + # JVM variant, e.g. client or server, defaults to server + BUILD_CONFIG[JVM_VARIANT]=${BUILD_CONFIG[JVM_VARIANT]:-""} + + # Any extra args provided by the user + BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]=${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]:-""} + + BUILD_CONFIG[DOCKER]=${BUILD_CONFIG[DOCKER]:-"docker"} + + BUILD_CONFIG[TMP_SPACE_BUILD]=${BUILD_CONFIG[TMP_SPACE_BUILD]:-false} + + # If the wrong git repo is there allow it to be removed + BUILD_CONFIG[CLEAN_GIT_REPO]=false + + # By default dont backport JEP318 certs to < Java 10 + BUILD_CONFIG[USE_JEP319_CERTS]=false +} + +# Declare the map of build configuration that we're going to use +declare -ax BUILD_CONFIG +export BUILD_CONFIG diff --git a/sbin/common/constants.sh b/sbin/common/constants.sh new file mode 100755 index 000000000..95e114850 --- /dev/null +++ b/sbin/common/constants.sh @@ -0,0 +1,28 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +export JDK8_VERSION="jdk8u"; +export JDK9_VERSION="jdk9u"; +export JDK10_VERSION="jdk10u"; +export JDK11_VERSION="jdk11"; +export JDKHEAD_VERSION="jdk"; + +export JDK8_CORE_VERSION="jdk8"; +export JDK9_CORE_VERSION="jdk9"; +export JDK10_CORE_VERSION="jdk10"; +export JDK11_CORE_VERSION="jdk11"; +export JDKHEAD_CORE_VERSION="${JDKHEAD_VERSION}"; +export AMBER_CORE_VERSION="amber"; \ No newline at end of file diff --git a/sbin/jtreg.sh b/sbin/jtreg.sh deleted file mode 100755 index e355b7727..000000000 --- a/sbin/jtreg.sh +++ /dev/null @@ -1,170 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# # -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck source=sbin/common-functions.sh -source "$SCRIPT_DIR/common-functions.sh" - -WORKING_DIR=$1 -OPENJDK_REPO_NAME=$2 -BUILD_FULL_NAME=$3 -VERSION=$4 -# shellcheck disable=SC2001 -JTREG_TEST_SUBSETS=$(echo "$5" | sed 's/:/ /') -JTREG_VERSION=${JTREG_VERSION:-4.2.0-tip} -JTREG_TARGET_FOLDER=${JTREG_TARGET_FOLDER:-jtreg} -JOB_NAME=${JOB_NAME:-OpenJDK} -NUM_PROCESSORS=${NUM_PROCESSORS:-$(getconf _NPROCESSORS_ONLN)} -TMP_DIR=$(dirname "$(mktemp -u)") -OPENJDK_DIR="$WORKING_DIR/$OPENJDK_REPO_NAME" -TARGET_DIR="$WORKING_DIR" - -determineBuildProperties - -checkIfDockerIsUsedForBuildingOrNot() -{ - if [[ -f /.dockerenv ]] ; then - echo "Detected we're in docker" - WORKING_DIR=/openjdk/build/ - OPENJDK_REPO_NAME=openjdk/ - OPENJDK_DIR="$WORKING_DIR/$OPENJDK_REPO_NAME" - TARGET_DIR=/openjdk/target/ - # Keep as a variable for potential use later - # if we wish to copy the results to the host - # shellcheck disable=SC2034 - IN_DOCKER=true - fi -} - -downloadJtregAndSetupEnvironment() -{ - # Download then add jtreg to our path - if [[ ! -d "${WORKING_DIR}/${JTREG_TARGET_FOLDER}" ]]; then - echo "Downloading Jtreg binary" - JTREG_BINARY_FILE="jtreg-${JTREG_VERSION}.tar.gz" - - # shellcheck disable=SC2046 - if ! (cd "$TMP_DIR" && wget https://ci.adoptopenjdk.net/job/jtreg/lastSuccessfulBuild/artifact/"$JTREG_BINARY_FILE"); then - echo "Failed to retrieve the jtreg binary, exiting" - exit - fi - - - cd "$WORKING_DIR" && tar xf "$TMP_DIR/$JTREG_BINARY_FILE" - fi - - echo "List contents of the jtreg folder" - ls "$WORKING_DIR/$JTREG_TARGET_FOLDER"/* - - export JT_HOME=$WORKING_DIR/$JTREG_TARGET_FOLDER - - export PATH=$JT_HOME/bin:$PATH - - # Clean up after ourselves by removing jtreg tgz - rm -f "$JTREG_BINARY_FILE" -} - -applyingJCovSettingsToMakefileForTests() -{ - echo "Apply JCov settings to Makefile..." - if [[ $VERSION == *8* ]]; then - cd "$OPENJDK_DIR/jdk/test" || exit - sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/jre\/lib\/rt.jar -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/src.zip -jcov\/include:*/" Makefile - else - cd "$OPENJDK_DIR/test" || exit - # TODO pass in correct jcov parameter for jdk9 and up - # sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/jdk\/classes\/ -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/..\/..\/jdk\/src\/share\/classes -jcov\/include:*/" TestCommon.gmk - fi - pwd - cd "$OPENJDK_DIR" || exit -} - -settingUpEnvironmentVariablesForJTREG() -{ - echo "Setting up environment variables for JTREG to run" - - # This is the JDK we'll test - export PRODUCT_HOME=$OPENJDK_DIR/build/$BUILD_FULL_NAME/images/j2sdk-image - echo "$PRODUCT_HOME" - ls "$PRODUCT_HOME" - - export JTREG_DIR=$WORKING_DIR/jtreg - export JTREG_INSTALL=${JTREG_DIR} - export JT_HOME=${JTREG_INSTALL} - export JTREG_HOME=${JTREG_INSTALL} - export JPRT_JTREG_HOME=${JT_HOME} - export JPRT_JAVA_HOME=${PRODUCT_HOME} - export JTREG_TIMEOUT_FACTOR=5 - export CONCURRENCY=$NUM_PROCESSORS -} - -runJtregViaMakeCommand() -{ - echo "Running jtreg via make command (debug logs enabled)" - if [ -z "$JTREG_TEST_SUBSETS" ]; then - make test jobs=10 LOG=debug - else - make test jobs=10 LOG=debug TEST="$JTREG_TEST_SUBSETS" - fi -} - -packageTestResultsWithJCovReports() -{ - echo "Package test output into archives..." - pwd - - cd "$OPENJDK_DIR/build/$BUILD_FULL_NAME/" || exit - - artifact="${JOB_NAME}-testoutput-with-jcov-reports" - echo "Tarring and zipping the 'testoutput' folder into artefact: $artifact.tar.gz" - tar -czf "$TARGET_DIR/$artifact.tar.gz" testoutput/ - - if [ -d testoutput ]; then - rm -fr "$WORKING_DIR/$OPENJDK_REPO_NAME/testoutput" - fi - cp -fr testoutput/ "$WORKING_DIR/testoutput/" - - cd "$WORKING_DIR" || exit -} - -packageOnlyJCovReports() -{ - echo "Package jcov reports into archives..." - pwd - - cd "$OPENJDK_DIR/build/$BUILD_FULL_NAME/" || exit - pwd - - artifact="${JOB_NAME}-jcov-results-only" - echo "Tarring and zipping the 'testoutput/../jcov' folder into artefact: $artifact.tar.gz" - tar -czf "$TARGET_DIR/$artifact.tar.gz" testoutput/*/JTreport/jcov/ - - cd "$WORKING_DIR" || exit -} - -packageReports() -{ - echo "Archiving your jtreg results (includes jcov reports)" - packageTestResultsWithJCovReports - packageOnlyJCovReports -} - -checkIfDockerIsUsedForBuildingOrNot -downloadingRequiredDependencies -downloadJtregAndSetupEnvironment -applyingJCovSettingsToMakefileForTests -settingUpEnvironmentVariablesForJTREG -runJtregViaMakeCommand -packageReports diff --git a/sbin/jtreg_prep.sh b/sbin/jtreg_prep.sh deleted file mode 100755 index 7e0e3e58b..000000000 --- a/sbin/jtreg_prep.sh +++ /dev/null @@ -1,169 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Purpose: This script was designed to do any+all setup required by the jtreg.sh script in order to run it. -# Tasks: Retrieve Java, unpack it if need-be, and store it locally in a specific location. If the location is blank, we put it in - -set -eu - -REPOSITORY=adoptopenjdk/openjdk-jdk8u -OPENJDK_REPO_NAME=openjdk -SHALLOW_CLONE_OPTION="--depth=1" - -JAVA_SOURCE="" -JAVA_DESTINATION="" -WORKING_DIR="" -USE_SSH=false -BRANCH="" - -parseCommandLineArgs() -{ - while [[ $# -gt 0 ]] && [[ ."$1" == .-* ]] ; do - opt="$1"; - shift; - case "$opt" in - "--" ) break 2;; - - "--source" ) - JAVA_SOURCE="$1"; shift;; - - "--destination" ) - JAVA_DESTINATION="$1"; shift;; - - "--ssh" | "-S" ) - USE_SSH=true;; - - "--repository" | "-r" ) - REPOSITORY="$1"; shift;; - - "--branch" | "-b" ) - BRANCH="$1"; shift;; - - "--working_dir" ) - WORKING_DIR="$1"; shift;; - - "--disable-shallow-git-clone" | "-dsgc" ) - SHALLOW_CLONE_OPTION=""; shift;; - - *) echo >&2 "Invalid option: ${opt}"; echo "This option was unrecognised. See the script jtreg_prep.sh for a full list."; exit 1;; - esac - done - - if [ -z "${JAVA_SOURCE}" ]; then - echo >&2 "jtreg_prep.sh failed: --source must be specified"; exit 1 - fi - - if [ -z "${WORKING_DIR}" ] ; then - echo "WORKING_DIR is undefined so setting to $PWD" - WORKING_DIR=$PWD - else - echo "Working dir is $WORKING_DIR" - fi - - if [ -z "${JAVA_DESTINATION}" ]; then - JAVA_DESTINATION="$WORKING_DIR/$OPENJDK_REPO_NAME/build/java_home/images" - fi - - if [ -z "${BRANCH}" ] ; then - echo "BRANCH is undefined so checking out dev" - BRANCH="dev" - fi -} - -# Step 1: Fetch OpenJDK, as that's where the tests live. -cloneOpenJDKRepo() -{ - if [ -d "$WORKING_DIR/$OPENJDK_REPO_NAME/.git" ] && [ "$REPOSITORY" == "adoptopenjdk/openjdk-jdk8u" ] ; then - # It does exist and it's a repo other than the AdoptOpenJDK one - cd "$WORKING_DIR/$OPENJDK_REPO_NAME" || exit - echo "Will reset the repository at $PWD in 10 seconds..." - sleep 10 - echo "Pulling latest changes from git repo" - git fetch --all - git reset --hard origin/$BRANCH - cd "$WORKING_DIR" || exit - elif [ ! -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] ; then - # If it doesn't exist, clone it - echo "Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" - if [[ "$USE_SSH" == "true" ]] ; then - GIT_REMOTE_REPO_ADDRESS="git@github.com:${REPOSITORY}.git" - else - GIT_REMOTE_REPO_ADDRESS="https://github.com/${REPOSITORY}.git" - fi - - if [[ "$SHALLOW_CLONE_OPTION" == "" ]]; then - echo "Git repo cloning mode: deep (preserves commit history)" - else - echo "Git repo cloning mode: shallow (DOES NOT contain commit history)" - fi - - GIT_CLONE_ARGUMENTS=("$SHALLOW_CLONE_OPTION" '-b' "$BRANCH" "$GIT_REMOTE_REPO_ADDRESS" "${WORKING_DIR}/${OPENJDK_REPO_NAME}") - - echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" - git clone "${GIT_CLONE_ARGUMENTS[@]}" - fi -} - -# Step 2: Retrieve Java -downloadJDKArtifact() -{ - if [ ! -d "${JAVA_DESTINATION}" ]; then - mkdir -p "${JAVA_DESTINATION}" - fi - cd "${JAVA_DESTINATION}" || exit - - #If it's a http location, use wget. - if [[ "${JAVA_SOURCE}" == http* ]]; then - # shellcheck disable=SC2046 - if ! (wget "${JAVA_SOURCE}"); then - echo "Failed to retrieve the JDK binary from ${JAVA_SOURCE}, exiting" - exit 1 - fi - else #Assume it's local or on a mounted drive. - if [ -f "${JAVA_SOURCE}" ] || [ -d "${JAVA_SOURCE}" ]; then - cp -r "${JAVA_SOURCE}" . - else - echo "The JDK artifact could not be found at the source location: ${JAVA_SOURCE}." - exit 1 - fi - fi -} - -# Step 3: Unpack Java if we need to. -unpackJDKTarArtifact() -{ - if [[ "$JAVA_SOURCE" == *\.tar\.gz ]]; then #If it's a tar file, unpack it. - cd "${JAVA_DESTINATION}" || exit - tar xf ./*.tar.gz - echo "The JDK artifact has been untarred at ${JAVA_DESTINATION}." - cd "${WORKING_DIR}" || exit - elif [ ! -d "${JAVA_SOURCE}" ]; then #If it's not a directory, then we don't know how to unpack it. - echo "The Java file you specified as source was copied to the destination, but this script doesn't know how to unpack it. Please add this logic to this script, or unpack it manually before running jtreg."; - fi -} - -# Step 4: Finish -printFinishedMessage() -{ - echo "$0 has finished successfully." -} - -################################################################## - -parseCommandLineArgs "$@" -cloneOpenJDKRepo -downloadJDKArtifact -unpackJDKTarArtifact -printFinishedMessage diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh new file mode 100755 index 000000000..c81828d60 --- /dev/null +++ b/sbin/prepareWorkspace.sh @@ -0,0 +1,356 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# This script prepares the workspace to build (Adopt) OpenJDK. +# See the configureWorkspace function for details +# It is sourced by build.sh +# +################################################################################ + +set -eu + + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/common/constants.sh" + +# Set default versions for 3 libraries that OpenJDK relies on to build +ALSA_LIB_VERSION=${ALSA_LIB_VERSION:-1.1.6} +FREETYPE_FONT_SHARED_OBJECT_FILENAME="libfreetype.so*" +FREEMARKER_LIB_VERSION=${FREEMARKER_LIB_VERSION:-2.3.28} + +# Create a new clone or update the existing clone of the OpenJDK source repo +# TODO refactor this for SRP +checkoutAndCloneOpenJDKGitRepo() +{ + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" + + # Check that we have a git repo of a valid openjdk version on our local file system + if [ -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ] && ( [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK10_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK11_CORE_VERSION}" ]) ; then + set +e + git --git-dir "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" remote -v + echo "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" + git --git-dir "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" remote -v | grep "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" | grep --quiet "${BUILD_CONFIG[REPOSITORY]}" + local isCorrectGitRepo=$? + set -e + + # If the local copy of the git source repo is valid then we reset appropriately + if [ "${isCorrectGitRepo}" == "0" ]; then + cd "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return + echo "Resetting the git openjdk source repository at $PWD in 10 seconds..." + sleep 10 + echo "Pulling latest changes from git openjdk source repository" + git fetch --all ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + git reset --hard "origin/${BUILD_CONFIG[BRANCH]}" + if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then + git checkout "${BUILD_CONFIG[TAG]}" + fi + git clean -ffdx + elif [ "${BUILD_CONFIG[CLEAN_GIT_REPO]}" == "true" ]; then + echo "Removing current git repo as it is the wrong type" + rm -rf "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + cloneOpenJDKGitRepo + else + echo "Incorrect Source Code for ${BUILD_CONFIG[OPENJDK_FOREST_NAME]}. This is an error, please check what is in $PWD and manually remove, exiting..." + echo "If this is inside a docker you can purge the existing source by passing --clean-docker-build" + exit 1 + fi + elif [ ! -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ] ; then + # If it doesn't exist, clone it + echo "Didn't find any existing openjdk repository at $(pwd)/${BUILD_CONFIG[WORKING_DIR]} so cloning the source to openjdk" + cloneOpenJDKGitRepo + fi + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + git remote set-branches --add origin "${BUILD_CONFIG[BRANCH]}" + git fetch --all ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + git reset --hard "origin/${BUILD_CONFIG[BRANCH]}" + if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then + git checkout "${BUILD_CONFIG[TAG]}" + fi + git clean -ffdx + + updateOpenj9Sources + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" +} + +# Set the git clone arguments +setGitCloneArguments() { + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" + local git_remote_repo_address; + if [[ "${BUILD_CONFIG[USE_SSH]}" == "true" ]] ; then + git_remote_repo_address="git@github.com:${BUILD_CONFIG[REPOSITORY]}.git" + else + git_remote_repo_address="https://github.com/${BUILD_CONFIG[REPOSITORY]}.git" + fi + + GIT_CLONE_ARGUMENTS=(${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} '-b' "${BUILD_CONFIG[BRANCH]}" "$git_remote_repo_address" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}") +} + +updateOpenj9Sources() { + # Building OpenJDK with OpenJ9 must run get_source.sh to clone openj9 and openj9-omr repositories + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]; then + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return + bash get_source.sh + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" + fi +} +# Clone the git repo +cloneOpenJDKGitRepo() +{ + setGitCloneArguments + + echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" + git clone "${GIT_CLONE_ARGUMENTS[@]}" +} + +# Create the workspace +createWorkspace() +{ + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}" || exit + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit +} + +# ALSA first for sound +checkingAndDownloadingAlsa() +{ + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + + echo "Checking for ALSA" + + FOUND_ALSA=$(find "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/" -name "installedalsa") + + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa/" || exit + + if [[ ! -z "$FOUND_ALSA" ]] + then + echo "Skipping ALSA download" + else + # TODO Holy security problem Batman! + #wget -nc ftp://ftp.alsa-project.org/pub/lib/alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + wget -nc https://ftp.osuosl.org/pub/blfs/conglomeration/alsa-lib/alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]] ; then + bzip2 -d alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + tar -xf alsa-lib-"${ALSA_LIB_VERSION}".tar --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa/" + rm alsa-lib-"${ALSA_LIB_VERSION}".tar + else + tar -xf alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa/" + rm alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + fi + fi +} + +# Freemarker for OpenJ9 +checkingAndDownloadingFreemarker() +{ + echo "Checking for FREEMARKER" + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/" || exit + FOUND_FREEMARKER=$(find "." -type d -name "freemarker-${FREEMARKER_LIB_VERSION}") + + if [[ ! -z "$FOUND_FREEMARKER" ]] ; then + echo "Skipping FREEMARKER download" + else + + wget -nc --no-check-certificate "https://www.mirrorservice.org/sites/ftp.apache.org/freemarker/engine/${FREEMARKER_LIB_VERSION}/binaries/apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/" || exit + tar -xzf "apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/" + rm "apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" + fi +} + +# Get Freetype +checkingAndDownloadingFreeType() +{ + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + echo "Checking for freetype at ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" + + FOUND_FREETYPE=$(find "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedfreetype/lib" -name "${FREETYPE_FONT_SHARED_OBJECT_FILENAME}" || true) + + if [[ ! -z "$FOUND_FREETYPE" ]] ; then + echo "Skipping FreeType download" + else + curl -L -o "freetype.tar.gz" "https://download.savannah.gnu.org/releases/freetype/freetype-${BUILD_CONFIG[FREETYPE_FONT_VERSION]}.tar.gz" + + rm -rf "./freetype" || true + mkdir -p "freetype" || true + tar xpzf freetype.tar.gz --strip-components=1 -C "freetype" + rm freetype.tar.gz + + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then + return; + fi + + cd freetype || exit + + + # We get the files we need at $WORKING_DIR/installedfreetype + # shellcheck disable=SC2046 + if ! (bash ./configure --prefix="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}"/installedfreetype "${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]}" && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} all && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} install); then + # shellcheck disable=SC2154 + echo "Failed to configure and build libfreetype, exiting" + exit; + else + # shellcheck disable=SC2154 + echo "Successfully configured OpenJDK with the FreeType library (libfreetype)!" + + if [[ ${BUILD_CONFIG[OS_KERNEL_NAME]} == "darwin" ]] ; then + TARGET_DYNAMIC_LIB_DIR="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}"/installedfreetype/lib/ + TARGET_DYNAMIC_LIB="${TARGET_DYNAMIC_LIB_DIR}"/libfreetype.6.dylib + + echo "Listing the contents of ${TARGET_DYNAMIC_LIB_DIR} to see if the dynamic library 'libfreetype.6.dylib' has been created..." + ls "${TARGET_DYNAMIC_LIB_DIR}" + + echo "Releasing the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" + install_name_tool -id @rpath/libfreetype.6.dylib "${TARGET_DYNAMIC_LIB}" + + # shellcheck disable=SC2181 + if [[ $? == 0 ]]; then + echo "Successfully released the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" + else + echo "Failed to release the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" + fi + fi + fi + fi +} + +# Certificate Authority Certs (CA Certs) +checkingAndDownloadCaCerts() +{ + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit + + echo "Retrieving cacerts file" + # Ensure it's the latest we pull in + rm -rf "cacerts_area" + mkdir "cacerts_area" || exit + cd "cacerts_area" || exit + + if [ "${BUILD_CONFIG[USE_JEP319_CERTS]}" == "true" ]; + then + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] + then + echo "Requested use of JEP319 certs" + local caLink="https://github.com/AdoptOpenJDK/openjdk-jdk10u/blob/dev/src/java.base/share/lib/security/cacerts?raw=true"; + mkdir -p "security" + curl -L -o "./security/cacerts" "${caLink}" + fi + else + git init + git remote add origin -f https://github.com/AdoptOpenJDK/openjdk-build.git + git config core.sparsecheckout true + echo "security/*" >> .git/info/sparse-checkout + git pull origin master + fi + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit +} + +# Download all of the dependencies for OpenJDK (Alsa, FreeType, CACerts et al) +downloadingRequiredDependencies() +{ + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then + echo "Windows or Windows-like environment detected, skipping downloading of dependencies...: Alsa." + else + echo "Downloading required dependencies...: Alsa, Freetype, Freemarker, and CaCerts." + echo "Checking and download Alsa dependency" + checkingAndDownloadingAlsa + + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]]; then + echo "Checking and download Freemarker dependency" + checkingAndDownloadingFreemarker + fi + fi + + if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then + if [ -z "${BUILD_CONFIG[FREETYPE_DIRECTORY]}" ]; then + echo "Checking and download FreeType Font dependency" + checkingAndDownloadingFreeType + else + echo "" + echo "---> Skipping the process of checking and downloading the FreeType Font dependency, a pre-built version provided at ${BUILD_CONFIG[FREETYPE_DIRECTORY]} <---" + echo "" + fi + else + echo "Skipping Freetype" + fi + + echo "Checking and download CaCerts dependency" + checkingAndDownloadCaCerts + +} + +function moveTmpToWorkspaceLocation { + if [ ! -z "${TMP_WORKSPACE}" ]; then + + echo "Relocating workspace from ${TMP_WORKSPACE} to ${ORIGINAL_WORKSPACE}" + + rm -rf "${ORIGINAL_WORKSPACE}" || true + + mkdir "${ORIGINAL_WORKSPACE}" + + # shellcheck disable=SC2086 + chmod 755 ${TMP_WORKSPACE}/workspace/* || true + chmod 755 "${ORIGINAL_WORKSPACE}" || true + + # shellcheck disable=SC2086 + cp -r ${TMP_WORKSPACE}/workspace/* "${ORIGINAL_WORKSPACE}" + rm -rf "${TMP_WORKSPACE}/workspace/" + rm -r "${TMP_WORKSPACE}" + + echo "Data at: ${ORIGINAL_WORKSPACE}" + ls -alh "${ORIGINAL_WORKSPACE}" + fi +} + + +relocateToTmpIfNeeded() +{ + if [ "${BUILD_CONFIG[TMP_SPACE_BUILD]}" == "true" ] + then + local tmpdir + tmpdir=$(mktemp -d 2>/dev/null || mktemp -d -t 'tmpdir') + export TMP_WORKSPACE="${tmpdir}" + export ORIGINAL_WORKSPACE="${BUILD_CONFIG[WORKSPACE_DIR]}" + + trap moveTmpToWorkspaceLocation EXIT SIGINT SIGTERM + + if [ -d "${ORIGINAL_WORKSPACE}" ] + then + cp -r "${BUILD_CONFIG[WORKSPACE_DIR]}" "${TMP_WORKSPACE}/workspace" + fi + BUILD_CONFIG[WORKSPACE_DIR]="${TMP_WORKSPACE}/workspace" + + fi +} + +################################################################## + +function configureWorkspace() { + createWorkspace + downloadingRequiredDependencies + relocateToTmpIfNeeded + checkoutAndCloneOpenJDKGitRepo +} + diff --git a/security/GenerateCertsFile.sh b/security/GenerateCertsFile.sh index 40611070c..97f77c936 100644 --- a/security/GenerateCertsFile.sh +++ b/security/GenerateCertsFile.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sign.sh b/sign.sh new file mode 100755 index 000000000..90dfa19e9 --- /dev/null +++ b/sign.sh @@ -0,0 +1,137 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +set -eu + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# shellcheck source=sbin/common/config_init.sh +source "$SCRIPT_DIR/sbin/common/config_init.sh" + +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/sbin/common/constants.sh" + +# shellcheck source=sbin/common/common.sh +source "$SCRIPT_DIR/sbin/common/common.sh" + +ARCHIVE="" +SIGNING_CERTIFICATE="" +WORKSPACE=$(pwd) +TMP_DIR_NAME="tmp" +TMP_DIR="${WORKSPACE}/${TMP_DIR_NAME}/" + +checkSignConfiguration() { + if [[ "${OPERATING_SYSTEM}" == "windows" ]] ; then + if [ ! -f "${SIGNING_CERTIFICATE}" ] + then + echo "Could not find certificate at: ${SIGNING_CERTIFICATE}" + exit 1 + fi + + if [ -z "${SIGN_PASSWORD+x}" ] + then + echo "If signing is enabled on window you must set SIGN_PASSWORD" + exit 1 + fi + fi +} + +# Sign the built binary +signRelease() +{ + case "$OPERATING_SYSTEM" in + "windows") + echo "Signing Windows release" + signToolPath=${signToolPath:-"/cygdrive/c/Program Files/Microsoft SDKs/Windows/v7.1/Bin/signtool.exe"} + + # Sign .exe files + FILES=$(find . -type f -name '*.exe') + echo "$FILES" | while read -r f; + do + echo "Signing ${f}" + "$signToolPath" sign /f "${SIGNING_CERTIFICATE}" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; + done + + # Sign .dll files + FILES=$(find . -type f -name '*.dll') + echo "$FILES" | while read -r f; + do + "$signToolPath" sign /f "${SIGNING_CERTIFICATE}" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; + done + ;; + "mac"*) + echo "Signing OSX release" + # Login to KeyChain + # shellcheck disable=SC2046 + # shellcheck disable=SC2006 + security unlock-keychain -p `cat ~/.password` + # Sign all files with the executable permission bit set. + FILES=$(find "${TMP_DIR}" -perm +111 -type f || find "${TMP_DIR}" -perm /111 -type f) + echo "$FILES" | while read -r f; do codesign -s "Developer ID Application: London Jamocha Community CIC" "$f"; done + ;; + *) + echo "Skipping code signing as it's not supported on $OPERATING_SYSTEM" + ;; + esac +} + +function parseArguments() { + parseConfigurationArguments "$@" + + while [[ $# -gt 2 ]] ; do + shift; + done + + SIGNING_CERTIFICATE="$1"; + ARCHIVE="$2"; +} + +function extractArchive { + rm -rf "${TMP_DIR}" || true + mkdir "${TMP_DIR}" + if [[ "${OPERATING_SYSTEM}" == "windows" ]]; then + unzip "${ARCHIVE}" -d "${TMP_DIR}" + elif [[ "${OPERATING_SYSTEM}" == "mac" ]]; then + gunzip -dc "${ARCHIVE}" | tar xf - -C "${TMP_DIR}" + else + echo "could not detect archive type" + exit 1 + fi +} + +if [ "${OPERATING_SYSTEM}" != "windows" ] && [ "${OPERATING_SYSTEM}" != "mac" ]; then + echo "Skipping code signing as it's not supported on ${OPERATING_SYSTEM}" + exit 0; +fi + +configDefaults +parseArguments "$@" +extractArchive + +# shellcheck disable=SC2012 +jdkDir=$(find "${TMP_DIR}" ! -path "${TMP_DIR}" -type d -exec basename {} \; | head -n1) + +cd "${TMP_DIR}/${jdkDir}" || exit 1 +signRelease + +cd "${TMP_DIR}" +createOpenJDKArchive "${jdkDir}" "OpenJDK" +archiveExtension=$(getArchiveExtension) +signedArchive="${TMP_DIR}/OpenJDK${archiveExtension}" + +cd "${WORKSPACE}" +mv "${signedArchive}" "${ARCHIVE}" +rm -rf "${TMP_DIR}" diff --git a/sbin/signalhandler.sh b/signalhandler.sh similarity index 62% rename from sbin/signalhandler.sh rename to signalhandler.sh index e8f3ecdf5..bcb04f2e0 100755 --- a/sbin/signalhandler.sh +++ b/signalhandler.sh @@ -1,10 +1,12 @@ #!/bin/bash + +################################################################################ # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -12,18 +14,13 @@ # See the License for the specific language governing permissions and # limitations under the License. # - -# For terminal colors -esc=$(echo -en "\033") -error="${esc}[0;31m" #red -normal=$(echo -en "${esc}[m\017") - +################################################################################ exit_script() { - if [[ -z ${KEEP} ]] ; then - docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER" | awk '{print $1 }' | xargs -I {} docker rm -f {} + if [[ -z ${KEEP_CONTAINER} ]] ; then + docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER_NAME" | awk '{print $1 }' | xargs -I {} docker rm -f {} fi - echo "${error}Process exited${normal}" + echo "Process exited" trap - SIGINT SIGTERM # clear the trap kill -- -$$ # Sends SIGTERM to child/sub processes }