Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installer windows #386

Merged
merged 65 commits into from Mar 28, 2024
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
7576290
don't create coverage and logs directory on install
karottenreibe Dec 19, 2023
0638508
rename linux tests
karottenreibe Dec 19, 2023
ee68cc5
implement windows environment variable step
karottenreibe Dec 19, 2023
49dc75c
add test for WindowsRegistry and document code
karottenreibe Dec 19, 2023
43896e5
add Windows test and refactor existing tests
karottenreibe Dec 20, 2023
fe25c08
add windows build
karottenreibe Dec 20, 2023
6ec4999
fix TS findings
karottenreibe Dec 20, 2023
ed160d4
fix linux tests
karottenreibe Dec 20, 2023
7188cbf
advise Windows users to restart after installation
karottenreibe Dec 20, 2023
02ee449
rename windows build step to "Test"
karottenreibe Dec 20, 2023
3f329e9
fix warning and improve error message
karottenreibe Dec 20, 2023
2085016
disable problematic test on Windows
karottenreibe Dec 20, 2023
db232f0
provide nice asserts for error reporter
karottenreibe Dec 20, 2023
6c93f90
add utility to mark directories as not writable during tests
karottenreibe Dec 20, 2023
d3e5f3a
convert to jlink build
karottenreibe Dec 20, 2023
d6457c7
include jlink installer in agent dist
karottenreibe Dec 20, 2023
b0acde9
remove incorrect test assertion
karottenreibe Dec 20, 2023
3d83554
fix TS finding
karottenreibe Dec 20, 2023
ef63bca
use normal java instead of GraalVM
karottenreibe Dec 20, 2023
0c4fcb4
update okhttp
karottenreibe Dec 20, 2023
db7c155
use jdk 17 also in docker build
karottenreibe Dec 20, 2023
1e463ae
fix docker build
karottenreibe Dec 20, 2023
2a9713a
add platform to coverage upload message
karottenreibe Dec 20, 2023
efa88e0
use java 17 in build
karottenreibe Dec 20, 2023
c0a2ca0
fix incorrect test enablement
karottenreibe Dec 20, 2023
b11e903
fix quoting jvm options
karottenreibe Dec 20, 2023
c006d38
do not strip debug info so we get line numbers in stack traces
karottenreibe Dec 20, 2023
ef491d0
try fixing windows tests that depend on read-only file
karottenreibe Dec 20, 2023
f0b312b
prevent verbose error messages
karottenreibe Dec 20, 2023
fcdf2b7
add tests for common Teamscale communication errors
karottenreibe Dec 20, 2023
668cc15
don't log useless stack traces of fatal error causes
karottenreibe Dec 20, 2023
201d33f
move utilities to their own package
karottenreibe Dec 20, 2023
5bc8e56
fix broken test
karottenreibe Dec 20, 2023
3ea0189
another attempt at making a directory read-only on Windows
karottenreibe Dec 20, 2023
a98463c
move test to correct package
karottenreibe Dec 20, 2023
74a2cd9
try fixing tests
karottenreibe Dec 21, 2023
6917bff
simplify marking windows files as read-only
karottenreibe Dec 21, 2023
fa92728
fix test on Windows
karottenreibe Dec 21, 2023
f7830e0
fix test on Linux
karottenreibe Dec 21, 2023
6cf1cd3
another attempt at fixing windows permissions in tests
karottenreibe Dec 21, 2023
0beb125
fix maven build on windows
karottenreibe Dec 21, 2023
1cc4d73
add missing dependency to maven builds
karottenreibe Dec 21, 2023
aae99e0
add more helpful error messages to assert
karottenreibe Dec 22, 2023
bc81721
fix test failure on Windows caused by wrong path separator
karottenreibe Dec 22, 2023
f8cefd4
fix more incorrect invocations of mvnw.cmd
karottenreibe Dec 22, 2023
6b27cd6
fix path to installer.bat
karottenreibe Jan 31, 2024
8c87488
also build windows installer
karottenreibe Jan 31, 2024
ea3ce81
rename scripts so they match usage help
karottenreibe Feb 2, 2024
974d64f
find source directory from jlink jvm
karottenreibe Feb 2, 2024
7d10448
fix installation directory on windows
karottenreibe Feb 2, 2024
1c3b077
ask user to manually delete install directory on windows
karottenreibe Feb 2, 2024
10516c8
make sure the profiler is not profiling the uninstaller
karottenreibe Feb 2, 2024
b7a1292
always advise to restart computer after uninstallation
karottenreibe Feb 2, 2024
ee38f2c
fix warnings
karottenreibe Feb 2, 2024
b7a0237
Merge branch 'master' into installer_windows
karottenreibe Feb 2, 2024
777bce5
fix tests since on Windows we don't remove the install directory
karottenreibe Feb 2, 2024
9706a93
fix warnings
karottenreibe Feb 2, 2024
94c15ac
export doesn't work in sh
karottenreibe Feb 7, 2024
407b5b1
Merge remote-tracking branch 'origin/master' into installer_windows
karottenreibe Mar 12, 2024
3e5bee6
rework
karottenreibe Mar 28, 2024
ca6d7b9
Merge remote-tracking branch 'origin/master' into installer_windows
karottenreibe Mar 28, 2024
eaffc8f
remove unnecessary dependsOn calls
karottenreibe Mar 28, 2024
eaa17c7
change formatter settings
karottenreibe Mar 28, 2024
3342917
reuse systemtest port convention
karottenreibe Mar 28, 2024
9c98428
remove system test clone
karottenreibe Mar 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
39 changes: 30 additions & 9 deletions .github/workflows/actions.yml
Expand Up @@ -8,18 +8,15 @@ on:
branches: '**'

jobs:
build:
name: Build & Deploy
build-linux:
name: Build Linux & Deploy
karottenreibe marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up GraalVM
uses: graalvm/setup-graalvm@v1
- uses: actions/setup-java@v4
with:
java-version: '17.0.8'
distribution: 'graalvm-community'
github-token: ${{ secrets.GITHUB_TOKEN }}
set-java-home: true
distribution: 'temurin'
java-version: '17'
- name: Build with Gradle
run: ./gradlew build
- name: Upload Release Assets
Expand Down Expand Up @@ -71,7 +68,31 @@ jobs:
accesskey: ${{ secrets.CQSE_TEAMSCALE_IO_ACCESSKEY }}
partition: 'Coverage'
format: 'JACOCO'
message: 'Coverage'
message: 'Linux Coverage'
files: '**/jacocoTestReport.xml'

test-windows:
name: Test on Windows
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '17'
- name: Build with Gradle
run: ./gradlew build
- name: Upload coverage to Teamscale
if: always() && github.event_name == 'push'
uses: cqse/teamscale-upload-action@v2.8.2
with:
server: 'https://cqse.teamscale.io'
project: 'teamscale-jacoco-agent'
user: ${{ secrets.CQSE_TEAMSCALE_IO_USER }}
accesskey: ${{ secrets.CQSE_TEAMSCALE_IO_ACCESSKEY }}
partition: 'Coverage Windows'
format: 'JACOCO'
message: 'Coverage Windows'
files: '**/jacocoTestReport.xml'

docker:
Expand Down
1 change: 1 addition & 0 deletions .idea/codeStyles/Project.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 1 addition & 2 deletions .idea/misc.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -5,6 +5,7 @@ We use [semantic versioning](http://semver.org/):
- PATCH version when you make backwards compatible bug fixes.

# Next Release
- [feature] add installer for Windows

# 33.1.2
- [fix] _teamscale-maven-plugin_: Revision and end commit could not be set via command line (user property)
Expand Down
4 changes: 2 additions & 2 deletions agent/build.gradle.kts
Expand Up @@ -84,8 +84,8 @@ distributions {
named("shadow") {
distributionBaseName.set("teamscale-jacoco-agent")
contents {
from(project(":installer").tasks["nativeCompile"]) {
include("installer")
from(project(":installer").tasks["jlink"]) {
into("installer")
}

from(tasks.readmeToPdf) {
Expand Down
9 changes: 9 additions & 0 deletions agent/src/dist/installer.bat
@@ -0,0 +1,9 @@
@echo off
set DIR="%~dp0"
set BAT_EXEC="%DIR:"=%\installer\installer-windows-x86_64\bin\installer.bat"

rem ensure that we don't accidentally profile the uninstaller
set JAVA_TOOL_OPTIONS=
set _JAVA_OPTIONS=

pushd %DIR% & %BAT_EXEC% %* & popd
8 changes: 8 additions & 0 deletions agent/src/dist/installer.sh
@@ -0,0 +1,8 @@
#!/bin/sh
DIR="${0%/*}"

# ensure that we don't accidentally profile the uninstaller
unset JAVA_TOOL_OPTIONS
unset _JAVA_OPTIONS

exec "$DIR/installer/installer-linux-x86_64/bin/installer" "$@"
3 changes: 2 additions & 1 deletion agent/src/docker/Dockerfile
@@ -1,6 +1,7 @@
# --- build image ---

FROM openjdk:8 as build
FROM openjdk:17 as build
RUN microdnf install findutils
karottenreibe marked this conversation as resolved.
Show resolved Hide resolved
ADD . /src
WORKDIR /src
RUN ./gradlew :agent:shadowJar
Expand Down
Expand Up @@ -3,6 +3,8 @@
import okhttp3.HttpUrl;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.DisabledOnOs;
import org.junit.jupiter.api.condition.OS;
import org.junit.jupiter.api.io.TempDir;

import java.io.IOException;
Expand Down Expand Up @@ -45,36 +47,44 @@ void successfulParsing() throws AgentOptionParseException, IOException {
void missingUsername() throws IOException {
Files.write(teamscalePropertiesPath, "url=http://test\naccesskey=key".getBytes(StandardCharsets.UTF_8));
assertThatThrownBy(
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining("missing the username");
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining(
"missing the username");
}

@Test
void missingAccessKey() throws IOException {
Files.write(teamscalePropertiesPath, "url=http://test\nusername=user".getBytes(StandardCharsets.UTF_8));
assertThatThrownBy(
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining("missing the accesskey");
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining(
"missing the accesskey");
}

@Test
void missingUrl() throws IOException {
Files.write(teamscalePropertiesPath, "username=user\nusername=user".getBytes(StandardCharsets.UTF_8));
assertThatThrownBy(
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining("missing the url");
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining(
"missing the url");
}

@Test
void malformedUrl() throws IOException {
Files.write(teamscalePropertiesPath, "url=$$**\nusername=user\nusername=user".getBytes(StandardCharsets.UTF_8));
assertThatThrownBy(
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining("malformed URL");
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining(
"malformed URL");
}

/** This test doesn't work on Windows since {@link java.io.File#setReadable(boolean)} does not work there. */
@DisabledOnOs(OS.WINDOWS)
@Test
void fileNotReadable() throws IOException {
Files.write(teamscalePropertiesPath, "url=http://test\nusername=user\nusername=user".getBytes(StandardCharsets.UTF_8));
Files.write(teamscalePropertiesPath,
"url=http://test\nusername=user\nusername=user".getBytes(StandardCharsets.UTF_8));
assertThat(teamscalePropertiesPath.toFile().setReadable(false)).isTrue();
assertThatThrownBy(
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining("Failed to read");
() -> TeamscalePropertiesUtils.parseCredentials(teamscalePropertiesPath)).hasMessageContaining(
"Failed to read");
}

}
Expand Up @@ -3,6 +3,9 @@
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.stream.Stream;

import static org.assertj.core.api.Assertions.assertThat;

/**
* Test Utilities
Expand All @@ -11,12 +14,15 @@ public class TestUtils {
/**
* Deletes all contents inside the coverage folder inside the agent directory
*/
@SuppressWarnings("ResultOfMethodCallIgnored")
public static void cleanAgentCoverageDirectory() throws IOException {
Path coverageDir = AgentUtils.getAgentDirectory().resolve("coverage");
if (Files.exists(coverageDir)) {
Files.list(coverageDir).forEach(path -> path.toFile().delete());
try (Stream<Path> stream = Files.list(coverageDir)) {
stream.forEach(path ->
assertThat(path.toFile().delete()).withFailMessage("Failed to delete " + path).isTrue());
}
Files.delete(coverageDir);
}
}

}
Expand Up @@ -12,6 +12,8 @@
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
Expand Down Expand Up @@ -54,12 +56,17 @@ public static void runMavenTests(String mavenProjectPath) throws IOException {

ProcessUtils.ExecutionResult result;
try {
String executable = "./mvnw";
List<String> arguments = new ArrayList<>();
if (SystemUtils.isWindows()) {
executable = Paths.get(mavenProjectPath, "mvnw.cmd").toUri().getPath();
Collections.addAll(arguments, "cmd", "/c", "mvnw.cmd");
} else {
arguments.add("./mvnw");
}
result = ProcessUtils.execute(
new ProcessBuilder(executable, "clean", "verify").directory(workingDirectory));

arguments.add("clean");
arguments.add("verify");

result = ProcessUtils.execute(new ProcessBuilder(arguments).directory(workingDirectory));
} catch (IOException e) {
throw new IOException(
"Failed to run ./mvnw clean verify in directory " + workingDirectory.getAbsolutePath(),
Expand Down
2 changes: 2 additions & 0 deletions gradle/libs.versions.toml
Expand Up @@ -50,6 +50,8 @@ spark = { module = "com.sparkjava:spark-core", version = "2.9.4" }
jcommander = { module = "com.beust:jcommander", version = "1.82" }
teamscaleLibCommons = { module = "com.teamscale:teamscale-lib-commons", version = "9.4.1" }
commonsCodec = { module = "commons-codec:commons-codec", version = "1.16.1" }
commonsLang = { module="org.apache.commons:commons-lang3", version="3.14.0" }
commonsIo = { module="commons-io:commons-io", version="2.15.1" }
slf4j-api = { module = "org.slf4j:slf4j-api", version = "2.0.12" }
jgit = { module = "org.eclipse.jgit:org.eclipse.jgit", version = "6.9.0.202403050737-r" }
okio = { module = "com.squareup.okio:okio", version = "3.8.0" }
Expand Down
83 changes: 57 additions & 26 deletions installer/build.gradle.kts
@@ -1,28 +1,78 @@
import org.beryx.jlink.util.JdkUtil

plugins {
application
com.teamscale.`java-convention`
com.teamscale.coverage
id("org.graalvm.buildtools.native") version "0.9.5"
id("org.beryx.jlink") version ("3.0.1")
}

tasks.jar {
manifest {
attributes["Main-Class"] = "com.teamscale.profiler.installer.RootCommand"
attributes(
//"Automatic-Module-Name" to "com.teamscale.profiler.installer",
karottenreibe marked this conversation as resolved.
Show resolved Hide resolved
"Main-Class" to "com.teamscale.profiler.installer.RootCommand",
)
}
}

java {
toolchain {
languageVersion.set(JavaLanguageVersion.of(17))
}
}

application {
applicationName = "installer"
mainClass.set("com.teamscale.profiler.installer.RootCommand")
mainClass = "com.teamscale.profiler.installer.RootCommand"
mainModule = "com.teamscale.profiler.installer"
applicationDefaultJvmArgs = listOf(
// Ensure that no stack traces are lost.
// See <https://stackoverflow.com/questions/2411487/nullpointerexception-in-java-with-no-stacktrace>
"-XX:-OmitStackTraceInFastThrow",
)
}

val ADOPTIUM_BINARY_REPOSITORY = "https://api.adoptium.net/v3/binary"
val RUNTIME_JDK_VERSION = "17.0.5+8"
jlink {
options = listOf(
"--compress", "2",
"--no-header-files",
"--no-man-pages",
"--dedup-legal-notices", "error-if-not-same-content"
)
launcher {
name = "installer"
}

targetPlatform("linux-x86_64") {
setJdkHome(
jdkDownload(
"$ADOPTIUM_BINARY_REPOSITORY/version/jdk-${RUNTIME_JDK_VERSION}/linux/x64/jdk/hotspot/normal/eclipse",
closureOf<JdkUtil.JdkDownloadOptions> {
archiveExtension = "tar.gz"
})
)
}
targetPlatform("windows-x86_64") {
setJdkHome(
jdkDownload(
"$ADOPTIUM_BINARY_REPOSITORY/version/jdk-${RUNTIME_JDK_VERSION}/windows/x64/jdk/hotspot/normal/eclipse",
closureOf<JdkUtil.JdkDownloadOptions> {
archiveExtension = "zip"
})
)
}
}

dependencies {
// we need this older version since newer versions are Kotlin-implemented and don't play nice with GraalVM
// okhttp 5 will add official GraalVM support but is currently still alpha https://square.github.io/okhttp/changelogs/changelog/
implementation("com.squareup.okhttp3:okhttp:3.14.9")
implementation(libs.teamscaleLibCommons)
implementation(libs.okhttp.core)
implementation(libs.commonsLang)
implementation(libs.commonsIo)
implementation(libs.picocli.core)
annotationProcessor(libs.picocli.codegen)
implementation("net.java.dev.jna:jna-platform:5.14.0")
karottenreibe marked this conversation as resolved.
Show resolved Hide resolved

testImplementation(libs.spark)
}
Expand All @@ -34,22 +84,3 @@ tasks.processResources {
}
}
}

graalvmNative {
binaries {
named("main") {
imageName.set("installer")
fallback.set(false)
// build an executable instead of a shared library
sharedLibrary.set(false)
buildArgs(
// Required for reading files from the filesystem. See https://github.com/oracle/graal/issues/1294
"-H:+AddAllCharsets",
// Required for HTTP requests
"--enable-http", "--enable-https",
// Required to include the app.properties file
"-H:IncludeResourceBundles=com.teamscale.profiler.installer.app"
)
}
}
}