Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test on CI/CD #65

Open
markkohdev opened this issue Apr 1, 2024 · 0 comments
Open

Flaky test on CI/CD #65

markkohdev opened this issue Apr 1, 2024 · 0 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@markkohdev
Copy link
Contributor

The java ConcurrencyTest sporadically fails on CI/CD. We should figure out how to make that test more stable.
A short-term workaround is simply re-running the failed build steps, this will usually resolve the issue within 1 retry

Error logs from a flaky run

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.spotify.voyager.jni.ConcurrencyTest
java.lang.RuntimeException: resource deadlock would occur: resource deadlock would occur
	at com.spotify.voyager.jni.Index.addItem(Native Method)
	at com.spotify.voyager.jni.ConcurrencyTest.lambda$itCanAddItemsInParallel$4(ConcurrencyTest.java:180)
	at java.base/java.lang.Thread.run(Thread.java:829)
Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.766 sec <<< FAILURE!
itCanAddItemsInParallel(com.spotify.voyager.jni.ConcurrencyTest)  Time elapsed: 1.446 sec  <<< FAILURE!
java.lang.AssertionError: Error in test: resource deadlock would occur: resource deadlock would occur
	at org.junit.Assert.fail(Assert.java:89)
	at com.spotify.voyager.jni.ConcurrencyTest.itCanAddItemsInParallel(ConcurrencyTest.java:200)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Running com.spotify.voyager.jni.IndexTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 139.354 sec
Running com.spotify.voyager.jni.StringIndexTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.[887](https://github.com/spotify/voyager/actions/runs/8511970940/job/23312645027?pr=63#step:6:888) sec
Running com.spotify.voyager.jni.utils.TinyJsonTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 sec

Results :

Failed tests:   itCanAddItemsInParallel(com.spotify.voyager.jni.ConcurrencyTest): Error in test: resource deadlock would occur: resource deadlock would occur

Tests run: 26, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:36 min
[INFO] Finished at: 2024-04-01T18:29:21Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project voyager: There are test failures.
Error:  
Error:  Please refer to D:\a\voyager\voyager\java\target\surefire-reports for the individual test results.
Error:  -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Error: Process completed with exit code 1.
@markkohdev markkohdev added bug Something isn't working good first issue Good for newcomers labels Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant