Skip to content
This repository has been archived by the owner on May 28, 2020. It is now read-only.

We found 1 test flaky #22

Open
shaker-project opened this issue May 27, 2020 · 0 comments
Open

We found 1 test flaky #22

shaker-project opened this issue May 27, 2020 · 0 comments

Comments

@shaker-project
Copy link

We run your test cases for 50 times and found that the following test is flaky [1] (i.e., they pass or fail non-deterministically):

  • clickOnNavigationDrawerItem_ChangeTheme

We ran these tests on a generic Android Emulator (AVD) with Android API version 28.

To reproduce failures on these tests, please run the following script on gist:
https://gist.github.com/shaker-project/080b2c5d71e5fadb373715b50e3c3123
If you didn't understand any point of the script we can explain.

We require that stress-ng is installed. If not, run $> sudo apt-get install stress-ng.

If you confirm this behavior, our suggestion is to add the annotation @testflaky on these tests. See https://developer.android.com/reference/androidx/test/filters/FlakyTest

If you wish for a repair proposal, please let us know.

[1] https://hackernoon.com/flaky-tests-a-war-that-never-ends-9aa32fdef359

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant