Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange behaviour for tests filtering #798

Open
IosephKnecht opened this issue May 24, 2023 · 2 comments
Open

Strange behaviour for tests filtering #798

IosephKnecht opened this issue May 24, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@IosephKnecht
Copy link

Describe the bug
When specifying a composite filter in marathon plugin configuration, subtraction works strangely.

Imagine that the following android tests are given:

.
└── androidTest
    └── java
        ├── annotation
        │   └── SomeTestAnnotation.kt
        ├── first_part
        │   ├── ExampleInstrumented1Test.kt    <-- annotated test by SomeTestAnnotation
        │   ├── ExampleInstrumented2Test.kt
        │   └── ExampleInstrumented3Test.kt
        └── second_part
            ├── ExampleInstrumented4Test.kt
            ├── ExampleInstrumented5Test.kt
            └── ExampleInstrumented6Test.kt

Next, specify the following filter in build.gradle.kts file:

marathon {
    // omitted code
    filteringConfiguration {
        val unionFilterList = listOf(
            TestPackageFilterConfiguration(regex = "first_part.*".toRegex()),
            TestPackageFilterConfiguration(regex = "second_part.*".toRegex())
        )

        val unionFilterConfiguration = CompositionFilterConfiguration(
            filters = unionFilterList,
            op = CompositionFilterConfiguration.OPERATION.UNION
        )

        val annotationFilterConfiguration = AnnotationFilterConfiguration(
            values = listOf("annotation.SomeTestAnnotation")
        )

        allowlist {
            add(
                CompositionFilterConfiguration(
                    filters = listOf(
                        unionFilterConfiguration,
                        annotationFilterConfiguration
                    ),
                    op = CompositionFilterConfiguration.OPERATION.SUBTRACT
                )
            )
        }
    }
}

It seems that when subtracting sets (ExampleInstrumented1Test, ExampleInstrumented2Test, ExampleInstrumented3Test, ExampleInstrumented4Test, ExampleInstrumented5Test, ExampleInstrumented6Test) and (ExampleInstrumented1Test) should have turned out (ExampleInstrumented2Test, ExampleInstrumented3Test, ExampleInstrumented4Test, ExampleInstrumented5Test, ExampleInstrumented6Test), however, for some reason it turns out an empty set and marathon does not plan a single test.

To Reproduce
Steps to reproduce the behaviour:

  1. extract zip with project marathon-filter-playground-main.zip
  2. execute in terminal from project folder ./gradlew :app:marathonDebugAndroidTest
  3. see output there will be something like this:
I 18:58:24.363 [main @coroutine#2] <com.malinskiy.marathon.Marathon> Scheduling 0 tests

Expected behavior
I would expect correct subtraction of sets when creating a composite filter.
P.S
Maybe I don't know something and it's not really a bug.

Logs and reports
marathon_filter_playground.log
marathon-filter-playground.zip

Devices (please complete the following information):

  • Device: Emulator
  • OS: Android 11

Additional context
Reproduced on marathon versions 0.7.6/0.8.0

@IosephKnecht IosephKnecht added the bug Something isn't working label May 24, 2023
@Malinskiy
Copy link
Member

Possibly fixed by #834

@IosephKnecht
Copy link
Author

@Malinskiy Great, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants