Spek gradle properties not working in Kotlin multiplatform project

[corlaez]

Spek Gradle properties don't seem to be working for a Kotlin multiplatform project i.e:

• spek2.execution.test.timeout
• spek2.discovery.parallel.enabled
• spek2.execution.parallel.enabled

Additionally this vmOption -DSPEK2_TIMEOUT=1 doesn't have any effect either. (Should this be a different issue?)

Update: For clarity, this happens in tests with JVM target.

Could this be a larger issue of multiplatform and gradle properties in general?

Reproduction example: https://github.com/corlaez/spekTimeoutConfigIgnored

[raniejade]

JS and Native are not supported for these properties. Spek doesn't officially support them yet.

[corlaez]

I don't know if this changes the assessment but it is happening in the JVM portion of a multiplatform project. I will add the reproduction in a sec.

[raniejade]

Ohh, interesting. Implementation wise it just checks the system properties. It might be a difference in how the system property is set from gradle.

[corlaez]

I updated the ticket since all gradle config in general are ignored.

[raniejade]

I just saw the example, gradle properties are not propagated to the JVM spawned for running the tests. You should be doing: https://github.com/spekframework/spek/blob/2.x/integration-test/build.gradle.kts#L105

[corlaez]

Awesome, I have changed the project to use systemProperty("", "") and Timeout works now.

However, parallel tests still seem sequential. Is there something about this test that prevents it to be run in parallel?

object TestSpec : Spek({
    Feature("Set") {
        val set by memoized { mutableSetOf<String>() }

        Scenario("adding items") {
            When("adding foo") {
                set.add("foo")
            }

            Then("it should have a size of 1") {
                assertEquals(1, set.size)
            }

            Then("it should contain foo") {
                assertTrue(set.contains("foo"))
            }
        }

        Scenario("empty") {
            Then("should have a size of 0") {
                assertEquals(0, set.size)
            }

            Then("should throw when first is invoked") {
                assertFailsWith(NoSuchElementException::class) {
                    set.first()
                }
            }
        }

        Scenario("getting the first item") {
            val item = "foo"
            Given("a non-empty set")  {
                set.add(item)
            }

            lateinit var result: String

            When("getting the first item") {
                result = set.first()
            }

            Then("it should return the first item") {
                assertEquals(item, result)
            }
        }
    }
})

[corlaez]

This alternative way to configure has the same problem. Timeout works, parallel doesn't.

tasks.getByName<org.jetbrains.kotlin.gradle.targets.jvm.tasks.KotlinJvmTest>("jvmTest") {
    systemProperty("spek2.discovery.parallel.enabled", "true")
    systemProperty("spek2.execution.parallel.enabled", "true")// tests still seem sequential
    // systemProperty("spek2.execution.test.timeout", "1")// this works if comment is removed, breaking tests.
}

[raniejade]

Parallelism is class level only (i.e if you have two classes, they can be executed in parallel), any scope under a class/spek is executed sequentially.

[corlaez]

Oh I see what you are saying.
I still don't see that happening though.

I am going to try on a non multiplatform and see if that helps

[corlaez]

or maybe they are too short. I will make them work harder (longer) and see if that shows a difference when run in parallel

[corlaez]

I sprinkled some Thread.sleep and I don't see a change (at least in multiplatform)

Additionally every so often, one of the tests is not detected. Maybe this deserves a different ticket altogether but I am not sure whether it's multiplatform exclusive or not yet.

[raniejade]

@corlaez where are you placing the sleeps?

[corlaez]

            Then("should throw when first is invoked") {
                assertFailsWith(NoSuchElementException::class) {
                    Thread.sleep(2000)
                    set.first()
                }
            }

            When("adding foo") {
                Thread.sleep(2000)
                set.add("foo")
            }

            When("getting the first item") {
                sleep(2000)
                result = set.first()
            }