coroutineTestScope: timeout for TestDispatcher cannot be configured

[CLOVIS-AI]

Which version of Kotest are you using
5.8.1

Reproducer:

class KotestTest : StringSpec({
	addTest(TestName("test"), disabled = false, config = TestConfig(timeout = 120.seconds, coroutineTestScope = true, testCoroutineDispatcher = true)) {
		withContext(Dispatchers.Default) {
			delay(90.seconds)
		}

		println("If this point is reached, the timeout was respected :)")
	}
})

This fails with

After waiting for 1m, the test coroutine is not completing, there were active child jobs: [UndispatchedCoroutine{Active}@39c77ef3]
kotlinx.coroutines.test.UncompletedCoroutinesError: After waiting for 1m, the test coroutine is not completing, there were active child jobs: [UndispatchedCoroutine{Active}@39c77ef3]

Yet, I passed TestConfig(timeout = 2.minutes), so the test should not have failed.

---

If I set coroutineTestScope = false, testCoroutineDispatcher = false, then the timeout is respected.

[OliverO2]

I haven't looked at the details, but isn't coroutineTestScope (which introduces its own notion of time) inherently incompatible with

• using Kotest's timeout configuration, and
• using another dispatcher like Dispatchers.Default inside?

Also, is there any documentation about testCoroutineDispatcher? I wonder, for example, what does it add if coroutineTestScope = true is present?

[CLOVIS-AI]

I haven't looked at the details, but isn't coroutineTestScope (which introduces its own notion of time) inherently incompatible with using Kotest's timeout configuration,

It does introduce its own notion of time that applies to the execution inside the test. However, the timeout set (passed to runTest) applies to the real time, not the fake time introduced within the test.

using another dispatcher like Dispatchers.Default inside?

Yes, but that's on purpose to create a reproducer. In my real project, there is something in the test that actually takes multiple minutes. I wanted a reproducer to show that Kotest doesn't honor the configured timeout, so I needed a concise way to spend time idling. As you mentioned, coroutineTestScope does time management and would skip the delay, I'm using Dispatchers.Default to get out of the fake time and force a "real" delay.

But indeed, you could replace that part of the reproducer by anything else that takes approximately 90 seconds in real time, and the reproducer would be unchanged.

[OliverO2]

Understood. To avoid misinterpretations, Thread::sleep could be used in such cases.

If I understand things correctly, the TestDispatcher from kotlinx-coroutines-test has its own timeout setting, defaulting to 10 seconds. The Kotest timeout is unrelated, uses withTimeout, or, if the test is marked as blocking, interrupts the test thread. My guess is that the TestDispatcher and Kotest timeouts were not designed to be used in combination.

I still wonder what the testCoroutineDispatcher parameter does and where that 1-minute timeout is coming from.

[CLOVIS-AI]

To avoid misinterpretations, Thread::sleep could be used in such cases.

I didn't because I wanted the reproducer to work in KMP, but you're right, that would have been clearer.

the TestDispatcher from kotlinx-coroutines-test has its own timeout setting, defaulting to 10 seconds.

Yes, though evidently, Kotest disables it, since my reproducer runs for longer than that without interruption.

I still wonder what the testCoroutineDispatcher parameter does and where that 1-minute timeout is coming from.

Same. It's blocking me on one of my project because one test is taking too long (but there's nothing we can do about it) and currently we just can't do anything since we don't know where it is configured.

[OliverO2]

If your use case is actually blocking code (test taking multiple minutes without cooperative cancellation in between), you should really use Kotest's timeout parameter in conjunction with blockingTest = true, and avoid the test dispatcher. That should do the trick.

[CLOVIS-AI]

The test does also interact with the test dispatcher. But that's not the point: Kotest exposes a timeout parameter that isn't honored when enabling the test dispatcher, and instead Kotest adds a 1-minute timeout which so far I haven't found a way to configure. Either this should honor the timeout configuration, or Kotest shouldn't add a timeout at all.

[OliverO2]

Actually, Kotest is not involved here. If coroutineTestScope is true, it wraps the test in a TestCoroutineInterceptor, which uses runTest from kotlinx-coroutines-test to run it.

Kotest does not provide additional configuration and invokes runTest with its default timeout. Contrary to the documentation in kotlinx-coroutines-test, this default is 1 minute, not 10 seconds.

[OliverO2]

Created Kotlin/kotlinx.coroutines#4099.

[CLOVIS-AI]

Kotest does not provide additional configuration and invokes runTest with its default timeout.

How can I tell Kotest to invoke it with the timeout I configured as part of the Kotest configuration?

[OliverO2]

Currently, you cannot. For an immediate solution, I'd consider invoking runTest directly. For an integrated fix, a PR would be the way to go.