transient error in testcpp_contextual_thread

[markus2330]

@sanssecours wrote in #1516:

Looks like there was a transient error in testcpp_contextual_thread.

valgrind --tool=helgrind bin/testcpp_contextual_thread

[sanssecours]

It seems the test failed once again 😢.

[markus2330]

Surprising that this is possible, the difference in waiting time is huge. I could not reproduce it even if I have the same waiting time for both threads. Only when I give the thread which should be faster more waiting time, the test case fails (as expected):

diff --git a/src/bindings/cpp/tests/testcpp_contextual_thread.cpp b/src/bindings/cpp/tests/testcpp_contextual_thread.cpp
index 3244653..9fe3c93 100644
--- a/src/bindings/cpp/tests/testcpp_contextual_thread.cpp
+++ b/src/bindings/cpp/tests/testcpp_contextual_thread.cpp
@@ -136,7 +136,7 @@ void activate1 (Coordinator & gc, KeySet & ks)
        c1.activate<Activate> ();
        ASSERT_TRUE (g_toggle);
        ASSERT_EQ (v1, 22);
-       std::this_thread::sleep_for (std::chrono::milliseconds (100));
+       std::this_thread::sleep_for (std::chrono::milliseconds (1));
        ASSERT_EQ (v1, 22);
        c1.deactivate<Activate> ();
        ASSERT_FALSE (g_toggle);
@@ -161,7 +161,7 @@ TEST (test_contextual_thread, activate)
 
        std::thread t1 (activate1, std::ref (gc), std::ref (ks));
        ASSERT_EQ (v, 10);
-       std::this_thread::sleep_for (std::chrono::milliseconds (50));
+       std::this_thread::sleep_for (std::chrono::milliseconds (2));
        ASSERT_TRUE (g_toggle);
        c.syncLayers ();
        ASSERT_EQ (v, 22);

Rewriting the test case to mutexes does not make much sense (there are already tests which test sequential execution). I'll increase the waiting times + disable the test for apple.

[mpranj]

I've just experienced this with a current PR.

 28/232 Test  #22: testcpp_contextual_thread .....................Child aborted***Exception:   1.26 sec

Running main() from /opt/gtest/googletest/src/gtest_main.cc

[==========] Running 8 tests from 1 test case.

[----------] Global test environment set-up.

[----------] 8 tests from test_contextual_thread

[ RUN      ] test_contextual_thread.instanciation

[       OK ] test_contextual_thread.instanciation (172 ms)

[ RUN      ] test_contextual_thread.activate

/home/jenkins/workspace/libelektra_PR-3291@2/src/bindings/cpp/tests/testcpp_contextual_thread.cpp:170: Failure

Value of: g_toggle

  Actual: false

Expected: true

terminate called without an active exception

😕

[markus2330]

Thank you for reporting. Yes, this transient error, together with many others (#2439) are still not fixed. Our current fix is to run failing tests again.

Can you check if this test really failed twice in a row? We could retry 2 times (3 in total).

[mpranj]

Can you check if this test really failed twice in a row?

No. It seems that the Jenkinsfile calls ctest directly instead of the workarount in run_all.

libelektra/scripts/jenkins/Jenkinsfile

Lines 1312 to 1314 in 0eb2a81

	sh """ctest -j ${env.CTEST_PARALLEL_LEVEL} --force-new-ctest-process \
	--output-on-failure --no-compress-output -T ${target} \
	-E 'testmod_(crypto_(botan|openssl)|dbus(recv)?|fcrypt|gpgme|zeromqsend)'"""

Maybe we add the same workaround to the Jenkinsfile so we don't waste too much time on this problem.

[markus2330]

I think we can simply change the Jenkinsfile to run make tests as all the logic for test exclusion is only about temporary errors (as far as I see).

[markus2330]

Doesn't fail anymore and the error itself is not very important.