Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scripts: twisterlib: handlers: early QEMU timeout #72610

Merged

Conversation

JordanYates
Copy link
Collaborator

@JordanYates JordanYates commented May 11, 2024

Terminate QEMU earlier if the test finishes and no output data is received for 1 second. This new timeout operates in parallel with the global test timeout.

For coverage testing, this can reduce the time spent running individual tests by up to 29 seconds, while still giving the full 30 extra seconds to dump gcov data if needed.

Should reduce the duration of long CI runs like this:
https://github.com/zephyrproject-rtos/zephyr/actions/runs/9036499613/job/24833501514

 INFO    - 1199/1790 mps2/an385                tests/kernel/mbox/mbox_usage/kernel.mailbox.usage  PASSED (qemu 31.285s)
INFO    - 1200/1790 mps2/an385                tests/kernel/mem_slab/mslab_concept/kernel.memory_slabs.concept PASSED (qemu 31.112s)
INFO    - 1201/1790 mps2/an385                tests/kernel/fifo/fifo_timeout/kernel.fifo.timeout PASSED (qemu 31.556s)
INFO    - 1202/1790 mps2/an385                tests/kernel/fifo/fifo_usage/kernel.fifo.usage     PASSED (qemu 31.445s)
INFO    - 1203/1790 mps2/an385                tests/kernel/mem_slab/mslab_threadsafe/kernel.memory_slabs.threadsafe PASSED (qemu 31.318s)

Terminate QEMU earlier if the test finishes and no output data is
received for 1 second. This new timeout operates in parallel with the
global test timeout.

For coverage testing, this can reduce the time spent running individual
tests by up to 29 seconds, while still giving the full 30 extra seconds
to dump gcov data if needed.

Signed-off-by: Jordan Yates <jordan@embeint.com>
@@ -879,6 +879,9 @@ def _thread(handler, timeout, outdir, logfile, fifo_fn, pid_fn,

while True:
this_timeout = int((timeout_time - time.time()) * 1000)
if timeout_extended:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For coverage testing, this can reduce the time spent

then it should be:

Suggested change
if timeout_extended:
if timeout_extended and harness.capture_coverage:

otherwise it will disable extended timeout for non-coverage 'late' output as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coverage is the primary reason for this change, but non-coverage can also benefit in a minor way. Is it likely that any tests print nothing for over a second after the end of the test and then start printing more info that should be captured?

@carlescufi carlescufi merged commit 44f5a5d into zephyrproject-rtos:main May 20, 2024
30 checks passed
@JordanYates JordanYates deleted the 240511_qemu_coverage_timeout branch May 20, 2024 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants