Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create tests for measuring glitches and latency #551

Open
gavv opened this issue Sep 22, 2023 · 0 comments
Open

Create tests for measuring glitches and latency #551

gavv opened this issue Sep 22, 2023 · 0 comments
Labels
help wanted An important and awaited task but we have no human resources for it yet much-needed This issue is needed most among other help-wanted issues rt-tests Real-time tests tests

Comments

@gavv
Copy link
Member

gavv commented Sep 22, 2023

Roc already has unit and integration tests, but they, by design, have limitations:

  • they are run frequently during development, so they should not take long
  • they are run everywhere (on developer machine, on CI), so should not be sensitive to real-time (e.g. tests should pass even on loaded machine like CI VM worker)
  • they are yes/no tests, i.e. should either succeed or fail

We want to develop a new set of real-time tests:

  • running on real hardware on unloaded system
  • running long if needed
  • providing metrics, not just success or failure
  • testing various real-time aspects of the toolkit, for example latency, number of glitches, etc

These new tests will live in a separate repo: rt-tests.

Currently, the new repo has a build system, and a skeleton of one test that implements full loopback: writes audio stream to roc sender, obtains the stream from roc receiver, and runs a very basic check.

Now we need to develop a few actual tests:

  • service quality test (measure number of glitches to check how roc copes with packet losses and delays)
  • latency test (measure delay from sender to receiver)
  • stress test (check invariants under intense load)
  • and others

Please refer to rt-tests repo issues for more detailed description of each test:

rt-tests repo uses C++17, cmake, and Google Test.

@gavv gavv added tests help wanted An important and awaited task but we have no human resources for it yet rt-tests Real-time tests labels Sep 22, 2023
@gavv gavv added this to Frontlog in kanban board Sep 22, 2023
@gavv gavv moved this from Frontlog to Help wanted in kanban board Sep 22, 2023
@gavv gavv added the much-needed This issue is needed most among other help-wanted issues label Sep 28, 2023
@gavv gavv pinned this issue Sep 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted An important and awaited task but we have no human resources for it yet much-needed This issue is needed most among other help-wanted issues rt-tests Real-time tests tests
Projects
kanban board
Help wanted
Development

No branches or pull requests

1 participant