tests: dynamic switch of emulated audio for HVM

[marmarek]

Use pulseaudio inside VM as that's more compatible.

This also tests starting HVM with a non-dom0 audiovm.

QubesOS/qubes-issues#8975

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.50%. Comparing base (ceea705) to head (8b9aba1).
Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #591      +/-   ##
==========================================
- Coverage   68.50%   68.50%   -0.01%     
==========================================
  Files          56       56              
  Lines       11232    11233       +1     
==========================================
  Hits         7695     7695              
- Misses       3537     3538       +1     

Flag	Coverage Δ
unittests	68.50% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[qubesos-bot]

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2024051221-4.2&flavor=pull-requests

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.2&build=2024050210-4.2&flavor=update

• system_tests_extra
  • TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
    self.assertNotEqual(vm.run('test -e /dev/vid... AssertionError: 0 == 0

Failed tests

8 failures

• system_tests_pvgrub_salt_storage

  • [unstable] TC_41_HVMGrub_fedora-39-xfce: test_000_standalone_vm (error)
    qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

  • [unstable] TC_41_HVMGrub_fedora-39-xfce: test_010_template_based_vm (error)
    qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

• system_tests_splitgpg

  • TC_10_Thunderbird_whonix-workstation-17: test_020_send_receive_inline_with_attachment (failure)
    dogtail.tree.SearchError: descendent of [application | Thunderbird]...

• system_tests_extra

  • [unstable] TC_00_QVCTest_debian-12-xfce: test_020_webcam (failure)
    AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

  • [unstable] TC_00_QVCTest_fedora-39-xfce: test_020_webcam (failure)
    AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

  • TC_00_QVCTest_whonix-gateway-17: test_010_screenshare (failure)
    self.assertNotEqual(vm.run('test -e /dev/vid... AssertionError: 0 == 0

  • [unstable] TC_00_QVCTest_whonix-gateway-17: test_020_webcam (failure)
    AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

  • [unstable] TC_00_QVCTest_whonix-workstation-17: test_020_webcam (failure)
    AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/98585#dependencies

1 fixed

• system_tests_dispvm
  • TC_20_DispVM_fedora-39-xfce: test_100_open_in_dispvm (failure)
    AssertionError: Timeout waiting for editor window

Unstable tests

• system_tests_basic_vm_qrexec_gui@hw1

  TC_20_AudioVM_Pulse_whonix-workstation-17/test_220_audio_play_pulseaudio (2/5 times with errors)

  • job 97185 AssertionError: only silence detected, no useful audio data
  • job 97620 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_222_audio_rec_unmuted_pulseaudio (2/5 times with errors)

  • job 97185 AssertionError: only silence detected, no useful audio data
  • job 97620 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_debian-12-xfce/test_223_audio_play_hvm (2/5 times with errors)

  • job 97620 AssertionError: only silence detected, no useful audio data
  • job 98341 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce/test_223_audio_play_hvm (2/5 times with errors)

  • job 97620 AssertionError: only silence detected, no useful audio data
  • job 98341 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce/test_225_audio_rec_unmuted_hvm (2/5 times with errors)

  • job 97620 AssertionError: only silence detected, no useful audio data
  • job 98341 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_PipeWire_debian-12-xfce/test_250_audio_playback_audiovm_pipewire (1/5 times with errors)

  • job 98601 AssertionError: only silence detected, no useful audio data

• system_tests_basic_vm_qrexec_gui

  TC_20_AudioVM_Pulse_whonix-workstation-17/test_220_audio_play_pulseaudio (2/5 times with errors)

  • job 97185 AssertionError: only silence detected, no useful audio data
  • job 97620 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17/test_222_audio_rec_unmuted_pulseaudio (2/5 times with errors)

  • job 97185 AssertionError: only silence detected, no useful audio data
  • job 97620 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_debian-12-xfce/test_223_audio_play_hvm (2/5 times with errors)

  • job 97620 AssertionError: only silence detected, no useful audio data
  • job 98341 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce/test_223_audio_play_hvm (2/5 times with errors)

  • job 97620 AssertionError: only silence detected, no useful audio data
  • job 98341 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce/test_225_audio_rec_unmuted_hvm (2/5 times with errors)

  • job 97620 AssertionError: only silence detected, no useful audio data
  • job 98341 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_PipeWire_debian-12-xfce/test_250_audio_playback_audiovm_pipewire (1/5 times with errors)

  • job 98601 AssertionError: only silence detected, no useful audio data

• system_tests_network

  VmNetworking_debian-12-xfce/test_010_simple_proxyvm (1/5 times with errors)

  • job 99274 subprocess.CalledProcessError: Command 'qubes.WaitForSession' retur...

• system_tests_pvgrub_salt_storage

  TC_41_HVMGrub_debian-12-xfce/test_000_standalone_vm (3/5 times with errors)

  • job 96359 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97092 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97197 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  TC_41_HVMGrub_fedora-39-xfce/test_000_standalone_vm (4/5 times with errors)

  • job 96359 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97092 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97197 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 98613 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  TC_42_PVHGrub_fedora-39-xfce/test_000_standalone_vm (3/5 times with errors)

  • job 96359 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97092 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97197 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  TC_41_HVMGrub_debian-12-xfce/test_010_template_based_vm (3/5 times with errors)

  • job 96359 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97092 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97197 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  TC_41_HVMGrub_fedora-39-xfce/test_010_template_based_vm (4/5 times with errors)

  • job 96359 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97092 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97197 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 98613 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  TC_42_PVHGrub_fedora-39-xfce/test_010_template_based_vm (3/5 times with errors)

  • job 96359 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97092 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...
  • job 97197 qubes.exc.QubesVMError: Cannot connect to qrexec agent for 120 seco...

• system_tests_splitgpg

  TC_10_Thunderbird_fedora-39-xfce/test_000_send_receive_default (3/5 times with errors)

  • job 97094 Exception: Failed to send message with error 'unknown'
  • job 97199 Exception: Failed to send message with error 'unknown'
  • job 97634 Exception: Failed to send message with error 'unknown'
  TC_10_Thunderbird_fedora-39-xfce/test_010_send_receive_inline_signed_only (3/5 times with errors)

  • job 97094 Exception: Failed to send message with error 'unknown'
  • job 97199 Exception: Failed to send message with error 'unknown'
  • job 97634 Exception: Failed to send message with error 'unknown'
  TC_10_Thunderbird_fedora-39-xfce/test_020_send_receive_inline_with_attachment (3/5 times with errors)

  • job 97094 Exception: Failed to send message with error 'unknown'
  • job 97199 Exception: Failed to send message with error 'unknown'
  • job 97634 Exception: Failed to send message with error 'unknown'

• system_tests_extra

  TC_00_QVCTest_debian-12-xfce/test_020_webcam (1/5 times with errors)

  • job 99260 self.assertTrue(frame)... AssertionError: b'' is not true
  TC_00_QVCTest_fedora-39-xfce/test_020_webcam (1/5 times with errors)

  • job 99260 self.assertEqual(len(img1), len(img2))... AssertionError: 2359296 != 0
  TC_00_QVCTest_whonix-gateway-17/test_020_webcam (1/5 times with errors)

  • job 98336 AssertionError: 'qubes-video-companion webcam' exited early (0): b'...
  TC_00_QVCTest_whonix-workstation-17/test_020_webcam (1/5 times with errors)

  • job 98596 AssertionError: 'qubes-video-companion webcam' exited early (0): b'...

• system_tests_usbproxy

  TC_20_USBProxy_core3_whonix-gateway-17/test_070_attach_not_installed_front (1/5 times with errors)

  • job 97173 qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...
  TC_20_USBProxy_core3_whonix-workstation-17/test_070_attach_not_installed_front (1/5 times with errors)

  • job 98333 qubesusbproxy.core3ext.QubesUSBException: Device attach failed: 202...
  TC_20_USBProxy_core3_debian-12-xfce/test_090_attach_stubdom (2/5 times with errors)

  • job 97608 AssertionError: 1 != 0 : Device connection failed
  • job 98333 AssertionError: 1 != 0 : Device connection failed
  TC_20_USBProxy_core3_fedora-39-xfce/test_090_attach_stubdom (2/5 times with errors)

  • job 97608 AssertionError: 1 != 0 : Device connection failed
  • job 98333 AssertionError: 1 != 0 : Device connection failed

• system_tests_network_ipv6

  VmIPv6Networking_debian-12-xfce/test_031_firewall_dynamic_block (1/5 times with errors)

  • job 97090 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError
  VmIPv6Networking_fedora-39-xfce/test_031_firewall_dynamic_block (1/5 times with errors)

  • job 97090 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^... AssertionError

• system_tests_dispvm

  TC_20_DispVM_fedora-39-xfce/test_100_open_in_dispvm (4/5 times with errors)

  • job 97625 AssertionError: Timeout waiting for editor window
  • job 98346 AssertionError: Timeout waiting for editor window
  • job 98606 AssertionError: Timeout waiting for editor window
  • job 99270 AssertionError: './open-file test.txt' failed with ./open-file test...

• system_tests_basic_vm_qrexec_gui_btrfs

  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (4/5 times with errors)

  • job 96348 AssertionError: only silence detected, no useful audio data
  • job 97081 AssertionError: only silence detected, no useful audio data
  • job 97186 AssertionError: only silence detected, no useful audio data
  • job 97621 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (4/5 times with errors)

  • job 96348 AssertionError: only silence detected, no useful audio data
  • job 97081 AssertionError: only silence detected, no useful audio data
  • job 97186 AssertionError: only silence detected, no useful audio data
  • job 97621 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_223_audio_play_hvm (1/5 times with errors)

  • job 97621 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_223_audio_play_hvm (1/5 times with errors)

  • job 97621 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_225_audio_rec_unmuted_hvm (1/5 times with errors)

  • job 97621 AssertionError: only silence detected, no useful audio data

• system_tests_basic_vm_qrexec_gui_ext4

  TC_00_Basic/test_141_libvirt_objects_reconnect (1/5 times with errors)

  • job 98343 AssertionError: libvirt event impl drain timeout
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (3/5 times with errors)

  • job 97082 AssertionError: only silence detected, no useful audio data
  • job 97187 AssertionError: only silence detected, no useful audio data
  • job 97622 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (3/5 times with errors)

  • job 97082 AssertionError: only silence detected, no useful audio data
  • job 97187 AssertionError: only silence detected, no useful audio data
  • job 97622 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_223_audio_play_hvm (2/5 times with errors)

  • job 97622 AssertionError: only silence detected, no useful audio data
  • job 98343 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_223_audio_play_hvm (2/5 times with errors)

  • job 97622 AssertionError: only silence detected, no useful audio data
  • job 98343 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_225_audio_rec_unmuted_hvm (2/5 times with errors)

  • job 97622 AssertionError: only silence detected, no useful audio data
  • job 98343 AssertionError: only silence detected, no useful audio data

• system_tests_basic_vm_qrexec_gui_zfs

  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (3/5 times with errors)

  • job 97096 AssertionError: only silence detected, no useful audio data
  • job 97175 AssertionError: only silence detected, no useful audio data
  • job 97636 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (3/5 times with errors)

  • job 97096 AssertionError: only silence detected, no useful audio data
  • job 97175 AssertionError: only silence detected, no useful audio data
  • job 97636 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_223_audio_play_hvm (2/5 times with errors)

  • job 97636 AssertionError: only silence detected, no useful audio data
  • job 98335 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_223_audio_play_hvm (2/5 times with errors)

  • job 97636 AssertionError: only silence detected, no useful audio data
  • job 98335 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_225_audio_rec_unmuted_hvm (2/5 times with errors)

  • job 97636 AssertionError: only silence detected, no useful audio data
  • job 98335 AssertionError: only silence detected, no useful audio data

• system_tests_network_updates

  TC_10_QvmTemplate_fedora-39-xfce/test_010_template_install (1/5 times with errors)

  • job 97631 AssertionError: libvirt event impl drain timeout
  TC_10_QvmTemplate_whonix-gateway-17/test_010_template_install (1/5 times with errors)

  • job 97201 AssertionError: libvirt event impl drain timeout

• system_tests_basic_vm_qrexec_gui_xfs

  TC_20_NonAudio_fedora-39-xfce-pool/test_000_start_shutdown (1/5 times with errors)

  • job 99268 raise exceptions.TimeoutError() from exc... TimeoutError
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_220_audio_play_pulseaudio (1/5 times with errors)

  • job 99268 raise exceptions.TimeoutError() from exc... TimeoutError
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_220_audio_play_pulseaudio (2/5 times with errors)

  • job 97188 AssertionError: only silence detected, no useful audio data
  • job 97623 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_whonix-workstation-17-pool/test_222_audio_rec_unmuted_pulseaudio (2/5 times with errors)

  • job 97188 AssertionError: only silence detected, no useful audio data
  • job 97623 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_debian-12-xfce-pool/test_223_audio_play_hvm (3/5 times with errors)

  • job 97623 AssertionError: only silence detected, no useful audio data
  • job 98344 AssertionError: only silence detected, no useful audio data
  • job 99268 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_223_audio_play_hvm (2/5 times with errors)

  • job 97623 AssertionError: only silence detected, no useful audio data
  • job 98344 AssertionError: only silence detected, no useful audio data
  TC_20_AudioVM_Pulse_fedora-39-xfce-pool/test_225_audio_rec_unmuted_hvm (2/5 times with errors)

  • job 97623 AssertionError: only silence detected, no useful audio data
  • job 98344 AssertionError: only silence detected, no useful audio data