client tests spuriously segfault and or fail

[legoktm]

Description

A tracking issue to discuss and investigate that our client tests regularly flakily segfault and fail. The failures usually error similar to RuntimeError: wrapped C/C++ object of type QPushButton has been deleted.

E.g.:

FAILED tests/gui/conversation/export/test_print_transcript_dialog.py::test_PrintTranscriptDialog_init - RuntimeError: wrapped C/C++ object of type QPushButton has been deleted
FAILED tests/gui/conversation/export/test_print_dialog.py::test_PrintDialog_init_sanitizes_filename - RuntimeError: wrapped C/C++ object of type QPushButton has been deleted

[legoktm]

I've been seeing a few QThread: Destroyed while thread is still running segfaults while fixing the functional tests to use the proxy v2, and came across https://blog.xmatthias.com/post/pytest-debug-segfault/ which suggested enabling Python's faulthandler and disabling pytest's. So now I have this stacktrace:

QThread: Destroyed while thread is still running
Fatal Python error: Aborted

Thread 0x000071bb773ff6c0 (most recent call first):
  File "/usr/lib64/python3.10/subprocess.py", line 1141 in communicate
  File "/usr/lib64/python3.10/subprocess.py", line 505 in run
  File "/usr/lib64/python3.10/subprocess.py", line 421 in check_output
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/sdk/__init__.py", line 117 in _rpc_target
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/sdk/__init__.py", line 148 in _send_json_request
  File "/home/user/github/freedomofpress/securedrop-client/client/tests/sdk/utils.py", line 66 in _send_json_request
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/sdk/__init__.py", line 847 in logout
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/logic.py", line 121 in call_api

Thread 0x000071bb76bfe6c0 (most recent call first):
  File "/usr/lib64/python3.10/subprocess.py", line 1141 in communicate
  File "/usr/lib64/python3.10/subprocess.py", line 505 in run
  File "/usr/lib64/python3.10/subprocess.py", line 421 in check_output
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/sdk/__init__.py", line 117 in _rpc_target
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/sdk/__init__.py", line 148 in _send_json_request
  File "/home/user/github/freedomofpress/securedrop-client/client/tests/sdk/utils.py", line 66 in _send_json_request
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/sdk/__init__.py", line 576 in download_submission
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/api_jobs/downloads.py", line 318 in call_download_api
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/api_jobs/downloads.py", line 148 in _download
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/api_jobs/downloads.py", line 136 in call_api
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/api_jobs/base.py", line 77 in _do_call_api
  File "/home/user/github/freedomofpress/securedrop-client/client/securedrop_client/queue.py", line 232 in process

Current thread 0x000071bbaefd1740 (most recent call first):
  <no Python frame>

Extension modules: yaml._yaml, PyQt5.QtCore, PyQt5.QtGui, PyQt5.QtWidgets, sqlalchemy.cprocessors, sqlalchemy.cutils, sqlalchemy.cresultproxy, PyQt5.QtSvg, markupsafe._speedups, PyQt5.QtTest (total: 10)
/usr/bin/xvfb-run: line 181: 107302 Aborted                 (core dumped) DISPLAY=:$SERVERNUM XAUTHORITY=$AUTHFILE "$@" 2>&1
make: *** [Makefile:82: test-functional] Error 1

Which indicates that the background sync job was (improperly?) terminated at the end of the test run while it was in progress and that triggered the segfault.

[rocodes]

Nice one!

I implemented some logic that may or may not be relevant on the wizard - basically connecting to the appwindow's finished signal, which is fired when the app is closed (although not when it's closed improperly in a crash) and killing in my case a QProcess not a QThread.

securedrop-client/client/securedrop_client/gui/conversation/export/export_wizard.py

Lines 65 to 67 in 2de3a3b

	# Sends cleanup signal to export if wizard is closed or completed.
	# (Avoid orphaned QProcess)
	self.finished.connect(self.export.end_process)

and

securedrop-client/client/securedrop_client/export.py

Lines 275 to 284 in 2de3a3b

	def end_process(self) -> None:
	"""
	Tell QProcess to quit if it hasn't already.
	Connected to the ExportWizard's `finished` signal, which fires
	when the dialog is closed, cancelled, or finished.
	"""
	self._cleanup_tmpdir()
	logger.debug("Terminate process")
	if self.process is not None and not self.process.waitForFinished(50):
	self.process.terminate()