tests fail with sqlite3 - "select ... for update" not supported

[cboltz]

lock_row() in helios/utils.py does

cursor.execute("select * from " + model._meta.db_table + " where id = %s for update", [pk])

When using a sqlite3 database, this leads to a test failure (see below) because select ... for update isn't supported in sqlite3.

This happens with latest master, with the DATABASES config changed to

'ENGINE': 'django.db.backends.sqlite3',

The tests work if I remove the for update part - but that also means that the locking doesn't happen anymore:

cursor.execute("select * from " + model._meta.db_table + " where id = %s", [pk])  # works with sqlite3, but looses locking

I'd recommend to use the for update part depending on the database engine.

.

Full test output:

python3 manage.py test
Creating test database for alias 'default'...
System check identified no issues (50 silenced).
....2021-11-14 06:40:12,542 ERROR Task helios.tasks.voter_file_process[8120c916-993e-4e9a-8ab6-cad9d0fe3d0b] raised unexpected: Exception('select * from helios_election where id = ? for update',)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/dev/shm/helios-server/helios/tasks.py", line 125, in voter_file_process
    voter_file.process()
  File "/dev/shm/helios-server/helios/models.py", line 823, in process
    utils.lock_row(Election, election.id)
  File "/dev/shm/helios-server/helios/utils.py", line 174, in lock_row
    cursor.execute("select * from " + model._meta.db_table + " where id = %s for update", [pk])
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 384, in execute
    raise Exception(query)
Exception: select * from helios_election where id = ? for update
F2021-11-14 06:40:12,670 ERROR Task helios.tasks.voter_file_process[51c585cb-883b-4e17-aa90-48fa131ffcf3] raised unexpected: Exception('select * from helios_election where id = ? for update',)
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/dev/shm/helios-server/helios/tasks.py", line 125, in voter_file_process
    voter_file.process()
  File "/dev/shm/helios-server/helios/models.py", line 823, in process
    utils.lock_row(Election, election.id)
  File "/dev/shm/helios-server/helios/utils.py", line 174, in lock_row
    cursor.execute("select * from " + model._meta.db_table + " where id = %s for update", [pk])
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 384, in execute
    raise Exception(query)
Exception: select * from helios_election where id = ? for update
F.2021-11-14 06:40:12,699 WARNING Not Found: /helios/elections/foobar
.2021-11-14 06:40:12,707 WARNING Not Found: /helios/t/test/foobar@bar.com/badsecret
.........2021-11-14 06:40:12,999 WARNING Forbidden (Permission denied): /helios/stats/
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/dev/shm/helios-server/helios/stats_views.py", line 27, in home
    user = require_admin(request)
  File "/dev/shm/helios-server/helios/stats_views.py", line 22, in require_admin
    raise PermissionDenied()
django.core.exceptions.PermissionDenied
2021-11-14 06:40:13,003 WARNING Forbidden (Permission denied): /helios/stats/
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/dev/shm/helios-server/helios/stats_views.py", line 27, in home
    user = require_admin(request)
  File "/dev/shm/helios-server/helios/stats_views.py", line 22, in require_admin
    raise PermissionDenied()
django.core.exceptions.PermissionDenied
...............................
======================================================================
FAIL: test_do_complete_election (helios.tests.ElectionBlackboxTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/dev/shm/helios-server/helios/tests.py", line 764, in test_do_complete_election
    election_id, username, password = self._setup_complete_election()
  File "/dev/shm/helios-server/helios/tests.py", line 598, in _setup_complete_election
    self.assertEqual(len(response.json()), NUM_VOTERS)
AssertionError: 0 != 4

======================================================================
FAIL: test_do_complete_election_private (helios.tests.ElectionBlackboxTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/dev/shm/helios-server/helios/tests.py", line 783, in test_do_complete_election_private
    election_id, username, password = self._setup_complete_election({'private_p' : "True"})
  File "/dev/shm/helios-server/helios/tests.py", line 598, in _setup_complete_election
    self.assertEqual(len(response.json()), NUM_VOTERS)
AssertionError: 0 != 4

----------------------------------------------------------------------
Ran 48 tests in 1.112s

FAILED (failures=2)
Destroying test database for alias 'default'...