pytest is unable to catch django's IntegrityError

[vanyakosmos]

pytest is unable to catch django's IntegrityError

# models.py
class A(models.Model):
    val = models.BooleanField(default=True)

class B(models.Model):
    a = models.ForeignKey(A, on_delete=models.CASCADE)

# tests.py
from django.db import IntegrityError
import pytest

@pytest.mark.django_db
def test_integrity():
    with pytest.raises(IntegrityError):
        # using object A with id=1 before creating it 
        B.objects.create(a_id=1)

After running pytest it will show two errors for test_integrity function:

• FAILURES: "DID NOT RAISE <class 'django.db.utils.IntegrityError'>"
• ERROR at teardown: "django.db.utils.IntegrityError: insert or update on table "core_b" violates foreign key constraint "core_b_a_id_2a5b83ce_fk_core_a_id"

Which is basically: "here is an error about not raising IntegrityError in test_foo, and here is an error for raising IntegrityError in test_foo, deal with it"

I can catch Integrity error w/o problems outside of pytest:

root@1f28fc583eed:/app# ./manage.py shell
Python 3.6.7 (default, Oct 24 2018, 22:47:56)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from core.tests import test_integrity
>>> test_integrity()
>>> exit()

pip list

``` Package Version ------------------- --------- apipkg 1.5 asn1crypto 0.24.0 atomicwrites 1.3.0 attrs 19.1.0 certifi 2019.3.9 cffi 1.12.3 chardet 3.0.4 codecov 2.0.15 coverage 4.5.4 cryptography 2.6.1 dj-database-url 0.5.0 Django 2.2.3 django-dbbackup 3.2.0 django-extensions 2.1.7 emoji 0.5.2 execnet 1.6.1 future 0.17.1 gunicorn 19.9.0 idna 2.8 more-itertools 7.0.0 pip 18.1 pluggy 0.11.0 psycopg2-binary 2.8.2 py 1.8.0 pycparser 2.19 pytest 4.5.0 pytest-cov 2.7.1 pytest-django 3.4.8 pytest-forked 1.0.2 pytest-mock 1.10.4 pytest-xdist 1.29.0 python-telegram-bot 12.0.0b1 pytz 2019.1 redis 3.2.1 regex 2019.4.14 requests 2.22.0 schedule 0.6.0 setuptools 40.4.3 six 1.12.0 sqlparse 0.3.0 tornado 6.0.2 urllib3 1.25.3 wcwidth 0.1.7 wheel 0.32.2 whitenoise 4.1.3 ```

Versions:

pytest: 4.5.0
django: 2.2.4

• a detailed description of the bug or suggestion
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[blueyed]

How does the stacktrace for django.db.utils.IntegrityError look like?
Please also try it with the latest pytest and pytest-django.

[vanyakosmos]

I already deleted example project, but here are error logs from another project where I test essentially the same thing: object created with id of related object that doesn't currently exist.

IntegrityError logs

______________________________ ERROR at teardown of TestReactionModel.test_safe_create_without_user ______________________________

self = <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>, sql = 'SET CONSTRAINTS ALL IMMEDIATE', params = None
ignored_wrapper_args = (False, {'connection': <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f7c95001898>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               psycopg2.errors.ForeignKeyViolation: insert or update on table "core_reaction" violates foreign key constraint "core_reaction_user_id_968339ea_fk_core_user_id"
E               DETAIL:  Key (user_id)=(111) is not present in table "core_user".

/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:82: ForeignKeyViolation

The above exception was the direct cause of the following exception:

self = <django.test.testcases.TestCase testMethod=__init__>

    def _post_teardown(self):
        """
        Perform post-test things:
        * Flush the contents of the database to leave a clean slate. If the
          class has an 'available_apps' attribute, don't fire post_migrate.
        * Force-close the connection so the next test gets a clean cursor.
        """
        try:
>           self._fixture_teardown()

/usr/local/lib/python3.6/site-packages/django/test/testcases.py:1009:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.6/site-packages/django/test/testcases.py:1177: in _fixture_teardown
    connections[db_name].check_constraints()
/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/base.py:246: in check_constraints
    self.cursor().execute('SET CONSTRAINTS ALL IMMEDIATE')
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:67: in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:76: in _execute_with_wrappers
    return executor(sql, params, many, context)
/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:84: in _execute
    return self.cursor.execute(sql, params)
/usr/local/lib/python3.6/site-packages/django/db/utils.py:89: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>, sql = 'SET CONSTRAINTS ALL IMMEDIATE', params = None
ignored_wrapper_args = (False, {'connection': <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f7c95001898>, 'cursor': <django.db.backends.utils.CursorWrapper object at 0x7f7c8e278898>})

    def _execute(self, sql, params, *ignored_wrapper_args):
        self.db.validate_no_broken_transaction()
        with self.db.wrap_database_errors:
            if params is None:
>               return self.cursor.execute(sql)
E               django.db.utils.IntegrityError: insert or update on table "core_reaction" violates foreign key constraint "core_reaction_user_id_968339ea_fk_core_user_id"
E               DETAIL:  Key (user_id)=(111) is not present in table "core_user".

/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py:82: IntegrityError

Updated pytest gang:

But the error is still present.
Also, there is another thing that I forgot to mention.

class A(Model):
    pass

class B(Model):
    a = ForeignKey(A, on_delete=CASCADE)

assert A.objects.all().count() == 0  # ok
try:
    b = B.object.create(a_id=1)
except IntegrityError:
    # will go here only if I run this code outside of pytest
    print('error')
    b = None
    # ... another logic to deal with situation
assert b is None  # will fail in pytest but will work if I run this code normally
assert b.a_id == 1  # will work in pytest
assert A.objects.count() == 0  # will work in pytest

Expected behavior

If there is no object A -> IntegrityError is thrown -> deal with it (eg create object A with id=1 and rerun).

Reality

Object B is created anyway despite having a foreign key constraint, IntegrityError (Key (a_id)=(1) is not present in table "app_a") is thrown but is not fetched by try/except. Everything is messed up.

[graingert]

have you tried:

# tests.py
from django.db import transaction, IntegrityError
import pytest

@pytest.mark.django_db
def test_integrity():
    with pytest.raises(IntegrityError):
        with transaction.atomic():
            B.objects.create(a_id=1)

[graingert]

https://docs.djangoproject.com/en/2.2/topics/db/transactions/#controlling-transactions-explicitly
Note particularly the admonition:

Avoid catching exceptions inside atomic

I believe you're seeing this because by default @pytest.mark.django_db tests run in an implicit atomic block

[vanyakosmos]

I've tried to wrap code into @transaction.atomic() like in your example, but it didn't work.

Errors are the same.

[blueyed]

@vanyakosmos
btw: pasting code is usually preferred to screenshots.

I think you need to use @pytest.mark.django_db(transaction=True) then probably. Otherwise you still have the outer atomic transaction.
You could also try to close that one before instead (since transactional tests are slow in general).

[vanyakosmos]

@pytest.mark.django_db(transaction=True) fixed everything. I didn't even need to add inner @trasaction.atomic. Thank you.

[blueyed]

For reference, this can be tested without transaction=True (which is slow):

@pytest.mark.django_db
def test_request_integrity():
    from django.db import IntegrityError
    from django.db import connection

    B.objects.create(a_id=1)
    with pytest.raises(IntegrityError) as excinfo:
        connection.check_constraints()
    assert 'Key (a_id)=(1) is not present in table "app_a"' in str(excinfo.value)

Django calls this itself in _fixture_teardown (for all DB connections): https://github.com/django/django/blob/2c66f340bb50ed6790d839157dff64456b497a43/django/test/testcases.py#L1171-L1177