Running ChannelsLiveServerTestCase does not create database for browser

[alfonsrv]

Using ChannelsLiveServerTestCase as described in documentation's Tutorial Part 4: Automated Testing yields the yellow error page connection to server at "127.0.0.1", port 5432 failed: FATAL: database "test" does not exist. Running database-altering code within a test function works and the database is available, but the frontend worker serving the browser request (using Daphne) seems to be totally detached from that database. Somehow the asgiref workers are working in another thread?

I created a small test repository with a small readme to reproduce the issue: https://github.com/alfonsrv/django-asgitest

Tried using macOS Ventura + Ubuntu 22.04 with Chrome.

[carltongibson]

Are you sure your environment variables are being set correctly? (I'd expect the test DB name to be test_testasgi from the sample .env file + settings.py you have in your repo.)

[alfonsrv]

Pardon me, coming from pytest I was unaware unittest already required a database and then simply cloned the already-existing one, prefixing it with test_. While the database error is gone, data created within a test function still cannot be used from the browser.

E.g. using the following code does not allow the user to log into the admin using the browser:

user_staff = User.objects.create(
    username='test',
    email='foo@bar.com',
    password=make_password('hunter42'),
    is_staff=True,
    is_superuser=True
)
self.client.login(
    username=user_staff.username, 
    password='hunter42'
)
# returns True

self.driver.get(self.live_server_url + "/admin/")
self.driver.find_element('name', 'username').send_keys(user_staff.username)
self.driver.find_element('name', 'password').send_keys('hunter42')
self.driver.find_element(By.CSS_SELECTOR, "input[type='submit']").click()
# error in admin - login does not work

[carltongibson]

Ok, but in general that works, so you need to put a breakpoint there and see why the login isn't working. (This isn't really a channels issue.)

[alfonsrv]

It's not a channels issue per-se, but what I assume is an asgiref issue / ChannelsLiveServerTestCase issue, since the intended use to test the frontend does not work as outlined in the docs.

It's like the database is out of sync between the sync test routing and the async server worker thread. I threw in a pdb in the test function, but everything there is working fine there. Not sure how I would debug the server function serving the frontend.

[carltongibson]

It works if you follow the steps in the tutorial. I'm not sure what else I can say given the info provided. Sorry.

[alfonsrv]

I just followed the steps in the tutorial and setting up a test database that is available via the frontend does not work as expected.

Users that are created as part of the regular Python functions are not available to the frontend (see chat.tests.DatabaseTests). The only reason the tutorial "works" is because it does not use any part of the ORM but unauthenticated views testing the mere websockets functionality.

One could argue that this is a sufficient for unit tests and the rest if out of scope for this library. However I would argue if you provide a dedicated test server for the use case, it should work with the ORM, because it's unlikely that it's used only with unauthenticated interactions.

[carltongibson]

OK, your test project has way too much going on to identify any issue. Can you trim it down to a minimal please, with a clear error case, rather than a sleep. (The initial ticket here was "Database not created", so if you can clarify that too, that would be great.

I'd move the user creation into setup, and then inspect the DB to make sure that's actually committed. Then compare Django's Admin Selenium test cases, which do this at length.

[alfonsrv]

OK, your test project has way too much going on to identify any issue

Not sure what exactly you are referring to. I simply implemented the tutorial boilerplate in chat and config-related settings as requested.

I slimmed down chat.tests – including a SyncDatabaseTests and AsyncDatabaseTests, both testing for the same result, only using the different servers. The former one works.

then inspect the DB to make sure that's actually committed. Then compare Django's Admin Selenium test cases, which do this at length

From what I can see in e.g. the Django admin_changelist test, the user is simply created, then used to log in thereafter. Nevertheless I added a simple check to test_user_model_creation that asserts the user is present before opening the browser.

[carltongibson]

Ok, thanks. I will try and have a look, and perhaps expand the tutorial example to show interactions with the DB.

[alfonsrv]

Heya! Have you had a chance to look into it?

[carltongibson]

Hi @alfonsrv — no, not as yet. It's on my list for later in the autumn, but short of someone (you maybe?) digging deeper (and highlighting either the error in the project or the bug in the test case) it'll have to wait until I can get to it.

[ashtonrobinson]

#2033

[alfonsrv]

@ashtonrobinson I tried using your PR but it leads to a ProgrammingError at /chat/test/ yellow error page, as if migrations didn't run for the frontend / test database.

But it seems like you're onto something – because using the async test case (chat.tests.AsyncDatabaseTests) I created, it appears to use the default database instead of the test database.

And users that are created as part of python manage.py createsuperuser are listed in the AsyncDatabaseTests frontend, while they are not when using the regular unit test SyncDatabaseTests.

[carltongibson]

From the PR...

a multiprocessing.Process object which takes care of creating a new process. This uses the SpawnProcess of multiprocessing class on Windows and macOS. Linux uses the Fork method...

Ah, spawn vs fork

This will be related to the discover runner updates in Django 4.1. https://docs.djangoproject.com/en/5.0/releases/4.1/#tests

See https://code.djangoproject.com/ticket/31169 — We'll likely need to use some of the added test setup steps.

Progress. 🎁

[carltongibson]

Just to add in the meantime, if the spawn startup is the issue running the tests with --parallel=1 should function as a workaround.

[alfonsrv]

That doesn't work either.

How can something as fundamental as tests (albeit in the context of database interactions) be kept broken for what is essentially a core Django library for this long?

[carltongibson]

@alfonsrv by it not being exercised. Django is built on volunteer effort, so short of someone picking it up and investigating it can take a while.

Since you're using this, perhaps you could be the person to work out the cause and resolve it.

[alfonsrv]

I tried looking into it, but I'm afraid it's a bit too advanced for my understanding.

Willing to sponsor and offer a bounty though.

[carltongibson]

@alfonsrv the first step would be a failing test case demonstrating the issue. If folks can instantly get to the problem it's much easier to work on.

[alfonsrv]

That exists: https://github.com/alfonsrv/django-asgitest – the test cases inherit the same base class, to run the same code both Sync and Async.

Mentioned the repo here previously. It has some boilerplate code due to following the channels tutorial – as I was asked to previously – but the test cases are very basic.

[carltongibson]

I really meant as a unit test against the channels test suite, but yes.

The bottom line is that until somebody can find the time to dig into it, this will sit there. It's on my list, but I have quite a few 🤹 other things which are more pressing to me, so it's not something I'm going to get to quickly.