[Bug]: TA fails run django unittest

[stratus-ss]

I've read the documentation

• I'm running the latest version of Tube Archivist and have read the release notes.
• I have read the how to open an issue guide, particularly the bug report section.

Operating System

RHEL 9.2

Your Bug Report

Describe the bug

Django's built in test framework fails when no tests are added to the code base of TA

Expected behavior

When creating a new Django app with nothing in it, you should be able to run the test command and have it tell you that there are no tests configured.

[root@kid-tube-clone test-django]# ls
core  manage.py  task1
[root@kid-tube-clone test-django]# cd ../
[root@kid-tube-clone ~]# cd test-django/
[root@kid-tube-clone test-django]# python3 manage.py test
Found 0 test(s).
System check identified no issues (0 silenced).

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

Attempting the same in TA results in the following:

[root@kid-tube-clone tubearchivist]# python3 manage.py test
Traceback (most recent call last):
  File "/root/testing/tubearchivist/tubearchivist/manage.py", line 23, in <module>
    main()
  File "/root/testing/tubearchivist/tubearchivist/manage.py", line 19, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/usr/local/lib/python3.9/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.9/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python3.9/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/testing/tubearchivist/tubearchivist/config/settings.py", line 20, in <module>
    from home.src.ta.config import AppConfig
  File "/root/testing/tubearchivist/tubearchivist/home/__init__.py", line 3, in <module>
    from .tasks import app as celery_app
  File "/root/testing/tubearchivist/tubearchivist/home/tasks.py", line 12, in <module>
    from home.src.download.queue import PendingList
  File "/root/testing/tubearchivist/tubearchivist/home/src/download/queue.py", line 10, in <module>
    from home.src.download.subscriptions import (
  File "/root/testing/tubearchivist/tubearchivist/home/src/download/subscriptions.py", line 8, in <module>
    from home.src.download.thumbnails import ThumbManager
  File "/root/testing/tubearchivist/tubearchivist/home/src/download/thumbnails.py", line 13, in <module>
    from home.src.es.connect import ElasticWrap, IndexPaginate
  File "/root/testing/tubearchivist/tubearchivist/home/src/es/connect.py", line 16, in <module>
    class ElasticWrap:
  File "/root/testing/tubearchivist/tubearchivist/home/src/es/connect.py", line 35, in ElasticWrap
    data: bool | dict = False,
TypeError: unsupported operand type(s) for |: 'type' and 'type'

Steps To Reproduce

1. download TA:

git clone https://github.com/tubearchivist/tubearchivist.git

Cloning into 'tubearchivist'...
remote: Enumerating objects: 10832, done.
remote: Counting objects: 100% (2091/2091), done.
remote: Compressing objects: 100% (469/469), done.
remote: Total 10832 (delta 1763), reused 1862 (delta 1602), pack-reused 8741
Receiving objects: 100% (10832/10832), 9.88 MiB | 17.48 MiB/s, done.
Resolving deltas: 100% (8033/8033), done.

2. Run the test command:

cd tubearchivist/tubearchivist
 
python3 manage.py test

This behaviour is preventing the creation of new test cases for any new PRs

Relevant log output

No logs, this is just attempting to run test cases

Anything else?

No response

[stratus-ss]

So I tracked this particular error down to the version of python. the single | operator was introduced in python 3.10. RHEL 9 runs 3.9 by default. I installed

python3.11
python3.11-devel
openldap-devel

Setup a new venv and this error was replaced by another error:

python3.11 manage.py test
... Redis connection failed, retry [0/10]
... Redis connection failed, retry [1/10]
... Redis connection failed, retry [2/10]
... Redis connection failed, retry [3/10]
... Redis connection failed, retry [4/10]
... Redis connection failed, retry [5/10]
... Redis connection failed, retry [6/10]
... Redis connection failed, retry [7/10]
... Redis connection failed, retry [8/10]
... Redis connection failed, retry [9/10]
Traceback (most recent call last):
  File "/app/tubearchivist/tubearchivist/manage.py", line 23, in <module>
    main()
  File "/app/tubearchivist/tubearchivist/manage.py", line 19, in main
    execute_from_command_line(sys.argv)
  File "/root/tube-archive-venv/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/root/tube-archive-venv/lib64/python3.11/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/root/tube-archive-venv/lib64/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/root/tube-archive-venv/lib64/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/tube-archive-venv/lib64/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/tubearchivist/tubearchivist/config/settings.py", line 20, in <module>
    from home.src.ta.config import AppConfig
  File "/app/tubearchivist/tubearchivist/home/__init__.py", line 3, in <module>
    from .tasks import app as celery_app
  File "/app/tubearchivist/tubearchivist/home/tasks.py", line 12, in <module>
    from home.src.download.queue import PendingList
  File "/app/tubearchivist/tubearchivist/home/src/download/queue.py", line 10, in <module>
    from home.src.download.subscriptions import (
  File "/app/tubearchivist/tubearchivist/home/src/download/subscriptions.py", line 8, in <module>
    from home.src.download.thumbnails import ThumbManager
  File "/app/tubearchivist/tubearchivist/home/src/download/thumbnails.py", line 21, in <module>
    class ThumbManagerBase:
  File "/app/tubearchivist/tubearchivist/home/src/download/thumbnails.py", line 24, in ThumbManagerBase
    CONFIG = AppConfig().config
             ^^^^^^^^^^^
  File "/app/tubearchivist/tubearchivist/home/src/ta/config.py", line 24, in __init__
    self.config = self.get_config()
                  ^^^^^^^^^^^^^^^^^
  File "/app/tubearchivist/tubearchivist/home/src/ta/config.py", line 29, in get_config
    config = self.get_config_redis()
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/tubearchivist/tubearchivist/home/src/ta/config.py", line 83, in get_config_redis
    raise ConnectionError("failed to connect to redis")
ConnectionError: failed to connect to redis

[bbilly1]

I was stuck at the same place before. As the application is setup now, this needs a working redis connection before any manage.py command can run. Last time I research this, I found various approaches:

• Run the tests in a complete setup, as in redis + ES + django, with something like docker exec -it tubearchivist python manage.py test, currently that will bring other problems, but theoretically, that should be solvable.
• Implement logic in the application to not connect to redis when running tests, with something like an env var that will skip these lines in question
• Use moking to simulate redis

Ultimately I don't have a solution, that's why this has been in #415 for a long time...

Some other expert out there has a solution?

[stratus-ss]

I'll see if I can come up with something simply for mock

[bbilly1]

v0.4.8 fixes that problem, now running pytest, with already a few tests. please help with extending that.