Lock issues

[iamshoXy]

Describe the issue

I've encountered several lock timeout errors (around 70) within a day. Is there any way to improve the "stale locks"?
Some team members have issues translating a specific string while others can update it.

There is also another case where a member cannot update a string from a newly imported translation file (about 30 minutes ago)

What can be done to reduce the occurrence or does it just happen sometimes? This unfortunately slows down the process abit

I already tried

• I've read and searched the documentation.
• I've searched for similar filed issues in this repository.

Steps to reproduce the behavior

Translated a string and clicked on "Save and continue", I cannot really provide reproduction steps right now.

Expected behavior

No response

Screenshots

Exception traceback

No response

How do you run Weblate?

Docker container

Weblate versions

• Weblate: 5.5.4-dev
• Django: 5.0.4
• siphashc: 2.4.1
• translate-toolkit: 3.13.0
• lxml: 5.2.1
• pillow: 10.3.0
• nh3: 0.2.17
• python-dateutil: 2.9.0.post0
• social-auth-core: 4.5.4
• social-auth-app-django: 5.4.1
• django-crispy-forms: 2.1
• oauthlib: 3.2.2
• django-compressor: 4.4
• djangorestframework: 3.15.1
• django-filter: 24.2
• django-appconf: 1.0.6
• user-agents: 2.2.0
• filelock: 3.14.0
• rapidfuzz: 3.9.0
• openpyxl: 3.1.2
• celery: 5.4.0
• django-celery-beat: 2.6.0
• kombu: 5.3.7
• translation-finder: 2.16
• weblate-language-data: 2024.5
• html2text: 2024.2.26
• pycairo: 1.26.0
• PyGObject: 3.48.2
• diff-match-patch: 20230430
• requests: 2.31.0
• django-redis: 5.4.0
• hiredis: 2.3.2
• sentry-sdk: 1.45.0
• Cython: 3.0.10
• misaka: 2.1.1
• GitPython: 3.1.43
• borgbackup: 1.2.8
• pyparsing: 3.1.2
• ahocorasick_rs: 0.22.0
• python-redis-lock: 4.0.0
• charset-normalizer: 3.3.2
• Python: 3.12.3
• Git: 2.39.2
• psycopg: 3.1.18
• psycopg-binary: 3.1.18
• phply: 1.2.6
• ruamel.yaml: 0.18.6
• tesserocr: 2.7.0
• boto3: 1.34.98
• zeep: 4.2.1
• aeidon: 1.14.1
• iniparse: 0.5
• mysqlclient: 2.2.4
• Mercurial: 6.7.2
• git-svn: 2.39.2
• git-review: 2.4.0
• PostgreSQL server: 16.2
• Database backends: django.db.backends.postgresql
• PostgreSQL implementation: psycopg3 (binary)
• Cache backends: default:RedisCache, avatar:FileBasedCache
• Email setup: django.core.mail.backends.smtp.EmailBackend: smtp.gmail.com
• OS encoding: filesystem=utf-8, default=utf-8
• Celery: redis://cache:6379/1, redis://cache:6379/1, regular
• Platform: Linux 6.1.0-18-amd64 (x86_64)

Weblate deploy checks

No response

Additional context

No response

[iamshoXy]

I have around 2.5k pending changes that I cannot commit and I have been waiting a few hours already :/

Could not lock the repository, another operation is in progress.

[nijel]

Can you check logs what it taking that long?

[iamshoXy]

I took a closer look at the logs but did not find anything specific.

WARNING/440] Handled exception: WeblateLockTimeoutError: Lock on lock:repo:1 could not be acquired in 120s

Here's the stack trace from Sentry:

WeblateLockTimeoutError: Lock on lock:repo:1 could not be acquired in 120s
  File "django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "contextlib.py", line 81, in inner
    return func(*args, **kwds)
  File "weblate/trans/views/edit.py", line 641, in translate
    response = handle_translate(request, unit, this_unit_url, next_unit_url)
  File "weblate/utils/ratelimit.py", line 132, in rate_wrap
    return function(request, *args, **kwargs)
  File "weblate/trans/views/edit.py", line 421, in handle_translate
    go_next = perform_translation(unit, form, request)
  File "weblate/trans/views/edit.py", line 335, in perform_translation
    saved = unit.translate(
  File "contextlib.py", line 81, in inner
    return func(*args, **kwds)
  File "weblate/trans/models/unit.py", line 1516, in translate
    saved = self.save_backend(
  File "weblate/trans/models/unit.py", line 1092, in save_backend
    self.commit_if_pending(author)
  File "weblate/trans/models/unit.py", line 1067, in commit_if_pending
    self.translation.commit_pending("pending unit", None)
  File "contextlib.py", line 81, in inner
    return func(*args, **kwds)
  File "weblate/trans/models/translation.py", line 558, in commit_pending
    return self.component.commit_pending(reason, user, skip_push=skip_push)
  File "weblate/trans/models/component.py", line 200, in on_link_wrapper
    return func(self, *args, **kwargs)
  File "weblate/trans/models/component.py", line 1967, in commit_pending
    with self.repository.lock:
  File "weblate/utils/lock.py", line 86, in __enter__
    self._enter_implementation()
  File "weblate/utils/lock.py", line 70, in _enter_redis
    raise WeblateLockTimeoutError(

The website was very busy at the beginning which may have caused this massive issue but things have calmed down now and the commits have been pushed. Unfortunately, it took quite a long time (like 8 hours or so until no error occurred again)

[nijel]

There had to be another operation in progress that should be visible in the logs.

[iamshoXy]

gunicorn stderr | [2024-05-17 10:19:19,947: INFO/360777] Acquired Lock('lock:lock:repo:1').
gunicorn stderr | [2024-05-17 10:19:19,948: INFO/360777] project/languages: updating repository
gunicorn stderr | [2024-05-17 10:19:23,394: INFO/360777] project/languages: update took 3.45 seconds
gunicorn stderr | [2024-05-17 10:19:23,399: INFO/360777] project/languages: repository up to date at bfc1ae78e39a567a4d3bea12517f62ee2a6254fd
gunicorn stderr | [2024-05-17 10:19:23,443: INFO/360777] Acquired Lock('lock:lock:repo:1').
gunicorn stderr | [2024-05-17 10:19:23,443: INFO/360777] project/languages: pushing to remote repo

Not an error but I wonder if there's an issue pushing here? Is there a way to get more logs about the process? After clicking on push, it just loads indefinitely

[nijel]

It basically just calls git push, maybe there is some networking issue?

[iamshoXy]

There were some network issues, so the push took longer than usual. I removed the remote repo to make sure it works, and it definitely does!

[github-actions]

The issue you have reported is now resolved. If you don’t feel it’s right, please follow its labels to get a clue for further steps.

• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.