Redis issue with previewgenerator -> 503 server

[pjrobertson]

EDIT

My redis-server seems to be having all kinds of problems this morning, so that appears to be the main root cause of the problem. But I think the bug report here is still valid. It seems like there is no way to change the redis settings for the previewgenerator app, or it does not honour the settings in config.php when they are updated. Perhaps there is something I am missing, but I think that needs looking into.

---

Original report:

My server suddently started returning 503s today. Disabling the previewgenerator app solved the problem. I originally thought it was a redis problem, because of the error message, but it was not. Removing the redis conf settings from settings.conf did not do anything. The only way to solve the problem was to disable the app using occ app:disable previewgenerator

Here's the log:

{
  "reqId": "7mugNd3ki2cFu7W0SOwc",
  "level": 3,
  "time": "2023-09-14T09:27:47+08:00",
  "remoteAddr": "13.40.122.214",
  "user": "--",
  "app": "core",
  "method": "GET",
  "url": "/index.php/apps/files/preview-service-worker.js",
  "message": "NOAUTH Authentication required.",
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/117.0",
  "version": "27.0.2.1",
  "exception": {
    "Exception": "RedisException",
    "Message": "NOAUTH Authentication required.",
    "Code": 0,
    "Trace": [
      {
        "file": "/var/www/nextcloud/lib/private/Memcache/Redis.php",
        "line": 72,
        "function": "get",
        "class": "Redis",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/L10N/Factory.php",
        "line": 346,
        "function": "get",
        "class": "OC\\Memcache\\Redis",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/L10N/Factory.php",
        "line": 483,
        "function": "findAvailableLanguages",
        "class": "OC\\L10N\\Factory",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/L10N/Factory.php",
        "line": 219,
        "function": "getLanguageFromRequest",
        "class": "OC\\L10N\\Factory",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/public/Util.php",
        "line": 227,
        "function": "findLanguage",
        "class": "OC\\L10N\\Factory",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/legacy/OC_Template.php",
        "line": 111,
        "function": "addTranslations",
        "class": "OCP\\Util",
        "type": "::"
      },
      {
        "file": "/var/www/nextcloud/lib/private/legacy/OC_Template.php",
        "line": 77,
        "function": "initTemplateEngine",
        "class": "OC_Template",
        "type": "::"
      },
      {
        "file": "/var/www/nextcloud/lib/private/legacy/OC_Template.php",
        "line": 321,
        "function": "__construct",
        "class": "OC_Template",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/index.php",
        "line": 74,
        "function": "printExceptionErrorPage",
        "class": "OC_Template",
        "type": "::"
      }
    ],
    "File": "/var/www/nextcloud/lib/private/Memcache/Redis.php",
    "Line": 72,
    "CustomMessage": "--"
  }
}```

[st3iny]

The error returned by redis indicates that Nextcloud attempted a connection without providing the correct password.

Please confirm that you are using an authenticated redis server and that you have set the correct password in config.php. Just to be sure that this is not the problem.

Also please keep in mind, that multiple config files might overwrite each other in some situations which also might lead to issues like this. E.g config/config.php and config/redis.config.php. If that is the case in your setup please make sure that you fix the redis config in all config files.

Otherwise, there indeed might be a bug somewhere in the code. Perhaps the password is not loaded and applied.

[pjrobertson]

The password is correct and I can login no problem using redis-cli. I think you are right that the issue is with caching.

Even if I comment out all the caching settings in config.php (to disable redis caching), the previewgenerator still seems to try to load redis. Even after restarting apache/php-fpm/redis. I don't have a redis.config.php file.

[joshtrichards]

It seems like there is no way to change the redis settings for the previewgenerator app, or it does not honour the settings in config.php when they are updated. Perhaps there is something I am missing, but I think that needs looking into.

The previewgenerator app doesn't interact with Redis.

The code path in question - in the provided stack trace - is in server. If there is/was a problem, it was/would be likely impacting any elements of server that access your Redis server. Every Redis retrieval would have been impacted.

Without more details about your environment it's impossible to do more than speculate about the underlying cause.

The only explanations I can come up with regard to why disabling the previewgenerator app seemed to fix the matter:

• timing (if some sort of caching issue somewhere)
• maybe FPM wasn't really fully restarted (?)
• disabling an app may do some OPcache clearing; I don't recall offhand

Even if I comment out all the caching settings in config.php (to disable redis caching), the previewgenerator still seems to try to load redis. Even after restarting apache/php-fpm/redis. I don't have a redis.config.php file.

Did you actually run any previewgenerator commands (occ preview:*) during this event?

Out of curiosity, was previewgenerator the very first app you tried disabling when this occurred?

My redis-server seems to be having all kinds of problems this morning, so that appears to be the main root cause of the problem. But I think the bug report here is still valid.

What kind of problems?