Wazuh API breaks every now and then and i cant figure out why

[antonisnyc94]

Im using wazuh 4.3.8 and this is the error im receiving in my wazuh-dashboard instance. The deployment is in k8s using this repo https://github.com/wazuh/wazuh-kubernetes.

2023/03/07 14:30:03 ERROR: [Local Client] [D API] One of the API child processes terminated abruptly. The API process pool is not usable anymore. Please restart the Wazuh API
Traceback (most recent call last):
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.3.8-py3.9.egg/wazuh/core/cluster/dapi/dapi.py", line 283, in execute_local_request
    task = loop.run_in_executor(pool, partial(self.run_local, self.f, self.f_kwargs,
  File "/var/ossec/framework/python/lib/python3.9/asyncio/base_events.py", line 814, in run_in_executor
    executor.submit(func, *args), loop=self)
  File "/var/ossec/framework/python/lib/python3.9/concurrent/futures/process.py", line 681, in submit
    raise BrokenProcessPool(self._broken)
concurrent.futures.process.BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/ossec/framework/python/lib/python3.9/site-packages/wazuh-4.3.8-py3.9.egg/wazuh/core/cluster/dapi/dapi.py", line 298, in execute_local_request
    raise exception.WazuhInternalError(900)
wazuh.core.exception.WazuhInternalError: Error 900 - One of the API child processes terminated abruptly. The API process pool is not usable anymore. Please restart the Wazuh API

Error: 3002 - Request failed with status code 500
    at Function._callee$ (https://infosec.shared/1/bundles/plugin/wazuh/wazuh.plugin.js:1:102662)
    at tryCatch (https://infosec.shared/1/bundles/plugin/reportsDashboards/reportsDashboards.plugin.js:1:341529)
    at Generator.invoke [as _invoke] (https://infosec.shared/1/bundles/plugin/reportsDashboards/reportsDashboards.plugin.js:1:345493)
    at forEach.prototype.<computed> [as throw] (https://infosec.shared/1/bundles/plugin/reportsDashboards/reportsDashboards.plugin.js:1:342654)
    at asyncGeneratorStep (https://odin.prod.infosec.shared/1/bundles/plugin/wazuh/wazuh.plugin.js:1:98937)
    at _throw (https://odin.prod.infosec.shared/1/bundles/plugin/wazuh/wazuh.plugin.js:1:99334)

and

INFO: Current API id [1513629884013]
INFO: Checking current API id [1513629884013]...
INFO: Current API id [1513629884013] has some problem: 3002 - Request failed with status code 500
INFO: Getting API hosts...
INFO: API hosts found: 1
INFO: Checking API host id [1513629884013]...
INFO: Could not connect to API id [1513629884013]: 3099 - ERROR3099 - One of the API child processes terminated abruptly. The API process pool is not usable anymore. Please restart the Wazuh API
INFO: Removed [navigate] cookie
ERROR: No API available to connect

wazuh-control status

root@wazuh-manager-master-0:/# /var/ossec/bin/wazuh-control status
wazuh-clusterd is running...
wazuh-modulesd is running...
wazuh-monitord is running...
wazuh-logcollector is running...
wazuh-remoted is running...
wazuh-syscheckd is running...
wazuh-analysisd is running...
wazuh-maild not running...
wazuh-execd is running...
wazuh-db is running...
wazuh-authd is running...
wazuh-agentlessd not running...
wazuh-integratord not running...
wazuh-dbd not running...
wazuh-csyslogd not running...
wazuh-apid is running...

Please advise how to troubleshoot further.

Thanks,
Tony