Move the plugin registry to stateless

[Desvelao]

Description

As part of the issue that pretends to remove the stateful of the Wazuh dashboard plugins, we need to replace the way the data of the plugin's registry is stored so this doesn't use a file in the filesystem of the application.

Currently, this data is stored in the wazuh-registry.json file located at `/data/wazuh/config/wazuh-registry.json.

The data related to the API hosts will be stored in an in-memory cache managed instead. Any action about the API host configuration should update the registry data of the host.

Tasks

• Analyze the stored data in the registry file Move the plugin registry to stateless #6387 (comment)
• Propose the new store system Move the plugin registry to stateless #6387 (comment)
• Implement the new store system based on in-memory-cache
  • Implement the cache on the ManageHosts service that will store information about the registry
  • Remove the UpdateRegistry file
  • Initializes the cache with the API hosts at start
  • Move and remove the logic to update the registry from the frontend side to the backend side

[Desvelao]

Analysis

Description

The usage of the file is referenced here: #1465 (comment)

The current look of this file:

{
  "name": "Wazuh dashboard",
  "app-version": "4.9.0",
  "revision": "00",
  "installationDate": "2024-02-26T16:28:27.840Z",
  "lastRestart": "2024-02-26T16:28:27.840Z",
  "hosts": {
    "imposter": {
      "cluster_info": {
        "manager": "wazuh-manager-master-0",
        "node": "master-node",
        "cluster": "wazuh",
        "status": "enabled",
        "allow_run_as": 2
      }
    }
  }
}

Stored data

• App metadata:
  • Name
  • Version
  • Revision
  • Installation date
  • last restart
• Cluster information of the API hosts:
  • Manager
  • Node name
  • Cluster name
  • Status
  • Allow run as

Usage

The App metadata is used:

• Frontend:
  • About application:
    • Version
    • Revision
    • Installation date
  • Mechanism to check if the plugin files are cached in the browser-side
• Backend:
  • GET /api/setup: returns the data stored in the registry file
  • GET /api/timestamp: unused by frontend side

The Name and lastRestart fields are not used. We could remove them if they are considered as not required.

The data related to the hosts is updated through the frontend or backend side (based on the current status of 4.9.0 development branch):

• Frontend:
  • Update in Server APIs when checking the connection to an API
  • Update in Server APIs table when using the refresh button
  • Update in API host selector when changing the API host
  • Update in API is down view when checking the connection
  • Get from /hosts/apis to retrieve the hosts data including the cluster_info

I am not sure because the cluster info is set through the frontend side instead of done by the backend side. All the actions use the GET /hosts/update-hostname/{id} API endpoint.

For this case, the data is coming from the GET /apis/check-api API endpoint

• Backend:
  • Update when checking the stored API of a client
  • Get from API endpoints:
    • GET /hosts/apis: returns data about host including the cluster info stored in the registry file.
  • Get the hosts data on the statistics job to decide the endpoints where to retrieve the data depending on whether the cluster is enabled or not

Note that the monitoring job checks the status of the cluster using the API endpoint. When the cluster is enabled, then it retrieves the cluster name data that is set as the clusterName of the documents. This reflects a different behavior of the statistics job.

Service

The management of registry file wazuh-registry.json is managed through the UpdateRegistry service. This has methods to:

• Read content
• Get data related to hosts (all or by ID)
• Write content
• Check if the host with ID exists
• Migrate to the registry
• Update the cluster info
• Update API extensions
• Remove host entries
• Remove orphan entries
• Get token by ID
• Update token by ID

Unused methods that could be removed:

• Get data related to hosts only for ID
• Check if the host with ID exist
• Migrate to registry
• Update API extensions
• Get token by ID
• Update token by ID

Conclusion

The current approach stores some data related to the plugin and cluster information about the API hosts.

The app metadata could be removed because:

• The installation and restart dates are not considered to keep
• Name is unused
• Version and revision can be found in the package.json instead of using the value stored in the registry file.

This leaves the data of the cluster information about the API host entries. Taking into account we want to remove the file wazuh-registry.json file of the filesystem, we could need another way to store this information.

The unused methods of the UpdateRegistry service and the unused API endpoints could be removed too.

[Desvelao]

Changes

• Remove unused API endpoint GET /api/timestamp
• Remove unused fields and unwanted from registry file
• Remove unused methods of UpdateRegistry service
• Adapted GET /api/setup to use version and revision information of plugin definition instead of stored in the registry file

[Desvelao]

Design

We will apply the following actions to the registry data:

• Remove the plugins metadata:
  • Name
  • Version
  • Revision
  • Installation date
  • last restart
• Move the cluster info to another store system based on in-memory cache

Note: Version and revision will be used from the package manifest instead.

The cluster info about the API hosts will be managed by the ManageHost service using an in-memory cache.

The cache stores information about the API host data:

interface IAPIHostRegistry {
  manager: string | null;
  node: string | null;
  status: string;
  cluster: string;
  allow_run_as: API_USER_STATUS_RUN_AS;
}

When the plugin starts, the ManageHosts service initiates the cache getting the data from the API hosts.

We will move the cache related to the allow_run_as for each API from the cacheAPIUserAllowRunAs to the cache of the registry.

This approach causes the UpdateRegistry service that managed the update of the registry file to be not needed anymore, so this will be removed.

The logic related to updating the cluster info in the registry and removing orphan entries in the registry is done by the frontend side, it is moved to the backend side.

[Desvelao]

Changes

• Update changelog of pull request Move the registry to in-memory cache #6481