Move the registry to in-memory cache #6481

Desvelao · 2024-03-06T11:18:00Z

Description

This pull request moves the registry to an in-memory cache and removes some logic done by the frontend side that should do the backend side.

Changes

Move the registry management to an in-memory cache managed by ManageHosts
service.
Initialize the registry data on the start
- Update the registry on adding, editing, or deleting an API host
Move the logic to check if the run_as is enabled to the ManageHosts
instead of cacheAPIUserAllowRunAs
Update the backend types of the core plugin
Remove the process to create/update the registry file when the plugin starts
Remove the Installation date from the About view
Remove the cacheAPIUserAllowRunAs of the ManageHosts service to use
the registry data
Remove the cacheAPIUserAllowRunAs property of Manage hosts
Remove the UpdateRegistry service
Remove API endpoint PUT /hosts/update-hostname/{id}
- Remove API endpoint (route and controller)
- Remove API request of frontend side
Remove logic to create the registry file on start
Remove API endpoint and usage of GET /hosts/remove-orphan-entries
- Remove API endpoint (route and controller)
- Remove API request of frontend side
Remove unused API endpoint GET /api/timestamp

Issues Resolved

#6387

Evidence

Test

TEST

With an API connection using an with run_as capability enabled with run_as disabled, edit the API connection setting run_as as true, save. The Run as column should display the run_as is enabled with a check

Context: @yenienserrano

6387-registry-in-memory-cache

Legend:
⚫: none
🟢: pass
🟡: warning
🔴: fail
⚪: not applicable

UI

Test	Chrome	Firefox	Safari
Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.	⚫	⚫	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.	⚫	⚫	⚫
Go to About, it only should display the version and revision of the application. No reference to installation date.	⚫	⚫	⚫

Details

⚫ Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to About, it only should display the version and revision of the application. No reference to installation date.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

6387-registry-in-memory-cache

Legend:
⚫: none
🟢: pass
🟡: warning
🔴: fail
⚪: not applicable

UI

Test	Chrome	Firefox	Safari
Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.	⚫	⚫	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.	⚫	⚫	⚫
Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.	⚫	⚫	⚫
Go to About, it only should display the version and revision of the application. No reference to installation date.	⚫	⚫	⚫

Details

⚫ Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

⚫ Go to About, it only should display the version and revision of the application. No reference to installation date.

Chrome - ⚫

Firefox - ⚫

Safari - ⚫

Check List

All tests pass
- yarn test:jest
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

- Enhance the logging messages - Minor enhancements

- Replace the logging services - Move services to core plugin - CacheAPIUserAllowRunAs - ManageHosts - ServerAPIClient (aka api-interceptor) - ServerAPIHostEntries (aka ) - UpdateConfigurationFile - UpdateRegistry - Adapt the usage of services in the main and check updates plugin from core plugin - Remove plugin logger from main and core plugins - Remove API endpoint - GET /utils/logs/ui - Remove unused code - Adapt the test suites

…to enhancement/6137-adapt-logging-method

… yarn.lock files

…to enhancement/6137-adapt-logging-method

- Remove GET /utils/logs endpoint - Remove related tests - Remove App logs application

- Create Configuration service - Create ConfigurationStore (backend and frontend side) - Register the plugin settings in the configuration services

…ration service - API endpoints: - GET /utils/configuration - PUT /utils/configuration - PUT /utils/configuration/files/{key} - DELETE /utils/configuration/files/{key}

…on service

… service

…ttings

Desvelao · 2024-03-12T08:11:09Z

Changes

Updated branch with the last changes of 4.9.0
Solve conflicts
Fix failing unit tests

plugins/main/server/controllers/wazuh-api.ts

yenienserrano · 2024-03-18T14:32:40Z

UI

Test	Chrome	Firefox	Safari
Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	⚫	🟢
Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	⚫	🟢
Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.	⚫	⚫	🟢
Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.	⚫	⚫	🟢
Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.	⚫	⚫	🟢
Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.	⚫	⚫	🟢
Go to About, it only should display the version and revision of the application. No reference to installation date.	⚫	⚫	🟢

Details

🟢 Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

🟢 Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

🟢 Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

🟢 Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

🟢 Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

🟢 Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

🟢 Go to About, it only should display the version and revision of the application. No reference to installation date.

Chrome - ⚫

Firefox - ⚫

Safari - 🟢

yenienserrano · 2024-03-18T16:52:37Z

when I delete an API I get an error message but if I press the refresh button the API seems to have been deleted correctly because it does not appear in the table.

Tostti

UI

Test	Chrome	Firefox	Safari
Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	🟢	⚫
Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.	⚫	🟢	⚫
Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.	⚫	🟢	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.	⚫	🟢	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.	⚫	🟢	⚫
Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.	⚫	🟢	⚫
Go to About, it only should display the version and revision of the application. No reference to installation date.	⚫	🟢	⚫
Changing run as status	⚫	🔴	⚫
Removing an API	⚫	🔴	⚫

Details

🟢 Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🟢 Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🟢 Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🟢 Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🟢 Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🟢 Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🟢 Go to About, it only should display the version and revision of the application. No reference to installation date.

Chrome - ⚫

Firefox - 🟢

Safari - ⚫

🔴 Changing run as status

Chrome - ⚫

Firefox - 🔴

Although the change is applied, it is not shown in the table

Safari - ⚫

🔴 Removing an API

Chrome - ⚫

Firefox - 🔴

If an API is removed (online or offline), this error is shown and the table have to be manually refreshed to show the change

Safari - ⚫

plugins/main/server/controllers/wazuh-api.ts

yenienserrano · 2024-03-18T17:18:12Z

when I edit an API I get an error message but if I press the update button the API seems to have been edited correctly.

…com/wazuh/wazuh-kibana-app into enhancement/6387-registry-stateless

…g the id

Desvelao · 2024-03-20T09:26:16Z

Changes

Fixed an error deleting the API host
Fixed an error when editing an API connection changing the ID. Related to Move the registry to in-memory cache #6481 (comment)
Fixed an error when enabling the Run as and this doesn't display as enabled in the API connection table: Related to Move the registry to in-memory cache #6481 (review)

…o enhancement/6387-registry-stateless

github-actions · 2024-04-03T07:41:05Z

Wazuh Core plugin code coverage (Jest) test	%	values
Statements	31.12%	( 286 / 919 )
Branches	29.88%	( 107 / 358 )
Functions	28.1%	( 86 / 306 )
Lines	30.71%	( 278 / 905 )

github-actions · 2024-04-03T07:41:11Z

Wazuh Check Updates plugin code coverage (Jest) test	%	values
Statements	76.44%	( 172 / 225 )
Branches	58.65%	( 61 / 104 )
Functions	61.7%	( 29 / 47 )
Lines	76.44%	( 172 / 225 )

github-actions · 2024-04-03T07:45:00Z

Main plugin code coverage (Jest) test	%	values
Statements	11.99%	( 4152 / 34612 )
Branches	7.77%	( 1764 / 22691 )
Functions	10.7%	( 881 / 8230 )
Lines	12.18%	( 4043 / 33179 )

asteriscos · 2024-04-03T09:42:24Z

UI

Test	Chrome	Firefox	Safari
Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.	🟢	⚫	⚫
Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.	🟢	⚫	⚫
Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.	🟢	⚫	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.	🟢	⚫	⚫
Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.	🟢	⚫	⚫
Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.	🟢	⚫	⚫
Go to About, it only should display the version and revision of the application. No reference to installation date.	🟢	⚫	⚫

Details

🟢 Go to Server APIs, and add an online API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

🟢 Go to Server APIs, and edit an offline API host that can be connected. The data about the cluster name or manager name should appear in the table.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

🟢 Go to Server APIs, and edit an online API host that can not be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

🟢 Go to Server APIs, change the cluster name or manager name of an offline API connection, and edit the API connection that can be connected. The new data about the cluster name or manager name should appear in the table.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

🟢 Go to Server APIs, change the cluster name or manager name of an offline API connection, and click on the Refresh button. The new data about the cluster name or manager name should appear in the table.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

🟢 Go to Server APIs, and add an offline API host that can be connected. The data about the cluster name or manager name should not appear in the table.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

🟢 Go to About, it only should display the version and revision of the application. No reference to installation date.

Chrome - 🟢

Firefox - ⚫

Safari - ⚫

Desvelao added 30 commits November 21, 2023 09:32

feat(logging): remove plugin logger from start tasks of main plugin

701b9aa

- Enhance the logging messages - Minor enhancements

feat(logging): minor fixes to logging messages in the main plugin

70f02f4

feat(logging): remove custom logger from endpoints of main plugin

c3ba9d1

fix(logging): remove parameter of addJobToQueue

7c34345

Merge branch 'master' of https://github.com/wazuh/wazuh-kibana-app in…

283788e

…to enhancement/6137-adapt-logging-method

feat(logging): removed constants related to log files path

842b4a3

fix(dependency): removed winston dependency of plugins and update the…

dad9274

… yarn.lock files

fix(logging): removed logs.level plugin setting

3914913

fix(logging): fixed monitoring and statistics backend tasks

20ed9bc

Merge branch 'master' of https://github.com/wazuh/wazuh-kibana-app in…

dba3e2f

…to enhancement/6137-adapt-logging-method

fix(monitoring): manage API host entries

2d0053b

feat(core): add description to core services classes

a9de346

feat(docs): add docs to core plugin

20b0f90

feat(docs): add description to core services

9ca647e

test: fix of check updates plugin

54187b5

test: fix of check updates plugin

efc99e2

feat(logging): remove the App logs application

d726682

- Remove GET /utils/logs endpoint - Remove related tests - Remove App logs application

fix: typo in UpdateRegistry service

87b2b18

feat: add InputFormPassword component

6042fe5

feat: add Configuration and ConfigurationStore services for core plugin

ac8b873

- Create Configuration service - Create ConfigurationStore (backend and frontend side) - Register the plugin settings in the configuration services

feat: adapt API endpoints related to configuration to use the Configu…

b0fd099

…ration service - API endpoints: - GET /utils/configuration - PUT /utils/configuration - PUT /utils/configuration/files/{key} - DELETE /utils/configuration/files/{key}

feat: adapt the monitoring task of main plugin to use the Configurati…

1edc282

…on service

fix: remove unwanted log

9d24c19

feat: adapt API endpoints related to sample data to the configuration…

322abe1

… service

feat: enhance plugin settings to define the mapping in the saved object

3fb4c1d

remove: remove buildIndexSettings service

60bd3e4

feat(configuration): remove reference to configuration file in App Se…

25c68fa

…ttings

feat(uilib): create buttons that opens flyouts

0c4f64a

fix: plugin settings definition

e61360e

Desvelao added 2 commits March 12, 2024 08:58

changelog: fix entry

3eec239

fix(reporting): tests

e816067

Tostti self-requested a review March 18, 2024 11:58

yenienserrano self-requested a review March 18, 2024 13:43

yenienserrano reviewed Mar 18, 2024

View reviewed changes

plugins/main/server/controllers/wazuh-api.ts Outdated Show resolved Hide resolved

Tostti requested changes Mar 18, 2024

View reviewed changes

plugins/main/server/controllers/wazuh-api.ts Outdated Show resolved Hide resolved

Merge branch '4.9.0' into enhancement/6387-registry-stateless

a08582c

Desvelao and others added 6 commits March 20, 2024 09:37

remove: console.log

cab4219

Merge branch 'enhancement/6387-registry-stateless' of https://github.…

c839d9c

…com/wazuh/wazuh-kibana-app into enhancement/6387-registry-stateless

Merge branch '4.9.0' into enhancement/6387-registry-stateless

65a0fb6

fix: error deleting API connection

721023c

fix(api-connections): error editing API connection entry when changin…

561a4ec

…g the id

fix(api-connections): error Run as enabled in the API connections table

7c944a4

changelog: add pull request entry

b6d34c4

Desvelao mentioned this pull request Mar 21, 2024

Move the plugin registry to stateless #6387

Closed

7 tasks

asteriscos and others added 2 commits March 22, 2024 19:50

Merge branch '4.9.0' into enhancement/6387-registry-stateless

4149028

Merge branch '4.9.0' of https://github.com/wazuh/wazuh-kibana-app int…

0c356b9

…o enhancement/6387-registry-stateless

asteriscos approved these changes Apr 3, 2024

View reviewed changes

Tostti approved these changes Apr 3, 2024

View reviewed changes

asteriscos merged commit 1da123b into 4.9.0 Apr 3, 2024
4 checks passed

asteriscos deleted the enhancement/6387-registry-stateless branch April 3, 2024 15:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move the registry to in-memory cache #6481

Move the registry to in-memory cache #6481

Desvelao commented Mar 6, 2024 •

edited

Desvelao commented Mar 12, 2024 •

edited

yenienserrano commented Mar 18, 2024 •

edited

yenienserrano commented Mar 18, 2024

Tostti left a comment

yenienserrano commented Mar 18, 2024

Desvelao commented Mar 20, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

asteriscos commented Apr 3, 2024

Move the registry to in-memory cache #6481

Move the registry to in-memory cache #6481

Conversation

Desvelao commented Mar 6, 2024 • edited

Description

Changes

Issues Resolved

Evidence

Test

TEST

6387-registry-in-memory-cache

UI

6387-registry-in-memory-cache

UI

Check List

Desvelao commented Mar 12, 2024 • edited

Changes

yenienserrano commented Mar 18, 2024 • edited

UI

yenienserrano commented Mar 18, 2024

Tostti left a comment

Choose a reason for hiding this comment

UI

yenienserrano commented Mar 18, 2024

Desvelao commented Mar 20, 2024

Changes

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

asteriscos commented Apr 3, 2024

UI

Desvelao commented Mar 6, 2024 •

edited

Desvelao commented Mar 12, 2024 •

edited

yenienserrano commented Mar 18, 2024 •

edited