Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Config and log management workflows #2844

Conversation

didier-wenzek
Copy link
Contributor

@didier-wenzek didier-wenzek commented Apr 26, 2024

Proposed changes

Update the log and config operations so those can be used by the workflow system,
enabling the ability to trigger a log or config update as a sub operation.

  • Make log and config message types similar to those used for the workflows
  • Dynamically register Restart operation
  • Dynamically register Software operations
  • Make the LogManagerActor acting on LogUploadCmd instead of MqttMessage
  • Make the ConfigManagerActor acting on ConfigSnapshotCmd & ConfigUpdateCmd instead of MqttMessage
  • Enable log operation workflows
  • Enable config operation workflows
  • Add system tests checking that LogUpload can be triggered as sub operations
  • Add system tests checking that LogUpload can be overridden
  • Add system tests checking that ConfigSnapshot can be triggered as sub operations
  • Add system tests checking that ConfigSnapshot can be overridden
  • Add system tests checking that ConfigUpdate can be triggered as sub operations
  • Add system tests checking that ConfigUpdate can be overridden

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#2836

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s)
  • I ran cargo fmt as mentioned in CODING_GUIDELINES
  • I used cargo clippy as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Copy link
Contributor

github-actions bot commented Apr 26, 2024

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
450 0 3 450 100 1h13m12.911536s

@didier-wenzek didier-wenzek force-pushed the feat/config-and-log-management-workflows branch from 1b59fad to 21e55cb Compare April 29, 2024 13:16
@reubenmiller reubenmiller changed the title Feat: Config and log management workflows feat: Config and log management workflows Apr 30, 2024
@didier-wenzek didier-wenzek force-pushed the feat/config-and-log-management-workflows branch from 340cf1a to 581d6d7 Compare May 3, 2024 11:44
Copy link

codecov bot commented May 3, 2024

Codecov Report

Attention: Patch coverage is 65.20270% with 206 lines in your changes are missing coverage. Please review.

Project coverage is 78.0%. Comparing base (0b974d5) to head (15b7e26).
Report is 3 commits behind head on main.

Additional details and impacted files
Files Coverage Δ
crates/common/mqtt_channel/src/topics.rs 86.2% <ø> (ø)
crates/core/c8y_api/src/json_c8y.rs 92.4% <ø> (ø)
crates/core/tedge_actors/src/builders.rs 96.6% <100.0%> (+19.6%) ⬆️
crates/core/tedge_actors/src/channels.rs 84.3% <ø> (+13.7%) ⬆️
crates/core/tedge_agent/src/agent.rs 0.0% <ø> (ø)
...tes/core/tedge_agent/src/software_manager/actor.rs 62.7% <100.0%> (+3.4%) ⬆️
...tedge_agent/src/tedge_operation_converter/tests.rs 94.5% <100.0%> (+0.6%) ⬆️
crates/core/tedge_api/src/lib.rs 100.0% <100.0%> (ø)
crates/core/tedge_api/src/workflow/mod.rs 60.5% <100.0%> (ø)
...s/c8y_mapper_ext/src/operations/config_snapshot.rs 90.5% <100.0%> (ø)
... and 20 more

... and 4 files with indirect coverage changes

@didier-wenzek didier-wenzek force-pushed the feat/config-and-log-management-workflows branch from 581d6d7 to 672fd42 Compare May 7, 2024 12:43
@didier-wenzek didier-wenzek force-pushed the feat/config-and-log-management-workflows branch from fa4db4f to a988410 Compare May 14, 2024 14:19
@didier-wenzek didier-wenzek force-pushed the feat/config-and-log-management-workflows branch from c1cb694 to 0bce80d Compare May 15, 2024 15:53
The builder is still expecting an MQTT connection, however the actor
message box is now free of any MQTT aspects

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
TedgeOperationConverterActor can now process any type of builtin
operations. Its builder still requires some knowledge of the available
operations. However this will be fixed in following steps so the agent
main can add any type of builtin operation such as log upload or config
update.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Now the workflow actor (aka TedgeOperationConverterActor) has no more
specific code related to the restart operation.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
To avoid name conflicts with TryFrom<Command>

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Now the workflow actor (aka TedgeOperationConverterActor) has no more
specific code related to any specific operation.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
This is a preparation step to make the ConfigManagerActor MQTT
independent and ready to be supported by agent workflows.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
…utput

This is a preparation step to make the ConfigManagerActor MQTT
independent and ready to be supported by agent workflows.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
A topic filter cannot be safely cast into a topic.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
The builder still expect an MQTT connection.
This will be removed when connecting ConfigManagerActor to the workflow
system.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
These operations can now be triggered as a sub-workflow.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Capability messages were erroneously sent twice for log upload, config
snapshot and config update.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
The operation file cache has to be redesigned as it's currently not
compatible with the workflow mechanism.

- The operation file cache runs on the main device while the workflow
  system runs on a child device: hence the communication must be over
  MQTT or HTTP.
- However, the operation-specific actors run behind the workflow actor
  and only the latter receives MQTT messages. A new mechanism is
  required for some messages to be pushed to the operation-specific
  actors.
- Furthermore, the config update workflow currently use the same state
  (executing) for two different stages (before and after cache).
  This prevents the config update operation to be properly handled by
  workflow.

I propose to disable - at least temporary, this cache for config
updates.

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Copy link
Contributor

@reubenmiller reubenmiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Great job (especially for the sustained effort!)

@didier-wenzek didier-wenzek added this pull request to the merge queue May 31, 2024
Merged via the queue into thin-edge:main with commit 08577c9 May 31, 2024
33 checks passed
@didier-wenzek didier-wenzek deleted the feat/config-and-log-management-workflows branch May 31, 2024 07:47
@reubenmiller reubenmiller added theme:configuration Theme: Configuration management theme:troubleshooting Theme: Troubleshooting and remote control labels May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme:configuration Theme: Configuration management theme:troubleshooting Theme: Troubleshooting and remote control
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants