You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug Triggers: on_deployment _update actions: resource_babysitter and mention_enricher sinks: Slack
Using on_deployment _update + resource_babysitter alone works just fine, but when used alongside mention_enricher produce multiple (i.e. four per pod) "empty" alerts as well as the actual "configured" alert. The "empty" alerts also triggers whenever there is any change to a deployment, even if the change is not in the field_to_monitor defined in resource_babysitter.
I have narrowed it down to being related to the trigger on_deployment_update trigger not working well with mention_enricher action. I have also tested that on_pod_update gives the same outcome. on_pod_create however will only create a single triggering resulting in a single alert.
I guess this has something to do with that a single change in a deployment will actually aggregate multiple changes to the deployment manifest. I.e. annotations.deployment.kubernetes.io/revision, kubectl.kubernetes.io/last-applied-configuration, creationTimestamp, generation e.t.c. And the mention_enricher action will be triggered for each of these changes instead of combining these changes into a single trigger?
Additionally: Alerts will be created even though the change does contain the specific field in specified in the resource_babysitter, but in this case we do not get the "configured" alert as expected. Removing the resource_babysitter action all together while still leaving the mention_enricher action in will give the same result.
apply to cluster using kubectl apply -f dummy-deployment.yaml.
Manually change the container image ...
to e.g. nginx:latest using kubectl edit deployment dummy-deployment
Expected behavior
Get a single alert in the tmp-robusta-test slack channel that is enriched with the slack mention @kristian, when the image of the dummy deployment is changed.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
The text was updated successfully, but these errors were encountered:
kristeey
changed the title
on-deployment _update + mention_enricher creates multiple alerts.on_deployment_update + mention_enricher creates multiple alerts.
Feb 13, 2024
Hi @kristeey,
I am guessing that the extra messages are for changes which matched the on_deployment_update trigger but where the babysitter did nothing because the change was not to the field spec.template.spec.containers[0].image . In that case, only the mention_enricher did something.
This obviously is not the desired behaviour and we have a fix for it coming soon! (We are moving the filtering functionality to the trigger itself - on_deployment_update - so that it will impact all actions in the playbook.)
Describe the bug
Triggers:
on_deployment _update
actions:
resource_babysitter
andmention_enricher
sinks: Slack
Using
on_deployment _update
+resource_babysitter
alone works just fine, but when used alongsidemention_enricher
produce multiple (i.e. four per pod) "empty" alerts as well as the actual "configured" alert. The "empty" alerts also triggers whenever there is any change to a deployment, even if the change is not in thefield_to_monitor
defined inresource_babysitter
.I have narrowed it down to being related to the trigger
on_deployment_update
trigger not working well withmention_enricher
action. I have also tested thaton_pod_update
gives the same outcome.on_pod_create
however will only create a single triggering resulting in a single alert.I guess this has something to do with that a single change in a deployment will actually aggregate multiple changes to the deployment manifest. I.e.
annotations.deployment.kubernetes.io/revision
,kubectl.kubernetes.io/last-applied-configuration
,creationTimestamp
,generation
e.t.c. And themention_enricher
action will be triggered for each of these changes instead of combining these changes into a single trigger?Additionally: Alerts will be created even though the change does contain the specific field in specified in the
resource_babysitter
, but in this case we do not get the "configured" alert as expected. Removing theresource_babysitter
action all together while still leaving themention_enricher
action in will give the same result.To Reproduce
1. Configure Robusta helm chart:
Custom Playbook:
Sink:
2. Apply a Dummy deployment to the cluster:
apply to cluster using
kubectl apply -f dummy-deployment.yaml
.to e.g.
nginx:latest
usingkubectl edit deployment dummy-deployment
Expected behavior
Get a single alert in the
tmp-robusta-test
slack channel that is enriched with the slack mention@kristian
, when the image of the dummy deployment is changed.Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
The text was updated successfully, but these errors were encountered: