Gitops with latest release 3.3.0 is not triggering anything

[abdnah]

I have configured gitops to my git repo and it was successful. I can see the file created with project name in my repo.

Then, i've added annotations to a deployment :

litmuschaos.io/gitops=true and litmuschaos.io/workflow=WORKFLOW_ID

Then created an event tracker :

apiVersion: eventtracker.litmuschaos.io/v1
kind: EventTrackerPolicy
metadata:
  name: eventtrackerpolicy-mydeployment
  namespace: litmus
spec:
  condition_type: "or"
  conditions:
  - key: "spec.replicas"
    value: "3"
    operator: GreaterThan
  - key: "spec.template.spec.containers[0].image"
    value: "attdemo.azurecr.io/nova/session/smartindexer:0.19.0"
    operator: EqualTo

Then to try to trigger, i've scaled up the deployment replicas to 4 then 5 and nothing was triggered. Can someone help please ??

[Nageshbansal]

hey @abdnah , could you please try with the this annotation:

litmuschaos.io/experimentId=<experimentID>

For experimentID you can use the workflowID from the experiment.yaml

[abdnah]

Hello @Nageshbansal,

Thanks a lot ! It looks much better but still not there yet; I'm getting following error:

level=info msg="Event Detected for experimentId: b7723aaa-2754-4c5e-8af9-d128b7125365, ResourceType: Deployment, ResourceName: live-session-smartindexer, ResourceNamespace: live-session"
ERROR    controller.eventtrackerpolicy    Reconciler error {"
reconciler group": "eventtracker.litmuschaos.io", 
"reconciler kind": "EventTrackerPolicy", 
"name": "eventtrackerpolicy-smartindexer", 
"namespace": "litmus", 
"error": "failed to get infra data: 
failed to get Secret: 
secrets "subscriber-secret" is forbidden: 
User "system:serviceaccount:litmus:event-tracker-sa" cannot get resource "secrets" in API group " " in the namespace "litmus"
} 

sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2                                                                                                                                                                                     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:227

Content of event-tracket-sa:

And it also keeps saying that the policy conditions are not matched with the changes in deployment, though the change was to scale up replicas to 5 instead of 3. So it should match with:

  conditions:
  - key: "spec.replicas"
    value: "3"
    operator: GreaterThan

[Nageshbansal]

Could you please check the RBACs

Also, sometimes it floods the logs with policy condition are not matched but there should be one matched instance, can you look into chaoscenter and see if the experiment is getting triggered.

Also, Could you please share the Statuses for even-tracker, it should be something like this for triggering:

Statuses:
  experiment_id:  4ca1121b-daf2-4ee6-a3e1-f3ea8c890af0
  is_triggered:   true
  Resource:       Deployment
  resource_name:  nginx
  Result:         ConditionPassed
  time_stamp:     Wednesday, 14-Feb-24 20:45:27 UTC

[abdnah]

The RBACs are well configured. I have both manager-role and event-tracker-cluster-role.

On ChaosCenter, it was not triggered. And here's the statuses:

[abdnah]

[abdnah]

Now it's triggered. I had to add the following in order for it to work:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: litmus
  name: secret-reader-role
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

and

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: event-tracker-sa-binding
  namespace: litmus
subjects:
- kind: ServiceAccount
  name: event-tracker-sa
  namespace: litmus
roleRef:
  kind: Role
  name: secret-reader-role
  apiGroup: rbac.authorization.k8s.io

[abdnah]

@Nageshbansal, thanks for your support, maybe you can check if above fix would be relevant for others.

I just have one question please, why nothing is stored at GitLab project ?
in previous releases, once a scenario/workflow is triggered, the manifest was saved in gitlab project, but it's not the case here ...

[Nageshbansal]

That sounds great. Not a problem at all.

Right in the previous release the experiment/scenario was saved in gitlab/github when the run was scheduled. In the latest release, we have a similar feature, but we're facing some issues with it ( working on them now).

You can track the progress of this through the following issue link: Issue-4434