fix: traverse generator tree when getting requeue time (#12407)

[rumstead]

Fixes #12407

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• Optional. My organization is added to USERS.md.
• I have signed off all my commits as required by DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My build is green (troubleshooting builds).

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: test-appset
  namespace: argocd
spec:
  generators:
    - merge:
        mergeKeys: ["name"]
        generators:
          - clusters:
              selector:
                matchLabels:
                  argocd.argoproj.io/secret-type: cluster
          - matrix:
              generators:
                - clusters:
                    selector:
                      matchLabels:
                        argocd.argoproj.io/secret-type: cluster
                      matchExpressions:
                        - key: kubernetes.cnp.io/environment
                          operator: In
                          values: ["dev"]
                - git:
                    repoURL: https://github.com/rumstead/argocd-playground.git
                    revision: HEAD
                    files:
                    - path: "manifests/appset/config.yaml"
  template:
    metadata:
      name: 'test-{{name}}'
    spec:
      project: default
      source:
        path: guestbook
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        targetRevision: '{{app.version}}'
      destination:
        namespace: default
        server: '{{server}}'
      syncPolicy:
        automated:
          prune: true

Before changes:

time="2023-02-13T19:33:45Z" level=info msg="created Application" app=test-dev appSet=test-appset
time="2023-02-13T19:33:45Z" level=info msg="created Application" app=test-admin appSet=test-appset
time="2023-02-13T19:33:45Z" level=info msg="end reconcile" applicationset=argocd/test-appset requeueAfter=0s
< ...changes in git never picked up... >

This means if I update any GIT config.yaml the applicationset controller never reconciles any changes. The application controller does poke the argo cd applications which the application set resources "own". It "looks like" the applicationsets are being reconciled because argo cd updates the reconciledAt on the application CR. Depending on your application controller reconciliation time, it could be too long.

After changes:

time="2023-02-13T20:30:50Z" level=info msg="unchanged Application" app=test-dev appSet=test-appset
time="2023-02-13T20:30:50Z" level=info msg="unchanged Application" app=test-admin appSet=test-appset
time="2023-02-13T20:30:50Z" level=info msg="end reconcile" applicationset=argocd/test-appset requeueAfter=3m0s
< ...changes made in git... >
time="2023-02-13T20:33:52Z" level=info msg="unchanged Application" app=test-admin appSet=test-appset
time="2023-02-13T20:33:52Z" level=info msg="unchanged Application" app=test-dev appSet=test-appset
time="2023-02-13T20:33:52Z" level=info msg="end reconcile" applicationset=argocd/test-appset requeueAfter=3m0s
< ...changes in git picked up... >

[codecov]

Codecov Report

Base: 47.78% // Head: 47.78% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (eac200f) compared to base (2b1d55b).
Patch coverage: 52.17% of modified lines in pull request are covered.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #12409   +/-   ##
=======================================
  Coverage   47.78%   47.78%           
=======================================
  Files         246      246           
  Lines       41940    41944    +4     
=======================================
+ Hits        20040    20045    +5     
+ Misses      19901    19898    -3     
- Partials     1999     2001    +2     

Impacted Files	Coverage Δ
applicationset/generators/cluster.go	80.27% <ø> (ø)
applicationset/generators/merge.go	53.24% <40.00%> (-4.37%)	⬇️
applicationset/generators/matrix.go	77.50% <66.66%> (+7.75%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[ishitasequeira]

@rumstead can you add an example test case to the description of the PR for testing?

[rumstead]

@rumstead can you add an example test case to the description of the PR for testing?

Is that what you are looking for?

[ishitasequeira]

@rumstead can you add an example test case to the description of the PR for testing?

Is that what you are looking for?

Yes. Thanks. I will test the PR out soon.

[ishitasequeira]

Overall PR looks good to me. I just have some code redundancy suggestions.

[rumstead]

Overall PR looks good to me. I just have some code redundancy suggestions.

Updated.

[rumstead]

@ishitasequeira what are the next steps in your opinion?

[ishitasequeira]

Thanks @rumstead. Apologies for taking time to get back to this.
The PR LGTM. @crenshaw-dev could you take 1 more look and help merge this?

[crenshaw-dev]

Overall looks great! A few small requests.

[crenshaw-dev]

/cherry-pick release-2.4
/cherry-pick release-2.5
/cherry-pick release-2.6

[gcp-cherry-pick-bot]

Cherry-pick failed with Merge error 53c3c49dc413c2a2a290478603909d1bc376df29 into temp-cherry-pick-44d8da-release-2.4

[crenshaw-dev]

Not surprising... I can't remember if we were even bundling appset as of 2.4.

[crenshaw-dev]

Nah, just conflicts. Will fix manually.

[crenshaw-dev]

/cherry-pick release-2.5

[ishitasequeira]

Not surprising... I can't remember if we were even bundling appset as of 2.4.

We started bundling appsets in 2.3

[crenshaw-dev]

/cherry-pick release-2.6

[crenshaw-dev]

Request to specify multiple cherry-picks in one comment: googleapis/repo-automation-bots#4983

[rumstead]

thanks for the reviews @ishitasequeira and @crenshaw-dev