[FR] Allow the Chart to create extra manifest

[pierluigilenoci]

Is your feature request related to a problem? Please describe.

Sometimes, accessories to the installation may be manifests to be installed (for example, a SecretProviderClass or Certificates).

Describe the solution you'd like

It would be nice to be able to do this directly via the Chart using the approach that Grafana, Prometheus or OAuth2 Proxy also uses.

Describe alternatives you've considered

I don't think there are alternatives

Additional context

No additional context

/kind feature

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[pierluigilenoci]

/remove-lifecycle stale

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[pierluigilenoci]

/remove-lifecycle stale

[wallrj]

@pierluigilenoci and @gplessis Please supply some concrete use cases for this feature.

I don't think Certificate can be among the extraManifests because that's a cert-manager CRD which will only be usable after the cert-manager CRDs have been installed and after the validating webhook has been deployed and is ready.

Please explain precisely why and how a SecretProviderClass would be deployed using this feature and explain why a Helm post install hook or another Helm chart or a set of pre-provisioned manifests would not also solve the problem for you.

https://cert-manager.io/docs/contributing/policy/#additional-configuration-options-for-the-helm-chart explains why we are reluctant to add new Helm chart features like this.

[pierluigilenoci]

@wallrj, the most common use case might be to install a ClusterIssuer.
The CRD installation precedence could be resolved with #6193.
Another example could be to create ConfigMap to create dashboards in Grafana.
Or any other software manifest to monitor the cert-manager.

[wallrj]

@wallrj, the most common use case might be to install a ClusterIssuer. The CRD installation precedence could be resolved with #6193.

I don't think that's a real example because even if the CRDs have already been installed, you can't create cert-manager custom resources (including ClusterIssuer) until the cert-manager webhook is ready. The explanation is here (although slightly out-of-date):

• https://cert-manager.io/docs/concepts/webhook/#webhook-connection-problems-shortly-after-cert-manager-installation

Another example could be to create ConfigMap to create dashboards in Grafana. Or any other software manifest to monitor the cert-manager.

Please be more precise about this use-case. Explain when, where and how Grafana gets installed. Explain why the ConfigMap isn't included in the Grafana chart. If we agree that the ClusterIssuer can not be included in the extraManifests of cert-manager, howdo you install those resources and why can't the grafana ConfigMap be installed in that way too?

[pierluigilenoci]

@wallrj, the way I see things, an application should be installed and configured using a single "artifact" rather than multiple elements.
From this comes the interest in having a chart that allows you to configure everything necessary, including the accessory parts, such as the Grafana dashboards I want to use to observe the cert-manager.

Obviously, I could insert the ConfigMaps to load the dashboards using the Grafana chart, but I find it more intelligent if they are together with the application that concerns them.
This could reduce cluttering.
It makes no sense for all the software's dashboards to be installed via Grafana because not all are present in all the clusters.

And then, when and if Grafana is installed does not matter.
Cert-manager would still be installed with everything it needs.

At this moment, to do so, a chart umbrella must be used that has the cert-manager dependency and installs the extra files that can't be installed with the base chart.
Or installed separately via one of the GitOps tools on the market; it doesn't matter which one.

However, I have a question: do you have any technical reason to be against this change? Because even if the use cases are minimal, it is a modification that has little or no impact, it makes the chart more dynamic without causing any damage.

Note: the ClusterIssuer could be installed post-install while the cert-manager pod could be programmed to become ready only after the validating webhook configuration is up and running by doing post-install checks.

[wallrj]

@wallrj, the way I see things, an application should be installed and configured using a single "artifact" rather than multiple elements. From this comes the interest in having a chart that allows you to configure everything necessary, including the accessory parts, such as the Grafana dashboards I want to use to observe the cert-manager.

Obviously, I could insert the ConfigMaps to load the dashboards using the Grafana chart, but I find it more intelligent if they are together with the application that concerns them. This could reduce cluttering. It makes no sense for all the software's dashboards to be installed via Grafana because not all are present in all the clusters.

And then, when and if Grafana is installed does not matter. Cert-manager would still be installed with everything it needs.

Ok. Thanks for taking time to explain. I guess that you use:

• https://github.com/grafana/helm-charts/blob/main/charts/grafana/README.md#sidecar-for-dashboards

If this is how you want to use Helm, you should create a feature request in https://github.com/helm/helm i.e.

helm install cert-manager jetstack/cert-manager \
  --values cert-manager.values.yaml \
  --user-supplied-manifests cert-manager-grafana-dashboard.yaml

So that you can use this technique with any Helm chart.
Rather than having to persuade every Helm chart to add the extraManifests value.

At this moment, to do so, a chart umbrella must be used that has the cert-manager dependency and installs the extra files that can't be installed with the base chart. Or installed separately via one of the GitOps tools on the market; it doesn't matter which one.

Those all sound like good solutions to this problem.
I would add

• https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/helmcharts/
• https://helm.sh/docs/topics/advanced/#post-rendering

However, I have a question: do you have any technical reason to be against this change? Because even if the use cases are minimal, it is a modification that has little or no impact, it makes the chart more dynamic without causing any damage.

The impact is not minimal.
The maintainers get dozens of requests to add new parameters to the Helm chart.
these all require discussion.
All require documentation.
All require maintenance and testing.

A feature like this is likely to cause support requests because it will be natural for users to try and use it to install ClusterIssuer,
which we have agreed is not possible.

Note: the ClusterIssuer could be installed post-install while the cert-manager pod could be programmed to become ready only after the validating webhook configuration is up and running by doing post-install checks.

The cert-manager chart already has a post-install hook blocks until the cert-manager webhook is Ready.
Read about it here:

• https://cert-manager.io/docs/releases/release-notes/release-notes-1.5/#helm-chart

[pierluigilenoci]

@wallrj, those ConfigMaps you linked from the Grafana documentation are precisely the ones I want to create as extra manifests from the cert-manager chart. It makes sense that all Grafana dashboards related to cert-manager are contained in the chart of the software itself without having to resort to external means.

I need to understand why I should ask for an additional feature in Helm to do something it already does—effectively complicating its use.

I don't need to persuade any Helm Chart maintainers. It is a good practice, which is, in fact, already used in many charts. So, I expect to find it already or hope that by requesting it, given the low impact of creating the PR to implement it, available maintainers will favorably embrace it.

The impact of implementation, documentation, and maintenance is so low that if you promise me a quick review and merge, I am willing to take on the effort (maximum 30 minutes of work).

Do you think Grafana or Prometheus maintainers are naive and open the gate to problems?

Suppose you are worried about someone using it to create ClusterIssuer. In that case, I can explicitly add, in the documentation for the option, that it is not recommended because it doesn't work (which, anyway, I would like to try because I'm not so convinced in light of #4234).

Why set up a more complex implementation when you can get a more straightforward solution for less?

[pierluigilenoci]

@wallrj any feedback?

[wallrj]

@pierluigilenoci Sorry for the radio silence.

I need to understand why I should ask for an additional feature in Helm to do something it already does—effectively complicating its use.

The alternative is to complicate every Helm chart instead.

It should be a more maintainable to implement helm install --user-supplied-manifests extra-objects/,
than to add Values.extraObjects to each Helm chart.

It should be more consistent to implement helm install --user-supplied-manifests extra-objects/,
than for each Helm chart to implement the feature slightly differently

It should be more user friendly to document helm install --user-supplied-manifests extra-objects/ thoroughly at helm.sh,
than to document it (or not document it) chart-by-chart in the chart README file, for example.

It should be more time efficient to discuss and plan this feature at https://github.com/helm/helm/,
than to repeat the discussion with the maintainers of every Helm chart.

Having said that, after investigating the various NetworkPolicy implementations, it might have been easier for us to simply add extraObjects and give users the flexibility to add any extra manifests they like, than to implement special values and templates for NetworkPolicy, PodDisruptionBudget etc.

Links of interest, in no particular order:

• Support Extras helm/helm#7077
• https://www.reddit.com/r/kubernetes/comments/yuw27c/deploying_with_helm_extra_manifests/
• https://artifacthub.io/packages/helm/argo/argo-cd#general-parameters
• https://tech.aabouzaid.com/2020/09/3-ways-to-customize-off-the-shelf-helm-charts-with-kustomize-kubernetes.html
• Add extraObjects value that allows creating supportive / adhoc resources traefik/traefik-helm-chart#595
• [prometheus] Accept template string as server.extraObjects prometheus-community/helm-charts#2139
• BREAKING CHANGE: Remove .Values.extraObjects aws/karpenter-provider-aws#3803
• extraObjects provisioner warning aws/karpenter-provider-aws#3705
• Add extraObjects to values #5997
• No matches for kind "ClusterTriggerAuthentication" if this CRD is referenced in extraObjects kedacore/charts#513
• Add extraObjects value that allows creating supportive / adhoc resources apache/couchdb-helm#72
• Add a new chart template that uses the conventions I have grown accustomed to colearendt/helm#38
• Included extraObjects example in flux2/values.yaml doesn't appear to work. fluxcd-community/helm-charts#183
• [BUG][Opensearch] ExtraManifests do not allow structured data opensearch-project/helm-charts#463
• generic: add extraObjects morremeyer/charts#320
• Add Extra Manifest kubecost/cost-analyzer-helm-chart#2325
• extraObjects does work with ApplicationSet in ArgoCd argoproj/argo-helm#1882
• [FR] Allow the Chart to create extra manifest grafana/helm-charts#1001
• [agent-operator] Add extraObjects functionality grafana/helm-charts#2192
• [tempo-distributed] Add the ability to add extraObjects grafana/helm-charts#2722
• feat(helm): add extra object option crossplane/crossplane#4664

[joebowbeer]

I have noticed that some fairly simple helm charts do provide an "extra resources" section (helm/helm#7077) where users can add your own stuff.

But we use kustomize, if needed, to customize helm charts, so we rarely have need for this feature.

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
/lifecycle stale

[gplessis]

/remove-lifecycle stale

[gplessis]

This has been implemented in #6424