issue#308 Allow document selector match many documents and skip empty templates

[mszygenda]

This PR builds on top of:

#328

Which partially addressed the issue:
#308

This PR addresses use-case in it's entirety.

Use-case:

I want to assert whether or not all manifests of type "Deployment":

• eg. define default resources section.
• eg. follow naming convention
• eg. set right namespace

I want this assertion to work for current and future deployments I might have in my chart.
I have to use document selector because I keep different types of k8s manifests in a single YAML files (eg. foo-application.yaml contains definition of Deployment as well as the Service)

It is resolved with introduction of two new properties on documentSelector level: matchMany and skipEmptyTemplates

Here's the updated documentation of the module

• documentSelector: DocumentSelector, optional. The path of the key to be match and the match value to assert. Using this information, helm-unittest will automatically discover the documents for asserting. Generally you can ignore this field if the template file render only one document.
  • path: string. The documentSelector path to assert.
  • value: any. The expected value.
  • matchMany: bool, optional. Set to true to allow matching multiple documents. Defaults to false which means selector has to match single document across all templates.
  • skipEmptyTemplates: bool, optional. Set to true to skip asserting templates which didn't render any matching documents. Defaults to false which means selector have to find at least one document in every template.

EXAMPLE:

suite: Document Selector is matching many documents
templates:
  - "*"
tests:
  - it: deployment names should end with -deployment suffix
    documentSelector:
      path: kind
      value: Deployment
      matchMany: true
      skipEmptyTemplates: true
    asserts:
      - matchRegex:
          path: metadata.name
          pattern: -deployment$

Happy to hear any feedback you might have. The code might need some styling fixes etc. but functionally it seems to be doing what it's supposed to do.

[mszygenda]

The core of the solution is that instead of relying on the "Index" of selected document we simply pass the array of "selected" documents to the assertion context.

I believe it's a bit simpler design to operate on the documents array directly instead of dealing with Indices array which then would have to be validated against indices going outside of boundary.

The context also includes unfiltered list of all documents (Docs) which makes assertions such as hasDocuments to stil work fine ignoring the by ignoring the Selection.

[mszygenda]

The idea is that the Selector should decide which documents in which templates should be asserted.

If the "skipEmptyTemplates" is true - it wont return templates which don't have any matching documents in the resulting map.

[mszygenda]

We should iterate only over selectedTemplates - That what's makes skipEmptyTemplates flag work.

[MikaelElkiaer]

@mszygenda Well done!

I finally got around to trying it out, and it works as expected.
Good idea to add the skipEmptyTemplates.

Regarding functionality, it looks to me that it is a nice extension to the current feature-set, without breaking or forcing it on anyone.
I have not spent much time looking into the code, so I dare not comment on code quality.

[mcg1969]

I would love to see this added, as a way to enforce a general hygeine tests across all resources of a specific type. Tremendous work @mszygenda

[mcg1969]

One nice addition, though of course it could easily be another PR, is to allow the value: field to be a string or a list of strings. This could enable, for instance, a set of conventions applied to both Deployment and StatefulSet objects.