Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

908: improve validate support #910

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

908: improve validate support #910

wants to merge 2 commits into from

Conversation

ademariag
Copy link
Contributor

@ademariag ademariag commented Dec 8, 2022 •
edited

Fixes issue #908

Proposed Changes

Overhaul of the validation support to greatly improve its use.
New proposed configuration:

    validate:
    - type: kubernetes
      fail_on_error: false
      verbose: true
      exclude:
        kinds:
        - HorizontalPodAutoscaler
        paths:
        - manifests/*-ingress.yml
      output_paths:
        - manifests/*.yml
      version: 1.26.0

Example output

kapitan compile -t tesoro prod-sockshop argocd
Rendered inventory (0.18s)
Compiled tesoro (0.07s)
Compiled prod-sockshop (0.28s)
Compiled argocd (0.81s)
Validated tesoro FAIL
INVALID [MutatingWebhookConfiguration]: manifest ./compiled/tesoro/manifests/tesoro-bundle.yml
---> ERROR ['webhooks', 0]: 'sideEffects' is a required property
---> ERROR ['webhooks', 0]: 'admissionReviewVersions' is a required property

Validated prod-sockshop OK (with 3 exclusion/s)
Validated argocd OK (with 5 exclusion/s)
Validated targets (7.70s)

Features

  • supports globbed paths on output_paths
  • uses https://github.com/kapicorp/kubernetes-json-schema instead of deprecated repo
  • improved output
  • only fails at the end of the validation to surface all errors at once
  • autodetects resource kind
  • silently ignore resources not available in the schema
  • does not output anything if there are no files to validate (for non kubernetes targets)
  • detects if schema exists by querying well known url for "all" kind
  • supports files which contain multiple yaml documents
  • support "fails_on_error" flag to make explicit whether to fail validation or not
  • exclusion based on
    • kind
    • file
    • annotation

@ademariag ademariag linked an issue Dec 8, 2022 that may be closed by this pull request
Improve validate to support globbed paths and autodetect kinds #908
Open
@ademariag ademariag marked this pull request as ready for review December 8, 2022 00:17
@ademariag ademariag requested a review from ramaro December 8, 2022 00:17
@Moep90
Copy link
Contributor

Moep90 commented Dec 23, 2022 •
edited

@ademariag we should make sure that some sort of globbing works as users may or may not have a nested directory structure.

  1. validate[*].exclude and validate[*].exclude.paths are seamingly required even If I dont want to exclude stuff
  2. validate[*].kind should also be validate[*].kinds[*] as user may or may not want to specify every kind
  3. A user may or may not want to specify a kind(s)
  4. If possible, can we make sure that 1.18 instead of 1.18.0 always uses the latest major version? In this case it would for example be 1.18.20
  5. fail_on_error: false should be the default - A user might want to see every error to fix them instead of re-run kapitan after every fix
  6. Validated <target_name> FAIL - should also print the kubernetes version on which it has failed
  7. output_paths might be a little confusing, as a user may expect to input files into the validator (personal opinion)
  8. print message on success, similar to kapitan compile

This....

My structure for example has compiled/<target_name>/k8s/loki/loki-distributed.yml

    validate: 
    - type: kubernetes 
      output_paths: 
        - "k8s/**/*.yml"
        - "k8s/**/*.yaml"
      exclude:
        paths: []
      version: 1.18.0

leads to:

$ kapitan validate
Unknown (Non-Kapitan) Error occured
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/danny/.env3_kapitan_fix_validate/lib/python3.10/site-packages/kapitan-0.31.0rc0-py3.10.egg/kapitan/targets.py", line 856, in schema_validate_kubernetes_output
    KubernetesManifestValidator(cache_dir).validate(validate_data, **kwargs)
  File "/home/danny/.env3_kapitan_fix_validate/lib/python3.10/site-packages/kapitan-0.31.0rc0-py3.10.egg/kapitan/validator/kubernetes_validator.py", line 116, in validate
    validate_files, errors, exclusions = self._validate_config_block(config)
  File "/home/danny/.env3_kapitan_fix_validate/lib/python3.10/site-packages/kapitan-0.31.0rc0-py3.10.egg/kapitan/validator/kubernetes_validator.py", line 50, in _validate_config_block
    kind = validate_instance.get("kind")
AttributeError: 'NoneType' object has no attribute 'get'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/danny/.env3_kapitan_fix_validate/lib/python3.10/site-packages/kapitan-0.31.0rc0-py3.10.egg/kapitan/targets.py", line 779, in schema_validate_compiled
    [p.get() for p in pool.imap_unordered(worker, validate_map.items()) if p]
  File "/home/danny/.env3_kapitan_fix_validate/lib/python3.10/site-packages/kapitan-0.31.0rc0-py3.10.egg/kapitan/targets.py", line 779, in <listcomp>
    [p.get() for p in pool.imap_unordered(worker, validate_map.items()) if p]
  File "/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
AttributeError: 'NoneType' object has no attribute 'get'


'NoneType' object has no attribute 'get'

How to reproduce?

  1. git clone https://github.com/kapicorp/kapitan.git
  2. python3 -m venv ~/.env3_kapitan_fix_validate
  3. source ~/.env3_kapitan_fix_validate/bin/activate
  4. python3 setup.py install

@MatteoVoges MatteoVoges added the enhancement enhancement to an existing feature label Apr 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ramaro ramaro Awaiting requested review from ramaro

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
enhancement enhancement to an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Improve validate to support globbed paths and autodetect kinds
3 participants
@ademariag @Moep90 @MatteoVoges