--dry-run does not validate the syntax correctly

[STgversluis]

Describe the bug
The --dry-run option should validate the syntax. That means that if no errors show up during the dry run, constraint violations shouldn't be an issue... but alas, they are.

How to reproduce
Steps to reproduce the behavior:
see log; actual files do not matter IMO - the issue is shown clearly

Expected behavior
Throw an error during dry-run, not once I deploy the config.

Log output
D:\MonacoV2\testdash>..\v2.6.0\monaco.exe deploy manifest.yaml -e acceptance
2023-09-06T17:15:28+02:00 info Loading manifest "D:\MonacoV2\testdash\manifest.yaml". Restrictions: groups=[], environments=["acceptance"]
2023-09-06T17:15:28+02:00 info Loading 1 projects...
2023-09-06T17:15:28+02:00 info Projects to be deployed (1):
2023-09-06T17:15:28+02:00 info - project
2023-09-06T17:15:28+02:00 info Environments to deploy to (1):
2023-09-06T17:15:28+02:00 info - acceptance
2023-09-06T17:15:28+02:00 info Deploying configurations to environment acceptance...
2023-09-06T17:15:28+02:00 info [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Deploying config
2023-09-06T17:15:28+02:00 info [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Updating existing object "*****************************************************************************************************************" with matching unique properties: (name = )
2023-09-06T17:15:28+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:30+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:31+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:32+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:33+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:34+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:35+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:36+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:37+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:38+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:39+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:40+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:41+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:42+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:43+02:00 warn [coord=project:builtin:tags.auto-tagging:PMA_Exec_Group] Failed to send HTTP request. Waiting for 1s before retrying...
2023-09-06T17:15:44+02:00 error Failed to deploy config project:builtin:tags.auto-tagging:PMA_Exec_Group: HTTP send request https://dynatrace.europe.stater.corp/e/acceptance/api/v2/settings/objects failed after 15 retries: (HTTP 400)!
Response was: [{"code":400,"error":{"code":400,"message":"Validation failed for 1 Validators.","constraintViolations":[{"path":"builtin:tags.auto-tagging/0/rules/0/attributeRule/conditions/0/caseSensitive","message":"Given property 'caseSensitive' with value: 'true' violates the following constraint: Must be of type boolean.","parameterLocation":"PAYLOAD_BODY","location":null}]},"invalidValue":{"description":"Grouping of PMA exec SOAP services","name":"","rules":[{"attributeRule":{"conditions":[{"caseSensitive":"true","key":"PROCESS_GROUP_NAME","operator":"CONTAINS","stringValue":""}],"entityType":"SERVICE","serviceToHostPropagation":false,"serviceToPGPropagation":false},"enabled":true,"type":"ME","valueNormalization":"Leave text as-is"}]}}]

Environment (please complete the following information):

• OS: Windows client
• Tool version Monaco 2.6.0

Additional context
Add any other context about the problem here.

[UnseenWizzard]

Hi @STgversluis , this is technically the same request as #1129 you had created.

Dry-run is currently not able to validate the the semantics of Dynatrace configuration payloads (counting the syntax defined in a Settings 2.0 Schema as Dynatrace 'semantics' here).

It only validates that the produced JSON payloads are syntactically correct JSON - it does not know what fields mean - what their expected content or type should be for the specific Dynatrace API/Settings schema.

The payload in this case is valid JSON, but this type of Setting expects that value of caseSensitive to be a boolean, not a string.

Assuming you're using a monaco parameter to set this value, to solve this it should be enough to make sure that your JSON template contains:

"caseSensitive": {{ .yourVariable }},

(without quotes around the template)

instead of

"caseSensitive": "{{ .yourVariable }}",

And your variable is defined as

yourVariable: true

(again no quotes)

---

BTW this is something Dynatrace support should be perfectly capable of helping you troubleshoot - and usually faster than we reply here.
If you run into further issues please don't hesitate to start a chat or open a support ticket instead of a github issue!

[STgversluis]

Hi @UnseenWizzard ,

As to your recommendation on calling support: my experiences in that are very volatile in the results. Sometimes the support engineer is indeed able to help, but in my latest attempt 2 days ago, I had to point the support engineer to the documentation pages I used...
From my PoV this was a bug or at least an enhancement request to improve the dry-run process. We may be alone in this but we need a solid dry-run process in place before we release this to our developers with some degree of trust that they will not wreck anything.
And to me, issue 1129 is not the same - that deals with not connecting to the environment for checks. This issue deals with simple schema checks, which I thought to be a bug or at least an enhancement req. But that's my view on things.
One final question: I just had another issue. When I use
type: builtin:tags.auto-tagging
as it shows in a download, I get the following error in a dry-run: unknown API: builtin:tags.auto-tagging
but when I use
type: auto-tag
all is well. Should I report that as a new issue, to eliminate this discussion and keep things separate?

And just to be sure: if I seem overly critical, that's from my intentions to help make Monaco even better. Speaking from 20+ years of experience in monitoring solutions, Monaco is one of the best things since Apfelstrudel - it just needs tweaking.

[UnseenWizzard]

HI @STgversluis ,

And just to be sure: if I seem overly critical, that's from my intentions to help make Monaco even better. Speaking from 20+ years of experience in monitoring solutions, Monaco is one of the best things since Apfelstrudel - it just needs tweaking.

No worries! You're feedback is appreciated and I hope my replies don't seem too dismissive!

As you certainly know, Dynatrace takes bugs very seriously and side-stepping support requests to log bugs directly isn't quite the way to go.
In both cases you've created so far, monaco doesn't behave as you expected, but it does behave as designed.
As such, while I do agree that documentation can be clarified and that what you're asking for are valuable improvements (and something we've been wanting to introduce for quite a while), they're not bugs.

I'm sorry to hear your experience with Support was spotty - in general monaco related questions should be routed to support engineers who are knowledgeable about it. I hope you left feedback on that, and I'll follow up internally as well!

---

From my PoV this was a bug or at least an enhancement request to improve the dry-run process. We may be alone in this but we need a solid dry-run process in place before we release this to our developers with some degree of trust that they will not wreck anything.
And to me, issue 1129 is not the same - that deals with not connecting to the environment for checks. This issue deals with simple schema checks, which I thought to be a bug or at least an enhancement req. But that's my view on things.

Sorry if I wasn't clear in my reply. Yes these are functionally not necessarily the same ask, but from a technical and design perspective they are.
Dynatrace is able to validate payloads - offering specific APIs or flags - and especially for Settings what is or isn't valid is subject to change between schema versions.
It's thus by design that payload validation is not re-implemented in Monaco, even for the case of Settings schemas.

Both the cases of syntactic Settings schema validation as well as general validation whether a given config is valid for the environment technically have the same solution: a real dry run that calls Dynatrace APIs to execute validation.
As mentioned on #1129 this feature is tracked internally already, and will be the solution to this request as well.

---

One final question: I just had another issue. When I use
type: builtin:tags.auto-tagging
as it shows in a download, I get the following error in a dry-run: unknown API: builtin:tags.auto-tagging
but when I use
type: auto-tag

The shorthand syntax of type: XYZ is not applicable to Settings schemas, it only works for Config API types like auto-tag.
You will need to write the full version:

type:
    settings:
      schema: builtin:tags.auto-tagging
      scope: environment

(tags a bad example for this as they only apply environment-wide AFAIK, but for most settings the scope is very relevant and can thus not be omitted)

see: https://www.dynatrace.com/support/help/shortlink/configuration-as-code-yaml-configuration#type-setting

as it shows in a download,

Where exactly did you encounter that shorthand? As said, this is never valid for a Setting.

On download you should actually get a full configuration including the otherwise optional schemaVersion.