Is it possible to validate that provided examples are meet specification by themselves?

[akadnikov]

Hello!

Is it possible to validate that provided examples (with template features) are actually meet specification by themselves?
If not then do you have plan to implement this feature?

[lbroudoux]

Hello @akadnikov!

Microcks don't do example validation during the import phase. Until now, we considered it the responsibility of the design/governance/linting tool and process to do so before importing. That said, we had several demands on that point lately and we may consider it in the future. Maybe it can be interesting to document your thoughts on that in an issue to see if there's interest out there.

Cheers,

[akadnikov]

I think, to close the contract testing cycle it is necessary
for the "Test" phase: not only validate the response from the provider but also requests from microcks,
for the "Mock" phase: validate that the mock behaves according to the specification.
Preferably that this be implemented at the import stage, but this check can also be done at execution phase. For example, for openAPI this can be done using the swagger-request-validator library

BR

[lbroudoux]

Thanks. We have our validation library that uses imported/re-normalized resources so it shouldn't be an issue. I also think that making the validation at the import stage is better. However, I don't know how to behave if an error occurs:

• Just remove the bad sample (whether a conformance error is detected in the request or in response),
• Drop the whole imported specification/artifact as the whole is not compliant.

WDYT?

[akadnikov]

I think the specification should be rejected because everything is based on examples, but on import stage it may be difficult to make this checks according with multiple variants of response templating features. Perhaps it would be simpler and more reliable to add such a check at execution time, after templating process, maybe at least in the form of some warnings

[yada]

Maybe an excellent new capability to add or link to the AI copilot/assistant: suggest examples corrections or new examples based on import validation?

[akadnikov]

It will be nice to have additional option)

[Apoorva64]

@lbroudoux Redocly Cli (https://redocly.com/docs/cli/commands/lint/) already does this type of linting (https://redocly.com/docs/cli/rules/no-invalid-media-type-examples/)
AysncApi rules (https://redocly.com/docs/cli/guides/lint-asyncapi/#asyncapi-rules)

[akadnikov]

I guess the problem with using linters is that they won't understand the microcks specific template syntax In spec

[Apoorva64]

@lbroudoux we could create custom rules for microcks using redocly with https://redocly.com/docs/cli/custom-plugins/custom-rules/

[lbroudoux]

I don't know redocly but it looks like an excellent platform. It could be nice to have custom rules for Microcks!