Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for structured html-validate output (OSOE-770) #337

Closed
Piedone opened this issue Jan 11, 2024 · 4 comments · Fixed by #364
Closed

Add support for structured html-validate output (OSOE-770) #337

Piedone opened this issue Jan 11, 2024 · 4 comments · Fixed by #364
Assignees
Labels
enhancement New feature or request

Comments

@Piedone
Copy link
Member

Piedone commented Jan 11, 2024

html-validate, what we use for HTML validation under the hood with Atata.HtmlValidation, supports multiple formatters for the output, including JSON.

Based on this, add support for a structured HTML validation output, i.e. a collection of .NET objects (perhaps by deserializing the JSON output). This will be then useful to have custom assertions for.

Keep the simple text output too, but this output should from then on be the recommended way. Change all built-in features to use that.

Related: atata-framework/atata-htmlvalidation#8.

Jira issue

@Piedone Piedone added the enhancement New feature or request label Jan 11, 2024
@github-actions github-actions bot changed the title Add support for structured html-validate output Add support for structured html-validate output (OSOE-770) Jan 11, 2024
@DemeSzabolcs
Copy link
Member

Hello @AydinE,

We use HTML validation in our UI tests, see: https://github.com/Lombiq/UI-Testing-Toolbox/blob/68d72b676e7ba75882a4887df325e5b05ddd793e/Lombiq.Tests.UI/Docs/Tools.md#:~:text=HTML%20markup%20validation%20can%20be%20done%20with%20html%2Dvalidate%20via%20Atata.HtmlValidation.

I also don't fully get the relation with the other linked issue.

The relation is only this part: atata-framework/atata-htmlvalidation#8.

So if there is an HTML validation problem it renders like this:

  91:6   error  <button> is missing required "type" attribute  element-required-attributes
  91:93  error  Class "shepherd-button" duplicated             no-dup-class

✖ 2 problems (2 errors, 0 warnings)

More information:
  https://html-validate.org/rules/element-required-attributes.html
  https://html-validate.org/rules/no-dup-class.html

This is okay, but in the code, it's also just one big string. So if we want to ignore some rules from the code, we would have to write complex regexes, etc. to search for specific patterns to ignore. Instead of this, it would be nice to have an array of objects, which contains the rule id, description, etc. when there is an error.

A good example of that is what the accessibility checking does: https://github.com/Lombiq/UI-Testing-Toolbox/blob/68d72b676e7ba75882a4887df325e5b05ddd793e/Lombiq.Tests.UI/Docs/Tools.md#:~:text=Accessibility%20checking%20can%20be%20done%20with%20axe%20via%20Selenium.Axe%20for%20.NET.
During a UI test, if we have accessibility errors and you debug the code here (axeResult): https://github.com/Lombiq/UI-Testing-Toolbox/blob/dev/Lombiq.Tests.UI/Services/AccessibilityCheckingConfiguration.cs#L65-L66
You could see, that the problems are in an array of objects with descriptions, ids, etc.

@Piedone
Copy link
Member Author

Piedone commented May 12, 2024

Is this addressed?

Keep the simple text output too, but this output should from then on be the recommended way. Change all built-in features to use that.

@DemeSzabolcs
Copy link
Member

Is this addressed?

Keep the simple text output too, but this output should from then on be the recommended way. Change all built-in features to use that.

Yes, but we did in a way, that the output is still human readable, both the file and the output in VS. But at the same time, you can ignore rules easier error.RuleId is not "prefer-native-element". And yes, this is the default now.

@Piedone
Copy link
Member Author

Piedone commented May 12, 2024

OK, great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
3 participants