Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

taskcat lint uses all regions defined in .taskcat.yaml but doesn't output the region name in case of an error or warning #814

Open
wombelix opened this issue Sep 27, 2023 · 0 comments

Comments

@wombelix
Copy link
Contributor

Describe the bug
I have multiple regions defined in .taskcat.yaml, let's say us-east-1 on project level and us-east-2, us-west-1 and us-west-2 specifics tests. taskcat lint seem to use all regions defined in any level but if an error or warning occurs, the region name is not part of the output. In my case, I have a list of allowed EC2 Instances, all of them are supported in us-east-1, us-east-2 and us-west-2, but some are not available in us-west-1. In that case, taskcat lint throws a lot of warnings but I can't see in the output, that it's about the region us-west-1, which makes troubleshooting a little complicated.

To Reproduce
Steps to reproduce the behavior:

  1. Are you testing a QuickStart or Custom template? - Partner Solution (in development)
  2. Attach or link a copy of the template if possible (remove any sensitive info) - Not public yet
  3. Provide the parameters that you passed. (remove any sensitive info) - see end of the issue
  4. How did you install taskcat? (docker or pip3) - pip
  5. Are you using a profile, an instance role or access keys to run taskcat? - profile
  6. Is your AWS environment configured via aws configure? - no, isengard

Expected behavior
I would expect that taskcat provides the name of the region a specific error or warning belongs to. Even better would be, if the region can be set by a cli parameter or only regions on project level / the default test is taken into account if not configured otherwise.

Screenshots
n/a

**Version (Please make sure you are running the latest version of taskcat)

  • Taskcat Version: 0.9.41
  • Python Version (python3 required): 3.9.16

Additional context
I can technically configure to ignore instance type related warnings, but in case of a typo or something, it feels valuable to run an actual test. But 100% coverage of all instance types across all regions is not realistic, so testing just against one specific region, as sanity check that in general the list is correct, feels like a good approach.

Passed parameters

This is an example of a instance type allowed values list that works for example with us-east-1, because c6id is available in that region, and will throw warnings with us-west-1, which is expected and fine. But that I don't seem to have visibility in the taskcat lint output and can't limit the regions that should be included is something that could be improved.

Parameters:
  InstanceType:
    AllowedValues:
      - c6id.large
      - c6id.xlarge
      - c6id.2xlarge
      - c6id.4xlarge
      - c6id.8xlarge
      - c6id.12xlarge
      - c6id.16xlarge
      - c6id.24xlarge
      - c6id.32xlarge
      - c6id.metal

Warning

[WARN   ][WARN   ] :     line 62 [2030] [Check if parameters have a valid value] You must specify a valid allowed value for InstanceType
                                                              (c6id.large). Valid values are ["c1.medium", "c1.xlarge",
                                                              "c3.2xlarge", "c3.4xlarge", "c3.8xlarge", "c3.large", "c3.xlarge",
                                                              "c4.2xlarge", "c4.4xlarge", "c4.8xlarge", "c4.large", "c4.xlarge",
                                                              "c5.12xlarge", "c5.18xlarge", "c5.24xlarge", "c5.2xlarge",
                                                              "c5.4xlarge", "c5.9xlarge", "c5.large", "c5.metal", "c5.xlarge", ........... ]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant