cfn-lint silently ignores unrecognized arguments

[iainelder]

CloudFormation Lint Version

cfn-lint 0.78.1

What operating system are you using?

Ubuntu 20.04

Describe the bug

Today I got really confused about why cfn-lint was ignoring aws-sso-util's own validation spec.

Why are the outputs different?

• $ cfn-lint --template Macro-Test.yaml --region eu-central-1 --spec-override cfn-lint-spec.json 
  E3001 Invalid or unsupported Type SSOUtil::SSO::AssignmentGroup for resource AssignmentGroup in eu-central-1
  Macro-Test.yaml:25:5

• $ cfn-lint --template Macro-Test.yaml --region eu-central-1 --override-spec cfn-lint-spec.json

It should have been the second one. I had mixed up the name of the parameter!

The CLI just seems to accept and silently ignore whatever it doesn't understand.

$ cfn-lint --template Macro-Test.yaml --region eu-central-1 --fdsfdsafdsafdsafdsafsa
E3001 Invalid or unsupported Type SSOUtil::SSO::AssignmentGroup for resource AssignmentGroup in eu-central-1
Macro-Test.yaml:25:5

Expected behavior

Fail on the unknown argument like this:

$ cfn-lint --template Macro-Test.yaml --region eu-central-1 --spec-override cfn-lint-spec.json 
fatal: unrecognized argument: --spec-override

(Inspired by git version 2.41.0's handling of unknown arguments. Try git log --names-only.)

or even better:

cfn-lint: '--spec-override' is not an argument. See 'cfn-lint --help'.

The most similar arguments are:
        --override-spec

(Inspired by git version 2.41.0's handling of unknown subcommands. Try git lag.)

Reproduction template

Template:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS-SSO-Util-2020-11-08

Resources:
  AssignmentGroup:
    Type: SSOUtil::SSO::AssignmentGroup
    Properties:
      Name: ReadOnly
      InstanceArn: arn:aws:sso:::instance/ssoins-ffffffffffffffff
      Principal:
        - Type: USER
          Id: ffffffff-ffff-ffff-ffff-ffffffffffffffff
      PermissionSet:
        - arn:aws:sso:::permissionSet/ssoins-ffffffffffffffff/ps-ffffffffffffffff
      Target:
        - Type: AWS_OU
          Id: ou-zzzz-zzzzzzzz

Spec:

{
    "ResourceTypes": {
        "AWS::SSO::PermissionSet": {
            "Properties": {
                "InstanceArn": {
                    "Required": false
                },
                "InlinePolicy": {
                    "PrimitiveType": "Json"
                }
            }
        },
        "SSOUtil::SSO::AssignmentGroup": {
            "Properties": {
                "InstanceArn": {
                    "PrimitiveType": "String",
                    "Required": false,
                    "UpdateType": "Immutable"
                },
                "Name": {
                    "PrimitiveType": "String",
                    "Required": false,
                    "UpdateType": "Immutable"
                },
                "Principal": {
                    "Required": true,
                    "UpdateType": "Mutable"
                },
                "PermissionSet": {
                    "Required": true,
                    "UpdateType": "Mutable"
                },
                "Target": {
                    "Required": true,
                    "UpdateType": "Mutable"
                }
            }
        }
    }
}

[benkehoe]

This is tagged good first issue and indeed it's potentially a one-line change, but is there a reason it uses ArgumentParser.parse_known_args() instead of parse_args()?

cfn-lint/src/cfnlint/config.py

Line 315 in 86f0370

self.cli_args, _ = self.parser.parse_known_args(cli_args)

[kddejong]

I've been thinking about this one for a while. I don't think so... except maybe just a miss. But good point I would want to see this through the ringer a few times to make sure I'm not missing something.

[ArjunMenon-bit]

Hi,
I am new to the open source community and would love to contribute to this project. Have raised a PR for the same.
Thanks.

[kddejong]

This is tagged good first issue and indeed it's potentially a one-line change, but is there a reason it uses ArgumentParser.parse_known_args() instead of parse_args()?

cfn-lint/src/cfnlint/config.py

Line 315 in 86f0370

self.cli_args, _ = self.parser.parse_known_args(cli_args)

I don't believe so. I was debating rolling this change into v1 just in case we are to break something as a result of this change.