Skip to content

Commit

Permalink
Add support for .tf.json inputs (#370)
Browse files Browse the repository at this point in the history
* Add support for .tf.json inputs

* change HCL to source in docs where appropriate
  • Loading branch information
jason-fugue committed Nov 10, 2022
1 parent a58739c commit 9b32b5d
Show file tree
Hide file tree
Showing 14 changed files with 614 additions and 16 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
Regula supports the following file types:

- CloudFormation JSON/YAML templates
- Terraform HCL code
- Terraform source code
- Terraform JSON plans
- Kubernetes YAML manifests
- Azure Resource Manager (ARM) JSON templates _(in preview)_
Expand Down
4 changes: 4 additions & 0 deletions changes/unreleased/Added-20221104-153025.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
kind: Added
body: Support for .tf.json files - including those output by Terraform CDK's `cdktf
synth` command.
time: 2022-11-04T15:30:25.460048-04:00
2 changes: 1 addition & 1 deletion cmd/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Input types:
auto Automatically determine input types (default)
tf-plan Terraform plan JSON
cfn CloudFormation template in YAML or JSON format
tf Terraform directory or file
tf Terraform directory or file (either .tf or .tf.json format)
k8s Kubernetes manifest in YAML format
arm Azure Resource Manager (ARM) JSON templates (feature in preview)
`
Expand Down
2 changes: 1 addition & 1 deletion docs/src/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Regula is the policy engine that powers [Fugue](https://www.fugue.co), a SaaS pl

Regula has the following advantages:

- Support for the IaC tools and templates that you use: AWS CloudFormation YAML and JSON templates, including SAM templates and those generated by the AWS Cloud SDK; Terraform HCL files and plan files, including support for modules; Kubernetes YAML manifests; Azure Resource Manager (ARM) templates (_preview_)
- Support for the IaC tools and templates that you use: AWS CloudFormation YAML and JSON templates, including SAM templates and those generated by the AWS Cloud SDK; Terraform source and plan files, including support for modules and Terraform JSON generated by the Terraform CDK; Kubernetes YAML manifests; Azure Resource Manager (ARM) templates (_preview_)
- Easy installation and deployment with Homebrew, Docker, and pre-built binaries for all platforms
- Out-of-the-box libraries of rules that inspect AWS, Azure, Google Cloud, and Kubernetes resources for potential misconfigurations and compliance issues, including CIS Foundations Benchmarks checks
- Configurable settings, including waivers for designating exceptions on resources or even an entire IaC file, and enabling/disabling rules based on your team’s needs
Expand Down
2 changes: 1 addition & 1 deletion docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
Regula supports the following file types:

- CloudFormation JSON/YAML templates
- Terraform HCL code
- Terraform source code (.tf or .tf.json format)
- Terraform JSON plans
- Kubernetes YAML manifests
- Azure Resource Manager (ARM) JSON templates _(in preview)_
Expand Down
2 changes: 1 addition & 1 deletion docs/src/report.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ Each rule result in the JSON report lists the following attributes:
- `controls`: Compliance controls mapped to the rule
- `families`: Compliance families associated with the rule
- `filepath`: Filepath of the evaluated Terraform HCL file, Terraform JSON plan, CloudFormation template, Kubernetes manifest, or ARM template (_in preview_)
- `input_type`: `tf` (Terraform HCL), `tf_plan` (Terraform JSON plan), `cfn` (CloudFormation), `k8s` (Kubernetes), `arm` (Azure Resource Manager JSON; _in preview_)
- `input_type`: `tf` (Terraform source code), `tf_plan` (Terraform JSON plan), `cfn` (CloudFormation), `k8s` (Kubernetes), `arm` (Azure Resource Manager JSON; _in preview_)
- `provider`: `aws`, `azurerm`, `google`, `kubernetes`, `arm`
- `resource_id`: ID of the evaluated resource
- `resource_type`: Type of the evaluated resource
Expand Down
17 changes: 11 additions & 6 deletions docs/src/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Global Flags:

### Input

`regula run [input...]` supports passing in CloudFormation templates, Kubernetes manifests, Terraform HCL files, Terraform plan JSON files, and Azure ARM templates _(preview)_.
`regula run [input...]` supports passing in CloudFormation templates, Kubernetes manifests, Terraform source files, Terraform plan JSON files, and Azure ARM templates _(preview)_.

- **When run without any paths,** Regula will recursively search for IaC configurations within the working directory. Example:

Expand Down Expand Up @@ -95,7 +95,7 @@ If an [input type](#input-types) is set with `-t | --input-type`, Regula will on

#### Terraform input

Regula operates on Terraform HCL code and JSON plans.
Regula operates on Terraform source code and JSON plans.

You can pass in an HCL file or a directory containing an HCL file:

Expand Down Expand Up @@ -180,7 +180,7 @@ Regula operates on ARM templates formatted as JSON.
- `auto` -- Automatically determine input types (default)
- `tf-plan` -- Terraform plan JSON
- `cfn` -- CloudFormation template in YAML or JSON format
- `tf` -- Terraform directory or file
- `tf` -- Terraform directory or file (either .tf or .tf.json format)
- `k8s` -- Kubernetes manifest YAML
- `arm` -- Azure Resource Manager JSON _(preview)_

Expand Down Expand Up @@ -240,6 +240,11 @@ Regula operates on ARM templates formatted as JSON.

cdk synth | regula run

* Check Terraform CDK stacks using `cdktf synth`

cdktf synth
regula run cdktf.out/stacks/*

* Recurse through the working directory and exclude rule FG_R00275:

regula run --exclude FG_R00275
Expand Down Expand Up @@ -1075,7 +1080,7 @@ Global Flags:
-v, --verbose verbose output
```

`regula show input [file...]` accepts Terraform HCL files or directories, Terraform plan JSON, Kubernetes manifests, and CloudFormation templates in YAML or JSON. In all cases, Regula transforms the input file and displays the resulting JSON.
`regula show input [file...]` accepts Terraform source files or directories, Terraform plan JSON, Kubernetes manifests, and CloudFormation templates in YAML or JSON. In all cases, Regula transforms the input file and displays the resulting JSON.

This command is used to facilitate development of Regula itself. If you'd like to see the mock input, mock resources, or mock config for an IaC file so you can develop rules, see [`regula repl`](#repl) and [Test Inputs](development/test-inputs.md#viewing-test-inputs).

Expand Down Expand Up @@ -1150,7 +1155,7 @@ Example output, and comparison to original file:
- `auto` -- Automatically determine input types (default)
- `tf-plan` -- Terraform plan JSON
- `cfn` -- CloudFormation template in YAML or JSON format
- `tf` -- Terraform directory or file
- `tf` -- Terraform directory or file (either .tf or .tf.json format)
- `k8s` -- Kubernetes manifest YAML
- `arm` -- Azure Resource Manager JSON _(preview)_

Expand Down Expand Up @@ -1243,7 +1248,7 @@ The input is saved as `<iac filename without extension>_<extension>.rego` in the
- `auto` -- Automatically determine input types (default)
- `tf-plan` -- Terraform plan JSON
- `cfn` -- CloudFormation template in YAML or JSON format
- `tf` -- Terraform directory or file
- `tf` -- Terraform directory or file (either .tf or .tf.json format)
- `k8s` -- Kubernetes manifest YAML
- `arm` -- Azure Resource Manager JSON _(preview)_

Expand Down
9 changes: 7 additions & 2 deletions pkg/loader/tf.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ package loader
import (
"fmt"
"path/filepath"
"strings"

"github.com/sirupsen/logrus"
"github.com/spf13/afero"
Expand All @@ -31,7 +32,7 @@ import (
type TfDetector struct{}

func (t *TfDetector) DetectFile(i InputFile, opts DetectOptions) (IACConfiguration, error) {
if !opts.IgnoreExt && i.Ext() != ".tf" {
if !opts.IgnoreExt && !hasTerraformExt(i.Path()) {
return nil, fmt.Errorf("Expected a .tf extension for %s", i.Path())
}
dir := filepath.Dir(i.Path())
Expand Down Expand Up @@ -70,7 +71,7 @@ func (t *TfDetector) DetectDirectory(i InputDirectory, opts DetectOptions) (IACC
// First check that a `.tf` file exists in the directory.
tfExists := false
for _, child := range i.Children() {
if c, ok := child.(InputFile); ok && c.Ext() == ".tf" {
if c, ok := child.(InputFile); ok && hasTerraformExt(c.Path()) {
tfExists = true
}
}
Expand Down Expand Up @@ -141,3 +142,7 @@ func (c *HclConfiguration) RegulaInput() RegulaInput {
},
}
}

func hasTerraformExt(path string) bool {
return strings.HasSuffix(path, ".tf") || strings.HasSuffix(path, ".tf.json")
}
217 changes: 217 additions & 0 deletions pkg/loader/tf_test/cdktf.out.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
{
"content": {
"hcl_resource_view_version": "0.0.1",
"resources": {
"aws_apigatewayv2_api.posts_api_api-gw_B6634897": {
"_filepath": "tf_test/cdktf.out/posts-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_apigatewayv2_api",
"cors_configuration": {
"allow_headers": [
"content-type"
],
"allow_methods": [
"*"
],
"allow_origins": [
"*"
]
},
"id": "aws_apigatewayv2_api.posts_api_api-gw_B6634897",
"name": "sls-example-posts-development",
"protocol_type": "HTTP",
"target": "aws_lambda_function.posts_api_7D5242CA"
},
"aws_cloudfront_distribution.frontend_cf_6C82FC12": {
"_filepath": "tf_test/cdktf.out/frontend-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_cloudfront_distribution",
"comment": "Serverless example frontend for env=development",
"default_cache_behavior": {
"allowed_methods": [
"DELETE",
"GET",
"HEAD",
"OPTIONS",
"PATCH",
"POST",
"PUT"
],
"cached_methods": [
"GET",
"HEAD"
],
"forwarded_values": {
"cookies": {
"forward": "none"
},
"query_string": false
},
"target_origin_id": "s3Origin",
"viewer_protocol_policy": "redirect-to-https"
},
"default_root_object": "index.html",
"enabled": true,
"id": "aws_cloudfront_distribution.frontend_cf_6C82FC12",
"origin": [
{
"custom_origin_config": {
"http_port": 80,
"https_port": 443,
"origin_protocol_policy": "http-only",
"origin_ssl_protocols": [
"TLSv1.2",
"TLSv1.1",
"TLSv1"
]
},
"domain_name": "aws_s3_bucket_website_configuration.frontend_website-configuration_53A72F76",
"origin_id": "s3Origin"
}
],
"restrictions": {
"geo_restriction": {
"restriction_type": "none"
}
},
"viewer_certificate": {
"cloudfront_default_certificate": true
}
},
"aws_dynamodb_table.posts_storage_table_50F8EECB": {
"_filepath": "tf_test/cdktf.out/posts-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_dynamodb_table",
"attribute": [
{
"name": "id",
"type": "S"
},
{
"name": "postedAt",
"type": "S"
}
],
"billing_mode": "PAY_PER_REQUEST",
"hash_key": "id",
"id": "aws_dynamodb_table.posts_storage_table_50F8EECB",
"name": "sls-posts-development",
"range_key": "postedAt"
},
"aws_iam_role.posts_api_lambda-exec_B42627E0": {
"_filepath": "tf_test/cdktf.out/posts-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_iam_role",
"assume_role_policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Action\":\"sts:AssumeRole\",\"Principal\":{\"Service\":\"lambda.amazonaws.com\"},\"Effect\":\"Allow\",\"Sid\":\"\"}]}",
"id": "aws_iam_role.posts_api_lambda-exec_B42627E0",
"inline_policy": [
{
"name": "AllowDynamoDB",
"policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Action\":[\"dynamodb:Scan\",\"dynamodb:Query\",\"dynamodb:BatchGetItem\",\"dynamodb:GetItem\",\"dynamodb:PutItem\"],\"Resource\":\"aws_dynamodb_table.posts_storage_table_50F8EECB\",\"Effect\":\"Allow\"}]}"
}
],
"name": "sls-example-post-api-lambda-exec-development"
},
"aws_iam_role_policy_attachment.posts_api_lambda-managed-policy_460C9C52": {
"_filepath": "tf_test/cdktf.out/posts-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_iam_role_policy_attachment",
"id": "aws_iam_role_policy_attachment.posts_api_lambda-managed-policy_460C9C52",
"policy_arn": "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
"role": "sls-example-post-api-lambda-exec-development"
},
"aws_lambda_function.posts_api_7D5242CA": {
"_filepath": "tf_test/cdktf.out/posts-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_lambda_function",
"environment": {
"variables": {
"DYNAMODB_TABLE_NAME": "sls-posts-development"
}
},
"filename": "assets/posts_api_code_lambda-asset_7F9E9FED/5C0604B46739D015AEDB1BA83362F19D/archive.zip",
"function_name": "sls-example-posts-api-development",
"handler": "index.handler",
"id": "aws_lambda_function.posts_api_7D5242CA",
"role": "aws_iam_role.posts_api_lambda-exec_B42627E0",
"runtime": "nodejs14.x",
"source_code_hash": "5C0604B46739D015AEDB1BA83362F19D"
},
"aws_lambda_permission.posts_api_apigw-lambda_02C673B9": {
"_filepath": "tf_test/cdktf.out/posts-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_lambda_permission",
"action": "lambda:InvokeFunction",
"function_name": "sls-example-posts-api-development",
"id": "aws_lambda_permission.posts_api_apigw-lambda_02C673B9",
"principal": "apigateway.amazonaws.com",
"source_arn": "aws_apigatewayv2_api.posts_api_api-gw_B6634897/*/*"
},
"aws_s3_bucket.frontend_bucket_EFDC2F3F": {
"_filepath": "tf_test/cdktf.out/frontend-dev.tf.json",
"_provider": "aws",
"_tags": {
"hc-internet-facing": "true"
},
"_type": "aws_s3_bucket",
"bucket_prefix": "sls-example-frontend-development",
"id": "aws_s3_bucket.frontend_bucket_EFDC2F3F",
"tags": {
"hc-internet-facing": "true"
}
},
"aws_s3_bucket_policy.frontend_s3_policy_42C30805": {
"_filepath": "tf_test/cdktf.out/frontend-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_s3_bucket_policy",
"bucket": "aws_s3_bucket.frontend_bucket_EFDC2F3F",
"id": "aws_s3_bucket_policy.frontend_s3_policy_42C30805",
"policy": "{\"Version\":\"2012-10-17\",\"Id\":\"PolicyForWebsiteEndpointsPublicContent\",\"Statement\":[{\"Sid\":\"PublicRead\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":[\"s3:GetObject\"],\"Resource\":[\"aws_s3_bucket.frontend_bucket_EFDC2F3F/*\",\"aws_s3_bucket.frontend_bucket_EFDC2F3F\"]}]}"
},
"aws_s3_bucket_website_configuration.frontend_website-configuration_53A72F76": {
"_filepath": "tf_test/cdktf.out/frontend-dev.tf.json",
"_provider": "aws",
"_tags": {},
"_type": "aws_s3_bucket_website_configuration",
"bucket": "aws_s3_bucket.frontend_bucket_EFDC2F3F",
"error_document": {
"key": "index.html"
},
"id": "aws_s3_bucket_website_configuration.frontend_website-configuration_53A72F76",
"index_document": {
"suffix": "index.html"
}
},
"data.terraform_remote_state.cross-stack-reference-input-posts-dev": {
"_filepath": "tf_test/cdktf.out/frontend-dev.tf.json",
"_provider": "terraform",
"_tags": {},
"_type": "data.terraform_remote_state",
"backend": "local",
"config": {
"path": "cdktf-integration-serverless-example/terraform.posts-dev.tfstate"
},
"id": "data.terraform_remote_state.cross-stack-reference-input-posts-dev",
"workspace": "default"
},
"local_file.frontend_env_FADFC9DB": {
"_filepath": "tf_test/cdktf.out/frontend-dev.tf.json",
"_provider": "local",
"_tags": {},
"_type": "local_file",
"content": "S3_BUCKET_FRONTEND=aws_s3_bucket.frontend_bucket_EFDC2F3F\nREACT_APP_API_ENDPOINT=data.terraform_remote_state.cross-stack-reference-input-posts-dev",
"filename": "cdktf-integration-serverless-example/frontend/code/.env.production.local",
"id": "local_file.frontend_env_FADFC9DB"
}
}
},
"filepath": "tf_test/cdktf.out"
}

0 comments on commit 9b32b5d

Please sign in to comment.