Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

Commit

Permalink
s3 lambda functions (#36)
Browse files Browse the repository at this point in the history
  • Loading branch information
Puneeth-n committed Dec 10, 2021
1 parent c617fef commit 918b301
Show file tree
Hide file tree
Showing 15 changed files with 486 additions and 103 deletions.
97 changes: 61 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Terraform AWS module for AWS Lambda

## Introduction
## Introduction
This module creates an AWS lambda and all the related resources. It is a complete re-write of our internal terraform lambda module.

## Usage
## Usage
Checkout [examples](./examples) on how to use this module for various trigger sources.
## Authors

Expand All @@ -17,52 +17,77 @@ MIT Licensed. See LICENSE for full details.

| Name | Version |
|------|---------|
| terraform | >= 0.13 |
| aws | ~> 3.0 |
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | ~> 3.0 |

## Providers

| Name | Version |
|------|---------|
| aws | ~> 3.0 |
| <a name="provider_aws"></a> [aws](#provider\_aws) | ~> 3.0 |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_cloudwatch-log-subscription"></a> [cloudwatch-log-subscription](#module\_cloudwatch-log-subscription) | ./log_subscription/ | n/a |
| <a name="module_sqs_external"></a> [sqs\_external](#module\_sqs\_external) | ./triggers/sqs_external/ | n/a |
| <a name="module_triggered-by-api-gateway"></a> [triggered-by-api-gateway](#module\_triggered-by-api-gateway) | ./triggers/api_gateway/ | n/a |
| <a name="module_triggered-by-cloudwatch-event-schedule"></a> [triggered-by-cloudwatch-event-schedule](#module\_triggered-by-cloudwatch-event-schedule) | ./triggers/cloudwatch_event_schedule/ | n/a |
| <a name="module_triggered-by-cloudwatch-event-trigger"></a> [triggered-by-cloudwatch-event-trigger](#module\_triggered-by-cloudwatch-event-trigger) | ./triggers/cloudwatch_event_trigger/ | n/a |
| <a name="module_triggered-by-cloudwatch-logs"></a> [triggered-by-cloudwatch-logs](#module\_triggered-by-cloudwatch-logs) | ./triggers/cloudwatch_logs/ | n/a |
| <a name="module_triggered-by-cognito-idp"></a> [triggered-by-cognito-idp](#module\_triggered-by-cognito-idp) | ./triggers/cognito_idp/ | n/a |
| <a name="module_triggered-by-sqs"></a> [triggered-by-sqs](#module\_triggered-by-sqs) | ./triggers/sqs/ | n/a |
| <a name="module_triggered-by-step-function"></a> [triggered-by-step-function](#module\_triggered-by-step-function) | ./triggers/step_function/ | n/a |
| <a name="module_triggered_by_kinesis"></a> [triggered\_by\_kinesis](#module\_triggered\_by\_kinesis) | ./triggers/kinesis/ | n/a |

## Resources

| Name | Type |
|------|------|
| [aws_cloudwatch_log_group.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
| [aws_lambda_function.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) | resource |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| cloudwatch\_log\_retention | Enable Cloudwatch logs retention | `number` | `90` | no |
| cloudwatch\_log\_subscription | Cloudwatch log stream configuration | <pre>object({<br> enable : bool<br> filter_pattern : string<br> destination_arn : string<br> })</pre> | <pre>{<br> "destination_arn": "",<br> "enable": false,<br> "filter_pattern": ""<br>}</pre> | no |
| description | Lambda function description | `string` | `"Managed by Terraform"` | no |
| environment | Lambda environment variables | `map(string)` | `null` | no |
| file\_name | Lambda function filename name | `string` | `null` | no |
| function\_name | Lambda function name | `string` | n/a | yes |
| handler | Lambda function handler | `string` | n/a | yes |
| image\_config | Container image configuration values that override the values in the container image Dockerfile. | <pre>object({<br> command = list(string)<br> entry_point = list(string)<br> working_directory = string<br> })</pre> | `null` | no |
| image\_uri | ECR image URI containing the function's deployment package | `string` | `null` | no |
| kinesis\_configuration | https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_event_source_mapping | <pre>map(object({<br> batch_size = number<br> bisect_batch_on_function_error = bool<br> destination_config__on_failure__destination_arn = string<br> event_source_arn = string<br> maximum_batching_window_in_seconds = number<br> maximum_record_age_in_seconds = number<br> maximum_retry_attempts = number<br> parallelization_factor = number<br> starting_position = string<br> starting_position_timestamp = string<br> tumbling_window_in_seconds = number<br> }))</pre> | `{}` | no |
| layers | List of layers for this lambda function | `list(string)` | `[]` | no |
| memory\_size | Lambda function memory size | `number` | `128` | no |
| publish | Publish lambda function | `bool` | `false` | no |
| region | AWS region | `string` | n/a | yes |
| reserved\_concurrent\_executions | Reserved concurrent executions for this lambda function | `number` | `-1` | no |
| role | Lambda function role | `string` | n/a | yes |
| runtime | Lambda function runtime | `string` | `"nodejs14.x"` | no |
| sqs\_external | External SQS to consume | <pre>object({<br> batch_size = number<br> sqs_arns = list(string)<br> })</pre> | `null` | no |
| tags | Tags for this lambda function | `map(string)` | `{}` | no |
| timeout | Lambda function runtime | `number` | `300` | no |
| tracing\_config | https://www.terraform.io/docs/providers/aws/r/lambda_function.html | <pre>object({<br> mode : string<br> })</pre> | <pre>{<br> "mode": "PassThrough"<br>}</pre> | no |
| trigger | Trigger configuration for this lambda function | `any` | n/a | yes |
| vpc\_config | Lambda VPC configuration | <pre>object({<br> subnet_ids : list(string)<br> security_group_ids : list(string)<br> })</pre> | <pre>{<br> "security_group_ids": [],<br> "subnet_ids": []<br>}</pre> | no |
| <a name="input_cloudwatch_log_retention"></a> [cloudwatch\_log\_retention](#input\_cloudwatch\_log\_retention) | Enable Cloudwatch logs retention | `number` | `90` | no |
| <a name="input_cloudwatch_log_subscription"></a> [cloudwatch\_log\_subscription](#input\_cloudwatch\_log\_subscription) | Cloudwatch log stream configuration | <pre>object({<br> enable : bool<br> filter_pattern : string<br> destination_arn : string<br> })</pre> | <pre>{<br> "destination_arn": "",<br> "enable": false,<br> "filter_pattern": ""<br>}</pre> | no |
| <a name="input_description"></a> [description](#input\_description) | Lambda function description | `string` | `"Managed by Terraform"` | no |
| <a name="input_environment"></a> [environment](#input\_environment) | Lambda environment variables | `map(string)` | `null` | no |
| <a name="input_file_name"></a> [file\_name](#input\_file\_name) | Lambda function filename name | `string` | `null` | no |
| <a name="input_function_name"></a> [function\_name](#input\_function\_name) | Lambda function name | `string` | n/a | yes |
| <a name="input_handler"></a> [handler](#input\_handler) | Lambda function handler | `string` | `null` | no |
| <a name="input_image_config"></a> [image\_config](#input\_image\_config) | Container image configuration values that override the values in the container image Dockerfile. | <pre>object({<br> command = list(string)<br> entry_point = list(string)<br> working_directory = string<br> })</pre> | `null` | no |
| <a name="input_image_uri"></a> [image\_uri](#input\_image\_uri) | ECR image URI containing the function's deployment package | `string` | `null` | no |
| <a name="input_kinesis_configuration"></a> [kinesis\_configuration](#input\_kinesis\_configuration) | https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_event_source_mapping | <pre>map(object({<br> batch_size = number<br> bisect_batch_on_function_error = bool<br> destination_config__on_failure__destination_arn = string<br> event_source_arn = string<br> maximum_batching_window_in_seconds = number<br> maximum_record_age_in_seconds = number<br> maximum_retry_attempts = number<br> parallelization_factor = number<br> starting_position = string<br> starting_position_timestamp = string<br> tumbling_window_in_seconds = number<br> }))</pre> | `{}` | no |
| <a name="input_layers"></a> [layers](#input\_layers) | List of layers for this lambda function | `list(string)` | `[]` | no |
| <a name="input_memory_size"></a> [memory\_size](#input\_memory\_size) | Lambda function memory size | `number` | `128` | no |
| <a name="input_publish"></a> [publish](#input\_publish) | Publish lambda function | `bool` | `false` | no |
| <a name="input_region"></a> [region](#input\_region) | AWS region | `string` | n/a | yes |
| <a name="input_reserved_concurrent_executions"></a> [reserved\_concurrent\_executions](#input\_reserved\_concurrent\_executions) | Reserved concurrent executions for this lambda function | `number` | `-1` | no |
| <a name="input_role"></a> [role](#input\_role) | Lambda function role | `string` | n/a | yes |
| <a name="input_runtime"></a> [runtime](#input\_runtime) | Lambda function runtime | `string` | `"nodejs14.x"` | no |
| <a name="input_s3_bucket"></a> [s3\_bucket](#input\_s3\_bucket) | S3 bucket name where lambda package is stored | `string` | `null` | no |
| <a name="input_s3_key"></a> [s3\_key](#input\_s3\_key) | S3 key where lambda package is stored | `string` | `null` | no |
| <a name="input_s3_object_version"></a> [s3\_object\_version](#input\_s3\_object\_version) | S3 object version of the lambda package | `string` | `null` | no |
| <a name="input_sqs_external"></a> [sqs\_external](#input\_sqs\_external) | External SQS to consume | <pre>object({<br> batch_size = number<br> sqs_arns = list(string)<br> })</pre> | `null` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Tags for this lambda function | `map(string)` | `{}` | no |
| <a name="input_timeout"></a> [timeout](#input\_timeout) | Lambda function runtime | `number` | `300` | no |
| <a name="input_tracing_config"></a> [tracing\_config](#input\_tracing\_config) | https://www.terraform.io/docs/providers/aws/r/lambda_function.html | <pre>object({<br> mode : string<br> })</pre> | <pre>{<br> "mode": "PassThrough"<br>}</pre> | no |
| <a name="input_trigger"></a> [trigger](#input\_trigger) | Trigger configuration for this lambda function | `any` | n/a | yes |
| <a name="input_vpc_config"></a> [vpc\_config](#input\_vpc\_config) | Lambda VPC configuration | <pre>object({<br> subnet_ids : list(string)<br> security_group_ids : list(string)<br> })</pre> | <pre>{<br> "security_group_ids": [],<br> "subnet_ids": []<br>}</pre> | no |

## Outputs

| Name | Description |
|------|-------------|
| arn | AWS lambda arn |
| dlq | AWS lambda Dead Letter Queue details |
| function\_name | AWS lambda function name |
| invoke\_arn | AWS lambda invoke\_arn |
| qualified\_arn | AWS lambda qualified\_arn |
| queue | AWS lambda SQS details |
| sns\_topics | AWS lambda SNS topics if any |
| version | AWS lambda version |
| <a name="output_arn"></a> [arn](#output\_arn) | AWS lambda arn |
| <a name="output_dlq"></a> [dlq](#output\_dlq) | AWS lambda Dead Letter Queue details |
| <a name="output_function_name"></a> [function\_name](#output\_function\_name) | AWS lambda function name |
| <a name="output_invoke_arn"></a> [invoke\_arn](#output\_invoke\_arn) | AWS lambda invoke\_arn |
| <a name="output_qualified_arn"></a> [qualified\_arn](#output\_qualified\_arn) | AWS lambda qualified\_arn |
| <a name="output_queue"></a> [queue](#output\_queue) | AWS lambda SQS details |
| <a name="output_sns_topics"></a> [sns\_topics](#output\_sns\_topics) | AWS lambda SNS topics if any |
| <a name="output_version"></a> [version](#output\_version) | AWS lambda version |
103 changes: 103 additions & 0 deletions examples/s3/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
variable "function_name" {
type = string
}

data "aws_iam_policy_document" "assume_role" {
statement {
actions = ["sts:AssumeRole"]

principals {
type = "Service"
identifiers = ["lambda.amazonaws.com"]
}
}
}

# Do not use the below policy anywhere
data "aws_iam_policy_document" "policy" {
statement {
actions = ["*"]
resources = ["*"]
}
}

resource "aws_iam_role" "lambda" {
name = var.function_name
assume_role_policy = data.aws_iam_policy_document.assume_role.json
force_detach_policies = true
}

resource "aws_iam_role_policy" "lambda" {
name = var.function_name
role = aws_iam_role.lambda.id

policy = data.aws_iam_policy_document.policy.json
}

resource "random_pet" "bucket_name" {
}

resource "aws_s3_bucket" "b" {
bucket = "ct-${random_pet.bucket_name.id}"
acl = "private"
}

resource "aws_s3_bucket_object" "object" {
bucket = aws_s3_bucket.b.id
key = "foo/bar/baz/foo.zip"
source = "${path.module}/../../test/fixtures/foo.zip"
etag = filemd5("${path.module}/../../test/fixtures/foo.zip")
}


module "s3" {

source = "../../"

s3_bucket = aws_s3_bucket.b.id
s3_key = aws_s3_bucket_object.object.key
function_name = var.function_name
handler = "index.handler"
role = aws_iam_role.lambda.arn
trigger = {
type = "api-gateway"
}
environment = {
"LOREM" = "IPSUM"
}
region = "us-east-1"
tags = {
"Foo" : var.function_name
}
}

output "arn" {
description = "AWS lambda arn"
value = module.s3.arn
}

output "qualified_arn" {
description = "AWS lambda qualified_arn"
value = module.s3.qualified_arn
}

output "invoke_arn" {
description = "AWS lambda invoke_arn"
value = module.s3.invoke_arn
}

output "version" {
description = "AWS lambda version"
value = module.s3.version
}

output "dlq" {
description = "AWS lambda Dead Letter Queue details"
value = module.s3.dlq
}

output "queue" {
description = "AWS lambda SQS details"
value = module.s3.queue
}

107 changes: 107 additions & 0 deletions examples/s3_versioning/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
variable "function_name" {
type = string
}

data "aws_iam_policy_document" "assume_role" {
statement {
actions = ["sts:AssumeRole"]

principals {
type = "Service"
identifiers = ["lambda.amazonaws.com"]
}
}
}

# Do not use the below policy anywhere
data "aws_iam_policy_document" "policy" {
statement {
actions = ["*"]
resources = ["*"]
}
}

resource "aws_iam_role" "lambda" {
name = var.function_name
assume_role_policy = data.aws_iam_policy_document.assume_role.json
force_detach_policies = true
}

resource "aws_iam_role_policy" "lambda" {
name = var.function_name
role = aws_iam_role.lambda.id

policy = data.aws_iam_policy_document.policy.json
}

resource "random_pet" "bucket_name" {
}

resource "aws_s3_bucket" "b" {
bucket = "ct-${random_pet.bucket_name.id}"
acl = "private"
versioning {
enabled = true
}
}

resource "aws_s3_bucket_object" "object" {
bucket = aws_s3_bucket.b.id
key = "foo/bar/baz/foo.zip"
source = "${path.module}/../../test/fixtures/foo.zip"
etag = filemd5("${path.module}/../../test/fixtures/foo.zip")
}


module "s3" {

source = "../../"

s3_bucket = aws_s3_bucket.b.id
s3_key = aws_s3_bucket_object.object.key
s3_object_version = aws_s3_bucket_object.object.version_id
function_name = var.function_name
handler = "index.handler"
role = aws_iam_role.lambda.arn
trigger = {
type = "api-gateway"
}
environment = {
"LOREM" = "IPSUM"
}
region = "us-east-1"
tags = {
"Foo" : var.function_name
}
}

output "arn" {
description = "AWS lambda arn"
value = module.s3.arn
}

output "qualified_arn" {
description = "AWS lambda qualified_arn"
value = module.s3.qualified_arn
}

output "invoke_arn" {
description = "AWS lambda invoke_arn"
value = module.s3.invoke_arn
}

output "version" {
description = "AWS lambda version"
value = module.s3.version
}

output "dlq" {
description = "AWS lambda Dead Letter Queue details"
value = module.s3.dlq
}

output "queue" {
description = "AWS lambda SQS details"
value = module.s3.queue
}

5 changes: 4 additions & 1 deletion main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@
resource "aws_lambda_function" "lambda" {
filename = var.file_name
function_name = var.function_name
s3_bucket = var.s3_bucket
s3_key = var.s3_key
s3_object_version = var.s3_object_version
layers = var.layers
handler = var.handler
role = var.role
Expand All @@ -31,7 +34,7 @@ resource "aws_lambda_function" "lambda" {
publish = var.publish
source_code_hash = local.source_code_hash
image_uri = var.image_uri
package_type = var.file_name != null ? "Zip" : "Image"
package_type = var.image_uri != null ? "Image" : "Zip"

dynamic "image_config" {
for_each = var.image_config == null ? [] : [var.image_config]
Expand Down
26 changes: 26 additions & 0 deletions test/lambda_aws_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,32 @@ func TestLambda_apiGatewayTriggerExample(t *testing.T) {
require.Regexp(t, regexp.MustCompile("arn:aws:apigateway:us-east-1:lambda:path/*"), terraform.Output(t, terraformOptions, "invoke_arn"))
}

func TestLambda_s3Example(t *testing.T) {
t.Parallel()

functionName := fmt.Sprintf("lambda-%s", random.UniqueId())
exampleDir := "../examples/s3/"

terraformOptions := SetupExample(t, functionName, exampleDir, nil)
t.Logf("Terraform module inputs: %+v", *terraformOptions)
defer terraform.Destroy(t, terraformOptions)

TerraformApplyAndValidateOutputs(t, terraformOptions)
}

func TestLambda_s3VersioningExample(t *testing.T) {
t.Parallel()

functionName := fmt.Sprintf("lambda-%s", random.UniqueId())
exampleDir := "../examples/s3_versioning/"

terraformOptions := SetupExample(t, functionName, exampleDir, nil)
t.Logf("Terraform module inputs: %+v", *terraformOptions)
defer terraform.Destroy(t, terraformOptions)

TerraformApplyAndValidateOutputs(t, terraformOptions)
}

func TestLambda_kinesisTriggerBasicExample(t *testing.T) {
t.Parallel()

Expand Down

0 comments on commit 918b301

Please sign in to comment.