AWS RDS Lambda database provisioner

Introduction

This module provisions an AWS lambda function which creates a new database and optionally a new user in RDS instance within a VPC. Supported engines are postgres and mysql. A newly created user or a master user (in case when you don't need a new user) will be granted all permissions to the created database.

This module is aim to solve a cold-start problem - when you execute terraform apply and all your infrastructure is provisioned in one run. If are trying to solve a different problem, then you should be optimizing for Day 2 operations and provision a database by other means (e.g. using terraform postrgres provider).

Features:

• Master user password as well as new user password can be passed to the module either via
  • Module variables
  • Parameters in SSM Parameter Store (Recommended!)
  • Secrets in Secrets Manager (Recommended!)
• Lambda function execution logs are shipped to Cloudwatch
• No database or user will be created if they already exist

Notes on using secrets from AWS Secrets Manager:

• When referencing secrets stored in Secrets Manager, the /aws/reference/secretsmanager prefix must be used
• A secret must contain password in the password field or be a plain-text secret

Caveats:

• This lambda function needs internet access in order to comminitcate with AWS API. You need to associate this function with one or more private subnets in your VPC and make sure that their routing tables have a default route pointing to NAT Gateway or NAT Instance in a public subnet. Associating a lambda function with a public subnet doesn't give it internet connectivity or public IP address. More context: Give Internet Access to a Lambda Function in a VPC
• This lambda function DOES NOT DROP provisioned database or user on destroy in order to prevent accidental data loss. Please make sure to delete provisioned database and user manually.
• ENIs attached to a lambda function may cause DependencyViolation error when you try to destroy associated security groups and/or subnets. More context: Corresponding issue on github

Backlog:

• Support SSL connections to RDS

This module is backed by best of breed terraform modules maintained by Cloudposse.

Terraform versions

Terraform 0.12. Pin module version to ~> 1.0. Submit pull-requests to terraform012 branch.

Terraform 0.13. Pin module version to ~> 2.0. Submit pull-requests to master branch.

Usage

Simple usage example

The following example creates a database new_database and a user new_user with the passwords passed via variables.

  module "db_provisioner" {
    source  = "aleks-fofanov/rds-lambda-db-provisioner/aws"
    version = "~> 2.0"
    
    source    = "git::https://github.com/aleks-fofanov/terraform-aws-rds-lambda-db-provisioner.git?ref=master"
    name      = "stack"
    namespace = "cp"
    stage     = "prod"

    db_instance_id                = "prod-stack-db"
    db_instance_security_group_id = "sg-XXXXXXXX"
    db_master_password            = "XXXXXXXX"

    db_name          = "new_database"
    db_user          = "new_user"
    db_user_password = "XXXXXXXX"

    vpc_config = {
      vpc_id             = "vpc-XXXXXXXX"
      subnet_ids         = ["subnet-XXXXXXXX", "subnet-XXXXXXXX"]
      security_group_ids = []
    }
  }

Example with passwords passed via SSM Parameters

This example creates a database new_database and a user new_user with the passwords passed via SSM Parameters.

module "db_provisioner" {
  source  = "aleks-fofanov/rds-lambda-db-provisioner/aws"
  version = "~> 2.0"
  
  name      = "stack"
  namespace = "cp"
  stage     = "prod"

  db_instance_id                       = "prod-stack-db"
  db_instance_security_group_id        = "sg-XXXXXXXX"
  db_master_password_ssm_param         = "/cp/prod/stack/database/master_password"
  db_master_password_ssm_param_kms_key = "alias/aws/ssm"

  db_name                            = "new_database"
  db_user                            = "new_user"
  db_user_password_ssm_param         = "/cp/prod/stack/database/new_user_password"
  db_user_password_ssm_param_kms_key = "alias/aws/ssm"

  vpc_config = {
    vpc_id             = "vpc-XXXXXXXX"
    subnet_ids         = ["subnet-XXXXXXXX", "subnet-XXXXXXXX"]
    security_group_ids = []
  }
}

Example without creating a new user

This example creates a database new_database without a new user with the master user password passed via SSM Parameter.

module "db_provisioner" {
  source  = "aleks-fofanov/rds-lambda-db-provisioner/aws"
  version = "~> 2.0"
  
  name      = "stack"
  namespace = "cp"
  stage     = "prod"

  db_instance_id                       = "prod-stack-db"
  db_instance_security_group_id        = "sg-XXXXXXXX"
  db_master_password_ssm_param         = "/cp/prod/stack/database/master_password"
  db_master_password_ssm_param_kms_key = "alias/aws/ssm"

  db_name = "new_database"

  vpc_config = {
    vpc_id             = "vpc-XXXXXXXX"
    subnet_ids         = ["subnet-XXXXXXXX", "subnet-XXXXXXXX"]
    security_group_ids = []
  }
}

Please refer to the examples folder for a complete example.

Requirements

Name	Version
terraform	>= 0.13.0
aws	>= 3.0
local	>= 2.1.0

Providers

Name	Version
aws	>= 3.0

Modules

Name	Source	Version
aggregated_policy	cloudposse/iam-policy-document-aggregator/aws	0.8.0
default_label	cloudposse/label/null	0.24.1

Resources

Name	Type
aws_cloudwatch_log_group.lambda	resource
aws_iam_policy.default	resource
aws_iam_role.lambda	resource
aws_iam_role_policy_attachment.basic_execution	resource
aws_iam_role_policy_attachment.default_permissions	resource
aws_iam_role_policy_attachment.vpc_access	resource
aws_lambda_alias.default	resource
aws_lambda_function.default	resource
aws_security_group.default	resource
aws_security_group_rule.egress_blocks	resource
aws_security_group_rule.egress_from_lambda_to_db_instance	resource
aws_security_group_rule.ingress_to_db_instance_from_lambda	resource
aws_db_instance.default	data source
aws_iam_policy_document.assume	data source
aws_iam_policy_document.default_permissions	data source
aws_iam_policy_document.lambda_kms_permissions	data source
aws_iam_policy_document.master_password_kms_permissions	data source
aws_iam_policy_document.master_password_secretsmanager_permissions	data source
aws_iam_policy_document.master_password_ssm_permissions	data source
aws_iam_policy_document.user_password_kms_permissions	data source
aws_iam_policy_document.user_password_secretsmanager_permissions	data source
aws_iam_policy_document.user_password_ssm_permissions	data source
aws_kms_key.lambda	data source
aws_kms_key.master_password	data source
aws_kms_key.user_password	data source
aws_lambda_invocation.default	data source
aws_partition.default	data source
aws_secretsmanager_secret.master_password	data source
aws_secretsmanager_secret.user_password	data source
aws_ssm_parameter.master_password	data source
aws_ssm_parameter.user_password	data source

Inputs

Name	Description	Type	Default	Required
allowed_egress_cidr_blocks	A list of CIDR blocks allowed to be reached from Lambda. Remember that Lambda needs to be able to communicate with AWS API	list(string)	[ "0.0.0.0/0" ]	no
attributes	Additional attributes, e.g. 1	list(string)	[]	no
db_instance_id	DB Instance Identifier	string	n/a	yes
db_instance_security_group_id	DB instance security group to add rules to. Rules will allow communication between Lambda and DB instance	string	null	no
db_master_password	DB Instance master password. The usage of this parameter is discouraged. Consider putting db password in SSM Parameter Store and passing its ARN to the module via db_master_password_ssm_parameter_arn parameter	string	null	no
db_master_password_ssm_param	Name of SSM Parameter that stores password for master user. This param takes precedence other db_master_password	string	null	no
db_master_password_ssm_param_kms_key	Identifier of KMS key used for encryption of SSM Parameter that stores password for master user	string	null	no
db_name	Database name that should be created	string	n/a	yes
db_user	Name of user that should be created and own (has all permission to) the provisioned database. If left empty, no user will be created	string	null	no
db_user_password	Password for the user that should be created and own (has all permission to) the provisioned database. Ignored if db_user is set to null	string	null	no
db_user_password_ssm_param	Name of SSM Parameter that stores password for provisioned user. This param takes precedence other db_user_password	string	null	no
db_user_password_ssm_param_kms_key	Identifier of KMS key used for encryption of SSM Parameter that stores password for provisioned user	string	null	no
delimiter	Delimiter to be used between namespace, name, stage and attributes	string	"-"	no
enabled	Defines whether this module should create resources	bool	true	no
invoke	Defines whether lambda function should be invoked immediately after provisioning	bool	true	no
kms_key	KMS key identifier. Accepts the same format as KMS key data source (https://www.terraform.io/docs/providers/aws/d/kms_key.html). If this configuration is not provided when environment variables are in use, AWS Lambda uses a default service key.	string	null	no
logs_kms_key_id	KMS Key Id for Lambda function logs encryption	string	null	no
logs_retention_days	Lambda function logs retentions in days	number	null	no
memory	Amount of memory in MB your Lambda Function can use at runtime	number	256	no
name	Solution name, e.g. 'app' or 'jenkins'	string	"rds"	no
namespace	Namespace (e.g. cp or cloudposse)	string	""	no
stage	Stage (e.g. prod, dev, staging)	string	""	no
tags	Additional tags (e.g. map(BusinessUnit,XYZ)	map(string)	{}	no
timeout	The amount of time your Lambda Function has to run in seconds	number	30	no
vpc_config	VPC configuration for Lambda function	object({ vpc_id = string subnet_ids = list(string) security_group_ids = list(string) })	n/a	yes

Outputs

Name	Description
lambda_function_arn	Lambda Function ARN
lambda_function_name	Lambda Function name
lambda_iam_policy_arn	Lambda IAM Policy ARN
lambda_iam_policy_id	Lambda IAM Policy ID
lambda_iam_policy_name	Lambda IAM Policy name
lambda_iam_role_arn	Lambda IAM Role ARN
lambda_iam_role_id	Lambda IAM Role ID
lambda_iam_role_name	Lambda IAM Role name

Authors

Module is created and maintained by Aleksandr Fofanov.

License

Apache 2 Licensed. See LICENSE for full details.

Help

Got a question?

File a GitHub issue.