Skip to content

Commit

Permalink
feat(elasticsearch): graduate to stable 馃殌 (aws#13900)
Browse files Browse the repository at this point in the history 
This PR includes a last minute API change that standardizes the way VPC configuration is passed to the domain. It also provides sane defaults, enabling users to simply pass `vpc` in order to connect a domain to a VPC.

In addition, I added a missing integration test for VPC enabled domains.

Resolves aws#10965

BREAKING CHANGE: `vpcOptions` was removed. Use `vpc`, `vpcSubnets` and `securityGroups` instead.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
@iliapolo @hollanddd
iliapolo authored and hollanddd committed Aug 26, 2021
1 parent 846ce94 commit c2f43c3
Show file tree
Hide file tree
Showing 6 changed files with 809 additions and 90 deletions.
36 changes: 30 additions & 6 deletions packages/@aws-cdk/aws-elasticsearch/README.md
View file
Expand Up @@ -6,7 +6,7 @@
Features | Stability
-----------------------------------|----------------------------------------------------------------
CFN Resources | ![Stable](https://img.shields.io/badge/stable-success.svg?style=for-the-badge)
Higher level constructs for Domain | ![Experimental](https://img.shields.io/badge/experimental-important.svg?style=for-the-badge)
Higher level constructs for Domain | ![Stable](https://img.shields.io/badge/stable-success.svg?style=for-the-badge)

> **CFN Resources:** All classes with the `Cfn` prefix in this module ([CFN Resources]) are always
> stable and safe to use.
Expand All @@ -15,11 +15,8 @@ Higher level constructs for Domain | ![Experimental](https://img.shields.io/badg
<!-- -->

> **Experimental:** Higher level constructs in this module that are marked as experimental are
> under active development. They are subject to non-backward compatible changes or removal in any
> future version. These are not subject to the [Semantic Versioning](https://semver.org/) model and
> breaking changes will be announced in the release notes. This means that while you may use them,
> you may need to update your source code when upgrading to a newer version of this package.
> **Stable:** Higher level constructs in this module that are marked stable will not undergo any
> breaking changes. They will strictly follow the [Semantic Versioning](https://semver.org/) model.
---

Expand Down Expand Up @@ -146,6 +143,33 @@ This sets up the domain with node to node encryption and encryption at
rest. You can also choose to supply your own KMS key to use for encryption at
rest.

## VPC Support

Elasticsearch domains can be placed inside a VPC, providing a secure communication between Amazon ES and other services within the VPC without the need for an internet gateway, NAT device, or VPN connection.

> Visit [VPC Support for Amazon Elasticsearch Service Domains](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html) for more details.
```ts
const vpc = new ec2.Vpc(this, 'Vpc');
const domainProps: es.DomainProps = {
version: es.ElasticsearchVersion.V7_1,
removalPolicy: RemovalPolicy.DESTROY,
vpc,
// must be enabled since our VPC contains multiple private subnets.
zoneAwareness: {
enabled: true,
},
capacity: {
// must be an even number since the default az count is 2.
dataNodes: 2,
},
};
new es.Domain(this, 'Domain', domainProps);
```

In addition, you can use the `vpcSubnets` property to control which specific subnets will be used, and the `securityGroups` property to control
which security groups will be attached to the domain. By default, CDK will select all *private* subnets in the VPC, and create one dedicated security group.

## Metrics

Helper methods exist to access common domain metrics for example:
Expand Down
113 changes: 71 additions & 42 deletions packages/@aws-cdk/aws-elasticsearch/lib/domain.ts
View file
Expand Up @@ -331,32 +331,6 @@ export interface CognitoOptions {
readonly userPoolId: string;
}

/**
* The virtual private cloud (VPC) configuration for the Amazon ES domain. For
* more information, see [VPC Support for Amazon Elasticsearch Service
* Domains](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html)
* in the Amazon Elasticsearch Service Developer Guide.
*/
export interface VpcOptions {
/**
* The list of security groups that are associated with the VPC endpoints
* for the domain. If you don't provide a security group ID, Amazon ES uses
* the default security group for the VPC. To learn more, see [Security Groups for your VPC]
* (https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) in the Amazon VPC
* User Guide.
*/
readonly securityGroups: ec2.ISecurityGroup[];

/**
* Provide one subnet for each Availability Zone that your domain uses. For
* example, you must specify three subnet IDs for a three Availability Zone
* domain. To learn more, see [VPCs and Subnets]
* (https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html) in the
* Amazon VPC User Guide.
*/
readonly subnets: ec2.ISubnet[];
}

/**
* The minimum TLS version required for traffic to the domain.
*/
Expand Down Expand Up @@ -513,14 +487,35 @@ export interface DomainProps {
readonly automatedSnapshotStartHour?: number;

/**
* The virtual private cloud (VPC) configuration for the Amazon ES domain. For
* more information, see [VPC Support for Amazon Elasticsearch Service
* Domains](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html)
* in the Amazon Elasticsearch Service Developer Guide.
* Place the domain inside this VPC.
*
* @see https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html
* @default - Domain is not placed in a VPC.
*/
readonly vpc?: ec2.IVpc;

/**
* The list of security groups that are associated with the VPC endpoints
* for the domain.
*
* Only used if `vpc` is specified.
*
* @default - VPC not used
* @see https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html
* @default - One new security group is created.
*/
readonly vpcOptions?: VpcOptions;
readonly securityGroups?: ec2.ISecurityGroup[];

/**
* The specific vpc subnets the domain will be placed in. You must provide one subnet for each Availability Zone
* that your domain uses. For example, you must specify three subnet IDs for a three Availability Zone
* domain.
*
* Only used if `vpc` is specified.
*
* @see https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html
* @default - All private subnets.
*/
readonly vpcSubnets?: ec2.SubnetSelection[];

/**
* True to require that all traffic to the domain arrive over HTTPS.
Expand Down Expand Up @@ -719,7 +714,7 @@ export interface IDomain extends cdk.IResource {
*
* @default maximum over 1 minute
*/
metricClusterIndexWriteBlocked(props?: MetricOptions): Metric;
metricClusterIndexWritesBlocked(props?: MetricOptions): Metric;

/**
* Metric for the number of nodes.
Expand Down Expand Up @@ -1002,8 +997,8 @@ abstract class DomainBase extends cdk.Resource implements IDomain {
*
* @default maximum over 1 minute
*/
public metricClusterIndexWriteBlocked(props?: MetricOptions): Metric {
return this.metric('ClusterIndexWriteBlocked', {
public metricClusterIndexWritesBlocked(props?: MetricOptions): Metric {
return this.metric('ClusterIndexWritesBlocked', {
statistic: Statistic.MAXIMUM,
period: cdk.Duration.minutes(1),
...props,
Expand Down Expand Up @@ -1177,7 +1172,7 @@ export interface DomainAttributes {
/**
* Provides an Elasticsearch domain.
*/
export class Domain extends DomainBase implements IDomain {
export class Domain extends DomainBase implements IDomain, ec2.IConnectable {
/**
* Creates a Domain construct that represents an external domain via domain endpoint.
*
Expand Down Expand Up @@ -1264,6 +1259,8 @@ export class Domain extends DomainBase implements IDomain {

private readonly domain: CfnDomain;

private readonly _connections: ec2.Connections | undefined;

constructor(scope: Construct, id: string, props: DomainProps) {
super(scope, id, {
physicalName: props.domainName,
Expand Down Expand Up @@ -1300,11 +1297,23 @@ export class Domain extends DomainBase implements IDomain {
props.zoneAwareness?.enabled ??
props.zoneAwareness?.availabilityZoneCount != null;


let securityGroups: ec2.ISecurityGroup[] | undefined;
let subnets: ec2.ISubnet[] | undefined;

if (props.vpc) {
subnets = selectSubnets(props.vpc, props.vpcSubnets ?? [{ subnetType: ec2.SubnetType.PRIVATE }]);
securityGroups = props.securityGroups ?? [new ec2.SecurityGroup(this, 'SecurityGroup', {
vpc: props.vpc,
description: `Security group for domain ${this.node.id}`,
})];
this._connections = new ec2.Connections({ securityGroups });
}

// If VPC options are supplied ensure that the number of subnets matches the number AZ
if (props.vpcOptions != null && zoneAwarenessEnabled &&
new Set(props.vpcOptions?.subnets.map((subnet) => subnet.availabilityZone)).size < availabilityZoneCount) {
if (subnets && zoneAwarenessEnabled && new Set(subnets.map((subnet) => subnet.availabilityZone)).size < availabilityZoneCount) {
throw new Error('When providing vpc options you need to provide a subnet for each AZ you are using');
};
}

if ([dedicatedMasterType, instanceType, warmType].some(t => !t.endsWith('.elasticsearch'))) {
throw new Error('Master, data and UltraWarm node instance types must end with ".elasticsearch".');
Expand Down Expand Up @@ -1491,10 +1500,11 @@ export class Domain extends DomainBase implements IDomain {
}

let cfnVpcOptions: CfnDomain.VPCOptionsProperty | undefined;
if (props.vpcOptions) {

if (securityGroups && subnets) {
cfnVpcOptions = {
securityGroupIds: props.vpcOptions.securityGroups.map((sg) => sg.securityGroupId),
subnetIds: props.vpcOptions.subnets.map((subnet) => subnet.subnetId),
securityGroupIds: securityGroups.map((sg) => sg.securityGroupId),
subnetIds: subnets.map((subnet) => subnet.subnetId),
};
}

Expand Down Expand Up @@ -1730,6 +1740,17 @@ export class Domain extends DomainBase implements IDomain {
accessPolicy.node.addDependency(this.domain);
}
}

/**
* Manages network connections to the domain. This will throw an error in case the domain
* is not placed inside a VPC.
*/
public get connections(): ec2.Connections {
if (!this._connections) {
throw new Error("Connections are only available on VPC enabled domains. Use the 'vpc' property to place a domain inside a VPC");
}
return this._connections;
}
}

/**
Expand Down Expand Up @@ -1778,3 +1799,11 @@ function parseVersion(version: ElasticsearchVersion): number {
throw new Error(`Invalid Elasticsearch version: ${versionStr}. Version string needs to start with major and minor version (x.y).`);
}
}

function selectSubnets(vpc: ec2.IVpc, vpcSubnets: ec2.SubnetSelection[]): ec2.ISubnet[] {
const selected = [];
for (const selection of vpcSubnets) {
selected.push(...vpc.selectSubnets(selection).subnets);
}
return selected;
}
6 changes: 3 additions & 3 deletions packages/@aws-cdk/aws-elasticsearch/package.json
View file
Expand Up @@ -107,12 +107,12 @@
"engines": {
"node": ">= 10.13.0 <13 || >=13.7.0"
},
"stability": "experimental",
"maturity": "experimental",
"stability": "stable",
"maturity": "stable",
"features": [
{
"name": "Higher level constructs for Domain",
"stability": "Experimental"
"stability": "Stable"
}
],
"awscdkio": {
Expand Down

0 comments on commit c2f43c3

Please sign in to comment.