Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(neptune): high level constructs for db clusters and instances #12763

Merged
merged 7 commits into from Mar 1, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
71 changes: 63 additions & 8 deletions packages/@aws-cdk/aws-neptune/README.md
Expand Up @@ -21,33 +21,88 @@

<!--END STABILITY BANNER-->

Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy to build and run applications that work with highly connected datasets. The core of Neptune is a purpose-built, high-performance graph database engine. This engine is optimized for storing billions of relationships and querying the graph with milliseconds latency. Neptune supports the popular graph query languages Apache TinkerPop Gremlin and W3C’s SPARQL, enabling you to build queries that efficiently navigate highly connected datasets.

The `@aws-cdk/aws-neptune` package contains primitives for setting up Neptune database clusters and instances.

```ts nofixture
import * as neptune from '@aws-cdk/aws-neptune';

cornerwings marked this conversation as resolved.
Show resolved Hide resolved
## Starting a Neptune Database
cornerwings marked this conversation as resolved.
Show resolved Hide resolved

To set up a Neptune database, define a `DatabaseCluster`. You must always launch a database in a VPC.

```ts
const cluster = new DatabaseCluster(this, 'Database', {
vpc,
instanceProps: {
instanceType: InstanceType.R5_LARGE,
}
const cluster = new neptune.DatabaseCluster(this, 'Database', {
vpc,
instanceType: neptune.InstanceType.R5_LARGE
});
```

Your cluster will be empty by default.
By default only writer instance is provisioned with this construct.

## Connecting

To control who can access the cluster, use the `.connections` attribute. Neptune databases have a default port, so
you don't need to specify the port:

```ts
```ts fixture=with-cluster
cluster.connections.allowDefaultPortFromAnyIpv4('Open to the world');
```

The endpoints to access your database cluster will be available as the `.clusterEndpoint` and `.clusterReadEndpoint`
attributes:

```ts
```ts fixture=with-cluster
const writeAddress = cluster.clusterEndpoint.socketAddress; // "HOSTNAME:PORT"
```

## Customizing parameters

Neptune allows configuring database behavior by supplying custom parameter groups. For more details, refer to the
following link: <https://docs.aws.amazon.com/neptune/latest/userguide/parameters.html>

```ts
const clusterParams = new neptune.ClusterParameterGroup(this, 'ClusterParams', {
description: 'Cluster parameter group',
parameters: {
neptune_enable_audit_log: '1'
},
});

const dbParams = new neptune.ParameterGroup(this, 'DbParams', {
description: 'Db parameter group',
parameters: {
neptune_query_timeout: '120000'
},
});

const cluster = new neptune.DatabaseCluster(this, 'Database', {
vpc,
instanceType: neptune.InstanceType.R5_LARGE,
clusterParameterGroup: clusterParams,
parameterGroup: dbParams,
});
```

## Adding replicas

`DatabaseCluster` allows launching replicas along with the writer instance. This can be specified using the `instanceCount`
attribute.

```ts
const cluster = new neptune.DatabaseCluster(this, 'Database', {
vpc,
instanceType: neptune.InstanceType.R5_LARGE,
instances: 2
});
```

Additionally it is also possible to add replicas using `DatabaseInstance` for an existing cluster.

```ts fixture=with-cluster
const replica1 = new neptune.DatabaseInstance(this, 'Instance', {
cluster,
instanceType: neptune.InstanceType.R5_LARGE
});
```
107 changes: 24 additions & 83 deletions packages/@aws-cdk/aws-neptune/lib/cluster.ts
Expand Up @@ -9,33 +9,6 @@ import { CfnDBCluster, CfnDBInstance } from './neptune.generated';
import { IClusterParameterGroup, IParameterGroup } from './parameter-group';
import { ISubnetGroup, SubnetGroup } from './subnet-group';

/**
* Backup configuration for Neptune databases
*
* @default - The retention period for automated backups is 1 day.
* The preferred backup window will be a 30-minute window selected at random
* from an 8-hour block of time for each AWS Region.
*/
export interface BackupProps {

/**
* How many days to retain the backup
*/
readonly retention: Duration;

/**
* A daily time range in 24-hours UTC format in which backups preferably execute.
*
* Must be at least 30 minutes long.
*
* Example: '01:00-02:00'
*
* @default - a 30-minute window selected at random from an 8-hour block of
* time for each AWS Region. To see the time blocks available, see
*/
readonly preferredWindow?: string;
}

/**
* Properties for a new database cluster
*/
Expand All @@ -55,13 +28,23 @@ export interface DatabaseClusterProps {
readonly port?: number;

/**
* Backup settings
* How many days to retain the backup
*
* @default - Backup retention period for automated backups is 1 day
cornerwings marked this conversation as resolved.
Show resolved Hide resolved
*/
readonly backupRetention?: Duration;

/**
* A daily time range in 24-hours UTC format in which backups preferably execute.
*
* Must be at least 30 minutes long.
*
* Example: '01:00-02:00'
*
* @default - Backup retention period for automated backups is 1 day.
* Backup preferred window is set to a 30-minute window selected at random from an
* 8-hour block of time for each AWS Region, occurring on a random day of the week.
* @default - a 30-minute window selected at random from an 8-hour block of
* time for each AWS Region. To see the time blocks available, see
*/
readonly backup?: BackupProps;
readonly preferredBackupWindow?: string;

/**
* The KMS key for storage encryption.
Expand Down Expand Up @@ -113,13 +96,6 @@ export interface DatabaseClusterProps {
*/
readonly associatedRoles?: iam.IRole[];

/**
* Indicates whether the DB cluster is IAM auth enabled.
*
* @default - false, connections to database don't require IAM auth
*/
readonly iamAuthEnabled?: boolean;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to test this before adding support for it, found some conflicting information where AWS CLI says IAM authentication is not supported by Neptune at the cluster level but is supported at instance level (whereas CFN has no flag to pass this value at instance level). Wanted to check if SDK documentation is incorrect or if it is a really an instance specific setting and if so whether CFN support for IAM needs to be fixed.


/**
* Indicates whether the DB cluster should have deletion protection enabled.
*
Expand Down Expand Up @@ -244,38 +220,12 @@ export interface DatabaseClusterAttributes {
readonly readerEndpointAddress: string;
}


/**
* A new or imported clustered database.
*/
abstract class DatabaseClusterBase extends Resource implements IDatabaseCluster {
/**
* Identifier of the cluster
*/
public abstract readonly clusterIdentifier: string;

/**
* The endpoint to use for read/write operations
*/
public abstract readonly clusterEndpoint: Endpoint;

/**
* Endpoint to use for load-balanced read-only operations.
*/
public abstract readonly clusterReadEndpoint: Endpoint;

/**
* Access to the network connections
*/
public abstract readonly connections: ec2.Connections;
}

/**
* Create a clustered database with a given number of instances.
*
* @resource AWS::Neptune::DBCluster
*/
export class DatabaseCluster extends DatabaseClusterBase {
export class DatabaseCluster extends Resource implements IDatabaseCluster {

/**
* The default number of instances in the Neptune cluster if none are
Expand All @@ -287,7 +237,7 @@ export class DatabaseCluster extends DatabaseClusterBase {
* Import an existing DatabaseCluster from properties
*/
public static fromDatabaseClusterAttributes(scope: Construct, id: string, attrs: DatabaseClusterAttributes): IDatabaseCluster {
class Import extends DatabaseClusterBase implements IDatabaseCluster {
class Import extends Resource implements IDatabaseCluster {
public readonly defaultPort = ec2.Port.tcp(attrs.port);
public readonly connections = new ec2.Connections({
securityGroups: [attrs.securityGroup],
Expand Down Expand Up @@ -336,12 +286,7 @@ export class DatabaseCluster extends DatabaseClusterBase {
/**
* The subnets used by the DB subnet group.
*/
public readonly vpcSubnets?: ec2.SubnetSelection;

/**
* The security groups used by DB cluster.
*/
protected readonly securityGroups: ec2.ISecurityGroup[];
public readonly vpcSubnets: ec2.SubnetSelection;

/**
* Subnet group used by the DB
Expand All @@ -362,9 +307,7 @@ export class DatabaseCluster extends DatabaseClusterBase {
super(scope, id);

this.vpc = props.vpc;
this.vpcSubnets = props.vpcSubnets ? props.vpcSubnets : {
subnetType: ec2.SubnetType.PRIVATE,
};
this.vpcSubnets = props.vpcSubnets ?? { subnetType: ec2.SubnetType.PRIVATE };

// Determine the subnet(s) to deploy the Neptune cluster to
const { subnetIds, internetConnectivityEstablished } = this.vpc.selectSubnets(this.vpcSubnets);
Expand All @@ -381,7 +324,7 @@ export class DatabaseCluster extends DatabaseClusterBase {
removalPolicy: props.removalPolicy === RemovalPolicy.RETAIN ? props.removalPolicy : undefined,
});

this.securityGroups = props.securityGroups ?? [
const securityGroups = props.securityGroups ?? [
new ec2.SecurityGroup(this, 'SecurityGroup', {
description: 'Neptune security group',
vpc: this.vpc,
Expand All @@ -395,7 +338,6 @@ export class DatabaseCluster extends DatabaseClusterBase {
throw new Error('KMS key supplied but storageEncrypted is false');
}

// Deletion protection
const deletionProtection = props.deletionProtection ?? (props.removalPolicy === RemovalPolicy.RETAIN ? true : undefined);

// Create the Neptune cluster
Expand All @@ -405,14 +347,13 @@ export class DatabaseCluster extends DatabaseClusterBase {
dbClusterIdentifier: props.dbClusterName,
dbSubnetGroupName: this.subnetGroup.subnetGroupName,
port: props.port,
vpcSecurityGroupIds: this.securityGroups.map(sg => sg.securityGroupId),
vpcSecurityGroupIds: securityGroups.map(sg => sg.securityGroupId),
dbClusterParameterGroupName: props.clusterParameterGroup?.clusterParameterGroupName,
deletionProtection: deletionProtection,
associatedRoles: props.associatedRoles ? props.associatedRoles.map(role => ({ roleArn: role.roleArn })) : undefined,
iamAuthEnabled: props.iamAuthEnabled,
// Backup
backupRetentionPeriod: props.backup?.retention?.toDays(),
preferredBackupWindow: props.backup?.preferredWindow,
backupRetentionPeriod: props.backupRetention?.toDays(),
preferredBackupWindow: props.preferredBackupWindow,
preferredMaintenanceWindow: props.preferredMaintenanceWindow,
// Encryption
kmsKeyId: props.kmsKey?.keyArn,
Expand Down Expand Up @@ -465,7 +406,7 @@ export class DatabaseCluster extends DatabaseClusterBase {

this.connections = new ec2.Connections({
defaultPort: ec2.Port.tcp(port),
securityGroups: this.securityGroups,
securityGroups: securityGroups,
});
}
}
58 changes: 19 additions & 39 deletions packages/@aws-cdk/aws-neptune/lib/instance.ts
Expand Up @@ -8,7 +8,7 @@ import { IParameterGroup } from './parameter-group';

/**
* Possible Instances Types to use in Neptune cluster
* used for defining {@link InstanceProps.instanceType}.
* used for defining {@link DatabaseInstanceProps.instanceType}.
*/
export enum InstanceType {
/**
Expand Down Expand Up @@ -114,43 +114,6 @@ export interface DatabaseInstanceAttributes {
readonly port: number;
}

/**
* A new or imported database instance.
*/
abstract class DatabaseInstanceBase extends cdk.Resource implements IDatabaseInstance {
/**
* Import an existing database instance.
*/
public static fromDatabaseInstanceAttributes(scope: Construct, id: string, attrs: DatabaseInstanceAttributes): IDatabaseInstance {
class Import extends DatabaseInstanceBase implements IDatabaseInstance {
public readonly defaultPort = ec2.Port.tcp(attrs.port);
public readonly instanceIdentifier = attrs.instanceIdentifier;
public readonly dbInstanceEndpointAddress = attrs.instanceEndpointAddress;
public readonly dbInstanceEndpointPort = attrs.port.toString();
public readonly instanceEndpoint = new Endpoint(attrs.instanceEndpointAddress, attrs.port);
}

return new Import(scope, id);
}

/**
* @inheritdoc
*/
public abstract readonly instanceIdentifier: string;
/**
* @inheritdoc
*/
public abstract readonly instanceEndpoint: Endpoint;
/**
* @inheritdoc
*/
public abstract readonly dbInstanceEndpointAddress: string;
/**
* @inheritdoc
*/
public abstract readonly dbInstanceEndpointPort: string;
}

/**
* Construction properties for a DatabaseInstanceNew
*/
Expand Down Expand Up @@ -201,7 +164,24 @@ export interface DatabaseInstanceProps {
*
* @resource AWS::Neptune::DBInstance
*/
export class DatabaseInstance extends DatabaseInstanceBase implements IDatabaseInstance {
export class DatabaseInstance extends cdk.Resource implements IDatabaseInstance {

/**
* Import an existing database instance.
*/
public static fromDatabaseInstanceAttributes(scope: Construct, id: string, attrs: DatabaseInstanceAttributes): IDatabaseInstance {
class Import extends cdk.Resource implements IDatabaseInstance {
public readonly defaultPort = ec2.Port.tcp(attrs.port);
public readonly instanceIdentifier = attrs.instanceIdentifier;
public readonly dbInstanceEndpointAddress = attrs.instanceEndpointAddress;
public readonly dbInstanceEndpointPort = attrs.port.toString();
public readonly instanceEndpoint = new Endpoint(attrs.instanceEndpointAddress, attrs.port);
}

return new Import(scope, id);
}


/**
* The instance's database cluster
*/
Expand Down
14 changes: 14 additions & 0 deletions packages/@aws-cdk/aws-neptune/rosetta/default.ts-fixture
@@ -0,0 +1,14 @@
import { Duration, Stack } from '@aws-cdk/core';
import { Construct } from 'constructs';
import * as ec2 from '@aws-cdk/aws-ec2';
import * as neptune from '@aws-cdk/aws-neptune';

class Fixture extends Stack {
constructor(scope: Construct, id: string) {
super(scope, id);

const vpc = new ec2.Vpc(this, 'VPC', { maxAzs: 2 });

/// here
}
}