Skip to content

Commit

Permalink
Refactor code:
Browse files Browse the repository at this point in the history
- remove `MultipartContentType`
- remove `MultipartUserDataPartWrapperOptions`
- remove `IMultipart`
- rename `MultipartUserDataPart` -> `MultipartBody`
- other removals
- restructure other classes
- moved part rendering to part class
- set default separator to hard codeded string
- added validation of boundry
  • Loading branch information
rsmogura committed Mar 4, 2021
1 parent a013138 commit f50d10b
Show file tree
Hide file tree
Showing 5 changed files with 233 additions and 134 deletions.
42 changes: 39 additions & 3 deletions packages/@aws-cdk/aws-ec2/README.md
Expand Up @@ -981,11 +981,47 @@ instance.userData.addExecuteFileCommand({
asset.grantRead( instance.role );
```

### Multipart user data

In addition to above the `MultipartUserData` can be used to change instance startup behavior. Multipart user data are composed
from separate parts forming archive. The moment, and behavior of each part can be controlled with `Content-Type`, and it's wider
than executing shell scripts.
from separate parts forming archive. The most common parts are scripts executed during instance set-up. However, there are other
kinds, too.

The advantage of multipart archive is in flexibility when it's needed to add additional parts or to use specialized parts to
fine tune instance startup. Some services (like AWS Batch) supports only `MultipartUserData`.

The parts can be executed at different moment of instance start-up and can server different purposes. This is controlled by `contentType` property.

However, most common parts are script parts which can be created by `MultipartUserData.fromUserData`, and which have `contentType` `text/x-shellscript; charset="utf-8"`.


In order to create archive the `MultipartUserData` has to be instantiated. Than user can add parts to multipart archive using `addPart`. The `MultipartBody` contains methods supporting creation of body parts.

If the custom parts is required, it can be created using `MultipartUserData.fromRawBody`, in this case full control over content type,
transfer encoding, and body properties is given to the user.

Below is an example for creating multipart user data with single body part responsible for installing `awscli`

Some services (like AWS Batch) allows only `MultipartUserData`.
```ts
const bootHookConf = ec2.UserData.forLinux();
bootHookConf.addCommands('cloud-init-per once docker_options echo \'OPTIONS="${OPTIONS} --storage-opt dm.basesize=40G"\' >> /etc/sysconfig/docker');

const setupCommands = ec2.UserData.forLinux();
setupCommands.addCommands('sudo yum install awscli && echo Packages installed らと > /var/tmp/setup');

const multipartUserData = new ec2.MultipartUserData();
// The docker has to be configured at early stage, so content type is overridden to boothook
multipartUserData.addPart(ec2.MultipartBody.fromUserData(bootHookConf, 'text/cloud-boothook; charset="us-ascii"'));
// Execute the rest of setup
multipartUserData.addPart(ec2.MultipartBody.fromUserData(setupCommands));

new ec2.LaunchTemplate(stack, '', {
userData: multipartUserData,
blockDevices: [
// Block device configuration rest
]
});
```

For more information see
[Specifying Multiple User Data Blocks Using a MIME Multi Part Archive](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bootstrap_container_instance.html#multi-part_user_data)
Expand Down
214 changes: 102 additions & 112 deletions packages/@aws-cdk/aws-ec2/lib/user-data.ts
Expand Up @@ -278,34 +278,28 @@ class CustomUserData extends UserData {
}

/**
* Suggested content types, however any value is allowed.
* Options when creating `MultipartBody`.
*/
export type MultipartContentType = 'text/x-shellscript; charset="utf-8"' | 'text/cloud-boothook; charset="utf-8"' | string;

/**
* Options when creating `MultipartUserDataPart`.
*/
export interface MultipartUserDataPartOptions {
export interface MultipartBodyOptions {

/**
* `Content-Type` header of this part.
*
* For Linux shell scripts use `text/x-shellscript`
* Some examples of content types:
* * `text/x-shellscript; charset="utf-8"` (shell script)
* * `text/cloud-boothook; charset="utf-8"` (shell script executed during boot phase)
*
* For Linux shell scripts use `text/x-shellscript`.
*/
readonly contentType: MultipartContentType;
readonly contentType: string;

/**
* `Content-Transfer-Encoding` header specifying part encoding.
*
* @default undefined - don't add this header
*/
readonly transferEncoding?: string;
}

/**
* Options when creating `MultipartUserDataPart`.
*/
export interface MultipartUserDataPartOptionsWithBody extends MultipartUserDataPartOptions {
/**
* The body of message.
*
Expand All @@ -314,66 +308,32 @@ export interface MultipartUserDataPartOptionsWithBody extends MultipartUserDataP
readonly body?: string,
}

/**
* Options when creating `MultipartUserDataPartWrapper`.
*/
export interface MultipartUserDataPartWrapperOptions {
/**
* `Content-Type` header of this part.
*
* For Linux shell scripts typically it's `text/x-shellscript`.
*
* @default 'text/x-shellscript; charset="utf-8"'
*/
readonly contentType?: MultipartContentType;
}

/**
* Interface representing part of `MultipartUserData` user data.
*/
export interface IMultipart {
/**
* The body of this MIME part.
*/
readonly body: string | undefined;

/**
* `Content-Type` header of this part.
*/
readonly contentType: string;

/**
* `Content-Transfer-Encoding` header specifying part encoding.
*
* @default undefined - don't add this header
*/
readonly transferEncoding?: string;
}

/**
* The base class for all classes which can be used as {@link MultipartUserData}.
*/
export abstract class MultipartUserDataPart implements IMultipart {
export abstract class MultipartBody {

/**
* Constructs the new `MultipartUserDataPart` wrapping existing `UserData`. Modification to `UserData` are reflected
* Constructs the new `MultipartBody` wrapping existing `UserData`. Modification to `UserData` are reflected
* in subsequent renders of the part.
*
* For more information about content types see `MultipartUserDataPartOptionsWithBody`
* For more information about content types see {@link MultipartBodyOptions.contentType}.
*
* @param userData user data to wrap into body part
* @param contentType optional content type, if default one should not be used
*/
public static fromUserData(userData: UserData, opts?: MultipartUserDataPartWrapperOptions): MultipartUserDataPart {
opts = opts || {};
return new MultipartUserDataPartWrapper(userData, opts);
public static fromUserData(userData: UserData, contentType?: string): MultipartBody {
return new MultipartBodyUserDataWrapper(userData, contentType);
}

/**
* Constructs the raw `MultipartUserDataPart` using specified body, content type and transfer encoding.
* Constructs the raw `MultipartBody` using specified body, content type and transfer encoding.
*
* When transfer encoding is specified (typically as Base64), it's caller responsibility to convert body to
* Base64 either by wrapping with `Fn.base64` or by converting it by other converters.
*/
public static fromRawBody(opts: MultipartUserDataPartOptionsWithBody): MultipartUserDataPart {
return new MultipartUserDataPartRaw(opts);
public static fromRawBody(opts: MultipartBodyOptions): MultipartBody {
return new MultipartBodyRaw(opts);
}

protected static readonly DEFAULT_CONTENT_TYPE = 'text/x-shellscript; charset="utf-8"';
Expand All @@ -382,51 +342,76 @@ export abstract class MultipartUserDataPart implements IMultipart {
public abstract get body(): string | undefined;

/** `Content-Type` header of this part */
public readonly contentType: string;
public abstract get contentType(): string;

/**
* `Content-Transfer-Encoding` header specifying part encoding.
*
* @default undefined - don't add this header
*/
public readonly transferEncoding?: string;
public abstract get transferEncoding(): string | undefined;

public constructor(props: MultipartUserDataPartOptions) {
this.contentType = props.contentType;
this.transferEncoding = props.transferEncoding;
public constructor() {
}

/**
* Render body part as the string.
*
* Subclasses should not add leading nor trailing new line characters (\r \n)
*/
public renderBodyPart(): string {
const result: string[] = [];

result.push(`Content-Type: ${this.contentType}`);

if (this.transferEncoding != null) {
result.push(`Content-Transfer-Encoding: ${this.transferEncoding}`);
}
// One line free after separator
result.push('');

if (this.body != null) {
result.push(this.body);
// The new line added after join will be consumed by encapsulating or closing boundary
}

return result.join('\n');
}
}

/**
* The raw part of multi-part user data, which can be added to {@link MultipartUserData}.
*/
class MultipartUserDataPartRaw extends MultipartUserDataPart {
private _body : string | undefined;
class MultipartBodyRaw extends MultipartBody {
public readonly body: string | undefined;
public readonly contentType: string;
public readonly transferEncoding: string | undefined;

public constructor(props: MultipartUserDataPartOptionsWithBody) {
super(props);
this._body = props.body;
}
public constructor(props: MultipartBodyOptions) {
super();

public get body(): string | undefined {
return this._body;
this.body = props.body;
this.contentType = props.contentType;
}
}

/**
* Wrapper for `UserData`.
*/
class MultipartUserDataPartWrapper extends MultipartUserDataPart {
public constructor(public readonly userData: UserData, opts: MultipartUserDataPartWrapperOptions) {
super({
contentType: opts.contentType || MultipartUserDataPart.DEFAULT_CONTENT_TYPE,
// Force Base64 in case userData will contain UTF-8 characters
transferEncoding: 'base64',
});
class MultipartBodyUserDataWrapper extends MultipartBody {

public readonly contentType: string;
public readonly transferEncoding: string | undefined;

public constructor(public readonly userData: UserData, contentType?: string) {
super();

this.contentType = contentType || MultipartBody.DEFAULT_CONTENT_TYPE;
this.transferEncoding = 'base64';
}

public get body(): string {
// Wrap rendered user data with Base64 function, in case data contains tokens
// Wrap rendered user data with Base64 function, in case data contains non ASCII characters
return Fn.base64(this.userData.render());
}
}
Expand All @@ -438,9 +423,11 @@ export interface MultipartUserDataOptions {
/**
* The string used to separate parts in multipart user data archive (it's like MIME boundary).
*
* This string should contain [a-zA-Z0-9] characters only, and should not be present in any part, or in text content of archive.
* This string should contain [a-zA-Z0-9()+,-./:=?] characters only, and should not be present in any part, or in text content of archive.
*
* @default `+AWS+CDK+User+Data+Separator==`
*/
readonly partsSeparator: string;
readonly partsSeparator?: string;
}

/**
Expand All @@ -452,63 +439,66 @@ export interface MultipartUserDataOptions {
*/
export class MultipartUserData extends UserData {
private static readonly USE_PART_ERROR = 'MultipartUserData does not support this operation. Please add part using addPart.';
private static readonly BOUNDRY_PATTERN = '[^a-zA-Z0-9()+,-./:=?]';

private parts: IMultipart[] = [];
private parts: MultipartBody[] = [];

private opts: MultipartUserDataOptions;

constructor(opts: MultipartUserDataOptions) {
constructor(opts?: MultipartUserDataOptions) {
super();

let partsSeparator: string;

// Validate separator
if (opts?.partsSeparator != null) {
if (new RegExp(MultipartUserData.BOUNDRY_PATTERN).test(opts!.partsSeparator)) {
throw new Error(`Invalid characters in separator. Separator has to match pattern ${MultipartUserData.BOUNDRY_PATTERN}`);
} else {
partsSeparator = opts!.partsSeparator;
}
} else {
partsSeparator = '+AWS+CDK+User+Data+Separator==';
}

this.opts = {
...opts,
partsSeparator: partsSeparator,
};
}
/**
* Adds existing `UserData`. Modification to `UserData` are reflected in subsequent renders of the part.
*
* For more information about content types see `MultipartUserDataPartOptionsWithBody`
*/
public addUserDataPart(userData: UserData, opts?: MultipartUserDataPartWrapperOptions): this {
this.parts.push(MultipartUserDataPart.fromUserData(userData, opts));

return this;
}

/**
* Adds the 'raw' part using provided options.
* Adds a part to the list of parts.
*/
public addPart(opts: MultipartUserDataPartOptionsWithBody): this {
this.parts.push(MultipartUserDataPart.fromRawBody(opts));
public addPart(part: MultipartBody): this {
this.parts.push(part);

return this;
}

public render(): string {
const boundary = this.opts.partsSeparator;

// Now build final MIME archive - there are few changes from MIME message which are accepted by cloud-init:
// - MIME RFC uses CRLF to separate lines - cloud-init is fine with LF \n only
var resultArchive = `Content-Type: multipart/mixed; boundary="${boundary}"\n`;
resultArchive = resultArchive + 'MIME-Version: 1.0\n';
// Note: new lines matters, matters a lot.
var resultArchive = new Array<string>();
resultArchive.push(`Content-Type: multipart/mixed; boundary="${boundary}"`);
resultArchive.push('MIME-Version: 1.0');

// Add new line, the next one will be boundary (encapsulating or closing)
// so this line will count into it.
resultArchive.push('');

// Add parts - each part starts with boundary
this.parts.forEach(part => {
resultArchive = resultArchive + '\n--' + boundary + '\n' + 'Content-Type: ' + part.contentType + '\n';

if (part.transferEncoding != null) {
resultArchive = resultArchive + `Content-Transfer-Encoding: ${part.transferEncoding}\n`;
}

if (part.body != null) {
resultArchive = resultArchive + '\n' + part.body;
}
resultArchive.push(`--${boundary}`);
resultArchive.push(part.renderBodyPart());
});

// Add closing boundary
resultArchive = resultArchive + `\n--${boundary}--\n`;
resultArchive.push(`--${boundary}--`);
resultArchive.push(''); // Force new line at the end

return resultArchive;
return resultArchive.join('\n');
}

public addS3DownloadCommand(_params: S3DownloadOptions): string {
Expand Down

0 comments on commit f50d10b

Please sign in to comment.