Bypass userdata limit with s3 plus follow ups #268
Conversation
|
This isn't really a natural order of things I would do to fix something but anyways E2E tests are passing against the cluster. |
| "stack.json": stackTemplate, | ||
| "userdata-controller": c.UserDataController, | ||
| "userdata-worker": c.UserDataWorker, | ||
| "userdata-etcd": c.UserDataEtcd, |
There was a problem hiding this comment.
etcd userdata can't be moved to s3, have to stay in CF template to be excluded from updates
There was a problem hiding this comment.
Good catch! I'll fix it. Anyways, it is uploaded but not read from anywhere now.
Thanks for the notice 👍 |
Current coverage is 57.20% (diff: 38.68%)
|
|
Unit/integration tests are passing. |
|
|
||
| templateURL, err := c.UploadTemplate(s3Svc, s3URI, stackBody) | ||
| if err != nil { | ||
| func (c *Provisioner) uploadTemplates(s3Svc S3ObjectPutterService, uploads map[string]string) (*string, error) { |
There was a problem hiding this comment.
@redbaron It seems to me that you've used uploadTemplates to upload not only a template but also files referenced from it(e.g. cloud-config-worker and cloud-config-controller).
If so, I'd rather like to change its signature to something like uploadStackAssets(s3Svc S3ObjectPutterService, stackTemplate string, cloudConfigs: map[string]string) for now and extend/introduce a type like StackAssets in the future.
Then, if filename == "stack.json" { can be removed hence less complexity.
|
I tested 9fce0cb change locally,seems to work, consider adding here if you agree EDIT: just noticed it misses comment with userdata hash to trigger userdata update when real userdata changes on s3 bucket :) |
Finish the work to bypass the userdata limit of 16KB in size with node pools support including spot fleet backed ones
mumoshu
force-pushed
the
bypass-userdata-limit-with-s3-plus-follow-ups
branch
from
7102d59
to
2d348a0
Compare
|
@redbaron Thanks for wrapping it into a systemd unit 👍 I will test it out later. Btw,
Any idea how we could improve it? |
mumoshu
changed the title
…a-limit-with-s3-plus-follow-ups Bypass userdata limit with s3 plus follow ups
@redbaron FYI this is what I have so far.
Unit tests are failing/won't even compile but I've tested this by creating an actual cluster including 1 main cluster with 2 node pools including 1 asg-backed pool and 1 spot-fleet-backed pool.
If interested, please see my latest commit mumoshu@c2358d7 for what were changed since your work. Basically I've refactored
cfnstack/so thatdestroy.go) anymore