Skip to content
This repository has been archived by the owner on Sep 30, 2020. It is now read-only.

Bypass 16kb userdata limit by uploading to s3 bucket #255

Conversation

redbaron
Copy link
Contributor

@redbaron redbaron commented Jan 17, 2017

Crude fix for #252 .

This is a best-effort rebase+squash of our internal branch on top of master. It seems to solve this unexpected blocker for us, but code is brittle and moderately ugly, but not much more that already existing code around there :) I have no capacity to refine it further. It breaks nodepools (another reason to get rid of them!).

Probably best thing to do is to make cluster.yml to accept s3Bucket config option and remove s3-uri flag, making presence of a readable bucket for cluster needs a hard requirement.

As I said, I wont be able to make it better, please adopt it, edit it to your needs and fix it,throw away it or whatever, but fix 16KB problem in master one way or another It is a major blocker for us so had to be solved ASAP, I imagine other users feel this pain too.

No testing of this PR is done whatsoever, just hacked it so that it compiles. Internal branch this PR is based on seems to be working fine.

@redbaron
Copy link
Contributor Author

small comment on PR itself: all it is doing is plumbing stackConfig (which includes what of *Cluster flavours, I lost track which one)all the way up to a point where CF template upload is happening.

@mumoshu mumoshu added this to the v0.9.4-rc.1 milestone Jan 18, 2017
@redbaron redbaron force-pushed the bypass-userdata-limit-with-s3 branch from 08d32e9 to efda1a2 Compare January 18, 2017 11:44
Seems to be an rebase artifact
@@ -577,7 +577,7 @@ func (c Cluster) stackConfig(opts StackTemplateOptions, compressUserData bool) (
stackConfig.Config.TLSConfig = compactAssets
}

if stackConfig.UserDataWorker, err = userdatatemplate.GetString(opts.WorkerTmplFile, stackConfig.Config, compressUserData); err != nil {
if stackConfig.UserDataWorker, err = userdatatemplate.GetString(opts.WorkerTmplFile, stackConfig.Config, false); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'd been bitten by this myself 😛

@mumoshu
Copy link
Contributor

mumoshu commented Jan 18, 2017

Thanks for the follow-ups!
I'm testing/refactoring this in my internal branch.
I'll merge this and/or my branch including your commit probably after #227 and/or #169

@mumoshu
Copy link
Contributor

mumoshu commented Jan 24, 2017

Closing this in favor of #268(It is almost ready to be merged!

@mumoshu mumoshu closed this Jan 24, 2017
@kylehodgetts kylehodgetts deleted the bypass-userdata-limit-with-s3 branch April 8, 2018 10:07
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants