Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monolithic releases #3

Open
domenic opened this issue Oct 10, 2014 · 34 comments
Open

Monolithic releases #3

domenic opened this issue Oct 10, 2014 · 34 comments

Comments

@domenic
Copy link

domenic commented Oct 10, 2014

From #2:

A much bigger concern to me is the overall decline in activity and the decline in new active committers, and at a time when overall ecosystem growth and interest in node is soaring. I'd look at people down the list and ask what may have turned them off from the project or, more importantly, what might have encouraged them to stay and potentially even increase their contributions.

I contributed the vm module to Node 0.11.2 or something like that. My purpose was so that consumers of jsdom could stop relying on a compiled add-on. I would have been interested in contributing more to core, but quickly lost interest when it became clear that Node 0.12 was never going to be released, and so none of the contributions I made would ever be usable by people I work with (e.g. jsdom consumers, or clients of the consultancy where I worked, or fellow open-source developers). That's what turned me off from contributing.

From this perspective the correct approach IMO is to abolish the even/odd tick-tock and just release things that are ready when they are ready. Use feature-flags to e.g. keep vm in node@beta until the bugs have been shaken out, but once it's un-bugged, move it into node immediately, instead of making its progress depend on a bunch of other features in a big "0.12" release. Do node@beta and node-stable releases on a time-based schedule (e.g. every 6 weeks), and whatever's ready after that time period is what gets shipped.

Honestly this whole stable/unstable release cycle feels like such a standards-bodies-esque mode of failure, e.g. waiting to release ES6 arrow functions until all the details of ES6 modules are finished, just because someone decided to bundle them all under the name "ES6." Browser vendors have realized this is not a good model of working, and started shipping features independent of spec version; even TC39 is moving to a feature-based model. I wish Node would too, instead of these monolithic bundled releases.

@stefanpenner
Copy link

I think you nailed it, good momentum is key to healthy contributions (from both new and existing contributors).

@mikeal
Copy link
Member

mikeal commented Oct 10, 2014

I actually agree with both of your points but I think they should be separated.

  1. Rate of release.
  2. even/odd numbering scheme.

Both are bad, but I don't think the slow rate of releases is directly related to the even/odd numbering scheme. At the same time, the even/odd number scheme is super confusing because the entire Node ecosystem uses semver.

@Fishrock123
Copy link
Member

the entire Node ecosystem uses semver.

Well, sorta-semver anyways. I'll leave this here: jshttp/style-guide#4

tl;dr: Semver means people should bump majors more, but people don't like upgrading majors.
(As a result of this and many other things express doesn't really use strict semver.)

@mikeal
Copy link
Member

mikeal commented Oct 10, 2014

Well, we should define what "using semver" means. I've probably never used it in the strictest sense, but he linear progression of version numbers I have stuck with, as has everyone else, because it gets enforced by npm.

This concurrent version number thing is very confusing for new people. We copied it from Linux before anyone even read the semver spec and npm existed. It's just straight up legacy and should be put out to pasture.

@Fishrock123
Copy link
Member

because it gets enforced by npm.

Not the best thing, but yeah, to a degree.

It's just straight up legacy and should be put out to pasture.

I actually like the idea behind it. However, I am inclined to agree.
Though Node should be 1.0'd before we do this, otherwise stuff like 0.13 is gona be super confusing haha.

@mikeal
Copy link
Member

mikeal commented Oct 10, 2014

Though Node should be 1.0'd before we do this

Absolutely, nothing would be more confusing than changing the versioning before declaring a 1.0. However, 1.0 should have been "declared" a long time ago :)

@FotoVerite
Copy link

And gifs no longer works. So totally agree.

@Qard
Copy link
Member

Qard commented Oct 10, 2014

It'd be nice to see Chromium/Firefox-style nightly/dev/prod release channels.

@mvolkmann
Copy link

I SO want to use arrow functions in Node now! Don't make us wait for other ES6 features to be finalized.

@domenic
Copy link
Author

domenic commented Oct 10, 2014

@mikeal I don't think they're as unrelated as that. The even/odd scheme---or really, any scheme that has an unstable branch where active development happens, leading up to a new release---is actively counterproductive to attempts to merge features in as soon as they are ready. The idea that 0.10 will get no new features, and thus we need to wait until 0.12 to get a new vm module---while the entire time a finished and polished vm module in the 0.11 branch is being held up by unfinished AsyncListener stuff---is the problem. If 0.10 was allowed to get new features, instead of being the stable no-new-features release train, I'd be much happier.

@domenic
Copy link
Author

domenic commented Oct 10, 2014

@mvolkmann that has nothing to do with the current thread (please re-read my post) and doesn't even make sense anyway since V8 doesn't have arrow functions.

@aheckmann
Copy link

The wait for 0.12 has caused me to think the project is running out of gas and losing focus. I disagree that adding new features in a minor release (0.10) is the right move because I like knowing new features / API changes are not going to impact me when I update a point release.

I too would like to see a schedule put in place to ship smaller, ready features more often.

@mvolkmann
Copy link

@domenic, you said "Honestly this whole stable/unstable release cycle feels like such a standards-bodies-esque mode of failure, e.g. waiting to release ES6 arrow functions until all the details of ES6 modules are finished, just because someone decided to bundle them all under the name "ES6."

That's where I got that idea that the Node team had decided not to enable use of arrow functions until other ES6 features were finalized.

@stefanpenner
Copy link

I disagree that adding new features in a minor release

Assuming these features are purely additive they should not disrupt existing code, if they do they should be considering breaking and clearly not included in a point release (and maybe not included at all)

@Qard
Copy link
Member

Qard commented Oct 10, 2014

Even if the feature is stable, it should stay marked as experimental until
the next minor release. It's easy for users to lose track of what versions
support a feature they want to use, if it could have come in a patch
release. Following semver properly should make it easier to track.

@sam-github
Copy link

I agree with @domenic, node should use semver (I also think the 0.12 release should be 1.0.0, in fact, as soon as node started to promise it would keep the API stable, years ago, it should have gone to 1.x, but thats ancient history now).

With semver, stable releases can have new features added. Right now, stalling good stable features from release for a year and a half (or even 6 months if we get release cycles that blessedly short), is not reasonable.

One thing this will force is more focus on bug fixing. Right now, AFAICT, features are flowing into 0.12 daily, probably contributing to its continuing instability.

What we really need, IMO, is release-blocking issues fixed in 0.12, and an embargo on merging anything to the release branch that is not release blocking so it can get released. But there is no list of such bugs.

@othiym23
Copy link

@aheckmann speaks for me here:

I disagree that adding new features in a minor release (0.10) is the right move because I like knowing new features / API changes are not going to impact me when I update a point release.

This is point on which @domenic and I have never agreed, but I just want to reiterate it, because I really don't think server-side programming platforms are directly comparable to web browsers. I am in favor of moving from the current version scheme to semver, but I would like everyone to err on the side of caution when marking changes as breaking.

@Fishrock123
Copy link
Member

but I would like everyone to err on the side of caution when marking changes as breaking.

Stability is good, but I don't think it should hold back enhancements.

If enhancements don't break stuff and are good to be shipped, I think we should consider shipping those sooner than never.

@jasonrhodes
Copy link

You can ship non-breaking enhancements by bumping a minor version—this is
exactly how semver theoretically works.

Currently, Node core doesn't use semver at all, and the even/odd release
cycle kills the ability to bump minor versions and pull in new features.
Seems like it's time to go 1.0 and embrace semver.

On Fri, Oct 10, 2014 at 11:05 PM, Jeremiah Senkpiel <
notifications@github.com> wrote:

but I would like everyone to err on the side of caution when marking
changes as breaking.

Stability is good, but I don't think it should hold back enhancements.

If enhancements don't break stuff and are good to be shipped, I think we
should consider shipping those sooner than never.


Reply to this email directly or view it on GitHub
#3 (comment)
.

@Fishrock123
Copy link
Member

Ok so I have a serious question.

If we do 1.x.x / semver with a quicker release cycle, which versions are we going to support for back-porting fixes?

@bnoordhuis
Copy link
Member

@Fishrock123 asks a great question.

Let me start out with the related issue of upgrading V8: a v1.0.0 release is a promise of stability but it's not feasible (nor desirable) to freeze V8 for all eternity. People want features, distros want security fixes and maintainers want support from upstream. But upgrading V8 inevitably breaks tons of add-ons.

As a conversation starter, I'm going to suggest allowing V8 upgrades (and libuv, openssl, etc.) with every minor (not major) release. TBD how often to put out minor releases.

I've become a fan of time-based releases; it's not perfect but it gives developers a deadline, users a date to look forward to and it avoids the "release when it's ready" quagmire that joyent/node is stuck in. How does a minor release every four months or thrice a year sound?

That brings me back to @Fishrock123's question, support for older releases. I suggest bug fix support for $minor-1 and security fix support for $minor-2. That's eight months of full support and one year of security fixes.

@Fishrock123
Copy link
Member

I'm going to suggest allowing V8 upgrades (and libuv, openssl, etc.) with every minor (not major) release
How does a minor release every four months or thrice a year sound?

If this actually becomes reality.. :D

avoids the "release when it's ready" quagmire that joyent/node is stuck in.

My observations so far is that "release when it's ready" just doesn't work in software, no matter what the software is.

Ok maybe if you're programming the mars rover. Then it might be applicable. :P

I suggest bug fix support for $minor-1 and security fix support for $minor-2. That's eight months of full support and one year of security fixes.

So what about major bumps? Inevitably there will be people too quagmired in complex legacy code to update for all the breaking changes.. (See 0.8..)

@bnoordhuis What's the sorta time-plan-idea for majors?

@jonathanong
Copy link

a time-based release sounds awesome! another approach is just to release a new version whenever v8 updates to a new "stable" minor version. i.e. it looks like v8 3.29.x took about 1.5 months.

i'd rather have a v1.0.0 quickly and do major releases whenever there's a possibility of a breaking change. i'd also rather core not be afraid of adding/removing/changing APIs, but they should make it easy on users via a utility like: https://github.com/dougwilson/nodejs-depd

@Fishrock123
Copy link
Member

Yes, using something like depd would be excellent. +1

@domenic
Copy link
Author

domenic commented Oct 13, 2014

@bnoordhuis glad you're a fan of time-based updates :). I agree for all the reasons you state.

IMO four months is way too long to wait for new features or for a V8 upgrade. Six weeks would be much better; as @jonathanong says, that is how often V8 releases. Four months would mean in some cases you wait a third of a year for something you've been using in Chrome for that long.

This is purely from a non-native module author and consumer perspective, though; I understand the way in which V8 upgrades cause breakage could be a cause for slowing things down. But I want to present a counterbalance to the "take things slow" crowd :). So with that in mind:

In a rapidly-updating world like this, I'd guess that native module authors would become more active in un-breaking their code, but there would still be some lag from native module authors and from those who consume them. So you'd probably need to have infrastructure in place to backport fixes efficiently. I might do security fixes only though, to reduce workload and indirectly help encourage the ecosystem to upgrade?

An abstraction layer like nan or similar bundled with core could help reduce the friction in order to make upgrading more seamless, and could even cause some "breaking" releases to convert into non-breaking if the differences are able to be abstracted behind a nan release. That is, you could go by a few six-week cycles before reaching a point where the changes are too big for nan to encompass.

@aredridel
Copy link

bundling nan or similar would help a ton, if we can provide an API surface that we bless as 'stable'. A good part of following semver is declaring what things are not supported interface. If v8's specifics are "out of zone", then it's easier to follow semver -- but reduces what one can do with the system and stay stable. Trade-offs.

@hunterloftis
Copy link

I agree with @domenic that the versioning scheme is tightly coupled to forward momentum. Even/odd turns the unstable branch into a second-class citizen, discouraging progress:

https://github.com/mongodb/js-bson/issues/64

Additionally, even/odd discourages selfish patches. Many of us write selfish PRs - we want to use a feature now or fix a bug now, for ourselves or our customers. That motivation disappears with even/odd versioning.

Finally, batching features together creates artificial delays. In some cases, I'm encouraging Heroku users to deploy Node 0.11 because it has features that can really benefit them today - and waiting for 0.12 could mean waiting indefinitely.

I respectfully disagree with @Qard that nightly/dev/prod release channels are a solution. I actually see a lot of confusion about Node versioning already, and I'd like to simplify and streamline it into a linear (semver) path, if possible.

Regarding release schedule, I think @bnoordhuis has the right idea with time-based releases, but I'd love to see updates more than three times a year. However, he would have a much better idea than I do about what's feasible in terms of core development. If node moves to a time-based, small-features-fast, semver-organized release schedule, I would be very interested in contributing to core myself.

@Qard
Copy link
Member

Qard commented Oct 16, 2014

You can do nightly/dev/prod with semver. Just use beta and alpha suffixes.

I'm not super adamant that is the best way. I just suggested it because it
seems to work for Chrome and Firefox.

@bnoordhuis
Copy link
Member

Regarding release schedule, I think @bnoordhuis has the right idea with time-based releases, but I'd love to see updates more than three times a year. However, he would have a much better idea than I do about what's feasible in terms of core development.

It's mostly a function of man power. We discussed a 6 week release schedule in yesterday's TC meeting but that raises the question of how long older releases can be supported.

Someone (I think it was @piscisaureus) suggested doing LTS releases. I think that's a good idea but the details of how often and how long to provide support are still TBD. Input welcome.

@Qard
Copy link
Member

Qard commented Oct 16, 2014

Perhaps you could set whatever the last minor version was as LTS whenever you bump major.

For example: 1.5.x is out and V8 changes everything again. You lock 1.5.x as the current LTS and bump to 1.6.x before introducing the breaking V8 changes.

If necessary, you could have a two-stage LTS system, where there's an actively supported LTS and a deprecation warning LTS. Both would receive security patches and such. If using the deprecation warning version, it's advisable to switch to active because the LTS window could shift off of it at any time.

@bjouhier
Copy link

@domenic Agree. It takes too long to get from 0.11 to 0.12 (still hoping we'll get there). It's not really a problem for us but it sends a bad signal.

Would be better to have a new stable branch at a regular rate (every 6 months would be good for us but faster would not hurt).

It is important to get updates on the stable branches (only bug fixes should be allowed). It is also important to encourage people to upgrade to the latest stable branch. This can be achieved with a policy like "bug fixes will be applied only to the last N stable branches and security fixes only to the last M stable branches" (M > N).

In our experience upgrades are rather smooth. The main pain point is V8 API changes.

@rlidwka
Copy link
Member

rlidwka commented Oct 20, 2014

Even-odd system is great. Only issue is that we rarely have releases nowadays, but it has nothing to do with the versioning system.

Another idea would be to bind node versioning with v8 versioning somehow. It will certainly make sense now when everybody waits for the es6 features.

@darrenderidder
Copy link

I generally like semver, but whether it goes with even/odd or stable / dev branches, an LTS version would be really important for the big enterprise-type users. A one-year rolling window for security patches is probably too short for the large international carriers we're deploying into, so I'd like to see security fixes back-ported to an LTS version if possible...

@mpareja
Copy link
Member

mpareja commented Nov 16, 2014

I can't help but think that a lot of these flow issues would be minimized by maintaining a visible backlog and appropriately breaking up the work into small independently deliverable chunks of value. Maybe I just don't have visibility into the right conversations, but I get the sense that work happens and is tracked in terms of monolithic features. Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests