New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Task Running #2
Comments
Having worked with Grunt, Gulp and Broccoli, my current go to is Broccoli. Mostly because I'm always setting up front-end build processes, testing and dev workflows and Broccoli was engineered specifically for that problem space. It's faster to deal with during dev builds than grunt, and there's generally less noodling with the Brocfile than I'd have to do with the others. The downside is it's not as general purpose as the other two. But that's also the upside got me at least. Grunt has the most mature community, but is dirt-slow. Gulp is faster than grunt (usually), but you end up doing a lot more work to set up your workflow than you will with Broccoli because of Broccoli's built in server and watching. At least if your developing UI. |
If I were doing something more general purpose, I suspect I'd use Gulp ATM. |
+1 for npm's task runner. I have found that Grunt files become unwieldy on large projects and is slower than Gulp. Having used Gulp on recent projects, I found that there are some cool ways to better organize your tasks, but a number of plugins tend to violate the Gulp guidelines and you need to research the good modules to create optimal Gulp recipes. Broccoli looks cool, but if you are looking for something that works equally well on *Nix and Windows, it doesn't appear to be quite ready for Windows. For me, each of these task runner tools create an extra layer that can be solved simpler by just utilizing npm run per Substack. Sticking with npm eliminates the need for additional dependencies which will reduce the overall boilerplate footprint and create a low barrier of adoption for those already familiar with Node. |
Makefiles I've said it before; Using a "flavor of the month" tool for a project that is meant to last a long time is a bad idea. Everything you can do in grunt/gulp/taskjs you can do in Makefiles, with the added benefit of decades of backing, knowledge, tutorials, and examples already out there. Given it's cross-project and cross-language, it allows others who may not be 100% familiar with JS build systems to get up and running quickly - one less tool to learn (I'm assuming most decent developers know Make already). If not make, then npm task runners would be my second choice. Because these can be simple calls out to a bash script that does what a Makefiles would have done for you otherwise. (And besides, Makefiles often call out to external scripts anyway - it's the Unix Philosophy!) |
You assume wrong. The majority of developers, particularly web developers, either don't have a build process or have their build processes handed to them by IDEs like Visual Studio. More importantly, they have absolutely no reason to know what a make file is, let alone how to write one. That doesn't make them bad developers. I see nothing wrong with Make, and I applaud the spirit of the suggestion, people are too happy for shiny things. But using Make with JavaScript projects is more "flavor of the month" (ala hipster) than anything else these days. It's pretty rare to see a JS project using Make, in my experience. If you're looking for "tried and true" in JavaScript-land, Grunt or simple |
@Blesh Thanks for calling me on that statement. I have a bias toward Make since I come from a C/C++ background.
Even more reason for using Makefiles; VS has a Makefile Project Wizard built in, whereas the others require a plugin to be runable from within that IDE. Although, but implying the majority of web developers use VS disregards the large numbers of devs who work in *nix environments (Linux, OSX, etc).
I was curious, so I did some basic searching on Github;
If my search-fu is correct (it probably isn't), it shows Grunt as by far and away the most popular build tool for Javascript, but Make still comes in before Gulp. |
Make has been around for decades and is still in constant use. There is a TON of tooling for make. Most IDEs either know how to assist you with make files, or have plugins that do. It's true that few JavaScript projects use it, but some do, and it may be worthwhile to encourage Make to spread in the JavaScript ecosystem. Contrast with Grunt, Gulp and Brocolli, the only one with a large ecosystem is Grunt, and it's quickly going out of fashion. Gulp and Brocolli are new things to learn for most JS developers. As long as you have to learn something new, shouldn't you have some assurance that it will still be useful in 2017? I think That said, make is something new to learn for many JavaScript developers. Maybe we should stick with Thoughts? |
Just gonna leave this here: https://gist.github.com/substack/8313379 |
My vote is for Gulp. It is something new to learn, but it is very intuitive. A Gulp File is very easy to read an understand what is going on. I have to agree with @clohr that you have to do research to find the good plugins/scripts because the community is still small. I do like the sound of make though. I personally never considered using it with JS. I'm going to have to look at a few examples. |
Arguably, so is Grunt, Gulp, etc - their syntax can be (almost) as opaque as make at times. I whole heartedly agree with
Now you're talking about implementing autoconf / cmake. I'm not so sure that's a good route to go down (it's the equivalent of implementing a gruntfile-builder package that autogenerates your gruntfile.js based on gruntfilelists.txt :P) It's always possible to alias npm tasks to make targets: // package.json
{
"scripts": {
"build": "make build"
}
} That way, it feels familiar to a JS user who might not have seen make before, but doesn't otherwise care about what Check out Building javascript projects with make for a great intro ;) |
One thought I have about using make is that it is a new language. If I imagine myself giving a Junior Developer that has started learning with JavaScript and they see this and see it is a completely different syntax they might be spooked a bit. Just a thought. |
I think we may be missing an important point... build processes are not necessarily appropriate for beginning developers to be _task_ed (pun intended) with writing. Build processes are more a "devops" thing than a "junior javascript developer just joing the team" kind of thing. That's not to be elitist and say that non-devops devs shouldn't be able to understand and even help manage these things. I'm just saying in the spectrum of important influences on choice of build systems, I wouldn't say "immediate familiarity to a junior dev" is high on my list. |
The caveat to that is; We may encounter non-junior / intermediate / senior JS devs who have never used Make before, but want to use the cloverfield project (in a way which requires altering the build system). In that case, I'd argue that forcing them to use/learn Make is a good thing, but it's almost guaranteed to turn some people off. The same way that using Grunt over Gulp might turn some people off, etc. |
I'd say "ability to use/execute the build system" and "ability to configure/customize/write the build system components" are two separate and almost orthagonal concerns. From my limited perspective, it seems the former is more what's at concern here with sentiments like "almost guaranteed to turn some people off". So I think the more important question there is what is the interface to the system, and how accessible is it to those with less experience. I can't imagine What it runs on under the covers should be debated separately, and the focus should be IMO less on "ease of familiarity" and more on "right fit". I linked to "task.js" earlier because for me, writing JS code to build my JS code is the most natural fit. If you're at a PHP shop, though, I bet PHP is the more natural fit. Same with a perl shop. I personally don't feel like |
Well thought out and articulate points! As I said earlier, I have a C/C++ background which gives me a favourable bias towards |
I agree that JavaScript feels like a more natural fit. So whatever is selected should be JS. |
👍 Yep. Let's clarify what our typical build system is trying to achieve:
What we really need is TodoMVC for build systemsLet's actually set up builds using the top contenders. Let's pick the Backbone TodoMVC source (we're not actually building an app here, this is an arbitrary, meaningless selection, no bikeshedding) and whip up the contenders. If you care about build system x, write a TodoMVC build with x and link to it here.To make the timing meaningful, we should try to use all the same tech in all the contender build systems. These are arbitrary tech selections -- these do not reflect the tech we'll promote in Cloverfield. No bikeshedding about this stuff here, please. =)
Since we're JavaScript guys, we can kick off the build process and time the results using JavaScript. We can worry about the exact timing code when we have a couple builds to compare. Criteria for judging:
Note also, this isn't a spec race. We're all going to weigh in with our opinions, which are informed by the somewhat-objective results. The fastest build is not an automatic winner, but it will help if we have some data backing our opinions up. Let's get building. =) |
Nope, I'm talking about running a single command to, for instance, add a CSS sprite compiler to the build, rather than choosing the tech, downloading the libs, and adding to the makefile manually. |
Absolutely agreed. 👍 |
I think we should keep it simple. Use |
One thing I love about |
This is totally doable. I LOLed at Substack's task.js. I think what's less important than the tools is that we support a high-quality workflow for developers.
|
Note, this is pretty basic stuff that just about every task runner should have lots of clear examples for. I'm totally cool with npm, and I don't think we'd get much backlash if we make that the standard, as long as the npm examples exit with an error status for continuous integration... |
There's a larger problem at play here in my mind: There are two camps:
The problem is that for people in category No. 2 above, the concepts and learning curves for all available technologies are daunting to say the least. They probably don't have access to mentors or even know were to look. So to what @getify was saying earlier that "Junior devs shouldn't be setting up builds" (paraphrasing), I'd say "that's only because it's so hard right now that they can't". Proof: A junior dev can build and deploy a C# project with the click of a mouse in Visual Studio. Furthermore, because they "shouldn't" doesn't mean they aren't going to try. And if they're going to try, it should be easy for them. Now I'm not saying the JS world should strive for "click to deploy". But I am saying that the available tooling isn't approachable or ergonomic enough, and that's why a "Junior Dev" can't be trusted with it. Not necessarily because the core concepts are hard. Personally, I think the work that @stefanpenner has done with Ember-CLI (built atop BroccoliJS) is absolute genius, and the larger JS community should strive to build upon that idea. What I think is a bad idea is getting all neckbeard about "what people should be using because it's the old way and it's 'established'". The old ways are good, and we should remember them, but they're not progress, and it's the inadequacies of the old ways that gave birth to all of these new task runners. If Make were perfect, Rake wouldn't exist. If Rake were perfect, Grunt wouldn't exist. Grunt, Gulp; Gulp, Broccoli. And so on. |
I think that all these tasks are better explained here: http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/ |
In closing, I'd like to offer exactly zero solutions to the problems I've presented. (╯°□°)╯︵ 🎤 |
Great idea - I'll see what I can do to put together a make example (although I do have to say; make's power really comes into play when there are a lot of files that need expensive compilation steps due to its intelligence in avoiding recompilation of untouched files). That's an excellent post, thanks for linking it - it even opened my eyes to some things I didn't know npm's task runner could do :)
I dispute the claim grunt, et al came about because of deficiencies in I would rephrase what you said to be:
|
Yep. I'm aware of that. What Cloverfield will be is something that takes most of the work and hard choices out of that. Things like which test suite should we use? Well, that kinda depends on your framework choice, right? For instance, I think if you're using Meteor, you really only have one choice that integrates well with meteor, but shouldn't This implies that Cloverfield should support multiple frameworks, and that the framework selection you make will have an impact on what Cloverfield does. Before any of you go crazy investing a lot of time in this, let me make it clear: Cloverfield is about supporting the future, not the framework of the minute. If you're happy with ember-cli, continue to use ember-cli and ignore this project. When you eventually realize that the web is eating all software because it's built on standards that have lasted the test of time, we'll welcome you back, and hopefully, somebody will have extended Cloverfield with some pretty nice Ember 6.0 support. Brace for it: That means I don't give a crap about supporting today's Angular custom element directives. They're getting shot in the head by Angular's team next year. ...But I do consider it important to think about how we might scaffold web components such that they're easy to use with Angular, or React, or Meteor. =) |
The thing is, they are hard. And we make them hard as a conscious choice. Technically, the only think you need to run a javascript project is It's us as the developers who are looking for more advanced options / features that don't exist by default who decide to add an extra tool or two or twenty. It's about the trade offs we make there - more complexity at build time for (possibly) better developer productivity during coding time, and (possibly) better performance at run time. A junior dev / someone new to JS would not be making those choices because they shouldn't need to. It's only when they get sufficiently advanced enough that they should be doing it. At that point, it's reasonable to assume they're able to learn another tool chain (Gulp / Grunt / Makefiles / etc), and that the tool they learn will be the one they stick with. This goes back to @ericelliott's implication that
Counter-proof: A junior dev can run a javascript application by running |
@jesstelford I'm not sure why, but I can't seem to get |
Hey @ericelliot I think the idea of comparing task runners in the same way todomvc compares frameworks is an awesome idea. It would be enlightening even beyond the scope of this project. I think it is worthy of its own repo, so if you don't start one I will. :) It would be nice to have a basic set of files to "build" and a clear set of tasks to accomplish. With some clever requirements, it could also measure some interesting metrics for each approach (number of required dependencies, timings, etc). |
I have created a "base" project which can be forked, and have the build tool added easily. This is what I used to build my |
I've created TodoTasks for this purpose. |
@jesstelford Can you open a TodoTasks PR with your base project? |
👍 Open: ericelliott/todotasks#1 |
Refer to the sample project, todotasks. @jesstelford, could you help us clean up the repo and make it conform more to this suggested spec? |
Not a complete answer to your questions @ericelliott but as someone who has gone through the build tool learning process recently (within the last year): Make - I found learning Rake easy and intuitive. I'm sure Make would be just as nice to work with. |
Thanks Adam! Your perspective is very helpful. =) |
I like the idea of |
The problem with Keep in mind that while Substack is a brilliant developer, he spends a whole lot of time building little one off, self contained, isolated modules (which is very good!). The purpose of this project is to ease the process of building complete applications. When you're working on a team (or for larger orgs, teams of teams!), it's really hard to keep everybody on the same page. If there's a prescribed way of doing things, that saves In other words, having no standards means that a lot of devs are wasting a lot of time reinventing the wheel. =) That's the problem that Cloverfield hopes to be able to solve for all application developers who decide to use it. |
While I'm quite late to the party, I just wanted to throw in a vote for NPM. I'm working on a games platform that's all Node -- and only back end services -- so that obviates Grunt, and we're managing the majority of the work that we'd otherwise use Gulp for with Jenkins instead. NPM's "scripts" and "bin" facilities (via package.json) allow us to do anything we need a module to do to install, run, test, and clean up after itself. To @Blesh's point about there being two types of developers, I can hang with the best of JS experts that know the value of continuous integration and rabid testing and automation, but I'm also prepared, if necessary, to accept that JavaScript may not always be the best tool for a job. If that were the case for a given module, my second choice would immediately be make, because it can actually do all of the things that these other tools have been built to do. It just happens to be the case that most people don't like reading man pages to learn how to use things. And then I read Building JavaScript Projects with Make, and I was like ... "uh, yeah, I can do that, too!" |
@elrasguno Rock on. I love the feedback. I'm really liking the |
@jesstelford I just looked at your |
I have some open PRs in the todotests repo if anybody would like to review them, in order, please! =) Note that in the npm example, the source gets compiled with Browserify and minified with working sourcemaps. It would be great if other build system examples could do the same. |
@vivainio raised an interesting point in the existing scaffolding tools thread.
It's true that both Any thoughts on using Does @jesstelford's make implementation of todotasks run on Windows? (I don't have a Windows box handy right now to test). |
Interesting. @isaacs (of npm / node fame) just posted a makefile tutorial gist. Check it out! |
@ericelliott ...and just the kind of example that won't work on Windows. |
Being new to the game, for me a good way to monitor/fish out errors in my code when I'm in development is very important so I can easily auto rebuild and possibly leave the file non minified so I can find where the issues are. Nodemon is very handy but I'm sure there is other solutions out there but been a bit of a newby, I've not had the experience to play with them all. |
@Cmdv Check out the npm implementation in the todotasks repository. It watches files for changes and automatically lints, rebuilds, and reruns unit tests on file saves. The build produces sourcemaps so you can easily find the source of bugs with correct filenames and line numbers. Ignore the actual code -- the important thing about that repo is the process. Type All of these tools support processes like that. The thing that's interesting about the |
Something else worth considering, at least for the
So it's hard to say "this is how it would look with |
I like the idea of mixing in wzrd or budo, assuming it's easy enough to integrate them into people's actual projects. They're a bit limited by default, but they're a great idea. For the rest of my reply, see my comment in the tech selection thread. |
Looks like we're going with npm scripts. Thanks for the lively discussions! =) |
Hey everybody... I'd love your input on composable scaffolds. =) |
Choosing a Task Runner for the Cloverfield Boilerplate Scaffold
Cloverfield aims to create a next generation JavaScript project boilerplate. That means we'll use the tools that coders in-the-know will be using over the next 1 - 3 years. We're starting with the 2016 edition. Read more.
The JS community has been splintering when it comes to task runner consensus. For a while, Grunt was the clear winner, but that isn't the case anymore. We need to make a decision on one of these options, because many of our generators are going to produce some sort of task runner config. Please do some research and weigh in by answering the questions below.
Gulp and Brocolli have entered the community radar, and there is a growing trend to lean more heavily on Unix pipes instead of Node streams or the heavy file i/o typical of Grunt setups.
And the long-time reigning champion that's been a part of Unix since the time of the ancient grey beards is starting to make inroads. See Building JavaScript Projects with Make.
Oh, and npm has a built-in task runner that lets you easily leverage Unix pipes. Substack thinks that's the way to go, and he's not alone.
This is a pretty confusing landscape for those who haven't investigated all of these possible solutions, so lets try to lay out the pros and cons of each.
Questions
Greener fields
The text was updated successfully, but these errors were encountered: