Bundling is too slow for development

[ashubham]

• Rollup Version: 1.1.2
• Operating System (or Browser): Mac OS
• Node Version: 10.11.0

How Do We Reproduce?

My Rollup config looks like this:

let plugins = [
    resolve({
        jsnext: true,
        browser: true
    }),
    commonjs(),
    includePaths({
        paths: ['app/.tmp/ts-out']
    }),
    json({
        compact: true
    })
];

const input = [
    'app/.tmp/ts-out/src/base/app.js'
];

export default [{
    input: input,
    treeshake: false,
    output: {
        dir: 'target/module',
        format: 'es',
        sourcemap: false
    },
    perf: true,
    plugins: plugins
}];

I run the rollup bundler using this command:

NODE_OPTIONS=--max_old_space_size=4096 ./node_modules/.bin/rollup -w -c app.rollup.config.js

Expected Behavior

It should be faster to build the bundle, like maybe <5s to be able to use it for development. This output is in the watch mode.

Actual Behavior

created target/module in 29.8s

BUILD: 18915ms, 957 MB / 2.62 GB

parse modules: 16553ms, 818 MB / 2.48 GB

• plugin 0 (node-resolve) - resolveId: 13ms, 297 kB / 2.24 GB
• plugin 0 (node-resolve) - resolveId (async): 113222ms, 3.76 GB / 2.48 GB
• plugin 1 (commonjs) - resolveId: 2ms, 3.42 kB / 2.48 GB
• plugin 1 (commonjs) - resolveId (async): 9ms, 504 kB / 2.48 GB
  load modules: 495723ms, 4.84 GB / 2.48 GB
• plugin 1 (commonjs) - load: 952ms, 478 kB / 2.48 GB
  generate ast: 867ms, 613 kB / 2.48 GB
  analyse ast: 8025ms, 718 MB / 2.48 GB
• plugin 1 (commonjs) - transform: 1ms, 2.06 kB / 1.76 GB
• plugin 1 (commonjs) - transform (async): 265ms, 1.22 MB / 1.76 GB
• plugin 1 (commonjs) - load (async): 2066ms, 6.05 MB / 2.47 GB
• plugin 3 (json) - transform: 1ms, 240 B / 1.77 GB

analyse dependency graph: 2048ms, 133 MB / 2.61 GB

mark included statements: 287ms, 1.74 MB / 2.61 GB

generate chunks: 23ms, 4.52 MB / 2.62 GB

GENERATE: 10654ms, -604 MB / 2.01 GB

render modules: 8269ms, 214 MB / 3.15 GB
render format: 125ms, 52.7 MB / 3.75 GB

Any help or pointers ? Do I need to do something more here ?

[TrySound]

Disable source maps in commonjs plugin to reduce generating time and memory usage.

commonjs({ sourceMap: false })

In the past output.indent also could take a lot. Not sure about now.

And still, rollup is primary production tool. It doesn't have fast and scalable development mode. In large projects it can be very slow for dev.

[ashubham]

@TrySound That does not seem to give any perf/memory boost. I tried setting output.indent: false too.

Is there a way to check if the cache is being properly used ? Because the timings seem to be similar on both the initial run and incremental build. Does it break when using code splitting ?

And still, rollup is primary production tool. It doesn't have fast and scalable development mode. In large projects it can be very slow for dev.

What is the ideal development workflow then ? Any pointers to the projects ? The starter project and guides seem to point to using rollup with the watch feature for dev.

[lukastaegert]

Note that there is no caching yet, even in watch mode, unless you explicitly pass a cache object: https://rollupjs.org/guide/en#cache

[ashubham]

@lukastaegert What about this from @Rich-Harris #1100 (comment)

Is this not correct anymore ?

[PepsRyuu]

Just throwing it out there, because it's designed for an app development workflow: https://github.com/PepsRyuu/nollup 👀

[ashubham]

I tried to manually use the Javascript API to use caching, I also marked one of the large dependencies as external to see if it reduces the build time.

• Observation 1: The initial build time reduced from ~30s to 13s. This leads me to the fact that presence of a huge module significantly increases build time. So need to workaround that.

Here is the time split:

{ '# BUILD': [ 13559.622425, 933477712, 951275504 ],
  '## parse modules': [ 12324.156036, 847256552, 865072848 ],
  '- plugin 0 (node-resolve) - resolveId': [ 443.580362, 40601368, 870454352 ],
  **'- plugin 0 (node-resolve) - resolveId (async)': [ 362087.5389020004, 8026905304, 870476224 ],**
  '- plugin 1 (commonjs) - resolveId': [ 70.59664999999995, 1119336, 869331152 ],
  '- plugin 1 (commonjs) - resolveId (async)': [ 36812.89819700007, 1238898800, 869113184 ],
  'load modules': [ 1438.9132919999988, 60619880, 870338912 ],
  '- plugin 1 (commonjs) - load': [ 74.07352999999999, 549176, 870335936 ],
  '- plugin 1 (commonjs) - transform': [ 92.32386399999992, 2442808, 870345848 ],
  '- plugin 1 (commonjs) - transform (async)': [ 2931.5796650000057, 187492576, 870366688 ],
  '- plugin 3 (json) - transform': [ 86.96251399999994, -4013536, 870369784 ],
  'generate ast': [ 1437.2192719999966, 107721512, 870391040 ],
  'analyse ast': [ 2392.0108089999994, 324457256, 870613744 ],
  '- plugin 1 (commonjs) - load (async)': [ 1032.5541279999995, 37037960, 855877640 ],
  '## analyse dependency graph': [ 988.112022, 79811872, 944885192 ],
  '## mark included statements': [ 218.101944, 1945520, 946831560 ],
  '## generate chunks': [ 27.069387, 4437080, 951269144 ] }

Note the huge node-resolve time.

• Observation 2: When re-running using the cache, the build time is still ~13s.

{ '# BUILD': [ 13236.415238, 640223992, 2376655424 ],
  '## parse modules': [ 11606.328256, 563715984, 2300151048 ],
  '- plugin 0 (node-resolve) - resolveId': [ 1.211242, 25432, 1823187560 ],
  '- plugin 0 (node-resolve) - resolveId (async)': [ 8907.691718, 474969384, 2296723128 ],
  '- plugin 1 (commonjs) - resolveId': [ 0.540151, 1720, 1737929488 ],
  '- plugin 1 (commonjs) - resolveId (async)': [ 8.136861, -15608096, 1722322112 ],
  'load modules': [ 333709.3811229999, 2759180680, 2298071472 ],
  '- plugin 1 (commonjs) - load': [ 659.3684709999998, 455232, 2298036696 ],
  'generate ast': [ 581.8961019999995, 135296, 2299194384 ],
  'analyse ast': [ 4410.857317999996, 516695184, 2300129264 ],
  '- plugin 1 (commonjs) - load (async)': [ 1168.739115, 5066024, 2296515784 ],
  '## analyse dependency graph': [ 1405.377913, 82839200, 2382990656 ],
  '## mark included statements': [ 181.395745, 1769672, 2384761024 ],
  '## generate chunks': [ 39.959361, -8108528, 2376652912 ] }

Now note the huge, load modules time. This puzzles me to why does the time increase in other phases.

Also, bundle.generate takes ~20s.

[ashubham]

FWIW, I tried usingParcel for development. It takes ~4s for incremental builds even with the big module included.

[syranide]

I'm interested in this too. I'm not seeing build-times this bad, but for comparison I'm getting incremental build times in Webpack to be about 200ms, in Rollup it's above 1000ms. I've disabled all the features I could find in Rollup, only sourcemaps should still be active (but they are in Webpack too). This is currently keeping me from switching to Rollup for development.

[viiiprock]

My setup used to bundle very fast but don't know where was it come from, maybe after update to 1.7 or 1.10 (automate by greenkeeper), it takes 5 minutes to bundle, very bad experience. I've spent time to migrate from webpack to rollup several months ago and now feel tired to think about another migration :(

[TrySound]

@viiiprock Show you config

[viiiprock]

Here is my config

import external from "rollup-plugin-peer-deps-external";
import typescript from "rollup-plugin-typescript2";
import commonjs from "rollup-plugin-commonjs";
import resolve from "rollup-plugin-node-resolve";
import alias from "rollup-plugin-alias";
import url from "rollup-plugin-url";
import svgr from "@svgr/rollup";
import babel from "rollup-plugin-babel";
import path from "path";
import pkg from "./package.json";

export default {
  input: "src/index.tsx",
  output: [
    {
      file: pkg.main,
      format: "cjs"
    },
    {
      file: pkg.module,
      format: "es"
    }
  ],
  cache: true,
  plugins: [
    external({
      includeDependencies: false
    }),
    url(),
    svgr(),
    resolve(),
    typescript({
      rollupCommonJSResolveHack: true,
      clean: true,
      declaration: true
    }),
    alias({
      "styled-components": path.resolve(__dirname, "node_modules", "styled-components")
    }),
    babel({
      plugins: ["babel-plugin-styled-components", "external-helpers"],
      exclude: "node_modules/**"
    }),
    commonjs()
  ]
};

[viiiprock]

Actually, you can find my config which was good before here
https://github.com/we-mak/w-design/blob/develop/rollup.config.js

[lukastaegert]

It would also be helpful if you could pin the issue down to an exact patch version.

[viiiprock]

I'm not sure but it was good at 1.7, currently I'm at 1.10.0

[PepsRyuu]

@viiiprock It's not a Rollup issue, it's an issue with rollup-plugin-typescript2. For some reason, when TypeScript was upgraded to 3.4, it got exponentially slower. I've confirmed this by downgrading TypeScript.

For cross reference, I found this issue: ezolenko/rollup-plugin-typescript2#148

[viiiprock]

Incredible, thanks @PepsRyuu

[stylejy]

@PepsRyuu You saved my day... Thanks soooo much...

[GrosSacASac]

Has anyone tried https://github.com/nathan/pax bundler ?
The problem is that it has a smaller pool of plugins so it cannot always replace rollup

[shellscape]

@GrosSacASac I'm not sure this is the proper place to discuss alternative bundlers. The context of this issue is @ashubham's issue. Alt bundler discussions are off-topic for this issue.

[shellscape]

Sorry folks, after the last snarky comment, we're going to lock this one. Maintainers may follow up with more information as it's available.

[shellscape]

It looks like the issue boils down to a problem with ezolenko/rollup-plugin-typescript2#148 and the author there is blocked. We'll open a new discussion internally to see if there's anything that can be done here. Closing for now.