Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Closure Compiler with offline template compiler #8550

Closed
6 tasks done
alexeagle opened this issue May 9, 2016 · 105 comments
Closed
6 tasks done

Use Closure Compiler with offline template compiler #8550

alexeagle opened this issue May 9, 2016 · 105 comments
Assignees
Labels
area: packaging Issues related to Angular's creation of npm packages feature Issue that requests a new feature freq2: medium

Comments

@alexeagle
Copy link
Contributor

alexeagle commented May 9, 2016

Angular's new offline compiler was demo'ed at ng-conf day 2 keynote by @robwormald. It generates tree-shakable ES6 code. Four steps are then required:

  • run a tree-shaker to remove ES6 modules not reachable from the entry point, in our demo we showed rollup
  • run a bundler to reduce down the module imports into a declarations in a single file; we showed system.js
  • down-level to ES5 - we showed this with TypeScript
  • minify to shorten symbol names, we used uglify.

This produced a 49k JS file, but requires a lot of configuration.

Google's Closure Compiler (https://developers.google.com/closure/compiler/) produces very small JS. It does all four steps required above, so the configuration should be a lot simpler. We also suspect we can get a smaller binary size for ng2-hello-world, around 36k.

Wiring this up requires:

  • add tsickle's closure annotation helper to `ngc
  • modify Angular's ES6 distribution to be closure-compiler compatible
  • ?? modify ES6 distro of our dependencies (rxjs, zone.js) to be closure-compiler compatible
  • figure out a minimal build (maybe a shell script) that successfully compiles the application together with the framework and its dependencies
  • document how we did it so others can repro
  • bundle externs with Angular to allow Protractor testing (the BrowserNodeGlobal type)
@evmar
Copy link
Contributor

evmar commented May 9, 2016

The part I think I most need your guidance on is where/how this fits into the larger angular build process.
Do we want to (1) build angular itself as a minified external bundle? or (2) compile angular+user code together into a single bundle? In the latter case we'll need to integrate into gulp or whatever.

@alexeagle
Copy link
Contributor Author

We do distribute Angular bundles, as ES5 with UMD module loader built-in.
However, it would be impossible to pick this back apart again to tree-shake
it, so our plan has been #2.
I'm not sure we should commit to building plugins for the several popular
userland build tools; others in the community will pick that up. I think a
simpler "npm run build && npm run closure-compiler" with shell scripts is
sufficient at this point.

On Mon, May 9, 2016 at 11:26 AM Evan Martin notifications@github.com
wrote:

The part I think I most need your guidance on is where/how this fits into
the larger angular build process.
Do we want to (1) build angular itself as a minified external bundle? or
(2) compile angular+user code together into a single bundle? In the latter
case we'll need to integrate into gulp or whatever.


You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#8550 (comment)

@alexeagle
Copy link
Contributor Author

FYI @jeffbcross maybe this can make your PWA demo for I/O

@jeffbcross
Copy link
Contributor

Yeah that would be awesome if we could have this working by the end of this week.

@alexeagle
Copy link
Contributor Author

alexeagle commented May 12, 2016

Some notes from the hacks I've had to make so far, in the hopes that others can reproduce:

Closure Compiler doesn't handle all ES6 module syntax, and discourages using it. So our strategy so far has been to use goog.module syntax. That's not a --module option to tsc, so we use tsickle to re-write it. That means angular and its dependencies have to be emitted as "Closure ES6".

Need to build closure compiler from HEAD, see https://github.com/google/closure-compiler/blob/master/README.md#building-it-yourself

To build angular into closure ES6, I have local edits in the ngc tool to run tsickle's closure annotation and convertCommonJsToGoogModule. See https://github.com/alexeagle/angular/tree/closure_hack2
Build angular the usual way with ./build.sh and then make the packages installable with for pkg in $(find dist/packages-dist -name package.json); do sed -i .bak 's/\$\$ANGULAR_VERSION\$\$/2.0.0-rc.2-snap/g' $pkg; done

To build rxjs into closure ES6, I have local edits in https://github.com/alexeagle/RxJS/tree/closure - then run npm run build_es6 to get the right files in dist/es6. Then npm run generate_packages to copy in a package.json.

We need a modification to tsickle to workaround ReactiveX/rxjs#1703 - see https://github.com/angular/tsickle/tree/closure

Now I make an example app. Snapshot at https://github.com/alexeagle/closure-compiler-angular-bundling
In the vendor directory I've already installed the locally built closure compiler.jar, angular2 packages, rxjs, and tsickle, with something like
npm install ../angular/dist/packages-dist/{common,compiler_cli,compiler,core,platform-browser,platform-server}
npm install ../rxjs/dist/es6

So you can just npm install and npm run build to reproduce the closure bundle :)

@dpsthree
Copy link
Contributor

So I would need to have a Java JRE installed to bundle a hello world Angular2 app?

@alexeagle
Copy link
Contributor Author

Closure Compiler is just one option for the
tree-shake/bundle/Es6-to-5/minify pipeline. If you choose the closure
compiler option, you take a JRE dependency for now. But there is also a JS
version in the works:
https://github.com/google/closure-compiler/blob/master/src/com/google/javascript/jscomp/gwt/client/GwtRunner.java

On Fri, May 13, 2016 at 9:44 AM dpsthree notifications@github.com wrote:

So I would need to have a Java JRE installed to bundle a hello world
Angular2 app?


You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#8550 (comment)

@alexeagle
Copy link
Contributor Author

I wrote some notes on the hacks required to make it work. https://docs.google.com/document/d/17m1GwzXraKgbCkCmH1JnY9IZzPy4cqlpCFVhvlZnOys/edit

@alexeagle alexeagle modified the milestones: 2.1.0, Angular 2 Final Jul 18, 2016
@jjudd
Copy link

jjudd commented Jul 20, 2016

We reproduced @alexeagle's work and with a more up to date version of Angular. The entire build process is automatic, so you can follow everything from source to end result. You can see it in action at https://github.com/lucidsoftware/closure-typescript-example. Instructions for running the example can be found in the README for the example repo. There is a Docker container for the project, so it should be possible for pretty much anyone to try it out.

The above example utilizes Angular 2, clutz, and tsickle. Closure JS is used in Angular 2 TS, which is compiled to Closure compatible JS.

The changes we made to the dependencies to get this to work are largely the same as Alex's. There are a few changes we added and a few changes that are no longer necessary. Regardless, the basic idea is the same. We use a modified ngc or tsickle on Angular 2 and its dependencies to produce Closure compatible versions of those dependencies.

Closure Compiler

Building from head is no longer necessary. Just use one of the versions from June or July.

Angular

https://github.com/lucidsoftware/angular/tree/closure-bundle

The bulk of the work in the Angular 2 source is modifying ngc to produce Closure compilable js by adding a few tsickle steps to tsc-wrapped. There are a few hoops that we jump through in order to use the modified source as input to the TypeScript compiler in one of the tsickle passes. Otherwise, this is pretty straight forward.

We also change certain Angular modules to be compiled to Closure compatible JS by putting an angularCompilerOptions block in the tsconfig.json like so

"angularCompilerOptions": {
  "googleClosureOutput": true
}

Then when you run build.sh those modules have the output you want.

A test utillity file that uses selenium-webdriver is modified because selenium-webdriver is missing and we didn't want to make it closure compatible.

RxJS

https://github.com/lucidsoftware/rxjs/tree/closure-hack

The build process of RxJS is modified to build with the Makefile from our project. About half the build process is in JS and the other half in Make. It's pretty janky.

The only other changes besides the build target are a few small things to make RxJS compile with TypeScript and Closure.

symbol-observable (RxJS dependency)

https://github.com/lucidsoftware/symbol-observable/tree/closure

RxJS now depends on a some code that used to be part of the RxJS source and is now its own module. We modify that module (which is JS) to be compatible with the Closure compiler. There are only two files, so I did this manually.

tsickle

https://github.com/lucidsoftware/tsickle/tree/ignore-type-comments

We modify Tsickle in two ways:

  1. Add a --ignoreTypesInComments option. This prevents tsickle from throwing an error when it encounters jsdoc in comments. This is needed to compile RxJS which has a bunch of jsdoc comments.
  2. Modify the pathToModuleName for the CLI to be the same as the pathToModuleName we use in ngc. That way modules are named the same when run through tsickle or ngc. This should be committed upstream because the current pathToModuleName sometimes produces invalid goog.module names.

@JamesHenry
Copy link
Contributor

  • run a tree-shaker to remove ES6 modules not reachable from the entry point, in our demo we showed rollup
  • run a bundler to reduce down the module imports into a declarations in a single file; we showed system.js
  • down-level to ES5 - we showed this with TypeScript
  • minify to shorten symbol names, we used uglify.

Whilst it perhaps cannot match the final output size of the Closure Compiler, webpack 2 (using typescript loader and standard optimize plugins) gets you all of the these steps.

I would argue that the benefit of devs already using (dare I say requiring) a tool like webpack in their stack, and therefore not having to add more (hacked) dependencies, is more important than shaving off extra bytes in the final payload.

Would love to hear your thoughts!

@alexeagle
Copy link
Contributor Author

@jjudd thank you so much for posting your repo and these notes, this is really helpful.

Do you have a minified app size you can share? Perhaps using Brotli compression so we have an apples-to-apples comparison with the hello world closure-ng2 example (which was 26.2k)

@JamesHenry we agree that webpack 2 is the way forward for most developers. Closure compiler is an expert tool and I doubt we can wrap it up in a package that 'just works' for developers who aren't already familiar with debugging the obscure ways its ADVANCED_OPTIMIZATIONS can bust your app.

@jjudd
Copy link

jjudd commented Jul 21, 2016

@alexeagle We're currently working on getting this to work with ADVANCED_OPTIMIZATIONS. When we get that working I'll go ahead and post some benchmarks.

@evmar
Copy link
Contributor

evmar commented Jul 22, 2016

Converted the tsickle issue 1 above into a bug report there.

@robwormald
Copy link
Contributor

closure compiler is now available on npm in pure javascript : https://www.npmjs.com/package/google-closure-compiler-js

@jjudd
Copy link

jjudd commented Aug 18, 2016

Wanted to provide an update:

We have the example repo and the beta Lucidchart editor working with advanced optimizations. We ran a short test using our version with simple optimizations, and our version using advanced optimizations is going out today or tomorrow.

The version of Angular used in the example repo has been upgraded to RC5 (HEAD as of Tuesday Aug 16).

The example repo and its accompanying Docker container have been updated in case anyone wants to try it out. Here's a link to the example repo https://github.com/lucidsoftware/closure-typescript-example

Bundle sizes

The final bundle size for the example project using advanced compilation are listed below. Note: the example repo includes code going from js -> ts using clutz and then ts -> js using tsickle. It is a bit bigger than just a simple hello world app.

Uncompressed: 112K
Brotli quality 10: 29K
Gzip: 34K

Notes on Getting This to Work

Here are the notes on what it took to get advanced optimizations working.In case I forgot something, I'm going to provide the links to all our forks, which have all of our commits.

Example repo

Externs files for Zone, Reflect, and Jasmine were included in the build process, so those function calls were not renamed.

To work around a Closure Compiler bug where static methods get lost, we have a really hacky sed command that we run on the built Angular js files, replacing all lines which include a static keyword with the line preceded by /** @nocollapse */. Here's a link to the Closure bug: google/closure-compiler#1776

Angular

The biggest change we made to Angular is making it use the Tsickle annotated output during compilation. We thought we did this last time, but we had a bug where the annotated output was being thrown away :D

There were a few places that Angular refers to functions as strings. These functions were referred to in other places using dot notation. Accordingly, some references were renamed and others were not. We change Angular to use dot notation everywhere for those functions, so that they are always renamed.

We also updated Angular/Tsickle to not annotate .d.ts files.

We made Angular play nicely with ES6, so it works in uncompiled mode. We change some places where .apply was used on a constructor to use the new keyword.

Tsickle

We fixed a bug in tsickle where the CLI did not always include all the source files during annotation. The list of filenames from the TypeScript Program should have been used, instead we were using a different list of filenames.

Links to all our forks of projects to make this work

Angular: https://github.com/lucidsoftware/angular/tree/closure-bundle
Tsickle: https://github.com/lucidsoftware/tsickle/tree/closure-bundle
RxJS: https://github.com/lucidsoftware/rxjs/tree/closure-hack
symbol-observable: https://github.com/lucidsoftware/symbol-observable/tree/closure
Clutz: https://github.com/lucidsoftware/clutz

@alexeagle
Copy link
Contributor Author

So we have a couple examples of this working, plus we use JSCompiler internally at Google so we have many apps doing this (though using the Bazel build system and some not-yet-released Bazel rules).

@robwormald @mprobst should we try to package this up in a more easily reproducible way for external users? I don't think we can claim success and close this issue yet.

@thelgevold
Copy link
Contributor

thelgevold commented Sep 20, 2016

The JS closure compiler looks really cool. However, I have seen issues where the process runs out of memory when targeting medium sized AOT projects via Rollup. Have you encountered this in your testing?

The Java version does not seem to suffer from this problem though.

google/closure-compiler-js#23

@alexeagle
Copy link
Contributor Author

I've heard that too. We have had to call node with
--max-old-space-size=4096 to get some tools to work in the past.

On Tue, Sep 20, 2016 at 11:27 AM Torgeir Helgevold notifications@github.com
wrote:

The JS closure compiler looks really cool. However, I have seen issues
where the process runs out of memory with targeting medium sized AOT
projects. Have you encountered this in your testing?

The Java version does not seem to suffer from this problem though.

google/closure-compiler-js#23
google/closure-compiler-js#23


You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub
#8550 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAC5I_FDlMdHvHO2YCvypXju9wR8onzuks5qsCWmgaJpZM4IaZIi
.

@thelgevold
Copy link
Contributor

I tried max-old-space-size with 8000 and 14000 on a Mac with 16GB of ram, but I still ran out of memory in my sample project.
Java version works though.

@alexeagle
Copy link
Contributor Author

oh, that seems bad :)
could you file a bug on https://github.com/google/closure-compiler/issues

On Tue, Sep 20, 2016 at 11:52 AM Torgeir Helgevold notifications@github.com
wrote:

I tried max-old-space-size with 8000 and 14000 on a Mac with 16GB of ram,
but I still ran out of memory in my sample project.
Java version works though.


You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
#8550 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAC5I25uOXJ-Pt8a9WzQQUO1zRx4j1YUks5qsCthgaJpZM4IaZIi
.

@thelgevold
Copy link
Contributor

Yup I have an active issue there already
google/closure-compiler-js#23

@kylecordes
Copy link

kylecordes commented Mar 16, 2017

@nosachamos Wild guess: something in the toolchain (probably tsickle or Closure?) did not anticipate a parameter named this. My suggestion is:

  • Edit your local copy of RxJS source, change that parameter name, confirm fixed. If so then...
  • Put in an issue (or even a PR) to RxJS to rename this this and any similar this parameters. Hopefully the RxJS team will see the merit in such a change;possibly a good word from the Angular team ( @alexeagle ?) on such an issue will help.
  • Put in an issue with a simple repro case to Closure, about the parameter named this... and it doesn't repro there, look elsewhere in the tool chain (tsickle?).

@nosachamos
Copy link

nosachamos commented Mar 16, 2017

@kylecordes Ops, I found my issue and deleted the comment I guess as you were typing.

For the record, my issue was that an argument named "this" was being removed during transpilation.

It turns out, closure was invoking that function using call, so this was actually set that way and needed not be passed in. If I rename "this" to "this2", the argument is kept. I guess the transpiler is smarter than I thought.

My issue was within ngrx effects, not rxjs. Ngrx/effects accesses the effects through their names, using the brackets notation, which is broken by the closure compiler when it renames all the things. To prevent the issue, in the constructor of your classes containing your effects, assign their names to "this" using brackets notation:

export class MyEffects {

  @Effect()
  doSomething$: Observable<Action> = this.actions$ ...

  constructor() {
    this['doSomething$'] = this.doSomething$;
  }

}

So I'm almost up and running on a real world app using the ngrx stack. I'm writing what I believe to be the last manual extern required for my case. If this all runs well, I'll create a branch on the sample app here with the ngrx stack as a demo. I'm using my own fork of ngrx/core and ngrx/store while my PRs for expanding wildcard exports are not merged, but other than that, I think this will go well.

I'll keep you guys posted.

@evmar
Copy link
Contributor

evmar commented Mar 16, 2017

Can you file a bug against tsickle with more information on the this issue? Our intent is that you shouldn't need to manually rename variables like this.

Is it possible to fix ngrx to not use brackets?

@kylecordes
Copy link

@nosachamos based on your deleted comment, and my maybe-should-be deleted comment, is there any issue to report? (per @evmar )

@nosachamos
Copy link

@kylecordes Maybe,
but what I think it's happening is that the transpiler can figure out the function has "this" set by the call invocation. So it can in this case safely remove the argument, as this will be whatever is passed to call. That would not be true if the argument gets renamed, so it keeps the argument. I think in "this" particular case, the transpiler is fine. In fact, renaming the argument doesn't fix the problem, as the real issue was the bracket access.

@evmar I believe so. This bracket access is quite central to their operation but I suspect annotating the underlying dictionary with closure's @dict annotation would suffice to fix the issue. I will give this a try and submit a PR to them if that works. Else, I'll submit a PR to their README with a note on the bracket workaround when using ngrx/effects with closure.

@evmar
Copy link
Contributor

evmar commented Mar 16, 2017

Note that any Closure type annotation added to TS code is stripped by Tsickle, so adding an @dict there won't help.

@awerlang
Copy link
Contributor

My issue was within ngrx effects, not rxjs. Ngrx/effects accesses the effects through their names, using the brackets notation, which is broken by the closure compiler when it renames all the things. To prevent the issue, in the constructor of your classes containing your effects, assign their names to "this" using brackets notation:

@nosachamos Won't the same issue happen with @Input() and @Output()? Another question would be: should closure rename decorated fields?

@nosachamos
Copy link

nosachamos commented Mar 16, 2017

@evmar I'm transpiling with removeComments: false and all my closure annotations are passed onto the ES6 code I feed closure, so that should be fine. I'm using @dict in other places, and it's fine.

@awerlang no, the @dict is a jsdoc style annotation, not an actual annotation like @input. These closure annotations go on comments.

@evmar
Copy link
Contributor

evmar commented Mar 16, 2017

@nosachamos our test case is this input https://github.com/angular/tsickle/blob/master/test_files/jsdoc/jsdoc.ts producing this output
https://github.com/angular/tsickle/blob/master/test_files/jsdoc/jsdoc.js
where you'll see we munge most annotations. Maybe there's a bug though.

@nosachamos
Copy link

nosachamos commented Mar 16, 2017

@evmar oh, I'm compiling with ngc and using

    "target": "es6",
    "module": "es2015",

and

  "angularCompilerOptions": {
    "skipMetadataEmit": true,
    "annotationsAs": "static fields",
    "annotateForClosureCompiler": true
  },

So I get just ES6, no goog.modules.. so everything seems to be working as intended, and I get the closure annotations. This is for RxJs. For packages that interact with angular (components, modules, etc), then skipMetadataEmit is set to false.

@kylecordes
Copy link

@nosachamos as I understand, the tsickle step can't do it's thing, and won't do its thing when run via ngc, unless you are using "module": "common". Tsickle enforces this limitation when run by its own CLI, does it gain the ability to work with es2015 module output when run via ngc? Do you see tsickle-generated JSDoc in the es2015 module output?

@nosachamos
Copy link

@kylecordes yeah, I get the same exact output you get in the demo repo you posted above compiling with tsc-wrapped and the rxjs-tsickle tsconfig.

@alexeagle
Copy link
Contributor Author

could we move bugs to another issue? it doesn't scale to address individual problems on the tracking issue. Thanks!

@nosachamos
Copy link

@alexeagle absolutely. I posted here initially because I thought this was relevant (a problem in how rxjs was declaring arguments named "this" that were being lost). There's nothing to follow up, but if something comes up we'll do in a separate issue.

@kylecordes
Copy link

@nosachamos Indeed... I just found that indeed, tsickle-via-ngc or -cli-wrapped can handle module:es2015.

@alexeagle Ah, oops. At 90+ comments, I didn't realize it was a tracking issue. Will make new issues if they come up.

@thelgevold
Copy link
Contributor

thelgevold commented Mar 22, 2017

Took another look at the generated bundle. Noticed that there is definitely room for further code reductions here.

It's not always easy for Closure to identify unnecessary code since the existing code structure sometimes makes it seem like the code is needed.

Based on the POC app in @alexeagle 's repo I picked a few unnecessarily included elements and made a few temporary tweaks to the source to force them out of the bundle.

Seems like there are a few repeating patterns here:

A service class may not be needed by the application, but if the service is added to a provider array along the way, it may cause the class to make it into the bundle. Examples of this are AnimationDriver and AnimationQueue. By removing the provider references to these classes I am able to get these classes out of the bundle.

The sample app in Alex's repo does not do any QueryList querying, but a lot of the Query related code is still in the bundle.
In this case Closure can't exclude the code because the code lives in a switch statement where multiple scenarios are handled.
Take a look at createViewNodes in core.js and the NodeType.Query clause in particular.

case NodeType.Query:
       nodeData = (createQuery());
       break;

The conditions in this method are resolved at runtime , so Closure has to include all the code to satisfy all the conditions in the switch. I removed this particular clause and the code was excluded from the bundle.

These are just two simple examples, but if you continue down this path, it will eventfully add up to noticeable reductions in size. I didn't go very far with this sample, but I probably removed roughly 1k by addressing 4-5 cases like this.

I also did a second experiment where I removed a bunch of code manually (without using Closure). I got my app down to 16.6k and I believe there is still room for removing more. Here is the sample: http://www.syntaxsuccess.com/viewarticle/minimal-angular-application . This particular experiment is a major hack, but at least it proves that you can stand up an Angular app with very little code.

To help with some of these challenges I was thinking it would perhaps make sense to extend AoT to generate more of the framework code - not just the view code.

Just thinking out loud here, but couldn't the ngc compiler in theory generate code for stuff like createViewNodes based on the needs of specific applications. That way if your app doesn't need Pipes, QueryList or whatever, those particular case statements would be omitted from the generated code.

Thoughts?

@thelgevold
Copy link
Contributor

thelgevold commented Mar 26, 2017

Based on what I learned from manually removing code in the bundles, I have applied similar modifications to the bundles in the closure fork.

I put the tweaks in a branch in case you are interested in seeing the results: https://github.com/thelgevold/closure-compiler-angular-bundling-old/tree/tweaking-src

Here is a diff of the original bundles vs the tweaked bundles: thelgevold/closure-compiler-angular-bundling-old@d1b3954

If you want to try it, just replace the default bundles with the modified ones. I am assuming Angular 4.0.0 for this experiment.

Here are the numbers with the new bundles:

The new size is now ~15k gzipped, and slightly smaller with brotli.

++ ls -alH dist/bundle.js dist/bundle.js.brotli dist/bundle.js.gz dist/bundle.js.map
-rw-r--r--  1 tor  staff   47487 Mar 26 13:58 dist/bundle.js
-rw-r--r--  1 tor  staff   13979 Mar 26 13:58 dist/bundle.js.brotli
-rw-r--r--  1 tor  staff   15482 Mar 26 13:58 dist/bundle.js.gz
-rw-r--r--  1 tor  staff  133888 Mar 26 13:58 dist/bundle.js.map
++ for script in dist/bundle.js node_modules/zone.js/dist/zone.min.js
++ gzip --keep -f node_modules/zone.js/dist/zone.min.js
++ bro --force --quality 10 --input node_modules/zone.js/dist/zone.min.js --output node_modules/zone.js/dist/zone.min.js.brotli
++ ls -alH node_modules/zone.js/dist/zone.min.js node_modules/zone.js/dist/zone.min.js.brotli node_modules/zone.js/dist/zone.min.js.gz
-rw-r--r--  1 tor  staff  29634 Mar 25 11:00 node_modules/zone.js/dist/zone.min.js
-rw-------  1 tor  staff   8759 Mar 26 13:58 node_modules/zone.js/dist/zone.min.js.brotli
-rw-r--r--  1 tor  staff   9516 Mar 25 11:00 node_modules/zone.js/dist/zone.min.js.gz

Obviously it's hard to replicate this without restructuring the Angular source, but it's at least interesting to see how small a basic app can get.

Not sure how useful this is, but it's at least a fun experiment :-)

In summary:
A lot of the savings are from removing code from switch/if-else structures. Closure can't currently infer that some of the options are not possible at runtime for a given application. Another case is provider arrays populated with unused services.

@kylecordes
Copy link

As I understand from @thelgevold 's work, there is considerable output size reduction, with relatively minor-to-moderate adjustment to Angular:

  • Accommodate fewer features with if and switch in the source code.
  • Pre-configured access again to fewer of those features, in the provider arrays built in.
  • Adjust the compiler (affecting both JIT and AOT) to generate the additional bits of code to wire up those features/providers in the generated code... if the features are actually used.

I'm interested in what someone on the core team thinks of this, whether it really is minor-to-moderate and whether there is interest in going down this path.

@alexeagle
Copy link
Contributor Author

Today we got a release of rxjs that works with closure compiler, and I've updated the example repo
https://github.com/alexeagle/closure-compiler-angular-bundling

The externs are cleaned up as well - Angular and Zone.js both distribute the needed externs in their respective packages.
We'll add some documentation and an announcement about the support shortly.

@buu700
Copy link

buu700 commented May 6, 2017

Now that all of this is out of the way, at this point do you have an idea of how it will end up integrating with the rest of the existing tooling?

e.g. might there eventually be an option added to @ngtools/webpack to automatically run the AOT output through Closure (and/or some documented configuration steps for webpack-closure-compiler)? Or is this something that will necessarily have to supersede webpack?

@vforv
Copy link

vforv commented May 21, 2017

Is possible to implement ngx-bootstrap with Closure?

https://github.com/valor-software/ngx-bootstrap

@angryagr
Copy link

Wanted to provide an update:

We have the example repo and the beta Lucidchart editor working with advanced optimizations. We ran a short test using our version with simple optimizations, and our version using advanced optimizations is going out today or tomorrow.

The version of Angular used in the example repo has been upgraded to RC5 (HEAD as of Tuesday Aug 16).

The example repo and its accompanying Docker container have been updated in case anyone wants to try it out. Here's a link to the example repo https://github.com/lucidsoftware/closure-typescript-example

Bundle sizes

The final bundle size for the example project using advanced compilation are listed below. Note: the example repo includes code going from js -> ts using clutz and then ts -> js using tsickle. It is a bit bigger than just a simple hello world app.

Uncompressed: 112K
Brotli quality 10: 29K
Gzip: 34K

Notes on Getting This to Work

Here are the notes on what it took to get advanced optimizations working.In case I forgot something, I'm going to provide the links to all our forks, which have all of our commits.

Example repo

Externs files for Zone, Reflect, and Jasmine were included in the build process, so those function calls were not renamed.

To work around a Closure Compiler bug where static methods get lost, we have a really hacky sed command that we run on the built Angular js files, replacing all lines which include a static keyword with the line preceded by /** @nocollapse */. Here's a link to the Closure bug: google/closure-compiler#1776

Angular

The biggest change we made to Angular is making it use the Tsickle annotated output during compilation. We thought we did this last time, but we had a bug where the annotated output was being thrown away :D

There were a few places that Angular refers to functions as strings. These functions were referred to in other places using dot notation. Accordingly, some references were renamed and others were not. We change Angular to use dot notation everywhere for those functions, so that they are always renamed.

We also updated Angular/Tsickle to not annotate .d.ts files.

We made Angular play nicely with ES6, so it works in uncompiled mode. We change some places where .apply was used on a constructor to use the new keyword.

Tsickle

We fixed a bug in tsickle where the CLI did not always include all the source files during annotation. The list of filenames from the TypeScript Program should have been used, instead we were using a different list of filenames.

Links to all our forks of projects to make this work

Angular: https://github.com/lucidsoftware/angular/tree/closure-bundle
Tsickle: https://github.com/lucidsoftware/tsickle/tree/closure-bundle
RxJS: https://github.com/lucidsoftware/rxjs/tree/closure-hack
symbol-observable: https://github.com/lucidsoftware/symbol-observable/tree/closure
Clutz: https://github.com/lucidsoftware/clutz

We reproduced @alexeagle's work and with a more up to date version of Angular. The entire build process is automatic, so you can follow everything from source to end result. You can see it in action at https://github.com/lucidsoftware/closure-typescript-example. Instructions for running the example can be found in the README for the example repo. There is a Docker container for the project, so it should be possible for pretty much anyone to try it out.

The above example utilizes Angular 2, clutz, and tsickle. Closure JS is used in Angular 2 TS, which is compiled to Closure compatible JS.

The changes we made to the dependencies to get this to work are largely the same as Alex's. There are a few changes we added and a few changes that are no longer necessary. Regardless, the basic idea is the same. We use a modified ngc or tsickle on Angular 2 and its dependencies to produce Closure compatible versions of those dependencies.

Closure Compiler

Building from head is no longer necessary. Just use one of the versions from June or July.

Angular

https://github.com/lucidsoftware/angular/tree/closure-bundle

The bulk of the work in the Angular 2 source is modifying ngc to produce Closure compilable js by adding a few tsickle steps to tsc-wrapped. There are a few hoops that we jump through in order to use the modified source as input to the TypeScript compiler in one of the tsickle passes. Otherwise, this is pretty straight forward.

We also change certain Angular modules to be compiled to Closure compatible JS by putting an angularCompilerOptions block in the tsconfig.json like so

"angularCompilerOptions": {
  "googleClosureOutput": true
}

Then when you run build.sh those modules have the output you want.

A test utillity file that uses selenium-webdriver is modified because selenium-webdriver is missing and we didn't want to make it closure compatible.

RxJS

https://github.com/lucidsoftware/rxjs/tree/closure-hack

The build process of RxJS is modified to build with the Makefile from our project. About half the build process is in JS and the other half in Make. It's pretty janky.

The only other changes besides the build target are a few small things to make RxJS compile with TypeScript and Closure.

symbol-observable (RxJS dependency)

https://github.com/lucidsoftware/symbol-observable/tree/closure

RxJS now depends on a some code that used to be part of the RxJS source and is now its own module. We modify that module (which is JS) to be compatible with the Closure compiler. There are only two files, so I did this manually.

tsickle

https://github.com/lucidsoftware/tsickle/tree/ignore-type-comments

We modify Tsickle in two ways:

  1. Add a --ignoreTypesInComments option. This prevents tsickle from throwing an error when it encounters jsdoc in comments. This is needed to compile RxJS which has a bunch of jsdoc comments.
  2. Modify the pathToModuleName for the CLI to be the same as the pathToModuleName we use in ngc. That way modules are named the same when run through tsickle or ngc. This should be committed upstream because the current pathToModuleName sometimes produces invalid goog.module names.

hi I installed your project but when do "make run" get Error :
'tools/@angular/tsc-wrapped/src/compiler_host.ts(55,32): error TS2345: Argument of type 'ts.Program' is not assignable to parameter of type 'ts.Program'.'
Could you help me?

@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Sep 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area: packaging Issues related to Angular's creation of npm packages feature Issue that requests a new feature freq2: medium
Projects
None yet
Development

No branches or pull requests