Reducing size of build

[okdistribute]

After the merge of branch that updates to iD 2 it seems like it's 50mb larger. Why?

[okdistribute]

asar and phantomjs? are they needed?

[okdistribute]

also is electron really 100MB? seems big.

[okdistribute]

can try this trick: electron/electron#2003 (comment)

[okdistribute]

from @gregor 😀

so the iD dist folder is 35Mb, most of that is the translations, but the imagery.json and presets also take up a lot. node_modules/id-mapeo is 45Mb, including a copy of dist, and I think the way electron-packager works with npm prune we are probably getting most of that 80Mb from iD. The asar format used by electron is not compressed (it’s like a tar) but it should be compressing most of those large json assets when we compress the exe. You can inspect the asar file in the packaged app with https://github.com/electron/asar or I think there are other tools for seeing folder sizes in the asar.
I know we had issues with npm prune in the past, because we were using some modules that listed some dependencies as both a regular dependency, and a devDependency, and unfortunatley in that case npm (used to?) remove those modules. That was a frustrating bug to work out…

[okdistribute]

OK I made it so the dist of id isn't copied, it didn't save that much space though as far as I can tell.

[okdistribute]

I took out modules to attempt to see which one(s) were the culprit..

• remove material-ui/icons: 138M (~0)
• remove id-mapeo: 129M (-9mb)
• remove bonjour: 138M (-0)
• remove mapeo-styles: 135M (-3mb)
• remove level and levelup: 138M (-0mb)
• remove hyperlog-*: 138M (-0mb)
• remove react-mapfilter: 130M (-8mb)
• remove pngjs: 138M (-0mb)
• remove lodash: 130M (-8mb)
• remove osm-p2p: 100M (~30mb)
  • On further inspection, I found that osm-p2p-db was containing the benchmark database data
• remove test/test-data: 110M (~20mb)

[gmaclennan]

I was looking into this with mapfilter-desktop.

Electron itself is about 33Mb (compressed in the installer). The rest comes from the node_modules folder being copied from the app. npm_prune is run I think to remove devDependencies, but maybe I turned this off at some point because one of our dependencies listed one of its dependencies as a devDependency, so npm prune was removing it. This was a really obscure bug that took a while to track down.

I reviewed the node_modules contents on mapfilter-desktop, and a big culprit is all the test folders, in particular hyperlog-sneakernet-sync which had 40Mb in test files. It looks like electron-builder excludes these and many other files by default: https://www.electron.build/configuration/contents#files (keep an eye on this because this could be a source of bugs in the future - accidentally excluding a required file from app packaging).

Although I have not seen examples of it as common practice, I think we should look at bundling the JS as part of the build process, and not including node_modules at all. I think this would also increase app startup time because all those require calls have an overhead. Easiest would be browserify, but we could also look at rollup which would give us the most efficient bundle I think. This would work for the render process, but for the main process we would not be able to bundle native deps. We could look at noderify and electron-link.

I can't see any reason why our bundled code would be much more than a few Mb, so we should be able to get the whole install package down to around 50Mb.

[okdistribute]

Build is significantly smaller now. Yay.

[hackergrrl]

yay!