Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In-place minification #638

Open
StoneCypher opened this issue Feb 2, 2020 · 3 comments
Open

In-place minification #638

StoneCypher opened this issue Feb 2, 2020 · 3 comments

Comments

@StoneCypher
Copy link

StoneCypher commented Feb 2, 2020

Issue type

  • Feature Request:
    • flag "dense output" that turns off the readable whitespace



Prerequisites

  • Can you reproduce the issue?:
    • yes
  • Did you search the repository issues?:
  • Did you check the forums?:
  • Did you perform a web search (google, yahoo, etc)?:
    • no need; i know what's wrong



Description

PEG parsers sometimes get really large because they're indented for human readability. I would like to add an option to change that.

My PEG grammar is about 34k. The parser it produces is about 1.7 meg. If you run that through Uglify3, it's about 101k, or a 94.2% reduction.

Thing is, I tried turning off symbol scrambling and relabelling, and it still dropped 93% in size. Almost all of that was just removing spaces

I can't find a minifier that can do this in under 2 minutes. Uglify takes about 2:40 on GH actions. My builds are [node 12,13][win,mac,linux][ff,ie,ch].

That means every time I commit, I spend about 45 minutes (spend because they cost money) on just minification.

I want to contribute a patch that fixes this. I don't know whether to work against 0.10 or 0.11

The preview pane from my editor is pretty amusing, as the code gets so deep (wasted spaces) that it entirely exits the preview panel for several hundred lines

image

The thing is, if you switch the thing from "for speed" to "for size," it stops wasting all that space, but then it also compiles a pretty slow parser

I'd like the fast parser's structure without the million wasted spaces (it's actually about 1.67 million wasted spaces in my current build)



Steps to Reproduce

  1. Compile a mid-sized grammar, such as this one
  2. Look at the output



Expected behavior:

  1. I expect to be able to turn off the 94% whitespace without having to create a slower parser
@StoneCypher
Copy link
Author

I have a patch ready for this, but it's buggy.

I went to contribute it, only to find out that the 0.11 I wrote it against is being thrown away, along with all of peg.js, in favor of some other parser written in a different language

@Seb35
Copy link

Seb35 commented Apr 1, 2020

I tested a very simple thing: apply .replace( /^ */gm, "" ) on the generated JS code and then minify it with terser. My PEG.js grammar is 46K, the generated JS code is 1.3M or 452K when removing initial spaces on each line (409K when removing all spaces) and 148K when minified (*); minifying the 1.3M file took about 23.3 seconds (sample = 5 runs from 22.9 s to 24.3 s) and minifying the 452K file took about 24.02 s (5 runs from 23.4 s to 24.9 s) (probably IO or other stuff could explain the slightly varying time).

So you can try such a simple instruction on your grammar, but I’m afraid that removing spaces will not change significantly the minification time. Depending on your goals, you can almost instantly divide by 3 the size, simply by removing initial spaces. Else given your figure of 2 minutes for uglify for a grammar of size similar to mine, terser seems to do better, you should try on your grammar.

(*) I remarked a few unused rules are completely dropped after minification, that’s fine :)

@StoneCypher
Copy link
Author

the problem is that'll interfere with things that actually have multiple spaces in their rules

my grammar is 34k and produces a 3.9 meg parser 🤣

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants