[REQUEST]: Support large assemblies given a filter that pares them down to a small assembly

[KSSAB]

Is your feature request related to a problem? Please describe

If you write horrible enough code, it can generate a large asm file, even if the parts of the assembly that are not filtered out by default are quite small: https://godbolt.org/z/Kf3417drT : https://i.imgur.com/sO2WdZX.png . For better or worse, if the output is too large, it is just not displayed, even if it would be quite short after filters: https://godbolt.org/z/89WK9x6sE : https://i.imgur.com/EOdDUYs.png

Describe the solution you'd like

Perhaps an option could be added to cog menu 'apply default filters on server side' (assuming the reasoning for asm size limit is to avoid network congestion). If this option is ticked (I am suggesting unticked by default to match current behavior), then each of the filters that are ticked by default (I think all of them?) are applied prior to size being checked, and the filter menu could be grayed out. If the location of the size check is arbitary between the two, it could be moved to after filters are applied (in afterCompilation instead of doCompilaton?).

Describe alternatives you've considered

I could write better code instead of expecting godbolt to process a kajillion lines

Additional context

It seems to me the asm is initialized in runCompiler inside of doCompilation :

compiler-explorer/lib/base-compiler.ts

Line 2368 in 72caf09

const [

; for the return value checkOutputFileAndDoPostProcess is called :

compiler-explorer/lib/base-compiler.ts

Line 1779 in 72caf09

async checkOutputFileAndDoPostProcess(asmResult, outputFilename, filters: ParseFiltersAndOutputOptions) {

, in turn calling postProcess

compiler-explorer/lib/base-compiler.ts

Line 3317 in 72caf09

async postProcess(result, outputFilename: string, filters: ParseFiltersAndOutputOptions) {

; While the filters are passed to objdump, they are passed to objdumpAndParseAssembly for an external parser , but as far as I can tell are not used for the path where an external parser is null:

compiler-explorer/lib/base-compiler.ts

Line 604 in 72caf09

const args = objdumper.getDefaultArgs(

. The result I believe is the length check inside of postProcess :

                  if (result.asmSize >= maxSize) {
                      result.asm =
                          '<No output: generated assembly was too large' +
                          ` (${result.asmSize} > ${maxSize} bytes)>`;
                      return result;
                  }

Applies after demangle / intel / binary (cog options) but before library, debug, comments (filter options).

After doCompilation, afterCompilation is called:

compiler-explorer/lib/base-compiler.ts

Line 2953 in 72caf09

const [result, optOutput, stackUsageOutput] = await this.doCompilation(

; where there are a couple paths that may call processAsm :

compiler-explorer/lib/base-compiler.ts

Line 3092 in 72caf09

async processAsm(result, filters, options) {

which may call ultimately parser.processAsm

compiler-explorer/lib/parsers/asm-parser.ts

Line 488 in 72caf09

processAsm(asmResult: string, filters: ParseFiltersAndOutputOptions): ParsedAsmResult {

which seems to apply these filters

I don't know much about ts/web dev, what I am wondering is if both of these (doCompilation and afterCompilation) are applied server side, or if say afterCompilation is applied client side? I.e. is the full asm transferred over net regardless of whether filters are used, and therefore the size limit has to do with avoiding DOS/network congestion? Otherwise, I see the comment:

// Start the execution as soon as we can, but only await it at the end.

At the top of afterCompilation, so perhaps the reasoning is to ensure their is something to do while the executor is working asynchronously, i.e. if parsing took place wholly before the executor could start, it would mean afterCompilation is more likely to wait on the executor with nothing else to do?

[partouf]

This size limitation in this case has to do with the load and time you're applying onto the machine CE is running on. The assembly in this case is processed by a native javascript parser, which becomes unwieldy when you process large amounts of text.

I don't know how random the chosen limit is currently, but in this case it seems to be "only" 300kb off, but it's still 64MiB, and that's not nothing.

I personally don't really feel like increasing the limit, but in this case even if we did, most likely you'll encounter different limits on processing time. So you'd be spending a lot of time on something you'd never see back in your browser.

You would be better off switching on binary mode, either binary object or linked: https://godbolt.org/z/GW8r1Mxz4
The binary mode will be parsed by a C++ program that does not have an incoming size limit, because it can usually handle large amounts of data.