Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linking with -s MAXIMUM_MEMORY=4GB and -s MODULARIZE=1 is very slow. #21796

Open
msorvig opened this issue Apr 22, 2024 · 2 comments
Open

Linking with -s MAXIMUM_MEMORY=4GB and -s MODULARIZE=1 is very slow. #21796

msorvig opened this issue Apr 22, 2024 · 2 comments

Comments

@msorvig
Copy link

msorvig commented Apr 22, 2024

Normally, the emscripten linker runs in a couple of seconds. When setting the MAXIMUM_MEMORY option to a value > 2GB the link time increases to several minutes.

The time looks to be spent by the "acorn" optimizer, and is an issue for MODULARIZE=1 builds which generate a lot of JS runtime code.

Version of emscripten/emsdk:
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.50 (9fdfa557c0c86afc55588846fbc8b19bd7205f85)
clang version 18.0.0 (https://github.com/llvm/llvm-project 14028ec0a62210d68a4dd7a046ac79c8c3b7727e)
Target: wasm32-unknown-emscripten
Thread model: posix

Failing command line in full:
/em++ -s INITIAL_MEMORY=50MB -s MAXIMUM_MEMORY=4GB -s EXPORTED_RUNTIME_METHODS=UTF16ToString,stringToUTF16,JSEvents,specialHTMLTargets,FS,callMain -s EXPORT_NAME=qtquickruntime_entry -s MAX_WEBGL_VERSION=2 -s FETCH=1 -s WASM_BIGINT=1 -s STACK_SIZE=5MB -s MODULARIZE=1 -s ALLOW_MEMORY_GROWTH -lembind [+ several .so files]

Full link command and output with -v appended:
(EMPROFILE=2 output)
profiler:INFO: start block "main"
profiler:INFO: start block "parse arguments"
profiler:INFO: block "parse arguments" took 0.001 seconds
profiler:INFO: start block "check_sanity"
profiler:INFO: block "check_sanity" took 0.000 seconds
profiler:INFO: start block "setup"
profiler:INFO: block "setup" took 0.003 seconds
profiler:INFO: start block "compile inputs"
profiler:INFO: start block "ensure_sysroot"
profiler:INFO: block "ensure_sysroot" took 0.000 seconds
profiler:INFO: block "compile inputs" took 0.001 seconds
profiler:INFO: start block "linker_setup"
profiler:INFO: block "linker_setup" took 0.068 seconds
profiler:INFO: start block "calculate linker inputs"
profiler:INFO: block "calculate linker inputs" took 0.683 seconds
profiler:INFO: start block "JS symbol generation"
profiler:INFO: block "JS symbol generation" took 0.089 seconds
profiler:INFO: start block "calculate system libraries"
profiler:INFO: block "calculate system libraries" took 0.001 seconds
profiler:INFO: start block "link"
profiler:INFO: block "link" took 0.503 seconds
profiler:INFO: start block "post link"
profiler:INFO: start block "emscript"
profiler:INFO: start block "get_metadata"
profiler:INFO: block "get_metadata" took 0.090 seconds
profiler:INFO: block "emscript" took 1.315 seconds
profiler:INFO: start block "binaryen"
profiler:INFO: start block "use_unsigned_pointers_in_js"
profiler:INFO: block "use_unsigned_pointers_in_js" took 199.120 seconds
profiler:INFO: block "binaryen" took 199.120 seconds
profiler:INFO: start block "final emitting"
profiler:INFO: block "final emitting" took 0.063 seconds
profiler:INFO: block "post link" took 200.499 seconds
profiler:INFO: block "main" took 201.853 seconds

@kripken
Copy link
Member

kripken commented Apr 23, 2024

Strange, use_unsigned_pointers_in_js is taking 200 seconds for you. I'm not sure why. When I test e.g.

EMPROFILE=2 ./emcc test/hello_libcxx.cpp -s MODULARIZE -sMAXIMUM_MEMORY=4GB -O3 -sLINKABLE -sALLOW_MEMORY_GROWTH

then I get 0.645 seconds on that phase. This emits 270 KB JS and 1.5 MB wasm so it is not tiny, but maybe yours is much larger or is different in some other way?

I also can't see any quadratic behavior or other possible issue in the pass code itself.

@msorvig
Copy link
Author

msorvig commented May 2, 2024

The .js file is 14MB in my case. The main part of it seems to be exported symbols from the linked to .so libraries, so I don’t think it can be reproduced with a small/standalone main.cpp. For example:

  function __ZN23QQuickFramebufferObject16releaseResourcesEv(
  ) {
  if (!wasmImports['_ZN23QQuickFramebufferObject16releaseResourcesEv'] || wasmImports['_ZN23QQuickFramebufferObject16releaseResourcesEv'].stub) abort("external symbol '_ZN23QQuickFramebufferObject16releaseResourcesEv' is missing. perhaps a side module was not linked in? if this function was expected to arrive from a system library, try to build the MAIN_MODULE with EMCC_FORCE_STDLIBS=1 in the environment");
  return wasmImports['_ZN23QQuickFramebufferObject16releaseResourcesEv'].apply(null, arguments);
  }
  __ZN23QQuickFramebufferObject16releaseResourcesEv.stub = true;

(the warning text is repeated for each function)

#21785 looks to be a relevant fix - I’ll update this issue after upgrading to a newer emsdk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants