Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve webpack configuration #9225

Merged
merged 19 commits into from May 2, 2024
Merged

Conversation

MaximeMulder
Copy link
Contributor

Brief summary of changes

Improve the webpack configuration to make building LORIS more memory and time efficient. The changes of this PR are described in this message.

Testing instructions

  1. Build LORIS using the new configuration (remove exisiting build files first)
  2. Run static analysis tools (js linter)

Links to related issues

@MaximeMulder
Copy link
Contributor Author

MaximeMulder commented Apr 26, 2024

Some Javascript lint tests are failing. While I get the error (although I disagree with it), I don't know why there is an error now but not in the previous configuration considering this is cut/pasted code.

@driusan
Copy link
Collaborator

driusan commented Apr 26, 2024

I merged this into your datetime-filter-element branch to test it and it worked, but there was an unexplained "undefined" printed to the console.

Compared to aces/main it seems to be a little faster.

This branch + datetime filter:

nix-shell:~/Code/Loris]$ time npm run compile

> loris@1.0.0 compile
> webpack --node-env=development

undefined

real	0m29.547s
user	0m49.412s
sys	0m4.070s

aces/main:

[nix-shell:~/Code/Loris]$ time npm run compile

> loris@1.0.0 compile
> webpack --node-env=development

webpack compiled successfully

real	0m34.214s
user	1m2.398s
sys	0m11.641s

@MaximeMulder
Copy link
Contributor Author

MaximeMulder commented Apr 26, 2024

I merged this into your datetime-filter-element branch to test it and it worked, but there was an unexplained "undefined" printed to the console.

Yup, this was a console.log statement I had forgotten to remove 😅 (I was working on it though), this is fixed in the last commit (although the PR is not usable yet, I still need to fix another bug).

@MaximeMulder
Copy link
Contributor Author

MaximeMulder commented Apr 26, 2024

(by @ridz1208 on the related issue)

@MaximeMulder on the project level there is usually a webpack-project.config.js file (see the CBIG repo for example) can you make sure to detail what would need to change in that file to remain functional ?

I have not tested this case yet but it is handled by the code and should be working.

I am having more troubles with the new config not picking up the custom configuration in the electrophysiology browser. I am not sure why as I use the same logic as the old code, so I guess this must come from ts-node or the update in the root tsconfig.json. Worst case I can just go back to vanilla JS, but it does feel kind of bad to go backwards because I did not manage to solve an isolated problem.

@MaximeMulder
Copy link
Contributor Author

MaximeMulder commented Apr 29, 2024

Okay, sooo despite this somewhat chaotic development, I think it is done and should be working !

In terms of functionality, everything should be the same as before and not break existing code. The only thing I have not tested yet (but which should work) is project-specific overrides, any tips on how to test that @ridz1208 ?

In terms of speed, the gain does not look that significant, but removing the cache: {type: 'filesystem'} (which instead puts the cache in memory) makes building LORIS go from ~2m45 to ~1m45 on my VM. Doing so causes no memory issue on my machine, which makes sense since this configuration is better optimized than our current one, but I'd like to get external opinions before committing this last change.

Also, I did not manage to cleanly integrate the electrophysiology_browser to the configuration, so I used a hack for this module. I don't think it is too problematic.

EDIT: Actually, the cache: {type: 'filesystem'} option is only used for 1 of the 20+ configuration objects of the current configuration, so I just removed it instead.

@MaximeMulder MaximeMulder marked this pull request as ready for review April 29, 2024 18:28
@MaximeMulder
Copy link
Contributor Author

Assigned @laemtl to test the project override !


// HACK: For some reason, the electrophysiology session view only compiles if
// it uses a separate (although possibly identical) configuration.
if (!target || target === 'electrophysiology_browser') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why we are checking for !target as well here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

target informs us of which single module we want to compile, but it can also be undefined if we want to compile all modules. We need to take into account both of these cases.

@driusan driusan merged commit a7d2728 into aces:main May 2, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Current webpack configuration leads to memory issues when building LORIS
3 participants