Out of memory issue when Rendering HTML for a large site

[jorik12345]

Hi all,

I have large Gridsome site that pulls data from Wordpress (Wordpress Source plugin). At this moment it contains about 8500 pages.

We are planning a large update and that means that we will go to about 12000 pages in total. When I read all the Gridsome stories this should be doable. The problem I now have is that Node / the V8 enigine is running out of memory when creating all the Pages.

I have already tried to increase the memory but when 30GB still isn't enough I am pretty sure that the problem lies somewhere else.

"build-full": "export NODE_OPTIONS=\"--max-old-space-size=30000\" && gridsome build"

I also tried to downsize the JSON output files when creating the pages but that also did not work.

This is the error that I am getting:

Compile assets - 144.65s
Execute GraphQL (12059 queries) - 1087.69s

<--- Last few GCs --->

[32183:0x7fbac8b00000]  1228046 ms: Mark-sweep (reduce) 1424.8 (1509.5) -> 1424.7 (1465.5) MB, 2109.2 / 0.0 ms  (average mu = 0.404, current mu = 0.013) last resort GC in old space requested
[32183:0x7fbac8b00000]  1230881 ms: Mark-sweep (reduce) 1424.7 (1465.5) -> 1424.7 (1465.0) MB, 2834.2 / 0.0 ms  (average mu = 0.211, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x100b8aa36 node::Abort() [/usr/local/Cellar/node@16/16.18.1/bin/node]
 2: 0x100b8acb1 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 3: 0x100cbf4ae v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 4: 0x100cbf47f v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 5: 0x100de7e2b v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 6: 0x100def573 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 7: 0x100dcc314 v8::internal::Factory::CodeBuilder::AllocateCode(bool) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 8: 0x100dcbb6e v8::internal::Factory::CodeBuilder::BuildInternal(bool) [/usr/local/Cellar/node@16/16.18.1/bin/node]
 9: 0x100dcc792 v8::internal::Factory::CodeBuilder::Build() [/usr/local/Cellar/node@16/16.18.1/bin/node]
10: 0x101242eeb v8::internal::RegExpMacroAssemblerX64::GetCode(v8::internal::Handle<v8::internal::String>) [/usr/local/Cellar/node@16/16.18.1/bin/node]
11: 0x1010407c8 v8::internal::RegExpCompiler::Assemble(v8::internal::Isolate*, v8::internal::RegExpMacroAssembler*, v8::internal::RegExpNode*, int, v8::internal::Handle<v8::internal::String>) [/usr/local/Cellar/node@16/16.18.1/bin/node]
12: 0x10105465a v8::internal::RegExpImpl::Compile(v8::internal::Isolate*, v8::internal::Zone*, v8::internal::RegExpCompileData*, v8::base::Flags<v8::internal::JSRegExp::Flag, int>, v8::internal::Handle<v8::internal::String>, v8::internal::Handle<v8::internal::String>, bool, unsigned int&) [/usr/local/Cellar/node@16/16.18.1/bin/node]
13: 0x101053f05 v8::internal::RegExpImpl::CompileIrregexp(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>, bool) [/usr/local/Cellar/node@16/16.18.1/bin/node]
14: 0x1010536c7 v8::internal::RegExpImpl::IrregexpPrepare(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>) [/usr/local/Cellar/node@16/16.18.1/bin/node]
15: 0x1010539b3 v8::internal::RegExpImpl::IrregexpExec(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>, int, v8::internal::Handle<v8::internal::RegExpMatchInfo>, v8::internal::RegExp::ExecQuirks) [/usr/local/Cellar/node@16/16.18.1/bin/node]
16: 0x101071d6f v8::internal::Runtime_RegExpExec(int, unsigned long*, v8::internal::Isolate*) [/usr/local/Cellar/node@16/16.18.1/bin/node]
17: 0x1012f8ef9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/usr/local/Cellar/node@16/16.18.1/bin/node]
18: 0x10134750c Builtins_RegExpPrototypeExecSlow [/usr/local/Cellar/node@16/16.18.1/bin/node]
19: 0x101347962 Builtins_RegExpPrototypeExec [/usr/local/Cellar/node@16/16.18.1/bin/node]

My question to you is if somebody ever experienced this and there is a solution to this problem.

Thanks.