[TwigComponent] out of memory

[choarau-craft]

Hello,

I've started to refactor a qui big application using twig components, but recently I ran into out of memories.
When I'm on the breach, I tried several things to find out what could cause those.
Replacing a simple twig component (without php file) by it's content fixed the memory issue when that component is called about a thousand of time in a loop.
So it's seems like components are adding a lot of memory consumption when used in loops.
Is that the expected behavior, is there any way to fix this ?

Thanks a lot

[WebMamba]

How many times do you loop?

[choarau-craft]

It's a double loop. In my case I'm starting to have memory issue when displaying a table with > 100 rows and about 10 Columns.
Without components I think I was able to display about 400 rows with the same amount of memory.
If I disable the profiler I can double the amount of data displayed i, both cases.

I wanted to use components as macros replacements as it's a lot cleaner and I really like how components are working.
In my small components. In each a single cell I can have up to 6-7 components used.
Components might not have been designed to be used this way ? I wasn't expected that much over memory consumption 😓

[smnandre]

What type of component are you using ? Anonymous ? Twig ? Or Live ?

Could you share a bit of the component code, maybe we can suggest you some ideas to lower the memory.

Is this also the case in production ? The profiler can cause a lot of memory usage in dev

[choarau-craft]

Thanks for reactivity.

I'm using both anonymous and non anonymous components. But no live components

Here is an example when using an anonymous component to display a cell.
If I use the code below I have a memory exception:

{% for k, place in episodeCollection.places %}
    <twig:Episodes:ProductionTable:Partials:Td :episodeClass="episodeClass"
    :place="place"
    :workflowPlaces="workflowPlaces"
    :workflowsPlacesLoop="workflowsPlacesLoop"/>
{% endfor %}

But if I include the code directly it's working fine with same amount of memory. I didn't even have to change sub components usage, but I assume this could result in even bigger memory usage improvements.

{% for k, place in episodeCollection.places %}
    {% include 'components/Episodes/ProductionTable/Partials/Td.html.twig' %}
{% endfor %}

The component itself set a bunch of variables then call other components.
I have just commented the props part to make include work.
Could have set the variables in a php code associated to the twig, but I didn't want to create extra files for simple logics

{#{% props#}
{#	episodeClass,#}
{#	place,#}
{#	workflowPlaces,#}
{#	workflowsPlacesLoop, %}#}

{% apply spaceless %}
	{% set workflowStart = (place == (workflowPlaces | first)) %}
	{% set lastPlace = (workflowPlaces | last) %}
	{% if lastPlace == constant('App\\Enum\\WorkflowPlacesEnum::final_package_validation_ok').value %}
		{% set lastPlace = (workflowPlaces[workflowPlaces|length -2]) %}
	{% endif %}

	{% set workflowEnd = (place == lastPlace) %}

	{% if workflowStart %}
		{{- setGlobalVariable('episode-row-workflowInProgress', true) -}}
	{% endif %}
	{% if workflowEnd %}
		{{- setGlobalVariable('episode-row-workflowInProgress', false) -}}
	{% endif %}

	{% if
		place not in workflowPlaces
		or attribute(episodeClass, place~'_type') is not defined %}
		<twig:Episodes:ProductionTable:Partials:TdEmpty :place="place"
															   :workflowInProgress="getGlobalVariable('episode-row-workflowInProgress')"/>
	{% elseif (not workflowsPlacesLoop.first) and workflowStart %}
		<twig:Episodes:ProductionTable:Partials:TdStartOfWorkflow
				:episodeClass="episodeClass"
				:place="place"
				:workflowPlaces="workflowPlaces"
				:workflowsPlacesLoop="workflowsPlacesLoop"/>
	{% elseif (not workflowsPlacesLoop.first) and workflowEnd %}
		<twig:Episodes:ProductionTable:Partials:TdEndOfWorkflow
				:episodeClass="episodeClass"
				:place="place"
				:workflowPlaces="workflowPlaces"
				:workflowsPlacesLoop="workflowsPlacesLoop"/>
	{% else %}
		<twig:Episodes:ProductionTable:Partials:TdDefault
				:episodeClass="episodeClass"
				:place="place"
				:workflowPlaces="workflowPlaces"
				:workflowsPlacesLoop="workflowsPlacesLoop"
				:workflowStart="workflowStart"
				:workflowEnd="workflowEnd"
		/>
	{% endif %}
{% endapply %}

[choarau-craft]

By replacing almost all sub components (7 out of 8) I was able to halves memory consumption (from 200M to 100M)
This is a huge difference only by using includes ...

[smnandre]

Just by curiosity could you share the "overall" content of your deepest components ?

twig:Episodes:ProductionTable:Partials:TdStartOfWorkflow

twig:Episodes:ProductionTable:Partials:TdEndOfWorkflow

?

But i can already answer: you won't ever compete with includes in that configuration, as Component do a lot more things and are renderered each one in an isolated context (leading to many duplication of vars).

[choarau-craft]

Most of those components are really simple ones rendering some twig like this one:

{#{% props#}
{#	episodeClass,#}
{#	place, %}#}

<td
		data-col-name="{{ place }}"
		class="
			{{- " cell-workflow-start " -}}
			{{- " cell-workflow-parallel-start " -}}
			{{- attribute(episodeClass, place~'_realDelivery') ? ' bg-light-green ' -}}
{#            {{- attribute(episodeClass, place~'_isCurrent') ? ' current-cell ' -}} #}
		"
>
	<i class="far fa-long-arrow-right fa-4x "></i>
</td>

I do understand that components add extra cpu/memory usage, but after rendering shouldn't the memory be freed before rendering other components ?

[choarau-craft]

It's like all components stays in memory until the whole file is rendered.

[smnandre]

It's hard to say without seeing more... 😅

If you have some time to investigate a bit and make some discoveries there, i'd be really happy to ear them and see what we could improve !

[choarau-craft]

I've tried quite a lots of thinks, without much success.
I'm currently replacing my most used components by includes.
I've tried to stream the response but I'm still running into out of memory. Flush in twig does not seems to free components memory usage.
It would be great to have a function, usable in twig template, to force memory cleanup.

By the way flush does not work inside twig component. Which seems quite logic, but it would be great to be able to make streamable components too.

If you have any idea I could test in my workspace I would be glad to try.

Thanks for you time

[smnandre]

@choarau-craft would you agree to share here (or privately if you prefer) a capture of the full twig component profiler page ? (let's say for a render with 100-ish components) ?

[choarau-craft]

Hi,

I won't have time this week, but will be available next week.
But I can't share code here, but I can send some data privately

[smnandre]

But I can't share code here, but I can send some data privately

You can find me on the Symfony Slack channel, or send me an email at smn.andre AT gmail.com if you want