Reduce Sequali and (slightly) FastQC memory footprint

[rhpvorderman]

• This comment contains a description of changes (with reason)

All I have done for FastQC is delete the huge amount of memory it uses to store the report data for aggregation. Any modules and MultiQC data that runs after the FastQC module can use this memory.

For sequali the changes are much more substantial. I thought hard about using clever methods, but these were always going to complicate the code a lot. Simply loading everything into memory and aggregating later is just so much simpler.

So I did the same as for FastQC, the data is not stored as a class variable but as a normal one that is passed to the class functions. This accomplishes the same as FastQC, when the variable is out of scope, the memory can be used again. On top of that I added a pruning function that removes all the data that is not used by MultiQC from each JSON sample dictionary immediately after loading. This pruning saves massive amounts of memory. According to memray this reduces the amount of memory used by sequali from 600+ MiB to just 150 MiB for 800 reports!

I did not add Sequali: in front of the title as the v1.22 release which contains sequali is not released yet.

[vladsavelyev]

Perfect!

That's something I started doing for some modules, e.g. Picard

MultiQC/multiqc/modules/picard/WgsMetrics.py

Lines 17 to 18 in 931de5d

	data_by_sample = dict()
	histogram_by_sample = dict()

but didn't get around to complete for all of them.

[rhpvorderman]

Thanks for merging!