Add Cross-Replicate Consolidation

[TheLostLambda]

Here are some pointers that I think might help with the implementation!

The goal is to generate an extra, consolidated CSV file when several datasets are uploaded and an "Advanced > Consolidation" option is ticked. If you uploaded three datasets, enabled consolidation, then ran PGFinder, it should download four files. Three of those will be the three it returns now (one CSV file per dataset), and the fourth will be the consolidation of those other three.

The place to loop into this is here:

pgfinder/lib/pgfinder/gui/shim.py

Lines 17 to 36 in a334000

	def run_analysis():
	from pyio import (
	cleanupWindow,
	consolidationPpm,
	enabledModifications,
	massLibrary,
	msData,
	ppmTolerance,
	)
	theo_masses = theo_masses_upload_reader(massLibrary.to_py())
	def analyze(virt_file):
	ms_data = ms_upload_reader(virt_file)
	matched = matching.data_analysis(
	ms_data, theo_masses, cleanupWindow, enabledModifications, ppmTolerance, consolidationPpm
	)
	return pgio.dataframe_to_csv_metadata(matched)
	return {f["name"]: analyze(f) for f in msData.to_py()}

• You add an import from the JS side in the pyio import that stores the bool for whether you should consolidate or not
• Then you check it's true and that you have more than one file in msData
• Then you call the consolidation function on that list of individual CSV outputs
  • You could also change analyze to return a tuple of the original dataframe (matched) as well as the CSV, then you just need to take a DataFrame and don't need to do any more CSV parsing

Writing the actual consolidation function should be relatively straightforward:

1. Stack / vcat the dataframes from each dataset
2. Group by structure
3. Compute mean and stddev for each group (for parameters like intensity and RT)
4. Write into a final table

@smesnage should be able to supply another reference file against which to check this process!

Finally, on the web side, you'll need to add that option to the "Advanced" options for toggling this feature.

1. Add the button to this column of the advanced options:

   pgfinder/web/src/routes/AdvancedOptions.svelte

   Lines 19 to 54 in a334000

   	<div class="flex flex-col justify-between aspect-square overflow-y-auto">
   	<div class="flex flex-col items-center">
   	<h5 class="pb-1 h5">PPM Tolerance</h5>
   	<input bind:value={ppmTolerance} class="input" type="number" step="1" min="0" />
   	</div>
   	<div class="flex flex-col items-center">
   	<h5 class="pb-1 h5">
   	Cleanup Window
   	<Tooltip style="inline ml-1" popupId="cleanupTooltip">
   	Set time window for in-source decay and salt adduct cleanup
   	</Tooltip>
   	</h5>
   	<input bind:value={cleanupWindow} class="input" type="number" step="0.1" min="0" />
   	</div>
   	<div class="flex flex-col items-center">
   	<h5 class="pb-1 h5">
   	Consolidation PPM
   	<Tooltip style="inline ml-1" popupId="consolidationTooltip">
   	During consolidation, structures with the lowest absolute ppm are selected over
   	those farther from the theoretical mass. However, if two or more matches have a
   	theoretical mass less than the consolidation ppm apart, then those matches are
   	retained, leaving several possible matches.
   	</Tooltip>
   	</h5>
   	<input
   	bind:value={consolidationPpm}
   	class="input"
   	type="number"
   	step="1"
   	min="0"
   	max={ppmTolerance}
   	/>
   	</div>
   	</div>

2. Add an exported property representing that boolean button state here, and bind it in the <input> object

   pgfinder/web/src/routes/AdvancedOptions.svelte

   Lines 5 to 10 in a334000

   	export let enabledModifications: Array<string>;
   	export let allowedModifications: Array<string> | undefined;
   	export let ppmTolerance: number;
   	export let cleanupWindow: number;
   	export let consolidationPpm: number;
   	export let advancedMode: boolean;

3. Add a bool to the Pyio type — used to shuttle data from the webUI to PGFinder:

   pgfinder/web/src/app.d.ts

   Lines 3 to 10 in a334000

   	declare type Pyio = {
   	msData: Array<VirtFile> | undefined;
   	massLibrary: VirtFile | undefined;
   	enabledModifications: Array<string>;
   	ppmTolerance: number;
   	cleanupWindow: number;
   	consolidationPpm: number;
   	};

4. Bind the boolean property here, connecting the pyio field to the button state in the <AdvancedOptions>:

   pgfinder/web/src/routes/+page.svelte

   Lines 117 to 124 in a334000

   	<AdvancedOptions
   	bind:enabledModifications={pyio.enabledModifications}
   	bind:ppmTolerance={pyio.ppmTolerance}
   	bind:cleanupWindow={pyio.cleanupWindow}
   	bind:consolidationPpm={pyio.consolidationPpm}
   	bind:advancedMode
   	{allowedModifications}
   	/>

That should be more than enough to get started with, and I'm happy to guide whoever through the implementation!