Using ICA sequentially leads to overwrites in the report

[scho97]

For the dataset that includes multiple data modalities (e.g., NTAD data with simultaneous M/EEG recordings), users might want to apply ICA denoising for each data types. For example,

- ica_raw: {n_components: 60, picks: meg}
- ica_autoreject: {picks: meg, apply: true}
- ica_raw: {n_components: 30, picks: eeg}
- ica_autoreject: {picks: eeg, apply: true}

However, using ICA preprocessing step sequentially led to following problems:

1. A *_ica.fif MNE object file tat was generated first gets overwritten by the latter one.
2. gen_html_data() only accepts one ICA object. This forces our report to only summarise and plot the most recent ICA step.
3. Applying plot_bad_ica() to ICAs of EEG leads to an error at L838-842, because raw objects contain all of mag, grad, and eeg in NTAD data, but we only detected ICA from EEG channels.

[matsvanes]

This is where more complex scenarios break the pipeline unfortunately! Who are these ambitious people that want to do simultaneous MEG + EEG??
As discussed yesterday, you may want to try and fit the ICA to EEG and MEG simultaneously. This is what we already do for magnetometers and gradiometers in MEG data, and I don't see a principled reason why it couldn't work in combination with EEG. But it would be worth comparing the individual decompositions with the combined one.

for point 3. the following might work:
plot_ica(ica, raw.copy().pick_types(eeg=True))

[scho97]

Thanks for the reply. I have tried fitting the ICA to EEG and MEG simultaneously. I sent you the results for the individual decompositions and the combined ones via Slack. I can try to add features and open pull requests that can take account for these complex scenarios if you think that will be helpful for the oslpy software.

For point 3, I think as long as we use the channel_type information from the ICA object rather than the Raw one for the if-statements , we can use plot_bad_ica() without much problem.

[matsvanes]

If we want to have multiple ICA objects in dataset, I think there's two options, and I'm not sure which is preferred:

1. by default, dataset["ica"] is an ica object, but if multiple exist, dataset["ica"]["meg"] and dataset["ica"]["eeg"] both contain an ica object.
2. Alternatively, we could have dataset["ica_meg"] and dataset["ica_eeg"] both containing an ica object.

In any case, we’d have to also make sure that the ica objects are saved with different file names, for example …eeg_ica.fif and ...meg_ica.fif.

What do you think @AJQuinn @cgohil8 ?

[scho97]

I think the former is better, because it will be easier to generalise in case other users might be interested in performing ICA on other channel types like grad. In that case, we can use dataset["ica"]["grad"], rather than adding dataset["ica_grad"], which will require more editing.

[matsvanes]

Fair point @scho97 ! I agree.

[cgohil8]

I think what you want to do could be possible without modifying the code. Why not do the following?

1. Run preproc until the ICA step.
2. Then have an ICA script for MEG which loads the fifs from step 1
3. Have a separate ICA script for EEG that saves to a different directory.

Then each ICA step has it's own directory with it's own report. Is this problematic/impractical?

I don't have a strong opinion. If you want to update the dataset['ica'] implementation then option one (adding dataset['ica']['eeg'], dataset['ica']['meg']) works for me.

[scho97]

Based on what I checked so far, I don't think there will be any bugs or problems when we create separate directories and reports for each ICA step. I think it is a matter of user preference about whether we want to create multiple directories or store them in one.
@matsvanes Do you have any preference on this?

[cgohil8]

If adding the option will be beneficial we should go with that. The one comment I have is the more special cases we add the more code and user options we'll have to maintain.

Another option is:

• Modify ica_raw to accept an argument to specify the name of the ICA object.
• Then write a custom function to do your ICA which does all the options your want and saves separate ICA objects.
  You wouldn't get the report panel with this option though.

I'm in favour of whatever make the code more usable, simpler or easier to maintain.

[scho97]

Yes, that could be another option. I guess it will be like implementing Andrew's customised lemon_ica function for multiple ICA steps.

I think editing the code will be helpful for my data, but I am not sure how many people will use the NTAD dataset or data that have simultaneous, multimodal data. I'll wait for Mats' opinion on this. I'm okay with using the unmodified version of the code for now.

[cgohil8]

I think the best option with be the write a wrapper with existing code to do what you want (and add small changes to the repo if neccessary), then if it would be helpful for other people we can simply include as a custom wrapper it in the repo.

[scho97]

Can you elaborate a bit more about a wrapper? Do you mean a wrapper function that would take dataset["ica"] and save separate ICA object and its report for each step? I'm not sure how a wrapper can do this without having some redundancy (e.g., it will need similar reporting codes to plot bad ICAs).

[cgohil8]

I haven't thought through the details, it might not be possible. It's unlikely you'll be able to write a wrapper that'll play nicely with the existing reporting. The wrapper could just save the separate ICA objects (one for MEG the other for EEG) and then you could have a separate script to plot/manually mark the ICA components.

[scho97]

Okay, it would work if I make the wrapper to save the objects and then a new script to plot ICA components. Yet, I don't think the wrapper will work nicely with the current reporting if it somehow gets included into the repo later on. Adding new scripts for reporting ICA separately seems to be adding an extra step in the pipeline. As you said, since this might be a special case, I'll make changes locally and add them only if they are necessary later on.

[cgohil8]

Is this still an issue?

@matsvanes @scho97

[scho97]

I believe it will still be an issue once I start on NTAD as we decided to do ICA separately on EEG and MEG. If keeping it open is cumbersome, I think we can close the issue and reopen later if necessary.

[cgohil8]

No, no problem. We'll leave it open.