Clean up secondary data in quasi-Newton acceleration

[uekerman]

In quasi-Newton acceleration, we distinguish primary data (data that is used to compute the acceleration and accelerated) and secondary data (data that is only accelerated). For IQN-ILS, we store W matrices for each secondary data and use the coefficient vector c from the primary data:

precice/src/acceleration/IQNILSAcceleration.cpp

Lines 209 to 219 in efd7fb6

	// Perform QN relaxation for secondary data
	for (int id : _secondaryDataIDs) {
	PtrCouplingData data = cplData.at(id);
	auto & values = data->values();
	PRECICE_ASSERT(_secondaryMatricesW[id].cols() == c.size(), _secondaryMatricesW[id].cols(), c.size());
	values = _secondaryMatricesW[id] * c;
	PRECICE_ASSERT(data->getSize() == data->getPreviousIterationSize(), data->getSize(), data->getPreviousIterationSize());
	values += data->previousIteration();
	PRECICE_ASSERT(data->getSize() == _secondaryResiduals[id].size(), data->getSize(), _secondaryResiduals[id].size());
	values += _secondaryResiduals[id];
	}

What should we do with secondary data in IQN-IMVJ? Seems like we silently don't do anything. We underrelax in the first iteration, but we don't do anything in IQNIMVJAcceleration::computeQNUpdate. Looking at the math, I am also not sure what we could do, but I did not yet think a lot about it.

Two options

1. Find a proper treatment of secondary data in IQN-IMVJ and implement it.
2. Forbid (implicit) configuration of secondary data for IQN-IMVJ, clean up (e.g. IQNIMVJAcceleration::computeUnderrelaxationSecondaryData), and document.

In both cases, we need to improve test coverage. There are currently no tests for secondary data at all.

Context

I stumbled over this while reviewing #1834. We are currently anyway improving documentation concerning secondary data in precice/precice.github.io#366.

[Fujikawas]

A possible solution according to Miriam:
Given that the current IMVJ is
$$x^p_{k+1} = \tilde{x}_k^p + [ (J^p_{\text{prev}})^{-1} + \underbrace{(W^p_k - (J^p_{\text{prev}})^{-1} V_k)}_{=: \tilde{W}^p_{k}} V_k^\dagger ] (-r_k)$$
with
$V_k = (\Delta r_i)_{i=1,2,\ldots}$: Matrix with residual differences (in primary data)
$W^p_k = (\Delta \tilde{x}^p_i)_{i=1,2,\ldots}$: Matrix with differences in primary data
$V_k^\dagger$ from solving $V_k^\dagger \cdot Q \cdot R = I \mbox{ with } V_k = Q \cdot R$

We could do the same to secondary data separately:
$$x^s_{k+1} = \tilde{x}_k^s + [ (J^s_{\text{prev}})^{-1} + \underbrace{(W^s_k - (J^s_{\text{prev}})^{-1} V_k)}_{=: \tilde{W}^s_{k}} V_k^\dagger ] (-r_k)$$
with
$V_k$, $V_k^\dagger$: same as above
$W^s_k = (\Delta \tilde{x}^s_i)_{i=1,2,\ldots}$: Matrix with differences in secondary data

Or combine the $W$ of both primary and secondary data:
$$x^t_{k+1} = \tilde{x}_k^t + [ (J^t_{\text{prev}})^{-1} + \underbrace{(W^t_k - (J^t_{\text{prev}})^{-1} V_k)}_{=: \tilde{W}^t_{k}} V_k^\dagger ] (-r_k)$$
with
$V_k$, $V_k^\dagger$: same as above
$W^t_k = \left(\array{ W^p_{k} \\ W^s_{k}}\right)$: Matrix with differences in both primary and secondary data

These two ways would result difference in restart with SVD-method etc.

[NiklasKotarsky]

If I have understood your comment correctly @Fujikawas then it would be enough to include the secondary data directly in $W_k$ to fix the issue in IMVJ? This would allow us to move the secondary data from the IQN-ILS and IMVJ class directly into the base class. Avoiding the need to touch the IMVJ class to fix the issue with the added benefit of only updating the coupling data in one place. (Currently secondary data is updated in the IQN-ILS class and not in the base class.) On top of this, it might also allow for an easier extension of IMVJ to waveform iterations. The interesting observation here mathematically is that $J^{-1}$ and $J^{-1}_k$ would be rectangular matrices.

[uekerman]

Sounds good to me 👍
Without having looked at all details yet, the following plan could make sense:

1. Increase test coverage of the existing IQN-ILS secondary data (and potentially add commented-out IQN-IMVJ tests)
2. Refactor IQN-ILS secondary data to the variant where everything is handled through one W matrix, move things to base class
3. Switch on / implement secondary data for IQN-IMVJ

After step 2, we could try using the same for the waveform implementation.
How does this sound? @Fujikawas @NiklasKotarsky

[Fujikawas]

Sounds good to me 👍 Without having looked at all details yet, the following plan could make sense:

1. Increase test coverage of the existing IQN-ILS secondary data (and potentially add commented-out IQN-IMVJ tests)

2. Refactor IQN-ILS secondary data to the variant where everything is handled through one W matrix, move things to base class

3. Switch on / implement secondary data for IQN-IMVJ

After step 2, we could try using the same for the waveform implementation. How does this sound?

Make sense for me, I opened two PRs above, one for tests and one for one W matrix for both ILS and IMVJ, think this method could work.
For the IQN clean-up, I still want to:

1. Rename the tests (this has been mentioned before IIRC)
2. Add test for Slide-restart, somehow this method is missed in the tests
3. Some other improvement of the comment contents

[NiklasKotarsky]

Sounds good to me 👍 Without having looked at all details yet, the following plan could make sense:

1. Increase test coverage of the existing IQN-ILS secondary data (and potentially add commented-out IQN-IMVJ tests)

2. Refactor IQN-ILS secondary data to the variant where everything is handled through one W matrix, move things to base class

3. Switch on / implement secondary data for IQN-IMVJ

After step 2, we could try using the same for the waveform implementation. How does this sound? @Fujikawas @NiklasKotarsky

Sounds good to me to 👍