Fix JacobianTest and CustomJacobianTest (Resolves #3470)

[mrdaybird]

Use output of the Forward method instead of input. Fixes #3470 (See issue for more details).

[rcurtin]

I think this is correct. At least, it passes all the tests, but, all layers tested with the Jacobian test do not use the first argument to Backward(). So what is needed is an audit: just need to look at all the layers, and see if they all use the convention that the first parameter to Backward() is the output from Forward(), not the input to Forward().

I'm not yet sure if this makes the Jacobian test succeed for the updated LogSoftMax implementation.

[rcurtin]

Went through and audited everything visually (quicker than I thought!).

• With this change the Jacobian test for the updated LogSoftMax layer passes.
• Many layers simply ignore the first parameter to Backward().
• Several layers use the first parameter just to get the batch size (.n_cols).
• A couple layers (HardTanH, LeakyReLU, PReLU) are such that Baclward() is correct regardless of whether the output from or input to Foward() is passed.
• I'm not 100% sure of Softmax but it looks like it takes the output. A test will tell us, I guess.
• The same is true of Softmin, but just glancing at the backward expression, I think it might be wrong. We'll find out when we apply a test, I guess.

I would definitely like to get these tests applied to all layers. I was surprised to see when I went through the tests that only Linear3D and LinearNoBias are tested with the Jacobian test in the tests/ann/layer/ directory

It could be useful in a follow-up PR to either clearly document what the first parameter to Backward() is, or change its name throughout the codebase, or both. I'm not sure what the best way forward there is.

[mrdaybird]

Also to be noted, many activation functions incorrectly implement the Deriv(x, y) method, having the same mistake as the Backward method, consequently corresponding layers also have the incorrect Backward method.

I think I will make a list of all layers/functions which maybe have been incorrectly implemented, so that they can be fixed and additional tests(including the Jacobian test) be added along with the fix.

[mlpack-bot]

Second approval provided automatically after 24 hours. 👍

[rcurtin]

Thanks again for finding this!