Issues with code example (CNN example and example in README)

[yanivnik]

Thank you for publishing the code. The method is interesting and seems like it could be helpful to me on a current research project.

For that reason I tried running the code examples, but I'm having some trouble getting them work, mainly around the shape of the projection matrix.

I tried running the "visualize_cnn_random.py" example (without any edit to the code), and I get the following error -

Traceback (most recent call last):
 ....
  File "splinecam/splinecam/wrappers.py", line 415, in verify
    fwd_val = self.layers.forward(A)
  File "splinecam/splinecam/wrappers.py", line 59, in forward
    return (self.Abw[...,:-1] @ x.T + self.Abw[...,-1:]).T
RuntimeError: mat1 and mat2 shapes cannot be multiplied (3072x1 and 2x1)

The input to the function x is with shape (1,2), the shape of the projection matrix T (and self.Abw) is (3072,2) .
It is unclear to me why the index is only accessed until the last element (self.Abw[...**,:-1**]) there. Could this be the cause of the issue?

Also, I looked at the code example in the README and tried using it, passing the domain parameter it as a list of 2D indices (As described in the paper and in one of the example notebooks), but it fails in another unclear exception -

Traceback (most recent call last):
.....  
File "splinecam/visualize_example.py", line 44, in <module>
    flag =  NN.verify()
...
File "splinecam/wrappers.py", line 415, in verify
    fwd_val = self.layers.forward(A)
....
  File "splinecam/splinecam/wrappers.py", line 698, in forward
    x = self.flat2img(x,self.input_shape)
  File "splinecam/splinecam/wrappers.py", line 689, in flat2img
    return x.reshape(x.shape[0],shape[0],shape[1],shape[2])
RuntimeError: shape '[1, 3, 32, 32]' is invalid for input of size 2

Is this an issue with the code? Or should the domain parameter be passed as something other than a list of 2D points? If so, could you please clarify this in the README?

Thank you so much for your assistance!

[ShaneXLi]

Hi @yanivnik. I also encounter the same error messages from running the two examples (visualize_cnn_random.py and visualize_mlp_random.py). I am just wondering if you have had any luck solving the issue?

[yanivnik]

@ShaneXLi I played around with some constants in the code to make it run, but unfortunately the resulting visualizations are still not very similar to the expected results presented in the paper. @AhmedImtiazPrio could you have a look at these demos?

[AhmedImtiazPrio]

Hi @yanivnik @ShaneXLi

Taking a look, sorry for all the delays with getting back to you. Is this happening everytime you run the current version of example codes?

[ShaneXLi]

Hi @yanivnik @ShaneXLi

Taking a look, sorry for all the delays with getting back to you. Is this happening everytime you run the current version of example codes?

Hi Ahmed. As the error suggested, the two matrices cannot be multiplied due to their matrix shapes. So this error happens consistently every time I run the current/latest version of example codes. Please take some time to review the codes.

[AhmedImtiazPrio]

It looks like an issue raised due to some changes that have been made to the splinecam library. I'll update the examples and ping here.

[AhmedImtiazPrio]

@yanivnik @ShaneXLi examples updated, please let me know if they are not working as expected!