Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datashift - Rigid and Segmented Registration #29

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

alexmorley
Copy link
Collaborator

@alexmorley alexmorley commented Aug 31, 2020

V1 working version of datashift.

Probably could use a) a couple more unit tests and b) better integration with the sorting flow.

Check for:

  • off-by-one errors
  • untested branches

alexmorley and others added 5 commits May 31, 2020 20:24
- eMouse simulation example
- Jenkins report builder
- Simple comparison analysis
* 'master' of github.com:rossant/pykilosort:
  Typed Configuration (MouseLand#28)
  Comparison between MATLAB & Python Versions (MouseLand#25)
  Add dockerfile for running tests with GPU (MouseLand#22)
@alexmorley alexmorley changed the title Datashift - Rigid Registration Datashift - Rigid and Segmented Registration Sep 14, 2020
@alexmorley alexmorley marked this pull request as ready for review September 14, 2020 00:18
counter = np.zeros(2, dtype=np.int32, order='F')

# filter the data with the temporal templates
Conv1D = cp.RawKernel(code, 'Conv1D')
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kushbanga was this Conv1D function the one you said had a bug?

@@ -0,0 +1,10 @@
0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0,7.5,17.6706,7.5,0,16,0
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These files are for regression tests.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the output is from the MATLAB version)


TEST_DATA_FOLDER = "tests/fixtures/data"

@pytest.mark.requires_gpu
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this one does require gpu...

@alexmorley
Copy link
Collaborator Author

Suggestion from @kushbanga. Let's add a test against the MATLAB version that the depth shift is exactly the same for each time segment.


# upsample the shift for each channel using interpolation
if len(ysamp) > 1:
# zero pad input so "extrapolation" tends to zero
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB: Makima vs zero-padded Akima

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try:

  • regression towards mean
  • mirror

) # this kernel is fixed as a variance of 1
dcs = cp.array(dcs)
# dcs = my_conv2_cpu(dcs, .5, [0,1,2])
for i in range(dcs.shape[0]):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conv bug in MATLAB??

image

@alexmorley
Copy link
Collaborator Author

@kushbanga I just pushed a bunch of fixes that I had sitting on my laptop just FYI!

@alexmorley
Copy link
Collaborator Author

more tests here alexmorley/kilosort-testing#2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant