Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add distance / dot product function for comparing spike train similarity #2192

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

h-mayorquin
Copy link
Collaborator

@h-mayorquin h-mayorquin commented Nov 10, 2023

Hey guys, after this weekend and yesterday and today of frustrations I decided to implement this. This needs some testing so I will put in draft.

Here is the property that I mentioned to you that you can compare to the agreement scores.

The agreement scores are:
1 / ((distance^2 / dot_product) + 1)

As you can see in this graph for the sortngs that @alejoe91 shared with me:

image

So on the left is the agreement scores calculated with the current formula on main, on the right there is the agrement scores calculate with the distance metric function from the PR.

They are not exactly the same. The distance metric penalizes non-matching burstings but as that is a corner case the overall structure is the same.

[EDIT 1]
I profiled the function against the make_agreement_scores in main at this point in time 2023 / 11 / 14:
Now I compared two sortings 10 hours long with 1000 units each. It takes the function around 3 minutes in this case.

These are the results:

/home/heberto/miniconda3/envs/neuroconv_env/bin/python /home/heberto/development/spikeinterface/bin/dev.py
durations=[36000.0], num_units=1000
function.__name__='make_agreement_scores'
--------------------------------------------------
Mean time over 5 iterations: 212.20 seconds
Mean excluding compliation 156.59 seconds
Std over 5 iterations: 111.38 seconds
Std excluding compliation over: 6.95 seconds
times=['434.61', '151.16', '158.51', '167.12', '149.58']
--------------------------------------------------
function.__name__='compute_distance_matrix'
--------------------------------------------------
Mean time over 5 iterations: 155.84 seconds
Mean excluding compliation 153.47 seconds
Std over 5 iterations: 5.50 seconds
Std excluding compliation over: 3.13 seconds
times=['165.32', '151.88', '158.58', '150.22', '153.20']
--------------------------------------------------
Speedup excluding compilation: 1.02
Speedup including compilation: 1.36

Gist:
https://gist.github.com/h-mayorquin/ecc2db36e768b8caf3a3f890b20611ff

@h-mayorquin h-mayorquin added the comparison Related to comparison module label Nov 10, 2023
@h-mayorquin h-mayorquin self-assigned this Nov 10, 2023
Copy link
Collaborator

@zm711 zm711 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple docstring/assert comments while you're working on the draft.

src/spikeinterface/comparison/comparisontools.py Outdated Show resolved Hide resolved
src/spikeinterface/comparison/comparisontools.py Outdated Show resolved Hide resolved
src/spikeinterface/comparison/comparisontools.py Outdated Show resolved Hide resolved
src/spikeinterface/comparison/comparisontools.py Outdated Show resolved Hide resolved
src/spikeinterface/comparison/comparisontools.py Outdated Show resolved Hide resolved
@alejoe91
Copy link
Member

@h-mayorquin I'm taking another look at this and it seems in good shape. Maybe we should add an option to the compare_* function to select the matching strategy?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comparison Related to comparison module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants