Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicit erosion #2665

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Explicit erosion #2665

wants to merge 4 commits into from

Conversation

graeme-winter
Copy link
Contributor

In dials.find_spots use explicit erosion based on a simpler kernel search rather than the distance-based method as currently implemented, for a small performance improvement and also for simpler understanding for future code archeologists

graeme-winter and others added 3 commits May 8, 2024 06:31
Work in progress, re-implement the erosion based on a simple kernel search
rather than the distance array, since the data are sparse and this allows a
far smaller number of calculations to be performed.
Copy link

codecov bot commented May 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.46%. Comparing base (b624db8) to head (abf0e1d).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2665      +/-   ##
==========================================
+ Coverage   78.40%   78.46%   +0.05%     
==========================================
  Files         613      613              
  Lines       75349    75958     +609     
  Branches    10761    10924     +163     
==========================================
+ Hits        59081    59604     +523     
- Misses      14087    14163      +76     
- Partials     2181     2191      +10     

@graeme-winter
Copy link
Contributor Author

On main:

Extracted 397830 spots
Removed 96808 spots with size < 3 pixels
Removed 54 spots with size > 1000 pixels
Calculated 300968 spot centroids
Calculated 300968 spot intensities
Filtered 298267 of 300968 spots by peak-centroid distance

Histogram of per-image spot count for imageset 0:
298267 spots found on 3600 images (max 5285 / bin)
         **********                  ************          
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
1                         image                         3600

--------------------------------------------------------------------------------
Saved 298267 reflections to strong.refl

real	5m19.734s
user	63m49.983s
sys	30m22.943s

This branch:

Extracted 397830 spots
Removed 96808 spots with size < 3 pixels
Removed 54 spots with size > 1000 pixels
Calculated 300968 spot centroids
Calculated 300968 spot intensities
Filtered 298267 of 300968 spots by peak-centroid distance

Histogram of per-image spot count for imageset 0:
298267 spots found on 3600 images (max 5285 / bin)
         **********                  ************          
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
************************************************************
1                         image                         3600

--------------------------------------------------------------------------------
Saved 298267 reflections to strong.refl

real	5m7.480s
user	61m0.185s
sys	29m2.833s

Comparison of output:

ws448 cmp :) $ diff dials.find_spots.log ../ref/dials.find_spots.log 
2c2
< DIALS 3.dev.1126-gc1010b948
---
DIALS 3.dev.1125-gb624db84b
11c11
<   experiments = ../ref/imported.expt
---
 experiments = imported.expt

ws448 cmp :( $ shasum strong.refl ../ref/strong.refl 
b98e56805360c2a9325b0a8f6985bbf7ba45cccd  strong.refl
b98e56805360c2a9325b0a8f6985bbf7ba45cccd  ../ref/strong.refl

=> I believe this gives the same output and is clearer to read. The ~ 15s reduction in wall clock is a nice to have which will be data set dependent.

I note in passing that this approach will be more GPU friendly also.

Add new comments to explain what is going on, leave the new code as a
reference implemntation but do not link it as I am not convinced that the
performance is better on all architectures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants