Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stbt.match: Add structure_weighting #530

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

wmanley
Copy link
Contributor

@wmanley wmanley commented Oct 19, 2018

We use Sobel edge detection to improve the matching. This is then
taken into account when constructing the heatmap. This more closely
matches human expectations at the cost of being slower to run.

The weighting is done by raising the values to a power before
multiplying them together. This is inspired by SSIM where seperate
brightness, contrast and structure terms are combined with the same
technique.

TODO:

  • Include example in tests where it makes a difference
  • Update debug HTML output

We use Sobel edge detection to improve the matching.  This is then
taken into account when constructing the heatmap.  This more closely
matches human expectations at the cost of being slower to run.

The weighting is done by raising the values to a power before
multiplying them together.  This is inspired by SSIM where seperate
brightness, contrast and structure terms are combined with the same
technique.

images.append(cv2.Sobel(image, cv2.CV_32F, 0, 1, ksize=3))
templates.append(cv2.Sobel(template, cv2.CV_32F, 0, 1, ksize=3))
weights.append(structure_weighting * 0.5)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do the x & y Sobel separately?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Specifying xorder=1, yorder=1 gives diagonal edges, which isn't what we want.

@drothlis
Copy link
Contributor

Misc. thoughts:

  • Sobel operation is a convolution with the Sobel kernel. A convolution in the spatial domain is the same as a multiplication in the frequency domain. cv2.matchTemplate already does an FFT. We could do the FFT a single time, then do the Sobel edge-detection and matchTemplate search. This would require un-picking matchTemplate (OpenCV's license is 3-clause BSD).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants