bounding box area of match

[RickyGunawan09]

hai thank you for making this good library for matching image but i have a question.

can i get the bounding box area of the matching image ? i think if that feature is added we can use lightglue for another use case like template matching maybe.

thank you

[Phil26AT]

Hey @RickyGunawan09, sorry for the late reply. Yes you can compute the bounding box. After matching the images, just compute the bounding box around the matched points in both images (taking min/max of all xy-coordinates from the matched points).

Something like this should work (second example from the demo notebook):

from matplotlib.patches import Rectangle
import matplotlib.pyplot as plt
image0 = load_image(images / "sacre_coeur1.jpg")
image1 = load_image(images / "sacre_coeur2.jpg")

feats0 = extractor.extract(image0.to(device))
feats1 = extractor.extract(image1.to(device))
matches01 = matcher({"image0": feats0, "image1": feats1})
feats0, feats1, matches01 = [
    rbd(x) for x in [feats0, feats1, matches01]
]  # remove batch dimension

kpts0, kpts1, matches = feats0["keypoints"], feats1["keypoints"], matches01["matches"]
m_kpts0, m_kpts1 = kpts0[matches[..., 0]], kpts1[matches[..., 1]]

viz2d.plot_images([image0, image1])
viz2d.plot_matches(m_kpts0, m_kpts1, color="lime", lw=0.2)
viz2d.add_text(0, f'Stop after {matches01["stop"]} layers')

tl = m_kpts1.min(dim=-2).values  # top-left corner
br = m_kpts1.max(dim=-2).values  # bottom-right corner
axes = plt.gcf().axes
axes[1].add_patch(Rectangle(tl, br[0] - tl[0], br[1]-tl[1], fill=False, color="lime"))