Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: make vector validation work with fp16 #1962

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: make vector validation work with fp16 #1962

wants to merge 1 commit into from

Conversation

chebbyChefNEQ
Copy link
Contributor

two issues fixed:

  • when using fp16 and cosine together, the distance we get is usually greater than 1e-6 even when we hit an exact match.
    • fixed by comparing the vectors instead of distance
  • fp16 can have edge cases where IVF partition is assigned wrong (very low percentage, fixable by just using more nprobes)
    • fixed by allowing lower validation threshold for fp16

westonpace
westonpace approved these changes Feb 19, 2024
View reviewed changes
python/python/lance/util.py
with_row_id=True,
)
passes += (
1 if (res[column].to_numpy(zero_copy_only=False)[0] == vec).all() else 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to worry about epsilon/precision? E.g. should this be < 1e-6?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@westonpace westonpace westonpace approved these changes

@wjones127 wjones127 wjones127 approved these changes

@eddyxu eddyxu Awaiting requested review from eddyxu

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@chebbyChefNEQ @westonpace @wjones127