Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add comparison workflow example #214

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

add comparison workflow example #214

wants to merge 2 commits into from

Conversation

pmeier
Copy link
Member

@pmeier pmeier commented Nov 20, 2023

Addresses #18. This shows two things:

  1. How to compare multiple components with each other.
  2. How to effectively use async programming to speed this up.

I don't intend to merge this PR as we need to have something like that as part of our documentation. But it can serve as base for future work.

@nenb You might also be able to use this for the comparisons you wanted to do in #205 (comment).

Here is the progress bar output on my machine:

100%|█████████████████████████████████████████████████████████████████████████| 4/4 [00:38<00:00,  9.61s/it]
100%|█████████████████████████████████████████████████████████████████████████| 4/4 [00:17<00:00,  4.31s/it]

So we are roughly twice as fast with utilizing async. This of course will vary a lot for different source storages / assistants / documents / prompts. So we cannot ever report a number here unless we create a fixed test scenario first.

@pmeier pmeier added the pr-status: do-not-merge 🛑 Do not merge (yet) label Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant