Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Contribute the WordLift Vector Store #13028

Merged
merged 39 commits into from
May 21, 2024
Merged

Conversation

ziodave
Copy link
Contributor

@ziodave ziodave commented Apr 22, 2024

Description

Add support for the WordLift Vector Store.

Fixes #12107

New Package?

Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?

  • Yes
  • No

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

  • Yes
  • No

Type of Change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • Added new unit/integration tests
  • Added new notebook (that tests end-to-end)
  • I stared at the code and made sure it makes sense

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added Google Colab support for the newly added notebooks.
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I ran make format; make lint to appease the lint gods

@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Apr 22, 2024
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

…feature/12107-add-support-for-the-wordlift-vector-store
@logan-markewich
Copy link
Collaborator

@ziodave Seems like the tests aren't working. Have you tried running them locally?

@ziodave ziodave marked this pull request as draft April 24, 2024 05:11
@ziodave
Copy link
Contributor Author

ziodave commented Apr 24, 2024

@ziodave Seems like the tests aren't working. Have you tried running them locally?

@logan-markewich yes, sorry I converted the PR to draft until we fix it. May I ask, we recreated the ubuntu-latest-unit-tester in our organization GH Runners so that we can have the GH Actions run on our fork, https://github.com/wordlift/llama_index/actions/runs/8800952266.

Oddly enough the tests pass there, is there a special configuration we need to apply to the GH Runner? This is basically the configuration we did:
image

cc @EthanWordlift

@logan-markewich
Copy link
Collaborator

logan-markewich commented Apr 25, 2024

@ziodave I don't think any special config is needed.

The errors in the test seem unrelated to env though

# Get the key to use for the operation.
>       key = await self.key_provider.for_query(query)
E       TypeError: object str can't be used in 'await' expression

@ziodave
Copy link
Contributor Author

ziodave commented Apr 25, 2024

@logan-markewich we're on it, we'll update the PR soon. Thanks!

@ziodave ziodave marked this pull request as ready for review April 25, 2024 12:11
@ziodave
Copy link
Contributor Author

ziodave commented Apr 25, 2024

@logan-markewich we're ready for review 🙏

@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels May 15, 2024
ziodave and others added 8 commits May 15, 2024 12:51
…2107-add-support-for-the-wordlift-vector-store-resync
…eature/12107-add-support-for-the-wordlift-vector-store-resync
…-resync' of https://github.com/wordlift/llama_index into feature/12107-add-support-for-the-wordlift-vector-store-resync
…-wordlift-vector-store-resync

Feature/12107 add support for the wordlift vector store resync
@ziodave
Copy link
Contributor Author

ziodave commented May 15, 2024

Reopening after fixing the issues. Local tests succeed. We need to check here too.

@ziodave ziodave reopened this May 15, 2024
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels May 15, 2024
@ziodave
Copy link
Contributor Author

ziodave commented May 15, 2024

We're checking the Python 3.10 test.

@ziodave
Copy link
Contributor Author

ziodave commented May 15, 2024

@logan-markewich we fixed the tests. Please let us know if you need more actions from our side.

Cheers.

@logan-markewich
Copy link
Collaborator

I think this looks good now. My remaining concern is just around the ID and metadata stuff

@ziodave
Copy link
Contributor Author

ziodave commented May 17, 2024

@logan-markewich I fixed the metadata issue, we're now sending it with the request. As per the entity ID please see my comment here.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label May 21, 2024
@logan-markewich logan-markewich merged commit feb45ef into run-llama:main May 21, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm This PR has been approved by a maintainer size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: add support for the WordLift VectorStore
3 participants