Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: use transformer.js for embeddings #74

Merged
merged 7 commits into from Mar 12, 2024
Merged

feat: use transformer.js for embeddings #74

merged 7 commits into from Mar 12, 2024

Commits on Mar 12, 2024

  1. feat: use transformer.js for embeddings 

    - added xenova/transformers.js
- added CustomHuggingFaceTransformersEmbedding to truncate the input text to maxTokens
- updated the vector-store test to use the new embedding
- added a test for the new embedding
- use the new embedding when creating the vector store
    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    22384a6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'alpha' into feat/embedding-transformersjs

    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    46d72b3 View commit details
    Browse the repository at this point in the history

  3. chore: remove unused console.log

    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    a631881 View commit details
    Browse the repository at this point in the history

  4. docs: describe how it works

    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    e816c65 View commit details
    Browse the repository at this point in the history

  5. ci: run e2e on windows

    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    0ff5fe2 View commit details
    Browse the repository at this point in the history

  6. ci: eslint

    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    6364d4b View commit details
    Browse the repository at this point in the history

  7. ci: don't use xvfb-run as this only works on Ubuntu

    @TimPietrusky
    TimPietrusky committed Mar 12, 2024
    Copy the full SHA
    9654f82 View commit details
    Browse the repository at this point in the history