fix: Loading of embedddings #22260
fix: Loading of embedddings #22260
Conversation
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the |
WalkthroughThe Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Review Details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Files selected for processing (1)
- ee/session_recordings/ai/embeddings_runner.py (3 hunks)
Additional comments not posted (3)
ee/session_recordings/ai/embeddings_runner.py (3)
6-6: Ensure the new importOptionalis used appropriately throughout the file.
26-26: The initialization of_encodingasNoneis a good practice for lazy loading.
206-206: Correct usage ofget_encoding()ensures that encoding is loaded lazily, aligning with the PR's objectives.
| def get_encoding() -> tiktoken.Encoding: | ||
| global _encoding | ||
| if not _encoding: | ||
| # NOTE: This does an API request so we want to ensure we load it lazily and not at startup | ||
| # tiktoken.encoding_for_model(model_name) specifies encoder | ||
| # model_name = "text-embedding-3-small" for this usecase | ||
| _encoding = tiktoken.get_encoding("cl100k_base") | ||
| return _encoding |
There was a problem hiding this comment.
The implementation of get_encoding supports lazy loading effectively. Consider adding error handling for the API request to tiktoken.get_encoding.
def get_encoding() -> tiktoken.Encoding:
global _encoding
if not _encoding:
try:
_encoding = tiktoken.get_encoding("cl100k_base")
except Exception as e:
logger.error("Failed to load encoding", error=str(e))
raise
return _encoding
Problem
We have a code path to load embeddings and it runs at startup which is pretty nasty because it depends on a request to openai. If that isn't available (noticed when locally devving) then the whole process fails :o
Changes
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Does this work well for both Cloud and self-hosted?
How did you test this code?