A method to choose the optimal chunk size for large document summarization.
See our blog post for a detailed introduction.
-
toy.ipynb shows a toy chunking comparison.
-
summarization.ipynb shows a practical large document summarization comparison.
Get connected to see how we could help to improve your LLM pipelines.
By Vectify AI