All notable changes to this project will be documented in this file.
The format is inspired by (but not strictly follows) Keep a Changelog, and this project adheres to Semantic Versioning.
Before you create a Pull Request, remember to update the Changelog with your changes.
- Run Tests from within the container
- Add model dict output indexing in graph
- Make lance upsert for added vectors
- Make vectors normalized in inmemory vector database for cosine measure.
- Add local cluster as tmux session
- At the end of the test, drop the collection instead of the database
- Force load vector indices during backfill
- Add nightly image for pre-release testing in the cloud environment
- Fix torch model fit and make schedule_jobs at db add
- Add requires functionality for all extension modules
- CI fails if CHANGELOG.md is not updated on PRs
- Update Menu structure and renamed use-cases
- Change and simplify the contract for writing new
_Predictordescendants (.predict_one,.predict) - Add file datatype type to support saving and reading files/folders in artifact_store
- Create models directly by importing package from auto and with decorator
@objectmodel,@torchmodel - Support Schema option for MongoDB
- Optimize LLM fine-tuning
- Sort out the llm directory structure
- Add cache support in inmemory vector searcher
- Add compute_kwargs option for model
- Add BulkWrite mongodb query
- Rename
_PredictortoModel - Allow developers to write
ListenersandGraphin a single formalism - Change unittesting framework to pure configuration (no patching configs)
- Add a simple REST server implementation
- Add reusable snippets that are reused across the docs
- Added snippet for connecting to superduperdb in docs
- Added support to serialize documents in a flat way "_leaves"
- Added
lazy_filedatatype - Optimized LLM finetuning usage experience
- Separate nightly release from sandbox
- Fixed a bug in refresh_after_insert for listeners with select None
- Refactor graph internal with input mapping
- Fixed a bug in Component init
- Fixed a bug in predict in db for missing ouptuts
- Fixed a bug in variable set
- Fixed the bug where select in listener is modified in schedule_jobs.
- LLM CI random errors
- VectorIndex schedule_jobs missing function.
- Fixed some bugs of the cdc RAG application
- Fixed open source RAG Pipeline
- Fixed vllm real-time task concurrency bug
- Fixed Post-Like feature
- Added CORS Policy regarding REST server implementation
- Fixed some bugs in multimodal usecase
- Fixed File datatype
- Fixed a bug in artifact store to skip duplicate artifacts
0.1.1 (2023-Feb-09)
- Test suite takes config from external .env file.
- Added support for multi key in model predict
- Support 3.10+ due to
dataclasssupported features - Updated the table creation method in MetaDataStore to improve compatibility across various databases.
- Replaced JSON data with String format before storage in SQLAlchemy.
- Implemented storage of byte data in base64 format.
- Migrated MongoDB Atlas vector search as a standalone searcher like lance.
- Deprecated Demo Image. Now Notebooks run in Colab.
- Replace dask with ray compute backend
- All training and validation parameters to be configured in
_Predictorattributes (.trainer,.train_X, etc.) - Docker build can include optional custom
requirements.txtpath
- Add Llama cpp model in extensions.
- Basic Ray server support to server models on ray cluster
- Add Graph mode support to chain models
- Simplify the testing of SQL databases using containerized databases
- Integrate Monitoring(cadvisor/Prometheus) and Logging (promtail/Loki) with Grafana, in the
testenv - Add
QueryModelandSequentialModelto make chaining searches and models easier. - Add
insert_to=<table-or-collection>to.predictto allow single predictions to be saved. - Support vLLM (running locally or remotely on a ray cluster)
- Support LLM service in OpenAI format
- Add lazy loading of artifacts by default
- Update connection uris in
sql_examples.ipynbto include snippets for Embedded, Cloud, and Distributed databases. - Fixed a bug related to using Clickhouse as both databackend and metastore.
0.1.0 (2023-Dec-05)
- Introduced Chinese version of README
- Updated paths for docker-compose.
0.0.20 (2023-Dec-04)
- Chop down large files from the history to reduce the size of the repo.
0.0.19 (2023-Dec-04)
- Add Changelog for tracking changes on the repo. It must be filled before any PR.
- Remove ci-pinned-dependencies and replaced them with actions with better cache management.
- Change logging mechanism from the default to loguru
- Update icons on the README.
- Reboot test-suite, with modular approach to toggling between SQL and MongoDB tests
- Add model-versioning of model-outputs
- Refactor OpenAI code to use the new features of the OpenAI API
- Fixes for dask worker compute delegation
- Wrap compute with abstraction as component of datalayer
- Simplify approach to project configuration
- Add services for vector-search and CDC for more comprehensive cluster mode
- Add a
Component.post_createhook to enable logic to incorporate model versions - Fix multiple issues with
ibis/ SQL code
- Add support for selecting whether logs will be redirected to the system output or directly to Loki
- Added libgl libraries in Dockerfile to correctly render the video in notebooks.
0.0.15 (2023-Nov-01)
- Updated readme by @fnikolai in #1196.
- Removed unused import by @jieguangzhou in #1205.
- Updated README.md with contributors by @thejumpman2323 in #1201.
- Added conditional builders in Dockerfile by @fnikolai in #1213.
- Optimized unit tests by @jieguangzhou in #1204.
- Updated README.md with announcement emoji by @thejumpman2323 in #1222.
- Launched announcement by @fnikolai in #1208.
- Added raw SQL in ibis by @thejumpman2323 in #1220.
- Added experimental keyword by @fnikolai in #1218.
- Added query table by @thejumpman2323 in #1212.
- Merged Ashishpatel26 main by @blythed in #1224.
- Bumped Version to 0.0.15 by @fnikolai in #1225.
- Fixed dependencies and makefile by @fnikolai in #1209.
- Fixed demo release by @fnikolai in #1210.