Releases: smallcloudai/refact
v1.6.2
Refact.ai Self-hosted
- Models Support: We've introduced support for gated models and the new llama3 model
- Even More Models: GPT4o and GPT4-turbo models are now available
Refact.ai Enterprise
- VLLM Speed Improvement: You are now able to experience faster processing times with our optimized VLLM
- VLLM LoRa-Less Mode: In cases where LoRa is not set up, VLLM will now operate 20% faster due to the new LoRa-less mode
- Empty Prompt and OOM Handling: We've addressed issues in VLLM that caused broken generations
v1.6.1
Context Switching Mechanism
We've implemented the context-switching mechanism, and it's available in our latest version of the VS Code plugin. Now, you can change the max context value for models depending on your needs — small context for less memory usage and faster operation or large context for deeper insights.
Model Deprecation
Our UI updates now flag models slated for removal. This ensures you're always working with the latest and most efficient models.
Factory Reset Fix
We've resolved issues with the factory reset process for when you need a fresh start.
v1.6.0
Refact.ai Self-hosted Updates:
Multiple Source Projects:
- Versatile Source Handling: In what used to be a single "sources" tab, you can now create multiple projects, each with its diverse sources.
- Fine-Tune on Demand: You can now fine-tune on your specific project.
Fine-Tune Enhancements:
- Unified Fine-Tuning Process: We've simplified the fine-tuning process by merging the filter and fine-tuning steps into one process.
- Multi-GPU Support: You can use multiple GPUs for faster fine-tuning!
- Simultaneous Fine-Tuning: Execute multiple fine-tuning processes concurrently to save time.
- Full Model Support: Load full model patched LoRA weights (note: requires substantial RAM).
Refact.ai Enterprise Updates:
- Customization Tab: Personalize system prompts and toolbox commands specifically for your team.
- Keycloak Integration: Secure user authentication with Keycloak, including a dedicated account page for users, ensuring both convenience and security. Check out the documentation for more information: https://docs.refact.ai/guides/keycloak/
v1.5.0
- Fine-tune Process Enhancements: We've made the fine-tuning process for starcoder models both faster and of higher quality with new default settings
- Fine-tune UI: The fine-tune setup has been moved to the Model Hosting tab for easier access
- Plugin Fine-tune Switching: VS Code and JetBrains plugins now support switching between fine-tuned models
- Chat Tab Redesign: The Chat tab has been temporarily hidden for a redesign and will be back in the next release
Compatibility Issues
- Plugin Support: Older versions of plugins will fall back to using the base model as they do not support the new fine-tuning capabilities. Make sure to update your plugin
v1.4.0
What's New
- WebGUI Chat: Now, we ship a chat UI with our docker image!
- Embeddings: From now on, in our docker, by default, we are starting the model responsible for the embeddings. That is necessary for the VecDB support.
- Shared Memory Issue Resolved: A critical performance issue related to shared memory has been fixed. For more details, check out the GitHub issue.
- Anthropic Integration: We've implemented an ability to add API keys to use third-party models!
- stable-code-3b: The list of available models is growing! This time, we added stabilityai/stable-code-3b!
- Optional API Key for OSS: Refact.ai Self-hosted version can now use an optional API key for security if deployed on a cloud.
- Build Information: In the settings, you can now find the About page, which includes information about packages that are used, versions, and commit hashes.
- LoRa Switch Fix: The issue with switching between LoRas (didn't show information in logs) is now fixed!
- VLLM Out-of-Memory (OOM) Fix: We've fixed an out-of-memory issue with VLLM for the Refact.ai Enterprise!
v1.3.1
Open-Source Updates:
- Memory Consumption Fix for local Cassandra.
- Unified Volume: One volume for all data, including the database.
- Encodings Fix for the fine-tuning process.
- Minor Fixes addressing various small issues.
Enterprise Updates:
- Tag Upgrade: Transition from
beta
tolatest
indocker-compose.yml
. Ensure to update your compose file. - Runpod Support:
- Local database integration.
- One storage solution for all data.
- Minor UI Fixes: Improvements and bug fixes.
v1.3.0
New Models
Expanding the list of available models with:
- Mistral
- 7B
- Mixtral
- 8x7B
- Deepseek
- 6.7B
- 33B
- Magicoder
- 6.7B
Statistics
We are introducing a new cool feature - user stats.
Check out the new page with informative charts to see the impact of having Refact!
Better Docker Flow
We've simplified our Docker image usage! Now, it's just one command:
docker run --rm --gpus all -p 8008:8008 -v perm-storage:/perm_storage -v
refact-database:/var/lib/cassandra smallcloud/refact_self_hosting:latest
UI Enhancements
- Improved Modal Window: From now on, we will have a more structured and organized interface for the list of models.
- User Seats Information: For the Refact Enterprise, the access control page now includes information about user seats – a feature exclusive to the enterprise plan.
General Improvements and Chat Handlers
v1.2.0
Deepseek-Coder Models:
We added support for the deepseek-coder family models, and you can use these models for completion and fine-tuning.
Codellama/7b:
Starting today, the codellama/7b
model is available for fine-tuning.
Faster Fine-Tuning:
Fine-tuning is now faster for GPUs with CUDA capabilities 8.0
and higher.
UI & Performance:
General UI and performance improvements.
v1.1.0
Model Updates
Starcoder 1b, 3b, and 7b models are now available for completion and finetuning.
Features
Upload LoRA
- LoRA Upload: LoRA upload feature is now available. You can now upload LoRA either via a direct link or by uploading the file.
Download Run
- "Download Run" feature allows downloading the best checkpoint.
- The "Download Checkpoint" feature allows you to download only the selected checkpoint.
UI & UX Improvements
- Model Hosting Tab: The selected finetune checkpoint associated with the model is now visible.
- Finetune Tab: The selected model for completion is now visible on the Finetuning page. From now on, you can easily view the model for which you've selected the checkpoint.
- Checkpoint Selection: You can now set the best checkpoint for a run.
Bug Fixes & Performance Improvements
- We've crushed some bugs and optimized Refact for even better performance.