CTranslate2 4.3.0

New features

• Support phi-3 (8k and 128k) (#1700 and #1680)

Fixes and improvements

• Fix regression Flash Attention (#1695)

CTranslate2 4.2.1

Note: Because of the increasing of package's size (> 100 MB), the release v4.2.0 was pushed unsuccessfully.

New features

• Support load/unload for generator/Whisper Attention (#1670)

Fixes and improvements

• Fix Llama 3 (#1671)

CTranslate2 4.2.0

New features

• Support Flash Attention (#1651)
• Implementation of gemm for FLOAT32 compute type with RUY backend (#1598)
• Conv1D quantization for only CPU (DNNL and CUDA backend is not supported) (#1601)

Fixes and improvements

• Fix bug tensor parallel (#1643)
• Use BestSampler when temperature is 0 (#1659)
• Fix bug gemma (#1660)
• Optimize loading/unloading time for Translator with cache (#1645)

CTranslate2 4.1.1

Fixes and improvements

• Fix classifiers in setup.py to push pypi package

CTranslate2 4.1.0

New features

• Support Gemma Model (#1631)
• Support Tensor Parallelism (#1599)

Fixes and improvements

• Avoid initializing unused GPU (#1633)
• Read very large tensor by chunk if the size > max value of int (#1636)
• Update Readme

CTranslate2 4.0.0

This major version introduces the breaking change while updating to cuda 12.

Breaking changes

Python

• Support cuda 12

New features

• Add feature to_device() in class StorageView in Python to move data between host <-> device

Fixes and improvements

• Implement Conv1D with im2col and GEMM to improvement in performance
• Get tokens in the range of the vocab size for LlaMa models
• Fix loss of performance
• Update cibuildwheel to 2.16.5

CTranslate2 3.24.0

New features

• Support of new option offset to ignore token score of special tokens

CTranslate2 3.23.0

New features

• Support Phi model

Fixes and improvements

• Fix the conversion for whisper without the "alignment_heads" in the "generation_config.json"
• Fix forward batch

CTranslate2 3.22.0

New features

• Support "sliding window" and "chunking input" for Mistral

Fixes and improvements

• Take into account the "generation_config.json" and fix "lang_ids" getter for Whisper converter
• Accept callback even on "generate_tokens" method
• Fix iomp5 linking with latest Intel OpenAPI on Ubuntu
• Fixed "decoder_start_token_id" for T5

CTranslate2 3.21.0

New features

• Minimal Support for Mistral (Loader and Rotary extension for long sequence). No sliding yet
• Support Distil-Whisper
• Support Whisper-large-v3