TensorRT support for openai/clip-vit-large-patch14-336

[junwang-wish]

Is there a fundamental technical limitation on not being able to support TensorRT for openai/clip-vit-large-patch14-336 ? Just wanna understand why most 768-dim embedding models are not supported according to https://clip-as-service.jina.ai/user-guides/server/#model-support

[ZiniuYu]

Thank you for your interest in clip-as-service and for reaching out with your question.

To answer your question, we want to clarify that clip-as-service aims to support as many models as possible, including the openai/clip-vit-large-patch14-336 model. However, due to the complex compatibility issues between TensorRT and other packages, such as hardware and CUDA version requirements, we are currently unable to support TensorRT for this model.

We appreciate your feedback and interest in TensorRT support, but at this moment, we don't have any concrete plans to add it in the near future due to our current capacity. However, we encourage you to continue exploring and experimenting with different solutions, and if you find a way to make TensorRT work with this model, please feel free to share it with the community so that others can benefit from your findings.

Thank you for your support and contribution to the open source community.