Background: this is a feature request from a customer project.

Recent models from OpenAI allow shortening the embeddings, e.g. with text-embedding-3-large the dimensions can be reduced from 3072 to 256 with still a better performance than ada-002 with 1536 dimensions. This has the potential benefit of reducing the index size (specifically vector space) without impacting performance.

To support this in the accelerator, the additional "dimensions" parameter can be added to the embeddings.create() call in embedding/aoai_embedding_model.py.