Release 3.1.0

What's Changed

• Hnsw algorithm by @javitonino in #2066
• Update h2 by @javitonino in #2069
• Max tokens for LLM by @lferran in #2065
• Do not deserialize empty nodes into RAM hnsw by @javitonino in #2070
• Query shard replicas as primary-secondary by @lferran in #2071
• Add global migration to purge orphan kbslugs keys from maindb by @jotare in #1963
• Support matryoshka embeddings by @jotare in #2063
• Adding AI tables storage by @bloodbare in #2018
• int order retuns only needed results by @hermeGarcia in #2076
• Back pressure: add more logging by @lferran in #2077
• Bump aiohttp and idna to fix vulnerabilities by @lferran in #2078
• Term query set by @javitonino in #2073
• Reuse field information when indexing paragraphs by @javitonino in #2081
• Bump rustls from 0.21.10 to 0.21.11 by @dependabot in #2082
• Fix pagination bug on catalog by @lferran in #2084
• Add origin source-id label alias by @lferran in #2085
• Support keep_markdown in models by @javitonino in #2087
• Feature-flag cleanup by @lferran in #2079
• Validate field ids against regex with fastapi / pydantic by @lferran in #2090
• Migrate to paragraphs v3 by @javitonino in #2089
• Completely remove vectorsets public api by @lferran in #2092
• Do not fail if apt-get update fails in CI by @javitonino in #2093
• Set to pending on field upload for existing resource by @lferran in #2095
• No vector metadata when opening by @hermeGarcia in #2091
• old version names by @hermeGarcia in #2096
• use rename by @hermeGarcia in #2097
• Cleanup ingest gRPC and id/slugs by @jotare in #2080
• Open index readers only need path, not config by @jotare in #2099
• Cleanup optionals from metadata file by @jotare in #2100
• Revert "Bigger k during layer>0 search" by @javitonino in #2101
• Fix identation on rollover by @lferran in #2104
• Ship pingcap cert by @javitonino in #2105
• Added limits to pagination by @lferran in #2108
• Fix fedora e2e by @lferran in #2109
• Be able to index a resource by storage referece by @lferran in #2094
• Set resource from storage tweaks by @lferran in #2111
• Faster HNSW insertion by @javitonino in #2102
• alias not rename by @hermeGarcia in #2115
• Add shard indexes metadata file by @jotare in #2103
• Deprecated unused OpStatus fields and don't compute counters anymore by @jotare in #2114
• Create indexes passsing its config by value by @jotare in #2116
• Less locking in ShardWriter by @jotare in #2117
• Store indexes file by write and rename by @jotare in #2120
• box readers + swap update by @hermeGarcia in #2122
• Concurrency by branch by @javitonino in #2126
• Remove previous ask resource feature by @lferran in #2124
• Stricter grpcio dependency requirements to ease installs by @lferran in #2123
• Add generated files on links by @bloodbare in #2125
• Fix S3 storage delete_object param name by @jotare in #2112
• Get semantic threshold from query endpoint by @lferran in #2127
• Limit the amount of vectorsets allowed by @jotare in #2128
• Update rust deps by @javitonino in #2130
• Onprem migrations on startup by @lferran in #2132
• Do not wait for nodes on onprem deploy by @lferran in #2134
• Set normalize_vectors as optional by @jotare in #2133
• CI wip by @javitonino in #2121
• Fix wrong usage of fastapi endpoint docs by @jotare in #2113
• Properly close asyncio loop by @lferran in #2135
• Add test for rollover resilient to modifications and deletions by @lferran in #2106
• prefer_markdown option in chat by @vangheem in #2098

Full Changelog: v3.0.3...v3.1.0