You need to use the identical question instruments to go looking vector indexes in addition to the remainder of your knowledge, providing you with the choice to go looking primarily based on similarities in your knowledge or by precise matches. This method is much like how large-scale search engines like google work and can assist discover and rank outcomes from massive semistructured knowledge units, for instance, trying to find related critiques on an e-commerce web site. Cloth requires a vector coverage for every Cosmos DB container, which defines measurement, dimensionality, and the underlying distance operate used to seek for related vectors. Search applied sciences like DiskANN require a excessive dimensionality, with not less than 1,000 dimensions (and a most of 4,096).
Querying Cosmos DB in Cloth
Once you question knowledge saved in Cosmos DB by Cloth’s OneLake, you’re working with a mirrored copy of your Cosmos DB knowledge. As you retailer knowledge, it’s copied throughout within the Delta Parquet format utilized in Cloth, permitting you to make use of any of the supported question instruments, together with the desktop Energy BI for advert hoc evaluation. Queries right here may be made throughout all of your operational knowledge, not simply Cosmos DB, treating it as a unified complete and nonetheless profiting from Cosmos DB’s characteristic set for purposes that want to make use of that knowledge.
This additionally permits you to reap the benefits of different Cloth options along with your Cosmos DB knowledge, for instance, utilizing it to rapidly add embeddings and a vector index to your knowledge, so it may be used as a part of the grounding knowledge for an AI software primarily based on retrieval-augmented era (RAG).
