Vector Database
Definition. A storage system purpose-built for high-dimensional vectors that supports approximate nearest-neighbour search at scale, usually through an index such as HNSW, IVF-PQ or DiskANN. Beyond the index it stores metadata for filtering, supports hybrid queries combining vector similarity with structured predicates, and exposes CRUD operations so embeddings can be updated as source documents change.
How it works. Define a collection with a fixed dimensionality matching your embedding model, a distance metric, and a metadata schema. Insert vectors in batches with their source identifiers. Build the index after bulk load rather than during it. At query time pass the query vector plus a metadata filter and a top-k value, then rerank the results with a cross-encoder if precision matters. Plan for reindexing: keep the raw text and the embedding model version alongside each vector so you can rebuild.
Example. A legal research product stores 4 million clause embeddings in a managed vector database with HNSW indexing. Queries filter on jurisdiction and effective date before the vector search, which reduces the candidate set from 4 million to 300,000 and cuts p95 latency from 240 ms to 60 ms. A nightly job re-embeds only documents whose hash changed, keeping incremental cost near zero.
Use case. Use a vector database whenever the corpus exceeds what you can hold in memory or when you need filtered semantic search, multi-tenant isolation, or live updates. For a few thousand documents a local index or even a NumPy array is sufficient and simpler.
Warnings. Approximate search trades recall for speed; tune the recall parameter against a labelled query set rather than trusting defaults. Deleting a document from the source system does not delete its vectors unless you implement that explicitly, which is a common cause of stale or leaked answers.
Limitations. Vector databases are poor at exact matching, numeric range queries and boolean logic, so they complement rather than replace a relational or search engine. Index build time and memory footprint grow with dimensionality, and multi-tenant filtering can degrade performance badly if the index is not partitioned by tenant.
| Pros | Cons |
|---|---|
| Supports similarity search and metadata filtering at scale. | Exact matches and numeric filters may still belong in a relational database. |