Embedding
Definition. A dense numeric vector, typically 384 to 3,072 dimensions, that represents the meaning of a piece of text in a continuous space where geometric proximity corresponds to semantic similarity. Embeddings are produced by an encoder model trained with contrastive objectives so that paraphrases land close together and unrelated text lands far apart. They are the substrate of semantic search, clustering, deduplication, recommendation and retrieval-augmented generation.
How it works. Choose an embedding model, chunk your corpus to a size the model handles well (often 256 to 512 tokens with overlap), embed every chunk once, and store the vectors with their source metadata in a vector index. At query time embed the question with the same model and retrieve the nearest neighbours by cosine similarity or dot product. Critically, the query and the documents must use the same model and the same normalisation, or distances become meaningless.
Example. A knowledge base of 60,000 support articles is chunked into 240,000 passages of about 400 tokens. Each is embedded into 1,536 dimensions and stored in a vector database with article ID, section heading and last-updated date. A user asks how to reset a two-factor device. Keyword search returns nothing because the article says authenticator app. The embedding query returns the correct article at rank one with a cosine similarity of 0.83, and the answer is generated from that passage.
Use case. Use embeddings for any task where you need to find things by meaning rather than exact wording: semantic search, related-content recommendations, ticket deduplication, clustering of feedback, anomaly detection on text, and as the retrieval half of RAG. They are also the cheapest way to build a classifier when you have few labelled examples, by comparing against labelled exemplar vectors.
Warnings. Embeddings are not interpretable and cannot be reversed to recover the original text reliably, but they do leak information and should be treated as sensitive data. Re-embedding an entire corpus after a model change is expensive and invalidates all stored vectors, so pin the model version in your schema.
Limitations. Embeddings compress meaning lossily: negation, numbers, and entity-specific detail are frequently lost, so a passage saying the policy does not apply can sit close to one saying it does. They have no notion of recency or authority, and similarity is not relevance, which is why production retrieval combines vector search with keyword search and a reranker.
| Pros | Cons |
|---|---|
| Finds semantic similarity across wording changes. | May lose negation, numbers, and specific entity details. |