Embeddings: definition

An embedding is a list of numbers that captures the meaning of a piece of text, so that passages about similar things sit close together and can be found by similarity rather than exact words.

Embeddings are what make "search by meaning" possible. Convert every document chunk into a vector once, convert the question into a vector at ask time, and the closest vectors are the most relevant passages, even if they share no keywords.

They are the retrieval half of RAG and the backbone of recommendation and deduplication systems. We use the same idea to catch near-duplicate content before it ever gets published.

Related terms

Retrieval-augmented generation (RAG): Retrieval-augmented generation is a technique that fetches relevant passages from your own documents at question time and hands them to the model, so its answer is grounded in your data instead of its training.
Vector database: A vector database stores embeddings and finds the closest matches to a query vector quickly, which is what lets a RAG system retrieve the right passages from millions of documents in milliseconds.
Large language model (LLM): A large language model is a neural network trained on huge amounts of text to predict the next token, which lets it write, summarise, classify, and answer questions across almost any topic.