Context window: definition

The context window is the maximum amount of text, measured in tokens, that a model can consider at once, covering the instructions, the conversation, and any documents you provide.

Everything the model "knows" in a given call has to fit in its context window: your system prompt, the chat history, and any data you paste or retrieve. Go over the limit and the oldest content falls out.

Larger windows let you show more at once, but filling them is slower and costlier, which is why retrieval (showing only the relevant few passages) usually beats dumping everything in.

Related terms

Token: A token is the unit a language model reads and writes in, roughly a word-piece of a few characters, and model pricing and limits are counted in tokens rather than words.
Retrieval-augmented generation (RAG): Retrieval-augmented generation is a technique that fetches relevant passages from your own documents at question time and hands them to the model, so its answer is grounded in your data instead of its training.
Inference: Inference is the act of running a trained model to get an output, and inference cost is what you pay each time the model answers, measured in tokens.