Token: definition

A token is the unit a language model reads and writes in, roughly a word-piece of a few characters, and model pricing and limits are counted in tokens rather than words.

Models do not see letters or whole words, they see tokens, which are chunks of text. "Automation" might be two or three tokens. As a rough guide, a token is about four characters of English, and a page of text is a few hundred tokens.

It matters because every model has a context limit in tokens and charges per token. Keeping prompts tight is not just tidy, it is directly cheaper and faster.

Related terms

Inference: Inference is the act of running a trained model to get an output, and inference cost is what you pay each time the model answers, measured in tokens.
Large language model (LLM): A large language model is a neural network trained on huge amounts of text to predict the next token, which lets it write, summarise, classify, and answer questions across almost any topic.