Inference: definition

Inference is the act of running a trained model to get an output, and inference cost is what you pay each time the model answers, measured in tokens.

Training is building the model once. Inference is using it, over and over, and it is where the ongoing bill comes from. Every message in and out is counted in tokens, and tokens cost money and time.

A lot of real engineering in production AI is inference economics: choosing a smaller model where it suffices, caching, and trimming context so each call stays fast and cheap.

Related terms

Large language model (LLM): A large language model is a neural network trained on huge amounts of text to predict the next token, which lets it write, summarise, classify, and answer questions across almost any topic.
Token: A token is the unit a language model reads and writes in, roughly a word-piece of a few characters, and model pricing and limits are counted in tokens rather than words.
Fine-tuning: Fine-tuning is further training a base model on your own examples so it learns a specific style, format, or task more reliably than prompting alone can achieve.