A typical autoregressive language model builds an answer token by token. It uses the available context to predict possible next pieces, a decoding procedure selects one and that piece joins the sequence used for the next step. There is no finished paragraph waiting to be retrieved simply because the result reads fluently.
That mechanism explains both the flexibility and some of the limits of an LLM. A system trained on varied language can produce many kinds of output through the same interface. But a plausible continuation is not automatically a checked fact, a successful calculation, or an instruction carried out in the world.
Tokens are the pieces the model handles
A token is a unit produced by a tokenizer. Depending on the tokenizer and text, it can correspond to a word, part of a word, punctuation, or another text fragment. Counting visible words is therefore not the same as counting model input.
The Google machine-learning introduction explains language modeling as estimating probabilities over tokens or token sequences. The model processes numerical representations, rather than reading letters in the way a person does. Its prediction supplies a distribution of possible next pieces; a decoding procedure selects among them.
Imagine the unfinished phrase “Please send the report by.” A time, date, or relative deadline might follow. Add “before our Friday meeting,” and the plausible continuation changes. The surrounding text helps determine which output fits. This is a miniature illustration of conditioning on context, not a claim that a real model has only those choices.
Selection is also why one prompt need not produce identical wording every time. Sampling can introduce variation. A more deterministic setting narrows that variation, but it does not turn the generation process into a truth test. Repeating the same incorrect answer consistently would still leave it incorrect.
Training changes the model; a prompt changes its context
During training, optimization adjusts numerical parameters to improve an objective across examples. During ordinary generation, the trained parameters are used to calculate outputs. Supplying an example in a prompt is different from retraining those parameters.
The GPT-3 research paper demonstrated useful performance on varied tasks with instructions and demonstrations provided in context, without gradient updates for each task. This is the idea behind few-shot prompting: show what the requested transformation looks like, then ask for another instance. The paper also reported limitations and methodological concerns. It did not establish universal competence.
For example, suppose you want support messages sorted into three categories. You could give the model the category definitions and a few hypothetical labeled messages. Those examples help specify the task inside that request. They do not establish a permanent new skill or guarantee correct handling of an unfamiliar case.
This distinction is useful when a conversation appears to teach the system something. The information may influence later turns because it remains available in context. Whether a product stores it separately, retrieves it later, or uses it for training is a separate system and policy question.
Attention connects parts of the sequence
Many familiar LLMs use transformer architectures. The original Attention Is All You Need paper introduced an architecture built around attention mechanisms for sequence transduction, initially evaluated on translation tasks. Attention allows representations at different positions to draw information from other positions, subject to the architecture’s constraints.
Consider “The box beside the chairs is heavy.” The sentence contains nearby plural and singular nouns, yet the verb relates to “box.” The example shows why merely considering the immediately previous word would be inadequate. Useful text generation needs relationships across the sequence.
Attention is a computational mechanism for combining information. The term should not be taken as evidence of conscious attention or a human reading experience. Nor does it by itself explain every capability. Training data, model size, objectives, and later tuning all affect the system a user encounters.
The original transformer included an encoder and decoder. A typical causal text generator uses a different arrangement and cannot use future generated tokens to predict the present one. Describing every transformer as the identical chatbot architecture would erase that distinction.
Why a chatbot follows instructions
Pretraining on text does not, by itself, make a model reliably act like a helpful assistant. Continuing a passage and answering the user’s request are related but different goals.
The InstructGPT paper describes additional training with human demonstrations and rankings of outputs. Its evaluated models were better at following the tested users’ intentions, while still making simple mistakes. This provides a concrete example of post-training changing how a pretrained model behaves.
When you ask for “three concise bullets,” the resulting format reflects more than knowledge of what a bullet looks like. The deployed model has also been shaped to respond to instructions. Such shaping can improve usefulness without supplying a guarantee that every sentence is supported.
The application can do more than the model
A chatbot product may add document retrieval, web search, stored memory, or tools. These change the information and actions available to the overall system. They should not be confused with facts already contained in the model’s parameters.
Suppose you ask about a policy issued this morning. A model’s fluent account cannot demonstrate that it has seen the policy. A retrieved document can make the policy available in context, but the answer must still be checked against it. Ask which document was opened and where the relevant statement appears.
Likewise, drafting “I have sent the email” is not evidence of a sent email. An actual sending tool and its result are required. The difference is especially useful when evaluating agents: text about an action and successful execution are separate events.
An LLM is therefore best understood as a trained text-generating model inside a larger application. To judge an answer, ask three concrete questions: what information was available, what transformation was requested, and what independent check would show success? That approach makes the next-token mechanism useful without asking it to do the work of verification.
