How Large Language Models Are Trained - And What "Training" Actually Means

Most explanations of how large language models are trained collapse several distinct processes into a single idea. Everything gets labeled "training," and the result is confusion - especially when people try to reason about accuracy, confidence, or model improvement.

In reality, there are two phases that actually change the model, and a third mechanism that only steers behavior at runtime. Treating these as separate is essential for understanding what language models can fix, what they cannot, and why some errors persist even after visible improvements.

The first phase: pretraining

Pretraining is the foundational stage. It is where the model learns the structure of language itself.

During pretraining, the model is exposed to a massive volume of text and optimized for a single objective: predicting the next token. There is no notion of truth, correctness, or understanding built into this process.

Once pretraining is complete, the model has a vast internal map of how language tends to continue - but no inherent mechanism for distinguishing solid knowledge from fragile inference.

The second phase: post-training and alignment

Post-training reshapes behavior rather than knowledge. Human feedback is introduced to steer which kinds of responses are preferred, discouraged, or avoided.

This phase influences tone, caution, instruction-following, and refusal behavior, but it does not install a truth detector. The model still generates language the same way.

The third component: runtime steering

Runtime steering is not training. It does not update the model or persist beyond the current interaction.

Instead, it biases which already-learned continuations are selected in the moment, explaining why the same model can behave differently across prompts, products, or conversations.