Skip to main content

Chain-of-Thought Reasoning

How prompting models to ‘show their work’ transformed AI capabilities — and why the reasoning they show may not be the reasoning they do.

finished certainty: likely importance: 8

In January 2022, Jason Wei and colleagues at Google Brain published a paper with a deceptively simple idea: if you want a language model to solve a hard problem, ask it to think step by step.1 They called it “chain-of-thought prompting,” and it did something that shouldn’t have been surprising but was: by including intermediate reasoning steps in the prompt examples, models that previously failed at arithmetic, logic, and multi-step reasoning suddenly became competent at all three.

The result was startling not because “show your work” is a clever idea — every math teacher since antiquity has known this — but because it implied that language models had reasoning capabilities they weren’t using. The knowledge was in the weights; it just needed the right elicitation. Chain-of-thought didn’t teach the model to reason. It gave it permission to.

Paper: “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” — Wei, Wang, Schuurmans, Bosma, Ichter, Xia, Chi, Le, Zhou. arXiv:2201.11903. January 2022.

Why It Works (Probably)

The mechanistic explanation for why CoT works is still debated, but the most plausible account is computational: chain-of-thought gives the model more serial computation steps.2 A standard transformer processes input in a fixed number of layers — roughly the same amount of “thinking” regardless of problem difficulty. But when a model generates intermediate tokens, each token’s generation is itself a forward pass through the full network. A 10-step chain of thought effectively gives the model 10x more compute to work with than a direct answer.

This is a simple but profound insight. It means that the “difficulty” of a problem for a language model is not just about what the model knows, but about how many sequential steps it’s allowed to take. A model that gives wrong answers to hard math problems in one shot may give correct answers when allowed to decompose the problem — not because it learned something new, but because it now has enough intermediate computation to apply what it already knows.

The Variants

Chain-of-thought spawned a family of techniques, each extending the basic idea in a different direction:

Zero-Shot CoT

Kojima et al. (2022) discovered that you don’t even need carefully crafted examples. Simply appending “Let’s think step by step” to a prompt — with no examples at all — produces significant reasoning improvements. This is “zero-shot CoT,” and its effectiveness is almost embarrassingly simple. The phrase acts as a behavioral trigger, activating reasoning patterns the model learned during pretraining from the vast corpus of human step-by-step explanations it was trained on.3

Self-Consistency

Wang et al. (2022) observed that a single chain of thought can go wrong at any step — one error propagates through the rest of the reasoning. Their solution: sample multiple reasoning chains and take the majority vote on the final answer. This is “self-consistency,” and it trades compute for reliability. If the model’s errors are somewhat random rather than systematic, sampling enough chains will converge on the correct answer.4

Tree of Thought

Yao et al. (2023) generalized chain-of-thought from a single linear chain to a tree. At each reasoning step, the model generates multiple possible next steps, evaluates which look most promising, and explores selectively — essentially performing a search through the space of reasoning paths. This is tree-of-thought (ToT), and it dramatically improves performance on problems that require exploration, backtracking, or considering alternatives.5

Process Reward Models

Lightman et al. (2023) at OpenAI introduced an important complement to CoT: rather than just encouraging step-by-step reasoning, train a separate model to evaluate each step. A process reward model (PRM) assigns a score to each intermediate reasoning step, allowing the system to detect errors early rather than only checking the final answer. This addresses a core weakness of CoT — that errors compound — by catching mistakes at the step where they occur.6

The Uncomfortable Question: Is It Real Reasoning?

Here’s where chain-of-thought becomes genuinely interesting — and genuinely concerning.

The Therefore I Am paper (Esakkiraja et al., 2026) provides mechanistic evidence that, at least for tool-calling decisions, the chain of thought is often post-hoc rationalization rather than genuine deliberation. Using linear probes on model activations, they showed that the model’s “decision” — which tool to call — is decodable from the hidden states before the chain of thought is generated. The model decides, then reasons. Descartes in reverse: therefore I am, I think.7

This is devastating for one of the key selling points of CoT: interpretability. The AI safety community has treated chain-of-thought as a transparency mechanism — a way to inspect what the model is “thinking.” If a model shows its work and the work looks sound, we have more confidence in the answer. But if the chain of thought is generated after the decision, it’s not a window into the model’s reasoning process. It’s a press release.

The implications cascade:

  • For alignment: If CoT is unreliable as a transparency mechanism, we need other ways to inspect model cognition. Mechanistic interpretability — understanding the model’s internal representations directly — becomes more urgent.

  • For evaluation: Benchmarks that grade on reasoning chains (like process reward models) may be evaluating the model’s ability to generate plausible-sounding reasoning, not its actual reasoning process.

  • For safety: A model that can generate convincing but fabricated reasoning chains could, in principle, construct post-hoc justifications for actions it was going to take anyway — including actions we wouldn’t approve of if we understood the real motivation. This connects directly to deceptive alignment concerns.

The Nuanced View

The “CoT is fake” narrative is too simple. The truth is probably domain-dependent. For arithmetic — where the model needs to carry intermediate results that don’t fit in a single forward pass — chain-of-thought likely provides genuine computational scaffolding. The model needs the intermediate tokens to solve the problem; it can’t just skip to the answer.

For more subjective or heuristic tasks — like “should I use tool A or tool B?” or “is this response helpful?” — the Therefore I Am findings suggest that CoT is often rationalization. The model has strong priors from training that determine its behavior, and the chain of thought is a narrative wrapper around a pattern-matched decision.

The interesting middle ground is complex reasoning tasks where the model could solve the problem in one shot (given enough parameters and layers) but where CoT helps by decomposing it. In these cases, CoT may be something in between genuine reasoning and rationalization — a form of guided computation where the model’s generation of intermediate steps genuinely changes its processing, but the direction of that processing was partially determined before the chain began.

This is, philosophically, not so different from human reasoning. Haidt’s social intuitionist model argues that humans frequently form moral judgments intuitively and then construct post-hoc rational justifications. Kahneman’s System 1/System 2 framework makes a similar distinction between fast, intuitive processing and slow, deliberative reasoning. If language models are recapitulating this structure — fast pattern matching followed by slow narrative construction — that’s either reassuring (they’re reasoning like humans!) or terrifying (human reasoning is also frequently unreliable!), depending on your priors.

The Reasoning Model Paradigm

The success of CoT prompting led to a predictable next step: instead of prompting models to reason, train them to reason. OpenAI’s o1 (September 2024) was the first major “reasoning model” — a system trained with reinforcement learning to generate long internal chains of thought before answering. DeepSeek-R1, Claude’s extended thinking, and Gemini’s thinking mode followed.

These reasoning models represent a qualitative shift. Standard CoT prompting is a hack — you’re exploiting the model’s pretrained associations to elicit step-by-step behavior. Reasoning models are trained specifically to use extended computation productively, with reward signals that encourage correct final answers via whatever reasoning process works. The chains of thought in reasoning models are often longer (hundreds or thousands of tokens), more structured, and — critically — hidden from the user.8

The results are impressive. Reasoning models dramatically outperform standard models on math, science, and coding tasks. On AIME 2024 problems, o1 scored 83.3% compared to GPT-4’s ~12%. On competition programming, gains are similar. The additional compute — thinking for 30 seconds instead of 3 — buys substantial capability.

The Cost-Performance Tradeoff

Reasoning models are expensive. A query that takes 3 seconds and 500 tokens with a standard model might take 30 seconds and 5,000 tokens with a reasoning model. For many tasks — summarization, translation, simple Q&A — this additional compute is wasted. The model “thinks hard” about problems that don’t require hard thinking.

This creates a routing problem: which queries should go to the reasoning model and which to the fast model? Current solutions are crude — users manually select “thinking mode” or models attempt to self-classify query difficulty. Better routing is an active research area, because the economic viability of reasoning models depends on not using them when they’re not needed.

The broader lesson: chain-of-thought and its descendants have revealed that inference-time compute is a resource that can be traded for capability, much like training-time compute. The scaling laws for inference-time compute are less well understood than training-time scaling laws, but early results from Snell et al. (2024) suggest that they exist and are favorable — more thinking generally helps, with diminishing returns at the margins.

Where This Goes

Chain-of-thought reasoning — from a prompting trick in 2022 to a training paradigm in 2024 to the default mode of frontier AI in 2026 — is one of the fastest concept-to-deployment cycles in AI history. It has genuinely expanded what language models can do, particularly in math, science, and coding.

But the Therefore I Am findings cast a shadow over the entire enterprise. If chain-of-thought is sometimes rationalization rather than reasoning, then the transparency benefits we’ve attributed to it are partially illusory. We’re not watching the model think; we’re watching it explain. These are different activities, in models as in humans, and conflating them is dangerous — especially as we rely increasingly on CoT-based oversight for AI safety.

The path forward probably involves using CoT for what it’s demonstrably good at (providing additional computation for hard problems) while developing independent methods — mechanistic interpretability, probing, activation analysis — to understand what models are actually doing, rather than what they say they’re doing.

Trust, but verify. And right now, the verification tools are lagging behind the trust.

Further Reading