The Original Claim

Let’s be precise about what Wei et al. claimed. They defined an emergent ability as one that is “not present in smaller models but is present in larger models.” They surveyed BIG-bench — a massive multi-task benchmark with over 200 tasks — and identified tasks where performance was at or near random for all model sizes below some threshold, then jumped sharply above it.

The data was drawn from multiple model families: GPT-3, PaLM, Chinchilla, LaMDA. The emergent pattern appeared across architectures and training recipes, suggesting it was a property of scale itself rather than any particular model design.²

The rhetorical power of the paper came from its framing. By analogy with emergence in physics — where macroscopic properties like temperature and pressure emerge from microscopic particle interactions — Wei et al. suggested that cognitive capabilities might similarly emerge from the statistical structure of large neural networks. This connected AI scaling to a venerable tradition in the philosophy of science and gave the results a narrative weight beyond their empirical content.

The paper was hugely influential. It was cited thousands of times. It shaped corporate strategy at labs that decided to keep scaling on the bet that new capabilities would appear. It informed government policy discussions about AI risk. It became a central piece of evidence in the “scaling is all you need” worldview.

The Mirage Hypothesis

Then Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo published “Are Emergent Abilities of Large Language Models a Mirage?” — and the discourse shifted.

Their argument was precise and devastating: the apparent phase transitions were artifacts of the metrics used to evaluate performance, not genuine discontinuities in the underlying capability. Specifically:

Non-linear metrics create apparent emergence. Many of the BIG-bench tasks used exact match accuracy — the answer is either perfectly correct or wrong, no partial credit. For a task like multi-digit arithmetic, a model that gets the first three digits right but the last one wrong scores the same as a model that outputs random garbage: 0%. As the model improves smoothly, its token-level accuracy increases gradually, but the exact-match score stays at 0% until the model crosses the threshold where it can get all digits right — then it jumps to 50%, 60%, 80%. The phase transition is in the metric, not the model.³

Changing to continuous metrics dissolves emergence. Schaeffer et al. demonstrated that when you replace exact-match accuracy with continuous metrics — Brier score, token-level accuracy, log-likelihood — the phase transitions disappear. Performance improves smoothly and predictably with scale. There’s no magical threshold. The model was getting gradually better all along; the metric just couldn’t see it until the improvement crossed a discontinuous boundary.

Emergence can be generated by choice of metric. In a clever demonstration, they showed you could make any smooth-improving task appear emergent by choosing the right metric threshold. Want to make “text summarization” look emergent? Define success as “ROUGE score > 0.4” and plot binary success/failure versus model size. You’ll see a phase transition that has nothing to do with a genuine capability jump.

Who’s Right?

I think both sides are partly right, and the synthesis is more interesting than either extreme.

Schaeffer et al. are right that many claimed emergent abilities are metric artifacts. The mathematical argument is sound, and their empirical demonstrations are convincing. Anyone citing Wei et al. as evidence for sudden capability jumps needs to grapple with the metric choice problem. The majority of BIG-bench “emergent” tasks probably reflect smooth improvement rendered invisible by discontinuous metrics.

Wei et al. are right that something interesting happens at scale. Even granting the metric critique, there are capabilities that seem qualitatively different above certain scales in ways that are hard to attribute purely to metric choice. Chain-of-thought reasoning in small models doesn’t just score poorly — it produces incoherent reasoning traces that are qualitatively different from the coherent traces produced by large models. There’s a genuine sense in which 10B-parameter models can’t do multi-step reasoning and 100B+ models can, regardless of how you measure the output.⁴

The truth probably involves a spectrum. Some capabilities improve smoothly with scale (factual recall, grammatical correctness, simple classification). Some show threshold effects that are mostly metric artifacts (multi-digit arithmetic, exact-format following). And some may show genuine qualitative transitions — capabilities that require a minimum level of representational complexity or circuit formation that only becomes available at certain scales.⁵

The analogy to physics is instructive but imperfect. Water-to-ice is a genuine first-order phase transition with a discontinuous order parameter. But most “emergence” in complex systems is more like the way traffic jams emerge from individual driving behavior — a qualitative macroscopic phenomenon arising from quantitative microscopic changes, without a sharp phase boundary. LLM capabilities may be more like traffic jams than ice crystals.

What The Scaling Laws Tell Us

The question of emergence is deeply connected to the scaling laws that have become the central organizing principle of modern AI research. Kaplan et al. (2020) showed that language model loss decreases as a smooth, predictable power law of compute, data, and parameters. Hoffmann et al. (2022) refined these laws with the Chinchilla scaling analysis.

The loss itself — the perplexity, the cross-entropy — is smooth. There are no phase transitions in loss curves. This is well-established and uncontroversial.⁶

But loss is an aggregate measure. It averages over all tokens, all tasks, all domains. A smooth decrease in aggregate loss is compatible with sharp transitions on specific tasks, just as a smooth decrease in national unemployment is compatible with sudden plant closures in specific towns. The scaling laws tell us that the average capability improves smoothly. They don’t tell us that every specific capability improves smoothly.

Ganguli et al. (2022) found this explicitly: while aggregate performance scales predictably, individual tasks can show highly non-linear behavior. Some tasks improve faster than the aggregate trend; some improve slower; a few show apparent thresholds. The aggregate smooth scaling and task-level heterogeneity are both real.

Why It Matters

The emergence debate isn’t merely academic. It has direct consequences for:

AI safety. If capabilities emerge unpredictably, then each scaling step is a roll of the dice — you might get harmless improvements, or you might suddenly get a model that can autonomously hack computer systems. The case for caution and extensive pre-deployment testing is much stronger if emergence is real. If capabilities scale smoothly, you can predict what the next model will be able to do, which makes AI evaluation and safety testing much more tractable.⁷

Compute allocation. If you believe in emergence, the rational strategy is to keep scaling until you hit the next capability threshold — spend $100M on training because the capability you want might be just around the corner. If you believe in smooth scaling, the rational strategy is to invest in evaluation infrastructure and task-specific measurement so you can predict whether more compute will actually help. Labs have collectively spent billions based partly on emergence-inspired reasoning.

AI governance. Should frontier AI systems require pre-deployment safety testing? If capabilities emerge unpredictably, yes — you literally can’t know what a model can do without testing it. If capabilities scale smoothly, testing is still valuable but less critical — you have some basis for predicting capabilities from smaller-scale experiments. The EU AI Act and similar regulations implicitly assume something like the emergence view.

My View

I lean toward a synthesis that takes the best of both sides. The mirage hypothesis is correct in its narrow claim: many apparent emergent abilities dissolve when you use better metrics. Schaeffer et al. performed a valuable service in forcing the field to be more rigorous about claims of emergence.

But I think the strong version of “emergence is entirely a mirage” goes too far. I’ve personally observed qualitative differences in model behavior at different scales that are hard to dismiss as metric artifacts. A 1B-parameter model doesn’t just do chain-of-thought poorly; it does it incoherently, in a way that suggests the underlying capability simply doesn’t exist yet. A 100B-parameter model produces chains of thought that are recognizably reasoning, even when they reach wrong conclusions. That qualitative shift is real, even if it’s continuous at the token-probability level.⁸

The resolution I find most convincing: capabilities accumulate smoothly in the model’s internal representations, but their expression on specific tasks can be non-linear due to the interaction between capability level and task difficulty. There’s no magical threshold where “reasoning” appears. But there is a threshold where the model’s reasoning capability becomes sufficient to solve a particular class of problems, and that threshold can look like emergence on a task-by-task basis.

This is less exciting than “phase transitions in intelligence” but more useful than “it’s all a mirage.” The practical implication: invest in evaluation, measure carefully, and expect gradual improvement with occasional surprises. Which, come to think of it, is just good engineering advice regardless of what you believe about emergence.