Skip to main content

Forecasting AI Progress

How prediction markets, scaling laws, and biological anchors attempt to quantify the most consequential uncertainty of our time.

finished certainty: likely importance: 7

In October 2022, Metaculus — the largest calibrated prediction platform — had its community median for “when will we achieve AGI?” sitting at 2040. By March 2024, it had shifted to 2027. That’s a thirteen-year shift in eighteen months.1 No major world event precipitated the change. No new theoretical breakthrough unlocked previously impossible capabilities. What happened was simpler: GPT-4 shipped, Claude got good, and thousands of forecasters updated on the evidence that scaling laws hadn’t hit a wall yet.

This is either the most important information in the world or the most overfit. Figuring out which is the central challenge of AI forecasting.

Why Forecasting Matters

AI forecasting is not an academic exercise. It directly determines:

  • How much we invest in safety research: If transformative AI is 30 years away, we have time for careful, methodical alignment work. If it’s 5 years away, we need to triage ruthlessly.

  • What governance structures we build: Regulations designed for a 2040 world will be obsolete if AGI arrives in 2028. The EU AI Act was largely written assuming current-generation capabilities; it may need wholesale revision.

  • Whether the current safety research agenda is viable: Many alignment approaches — scalable oversight, debate, recursive reward modeling — require significant further development. Whether there’s time to develop them depends on timelines.

The stakes create a perverse incentive: everyone involved in AI has strong reasons to believe timelines that favor their position. Researchers who work on long-term alignment want longer timelines (more time for their work to matter). Labs want shorter timelines (urgency justifies their pace). Investors want whatever timelines make their portfolio look best. Forecasting tries to cut through these biases with calibrated, track-recorded predictions.2

Three Frameworks

Scaling Laws and Extrapolation

The most empirically grounded approach to AI forecasting starts with scaling laws — the observation that model performance improves predictably as a function of compute, data, and parameters. Kaplan et al. (2020) at OpenAI showed that loss decreases as a power law of compute, with no sign of diminishing returns within the range tested. Hoffmann et al. (2022) at DeepMind refined this with “Chinchilla scaling laws,” showing that compute is most efficiently allocated by scaling data and parameters together.3

The forecasting extrapolation is straightforward: if performance scales predictably with compute, and compute availability is growing predictably (due to hardware improvements, investment, and cluster scaling), then you can project when models will reach various capability thresholds. Epoch AI does this systematically, tracking compute trends, hardware prices, and algorithmic efficiency improvements.

The weakness of this approach: scaling laws describe loss on training distribution, not capability on tasks we care about. The relationship between training loss and benchmark scores is noisy and nonlinear. Emergent abilities — capabilities that appear suddenly at certain scales — complicate any smooth extrapolation.4

Biological Anchors

Ajeya Cotra’s biological anchors framework (2020, updated 2022) takes a different approach: instead of extrapolating from current AI progress, it asks “how much compute would it take to train a model with brain-like capabilities?” and then estimates when that much compute will be affordable.5

The framework estimates the human brain performs roughly 10^15 FLOP/s, and that a human lifetime of learning involves ~10^24 FLOP. Depending on which “anchor” you use (and how efficiently AI can use compute relative to biology), transformative AI becomes affordable somewhere between 2030 and 2090, with a median around 2040–102050 in the original report. The 2022 update pulled this forward to ~2035–52040, driven by faster-than-expected hardware scaling and algorithmic improvements.6

The biological anchors framework is valuable less for its specific predictions than for making the structure of the argument explicit. It forces you to state your assumptions about compute requirements, hardware trends, and spending willingness. Disagreements become productive because they’re about specific parameters rather than vague intuitions.

Expert Surveys and Prediction Markets

The third pillar of AI forecasting is aggregating expert and crowd opinion. Major surveys include:

  • AI Impacts surveys: Periodic surveys of ML researchers. The 2022 survey found a median estimate of 2059 for “high-level machine intelligence” (HLMI), with significant variation by respondent demographics and question framing.

  • Metaculus: Calibrated crowd forecasting with track records. Currently the best-calibrated public forecasting platform for AI questions.

  • Prediction markets: Manifold and Polymarket allow real-money or play-money betting on AI outcomes. Markets theoretically aggregate private information, but thin trading on AI questions limits their reliability.

The Survey Framing Problem

Expert surveys suffer from extreme sensitivity to question framing. When Grace et al. (2018) asked ML researchers “when will AI be able to perform any task as well as a human?”, they got a median of 2061. When they asked “when will all human jobs be automatable?”, the median jumped to 2136. These should be approximately the same question, but the 75-year gap reveals how much the answers reflect connotation and anchoring rather than calibrated probability.

The Metaculus approach — decomposing vague questions into specific, operationalizable resolution criteria — is more rigorous but introduces its own problems. The criteria for “AGI” on Metaculus require passing specific benchmarks and capability tests, which means the question is really “when will a model pass these specific tests?” rather than “when will we build something that matters?”

Tetlock’s research on superforecasting suggests that the best forecasters are those who break big questions into smaller, more tractable components, update frequently on evidence, and maintain calibrated uncertainty. Applied to AI: rather than forecasting “when AGI,” forecast “when will models score 90% on GPQA?” or “when will an AI agent successfully complete a complex multi-week project?” These are answerable questions that, in aggregate, tell us something about the bigger one.

What the Forecasts Actually Say

Synthesizing across frameworks:

  • Scaling-law extrapolation suggests frontier capabilities will continue improving at roughly the current rate through at least 2027-2028, conditional on continued investment. What “improving” means at the capability frontier is uncertain.

  • Biological anchors give a median of ~2035–52040 for transformative AI, with wide confidence intervals (10th percentile around 2028, 90th percentile after 2060).

  • Metaculus community prediction for AGI currently sits around 2028-2030, having moved forward dramatically from 2040 in just two years.

  • Expert surveys give longer timelines (median ~2040–202060) but with enormous variance and known framing effects.

The interesting pattern: methods anchored on recent progress (scaling extrapolation, prediction markets) give shorter timelines than methods anchored on theoretical requirements (biological anchors, expert surveys). This could mean the first group is recency-biased, or that the second group is anchored on priors that need updating. Or both.8

The Track Record

How good is the forecasting community at predicting AI progress? The honest answer: mixed.

Successes: Metaculus has a good calibration curve overall — events they assign 70% probability happen about 70% of the time. Specific AI predictions that resolved well include the timeline for models passing the bar exam and achieving expert-level performance on various benchmarks.

Failures: Almost everyone — forecasters, experts, labs themselves — underestimated the pace of progress from 2020–42024. The jump from GPT-3 to GPT-4 was larger than most predicted. The speed of open-source catching up to frontier models was faster. The breadth of capabilities (from coding to image generation to scientific reasoning) surprised even optimists.

The meta-lesson: in periods of rapid capability growth driven by a known mechanism (scaling), extrapolation works well. In periods of paradigm shift (the transition to transformers, the discovery that RLHF makes models much more useful), extrapolation fails because the relevant variables change. We may be in a period where the current paradigm (scaling transformers with RLHF/DPO) is approaching diminishing returns, or we may be at the beginning of a new paradigm (agents, tool use, chain-of-thought). Depending on which, the next five years of progress could be incremental or transformative.

Anyone who tells you they know which is selling something.

Further Reading