Where AI Value Lands

Agentic Scaling Law

5 min read

Agentic Scaling Law

One-line

Jensen Huang’s claim (cited in karpathy-software-3) that the next leap in AI capability comes not from larger base models but from test-time compute: agents spawning sub-agents to reason, plan, and search. Reasoning is something you buy at inference, not just at training.

The argument

Classical scaling laws (Kaplan, Chinchilla) say capability grows roughly as a power law of training compute and dataset size. Huang’s framing introduces a second scaling axis:

  • Train-time scaling — bigger model, more data, more pre-training compute → marginal returns on next-token prediction.
  • Test-time scaling — at inference, spawn sub-agents, allocate more reasoning budget, use search and planning over tools → returns that compound on top of the base model.

If both axes scale, the effective intelligence available per task grows much faster than either axis alone suggests. This is the operational logic behind reasoning models (o1, o3, Claude with extended thinking), agent swarms, and the “AI factory” architecture: factories are sized for gigawatts of inference, not just training runs.

Where the claim comes from

karpathy-software-3 (Huang material cited as synthesis inside Karpathy’s lecture). Empirically anchored in the post-2024 wave of reasoning-tuned models and multi-agent orchestration frameworks.

Demand-side validation from Sutskever (2026). sutskever-age-of-research argues that pre-training data is exhausted and the frontier of scaling has already shifted to RL and inference-time compute — the same axis Huang named. Sutskever and Huang agree on direction but diverge sharply on methodology: aesthetic top-down research taste vs first-principles-physics co-design (ai-factory-huang). See post-scaling-research-pivot. The cross-source agreement on the axis materially strengthens the agentic-scaling claim; the methodological disagreement is the live question.

Relationship to scaling-wall — direct contradiction (partial)

This is the strongest bull counter to scaling-wall. The two framings disagree about what is scaling:

scaling-wall (Marcus, et al.)Agentic scaling law (Huang)
Axis under examinationTrain-time scaling of base modelsTest-time / agentic scaling
DiagnosisDiminishing returns on reasoning; architectural ceilingA different scaling axis exists and is just starting
Implication for L4–L5 of H1_L0-L7-ladderWall, not staircaseGlide path
Implication for L7Paradigm-shift betOn track

Both can be partially right. Marcus’s critique — that base-model scaling shows diminishing returns and lacks world models / compositionality — does not directly refute the agentic-scaling claim, because the latter operates on top of a (possibly mediocre) base model and adds reasoning at inference time. Conversely, agentic scaling cannot fix problems that are architectural in the base model: a sub-agent built on a model that lacks compositionality inherits that limitation.

The honest synthesis is probably:

  • Agentic scaling reshapes the wall, doesn’t remove it. Many tasks where naive scaling failed now work with reasoning + search. Many do not.
  • The wall is task-distributional. Domains with verifiable intermediate steps (code, math, formal logic) benefit most; domains requiring tacit world models (long-horizon planning in messy physical or social contexts) benefit least.

Why this matters for Where Value Lands

  • Restores the bottom of H2_u-curve-of-value — partially. If test-time compute is a real and durable scaling axis, then inference is the load-bearing economic activity, not training. AI factories (gigawatts of inference compute) become the substrate, and their margin profile is closer to “running tap water” than to “speculative R&D.” This reinforces Huang’s “extreme co-design” claim.
  • Sharpens the bull case for L7 — if you own the energy + silicon + model and inference is where the scaling happens, owning the factory captures most of the durable value.
  • But it also reinforces circular-ai-economy risk. If inference is the durable economic activity but ~70–80% of inference revenue is currently recycled VC pass-through, then agentic scaling is real and the financial flows around it are still fictitious. Both can be simultaneously true.

Why this might be wrong

  • The “scaling law” label is rhetorical. Power-law fits on test-time compute vs. capability are early; calling this a law imports more confidence than the empirical curves justify.
  • Cost scales too. Each sub-agent multiplies inference cost. Tokens-per-second-per-watt becomes the actual bottleneck, which is why Huang himself emphasizes the hardware co-design problem.
  • Verification cost can swamp generation gains. If autonomy-slider is right that human verification is the bottleneck, agentic scaling produces more output that humans cannot meaningfully review — gains in raw capability without gains in deployable capability.
  • Sub-agent error compounding. Multi-agent systems can amplify hallucinations rather than cancel them; empirical evidence on this is mixed and benchmark-dependent.

Open questions

  • Empirical decay curves: how does capability scale with inference compute on hard reasoning benchmarks? Is the curve sub-linear, linear, or power-law?
  • At what cost-per-token does agentic scaling stop being economically viable vs. just hiring a human?
  • Does agentic scaling materially help on the world-model / compositionality failures scaling-wall cites, or does it work only on tasks where intermediate steps are verifiable?

Graph View

Backlinks