Foundation Models

Confidence 0.88 · 34 sources · last confirmed 2026-10-25

Large pretrained models — typically transformer-based — that serve as the substrate for downstream AI applications via prompting, fine-tuning, or retrieval. Foundation models drive most modern generative-ai capability and are the locus of the “frontier” debate (capability, safety, transparency). In the software-3.0 framing they are the interpreter — the runtime that executes context-window programs.

Their capability profile is jagged — see jagged-frontier for the task-level observation (Dell’Acqua et al. 2026) and the cause-of-jaggedness mechanism (Andrej Karpathy in Sequoia AI Ascent 2026: “LLMs automate what you can verify” — verifiability + labs-care editorial choices about RL training data combine to produce uneven peaks and valleys).

Working definition

A foundation model is a model trained on broad data at scale that can be adapted to a wide range of downstream tasks. The term, coined by Stanford’s Center for Research on Foundation Models (CRFM), foregrounds the adaptation role — the model is a foundation other things are built on, not the end product.

Frontier model is a near-synonym foregrounding capability (top of the leaderboard). The two terms diverge in policy/regulatory contexts: “frontier” often implies the regulator-relevant subset of foundation models above a capability threshold.

Key claims

Industry has decisively taken the frontier

• ~90% of notable AI models in 2024 came from industry (vs. 60% in 2023); 91.18% in 2025 per the AI Index 2026 update. Academia remains the leading source of highly cited research, but not of new notable models.
• 2024: U.S. 40 notable models, China 15, Europe 3.
• 2025 (AI Index 2026): U.S. 59, China 35, South Korea 8, Europe 2.
• Top organizations 2025: OpenAI 20, Google 14, Alibaba 11, Anthropic 7, xAI 5, DeepSeek 4, LG AI Research 4, Meta 4, Tsinghua University 4 (only academic institution in the top 9), ByteDance / Moonshot / Nvidia 3 each.
• The U.S.-China performance gap has effectively closed: DeepSeek-R1 briefly matched the top U.S. model in Feb 2025; as of March 2026, Anthropic’s top model leads by just 2.7%.

Compute is scaling fast

• Training compute for notable models doubles every ~5 months.
• Dataset size for training LLMs doubles every ~8 months.
• Power required for training doubles annually.
• Global AI compute capacity reached 17.1M H100-equivalents by 2025, growing 3.3× per year since 2022 (AI Index 2026 §1.2).
• Nvidia: >60% of total compute; Google + Amazon supply most of the remainder; Huawei holds small but growing share.
• U.S. hosts 5,427 AI data centers — more than 10× any other country.
• TSMC fabricates almost every leading AI chip; TSMC-U.S. expansion began operating in 2025.
• AI data center power capacity: 29.6 GW — comparable to New York state at peak demand.
• Carbon emissions trajectory:
  • AlexNet (2012): 0.01 tons
  • GPT-3 (2020): 588 tons
  • GPT-4 (2023): 5,184 tons
  • Llama 3.1 405B (2024): 8,930 tons
  • Grok 4 (2025): 72,816 tons (AI Index 2026)
  • (Reference: average American = 18 tons/year.)
• Annual GPT-4o inference water use alone may exceed the drinking water needs of 1.2 million people.

Disclosure is dropping (AI Index 2026)

• Training code, dataset sizes, parameter counts increasingly withheld for the most resource-intensive systems including those from OpenAI, Anthropic, Google.
• Reported parameter counts have stayed near 1 trillion for three years as disclosure dropped — but training compute (estimable independently from hardware) has continued to rise.
• OLMo 3.1 Think 32B with ~90× fewer parameters than Grok 4 achieves comparable benchmark results via pruning, deduplication, curation alone — evidence that data quality and post-training matter as much as scale.

Open-weight catching up to closed-weight

• Performance gap between top closed-weight and top open-weight models on the Chatbot Arena Leaderboard: 8.0% in early Jan 2024 → 1.7% by Feb 2025 on some benchmarks.
• The frontier is also tightening overall: top-2 model gap = 0.7%, top-10 gap = 5.4% (down from 4.9% / 11.9% the year before).

U.S.-China performance gap narrowing fast

• End-2023 vs. end-2024 gaps on major benchmarks:
  • MMLU: 17.5pp → 0.3pp
  • MMMU: 13.5pp → 8.1pp
  • MATH: 24.3pp → 1.6pp
  • HumanEval: 31.6pp → 3.7pp

Transparency improving

• Foundation Model Transparency Index (CRFM): avg score among major developers 37% (Oct 2023) → 58% (May 2024). Substantial progress, ~42% gap remaining. See responsible-ai.

Smaller is mighty

• 142× model-size reduction for the same MMLU >60% threshold in two years: PaLM (540B params, 2022) → Phi-3-mini (3.8B, 2024).

Models converging to commodity status: the “rent vs own” framing (Chatterjee 2026 + Kokane 2026)

By April–May 2026, practitioner writing on agent harnesses had converged on a sharp framing of foundation models as commercial inputs to differentiated products rather than as the differentiator itself:

“Models are converging toward commodity status. They are rented from vendors whose competitors will outperform them within the year. They are swapped between, routed across, replaced quarterly. Two years ago, choosing a foundation model felt like choosing a database. Today it feels closer to choosing a CDN. The decision still matters, but the moat it produces lasts months, not years.” — Chatterjee 2026

The framing has three implications worth flagging:

1. The CDN analogy is sharper than prior wiki framings (database, library, API). It captures the speed of substitution: you do not architect a multi-quarter CDN selection process; you keep your stack CDN-agnostic and switch when the price/performance gradient justifies it.
2. Layer-4-swappable architectures are now the practitioner default. Kokane’s harness diagram explicitly labels the model layer as “swappable backends” — Azure OpenAI/GPT-5, Claude (Anthropic), Gemini/Llama/Ollama, AWS Bedrock — selected at runtime. This is consistent with the open-weight performance gap closing trend documented above (1.7% on Chatbot Arena by Feb 2025).
3. Where competitive advantage moves: from model selection to the harness layer (memory architecture, quality contracts, learned per-customer overrides, telemetry). The “rent vs own” aphorism: “The model is what you rent. The harness is what you own.” See agent-harness for the full treatment. As of May 2026 this framing now has its first empirical anchor: an optimised harness from Khattab et al. (DSPy team) transferred across five different foundation models without re-optimisation, lifting all of them — turning “plan for swap” from a prudential design hint into a measured property of the harness layer.

Caveat against over-commoditization narrative: Anthropic Managed Agents documented that Claude Sonnet 4.5’s “context anxiety” failure mode was absent on Opus 4.5 — same family, different reliability. Within-family variation can be large enough that the “models converging” framing needs nuance: the frontier converges in benchmark performance, but production-relevant reliability characteristics (long-horizon stability, refusal behaviour, cost/latency curves) can vary substantially even between sibling models. The “plan for swap, not for marriage” prescription survives the caveat — but harness designers should expect to discover model-specific quirks at production scale and budget for them.

The pricing scissors quantified ( AWS Sydney 2026)

Brovich puts numbers on the commodity dynamic above: training costs rising ~2.4× per year, inference costs falling ~10× per year, so the gap — the “pricing scissors” — opens by 12–24× per year. Practical consequence: creating a frontier model approaches the billions (a handful of firms can), while using one collapses toward zero. This is the economic engine behind the USE/COMPOSE/BUILD decision (already on the Allen page) — build is justified only where differentiation outweighs the collapsing cost of compose. A second AWS strategist independently delivering the same framework strengthens the wiki’s read that “rent the model, own the harness” is settled vendor doctrine, not one strategist’s pitch.

Cross-model deliverable comparison on real work (GDPval, OpenAI 2025)

GDPval supplies a rare head-to-head, deliverable-quality comparison across frontier families on real economic tasks (not academic benchmarks). On the gold subset, Claude Opus 4.1 was the best of GPT-4o / o4-mini / o3 / GPT-5 / Claude Opus 4.1 / Gemini 2.5 Pro / Grok 4, with 47.6% wins-or-ties vs human experts. The model personalities are the load-bearing detail for the commodity-vs-differentiated debate above: Claude excels at aesthetics and file formats (.pdf/.xlsx/.ppt), GPT-5 excels at accuracy and instruction-following; Gemini and Grok most often failed by not delivering or using the wrong format. So even as raw benchmark scores converge, models retain distinct deliverable profiles — a reason the “swap freely” framing needs the same nuance the within-family caveat raises. GDPval also confirms performance is improving roughly linearly over time, consistent with the frontier-keeps-moving picture.

Notable foundation model series (mentioned via AI Index 2025)

To be promoted to standalone entity pages when discussed in depth in another source. Currently noted here as a roster:

• OpenAI: GPT-4, GPT-4o, o1, o3 (test-time compute reasoning), SORA (video).
• Google DeepMind: Gemini family (Gemini-1.5-Flash-8B is the 280×-cost-reduction marker), Veo 2 (video).
• Anthropic: Claude 3 family (incl. Sonnet — implicit-bias study); Claude Sonnet 4.5 (predominant model in AEI 4 sample, exhibits “context anxiety” per Managed Agents post); Claude Opus 4.5 (released between AEI 4 and AEI 5 windows; the Managed Agents post notes the “context anxiety” failure mode is absent on Opus 4.5 — first evidence in this wiki of long-horizon reliability varying by model within a family, not just across families); Claude Opus 4.6 (released coincident with AEI 5 sample window, Feb 2026).
• Meta: Llama 3.1 405B (the 8,930-ton-CO2 marker), Movie Gen (video).
• Microsoft: Phi-3-mini (the 3.8B-param-MMLU marker).
• Mistral AI: French open-source.
• xAI: Grok family.

The commodity-utility, not platform value-capture thesis ( Lenny’s Podcast May 2026)

Evans (May 2026) poses the wiki’s most direct independent-analyst-altitude challenge to the foundation-models-as-the-value-capture-layer presumption:

“The models don’t seem to have network effects. So there doesn’t seem to be a winner-takes-all effect where one of these will run away ahead of the others. So you should have competition indefinitely. You have competition indefinitely. You don’t have really radical differentiation in what the product is. Then why would you have pricing power?”

The structural analogy: foundation models look more like AWS than Windows — every developer doesn’t have to standardise on one because the customers don’t care which one runs underneath. “If you’re an engineering company or a law firm buying a piece of software, you don’t care which cloud it runs on and you don’t have to standardize on AWS because that’s where all the software is and like the developers all standardize on AWS because all the customers use AWS. That’s not how it works. That’s how Windows OS works, but that’s not how AWS works.”

Sam Altman’s “AI on a meter like water or electricity” line gets Evans’s response: “my dear sweet child, you need me to explain the margin structure of the utility industry to you … when you watch television the TV company isn’t paying a percentage of your monthly bill to the electricity company, you know, when you wash your clothes, Bosch isn’t paying a percentage of the price of the washing machine.”

The telecom worked example Evans offers as the precedent: “Global mobile industry has revenue of about a trillion dollars a year, maybe a bit more now. And it spends about $200 billion a year on capex every year. Total telecoms is about 300 … if you look at a chart of mobile data consumption, it’s an exponential curve like perfect curve going straight up. And then the number now I think it’s about 1500 to 2000 times what it was in 2010 globally. And the stocks have gone nowhere in 25 years because it’s an X-growth low-margin commodity utility.”

The structural implication Evans names: “What is the steady state equilibrium point where all of these lines on the chart kind of get lined up and we don’t have this kind of weird crazy stuff going on, and then will you have pricing power or have you got like three or four or five companies kind of all selling the same thing — and so then you should have a pricing — you should have lower pricing and lower margins and the value got — stack.”

The under-asked question Evans poses (in response to “what’s a question about AI that nobody’s asking?“): “I’m not sure how many people are asking whether model labs have pricing power. I think a lot of people are just presuming that the situation today will continue or that of course they will.”

The thesis sits in productive tension with foundation-models-as-the-platform readings (Spiegel’s software-is-not-a-moat makes the same argument from the application-layer side at a different altitude; the agent-harness concept’s “harness wraps a (rented, swappable) foundation model” framing is structurally compatible — the model is the commodity, the harness is the differentiated layer). The pricing-power-or-not question is genuinely open in 2026; Evans is the wiki’s first explicit no-they-won’t-have-pricing-power voice at independent-analyst altitude.

The beneath-vs-above-the-model + open-source-substitution buyer view ( Snowflake 2026)

Ramaswamy (CEO, Snowflake — a major buyer and host of foundation models) corroborates the commodity-pressure thesis from the platform-vendor chair: open-source models are “getting better and better,” Snowflake hosts them alongside the frontier models, and “if they’re as good, our customers will want to use them instead of the frontier model.” He frames the open strategic question as what stays “beneath the model” vs “above the model” — will disruption land on the layer close to end-users, or at a lower layer? This is the same the-model-is-the-swappable-commodity, the differentiated value is elsewhere structure as Evans’s commodity-utility thesis and the agent-harness rented-model framing, now stated as a procurement reality (a buyer that will switch to open weights the moment they reach parity). It also pairs with the Ng optionality discipline — both treat model choice as fluid and lock-in as a liability.

The open-weight-at-scale switch, and China’s open-model lead ( Hugging Face, July 2026)

Delangue (Hugging Face CEO — host of nearly 3 million public open-weight models) turns the commodity-substitution thesis above into an operational adoption pattern: enterprises start on frontier APIs, then switch to open-weight models at production scale as frontier cost bites — the demand-side cost curve behind Ramaswamy’s “if they’re as good, customers will use them instead.” He adds two data points the wiki’s foundation-model material otherwise lacks: (a) the open-weight-vs-closed capability gap is narrow (echoing Huang’s Nemotron 3 Ultra 86% vs. Claude Opus 87% at ~10× lower cost); (b) per Hugging Face’s Spring 2026 report, Chinese models are ~41% of downloads — the plurality, surpassing the US — with named open families (GLM 5.2, Nemotron, OpenAI’s open GPT) as the off-the-shelf starting points teams then post-train. The full own-vs-rent / sovereignty / concentration-of-power treatment lives on the new open-source-ai concept; here it sharpens the “open-weight closing the gap” debate bullet below into a dated, quantified corpus datapoint.

The scaling story told from inside a Stanford course — including where it stopped ( CS329A, August 2026)

A compact instructional account of the three scaling axes (compute, dataset size, parameter count), with the trajectory given concretely — BERT 340M → GPT-2 1.5B → GPT-3 175B → PaLM 540B → GPT-4 “estimated… trillions” — and, notably, a stated end point: this held “until last year, where this was starting to hit some kind of a saturation point.” (Delivered September 2025.)

Two capabilities are named as what scale bought: few-shot and zero-shot learning — “earlier you had to fine-tune the model for [a] specific domain, but just by giving a few examples the model can follow that template” — and emergent behaviour, where “capabilities like reasoning only emerge in larger models.” Mirhoseini’s framing of why emergence matters epistemically is the part worth keeping: beyond “predictable scaling laws… there are new behaviours that appear in the model that they didn’t have before, and we never could predict that until we had these bigger models.” Chain of thought is the canonical case — LaMDA and GPT at ~7–8B “can’t really benefit from chain of thought, it doesn’t do anything for them” — with modular arithmetic and word-unscrambling appearing abruptly at scale.

The lecture is also unusually clear that ChatGPT’s advantage over GPT-3 was post-training, not size: high-quality fine-tuning data (“data that companies pay millions, or hundreds of millions of dollars, to buy”), instruction tuning, and RLHF, where “instead of just creating some supervised prompt and labels, we are creating a reward model out of human preferences,” with reward types weighted per objective (correctness, helpfulness, specificity, harmlessness). Their own verdict on alignment is worth recording verbatim: “we try to steer the AI models to follow the goals and preferences and values of humans. And this is still a big problem. It’s a big issue. We haven’t mastered it yet.”

The pivot the course is built on: “pre-training and then fine-tuning were the big pieces. But since a year and a half ago, it turns out inference is also a frontier for making the models much more capable” — with the self-improvement loop closing when test-time-scaled generations become training data for the next fine-tune. Per Lifecycle rules confidence lifts only 0.85 → 0.87: this is instructional synthesis of published results rather than new evidence, and the eleven-month publication lag makes its state-of-the-art claims dated.

Open weights as malleability, and the two-regime stack ( Sequoia, August 2026)

Sonya Huang’s contribution to this page is a mechanism rather than a market observation. Her argument for why owning weights can now beat a frontier API in-domain rests on malleability, not on the base model being better: “because the weights are available, they’re actually much more malleable than working with the closed APIs. And so you start with a baseline that’s already close to frontier” — naming Kimi K3 and GLM-5.2 as the 2026 releases that made the baseline close enough — “and then with strong post-training, prompt harness engineering, online learning, you can actually reach better than frontier performance.”

The corollary she states is the useful part for this page: in the closed-API regime “you’re not really having to collect a ton of data, you’re not really having to train your own models, so it’s a much simpler stack… a higher floor, but a lower ceiling, because you don’t actually have the ability to take your own data, to take online production data and improve your own intelligence.” The ceiling is set by the feedback loop, not by the model.

She also frames the strategic consequence as a layer shift: the contest “is increasingly becoming not just the race for the application layer but the race for the intelligence layer… So the product is the intelligence”, with applied-research teams inside application companies — she names Harvey, Factory, Glean, OpenEvidence, Sourcegraph — as “the newest neolabs.”

Unevidenced throughout, and investor content addressed to its own portfolio; see the source page’s scope warning and the parallel note on open-source-ai.

The size axis, and a definition that moves ( NVIDIA Research, 2025)

This page’s vocabulary splits foundation (adaptable substrate) from frontier (top of the leaderboard) — both capability-facing terms. The SLM literature adds a third axis that is deployment-facing, and defines it without reference to capability at all:

A SLM is a LM that can fit onto a common consumer electronic device and perform inference with latency sufficiently low to be practical when serving the agentic requests of one user. An LLM is a LM that is not a SLM.

Two consequences worth carrying on this page. First, “small” is device-indexed and therefore moves: the same definition yields ~10B parameters against 2025 consumer hardware and 10–30B (4-bit quantized, 16–32 GB) against 2026 laptops, per Sokolenko. Any fixed parameter threshold in a source is a dated instance, not a definition.

Second, it reframes what a large model’s extra parameters are for. The paper’s claim is that they encode information that a given deployment never touches — and that in agentic use specifically, the untouched fraction is most of it. That is a size-axis restatement of the jagged-frontier observation this page already carries: capability is uneven, and which peaks you need depends on the task.

See small-language-models for the argument, the evidence, and where it does not hold.

Debates and supersession

• Frontier consolidation vs. multi-model enterprise use (added 2026-09-21). Swisher predicts seven frontier labs becoming two; Krishna predicts enterprises using many models, open and closed. Different quantities (labs vs. deployed models), but the combination implies the enterprise’s frontier options narrow while open weights carry the variety. Open question: which is observed by 2027?

• “Frontier” vs. “foundation” framing. “Frontier” emphasizes capability gap; “foundation” emphasizes adaptation role. Different policy/regulation implications — frontier-model bills target capability thresholds; foundation-model bills target the broader pretraining-then-adapt pattern.

• Compute-scaling sustainability. Data-commons shrinkage (see responsible-ai) plus rising energy demands (driving nuclear-energy partnerships — Microsoft’s Three Mile Island, Google’s SMRs, Amazon’s SMRs) raise structural questions about the 5-month-compute-doubling trajectory continuing.

• Open-weight closing the gap. As open-weight performance catches closed-weight, the policy logic for restricting model release weakens — but so does the commercial moat for closed-weight providers. Open question how 2025–2026 plays out.

• Pricing power yes-or-no. Evans (May 2026) argues no (commodity-utility, AWS-not-Windows analogy); Sam Altman’s AI-on-a-meter line presumes yes. The wiki has no settled position; the question is one of the load-bearing open questions of 2026 model-economics. Evidence for “no” (2026-09-19): Bloomberg Originals reports OpenAI launching GPT-5.6 Luna in July 2026 as its cheapest model and, per one customer, cutting prices by about 80% to match Chinese open models, which were ahead of US models on OpenRouter usage from June. A frontier lab repricing to meet open-weight competitors behaves like a commodity supplier, not a platform. It is one price move, and the same film shows the frontier labs keeping the top end (“iPhone makes by far the most money”), so the answer may be both: pricing power at the frontier, commodity pricing below it.

• Open weights catching up, on a withheld capability (added 2026-10-03). The Open-weight closing the gap bullet asks how 2025–2026 plays out. Chow (Metalware), September 2026 relays an Anthropic report that an open-weight model now has Mythos-level cyber capability, with adversaries “only a couple months behind the US capabilities on the frontier.” The report (not ingested) gives about four months behind the US frontier on NIST CAISI’s aggregate cyber benchmarks, with US models tested with safeguards disabled, and 12% against Mythos Preview’s 14% on one exploit benchmark. Unlike the percentage-point gaps in Key claims, this gap concerns a capability a lab chose not to release, and it is stated in time rather than score. Chow’s “a couple months” is shorter than the report’s figure.

Related concepts

• generative-ai — the dominant use of foundation models today
• ai-benchmarks — how foundation-model capabilities are evaluated
• responsible-ai — the transparency, safety, and governance overlay
• enterprise-ai-adoption — the deployment context
• small-language-models — the same object class read along the size axis, with “small” defined by what fits a consumer device

One capability that did not scale (added 2026-08-30)

Veracode tested 100+ models across several years of sizes and vintages on 80 coding tasks and found security performance “flat, regardless of model size or training sophistication” — newer and larger models were “no better.” Against a period in which SWE-bench resolution ran from 1.96% (2023) to 87.1% (2026), this is a rare documented case of a capability dimension that scaling did not lift — most plausibly because the benchmarks driving scaling do not price a vulnerability. See ai-generated-code-quality.

Cost per task as a competitive axis (added 2026-09-19)

Bloomberg Originals adds a rough cost-per-task comparison to the corpus, where earlier sources compare benchmark scores or price per token. The same small-business website was built for about $50 with Claude Fable 5, $12 with Moonshot’s Kimi K3 (a 2.8-trillion-parameter open-weight model), and just over $4 with OpenAI’s GPT-5.6 Luna. It is one task, run once, and it prices the job, not the quality. But it states the trade-off buyers actually face: “you don’t need God to write your emails … it comes down to really a balance between cost and capability.”

The film’s performance chart has Chinese models closing on the US frontier through 2026 without matching it, consistent with the AI Index 2026 finding that the gap had “effectively closed”, with the top US model ahead by 2.7% in March 2026 and the US still producing more top-tier models. The film’s own summary: the US still has “the biggest, smartest, baddest AI models,” plus more compute and capital, while the competition that matters for adoption has moved to the price of good-enough.

How many frontier labs survive, and how many models an enterprise uses (added 2026-09-21)

Two September-2026 sources give opposite-looking pictures of market structure.

• The Atlantic Festival, September 2026: “There’s… seven frontier companies. There’s gonna be two.” She expects mergers, names Google and “probably Anthropic”, and says OpenAI’s economics are “just not sustainable.” An opinion, with no numbers.
• IBM on Bain’s Winning with AI, September 2026: enterprises will live in an “and world” of several LLMs plus open-weight models, chosen for technical, political or diversity reasons.

They are not strictly in conflict. Swisher predicts the number of labs training frontier models; Krishna describes the number of models an enterprise deploys, many of them open-weight derivatives. But if Swisher is right, the frontier choice in Krishna’s “and world” narrows to two suppliers, and the open-weight tier becomes the main source of variety.

A non-specialist account of scale and post-training (added 2026-10-12)

Episode 19 of the MIT Sloan CIO Symposium podcast gives the lay version of the story told technically above. Irving Wladawsky-Berger calls fluent conversation “almost an emergent behavior from scale” and describes post-training as “socializing the model”, because “the model in the wild was too wild for humans”. His image is domesticated wolves. Episode 20 asks why scale should produce qualitatively new capabilities, and he answers “No, I don’t think we know.” The episodes also carry a model’s own account of how it edits text (pattern recognition across “lots and lots of sentences”); that is a model’s self-report, not evidence of its mechanism.

Power: peak capacity and connection queues (added 2026-10-17)

The compute-scaling sustainability bullet under Debates treats energy as a question about the scaling trajectory. Varun Sivaram (Emerald AI) on BCG’s Pitching Progress (uploaded October 2026, recorded about November 2025) gives a narrower diagnosis: the binding constraint is peak grid capacity and the wait to connect, not energy overall. The grid is truly stressed “on the order of 1% per year or less,” and “there’s 100 gigawatts of existing stranded capacity that just doesn’t get used.” New data centres wait up to ten years for lines and substations sized for their worst moment. BCG’s Stevan Jovanovic puts US installed capacity at about 1,200 GW, used on average 40–50% of the time, against 80–100 GW of new data-centre demand.

The proposed remedy is a data centre that can cut its load when the grid is stressed: temporal flexibility (pause batchable work such as fine-tuning), spatial (move time-sensitive inference to another site) and resource (on-site batteries), with NVIDIA’s 96 MW Aurora facility in Virginia as the first commercial test. Two points bear on this page: Sivaram expects “a 50/50 split between training and inference” in power use through the end of the decade, and he treats inference as a range from latency-critical to batchable. A founder’s figures, recorded before Aurora ran. Compare the AI Index’s 29.6 GW of AI data-centre capacity and China’s routing of training to regions with surplus wind and solar.

Consumption pricing, Jevons, and open weights in offence (added 2026-10-25)

Two McKinsey sources from the same week add to the cost per task and open weights threads above.

• Pricing moved under enterprises’ feet. McKinsey’s Explainer on the economics of enterprise AI (Oct 2026): “A lot of the frontier companies in quarter 1 of this year switched from per seat licensing… to now all consumption based,” so the same use can cost “two to three times more… instantaneously.” Per-token prices are down about 90%, but use has grown faster (“your standard Jevons paradox”), multiplied by reasoning tokens, growing context windows and agents that call sub-agents. Lari Hämäläinen’s correction to the price headline: “it’s not just about cost per token, it’s really about cost per task.” The figures are McKinsey’s and unsourced in the video.
• Open weights catching up, on offence. Lewis & Nadeau (McKinsey, Inside the Strategy Room, Oct 2026): Charlie Lewis expects attackers to lead “for a period of time, a year, maybe two years,” and “open weight models will start catching up to where the frontier labs are today.” The same direction as Chow’s report of Mythos-level capability in open weights, from a consultant rather than a defence contractor.

Robotics foundation models are bound by data (added 2026-10-25)

Gao (Mecka) on Sequoia’s Training Data (Oct 2026): robot learning needs “pixels… actions, and… logic,” and there was no free corpus of it, so robotics pre-training data is becoming “quite a large market” in a way LLM pre-training data never was. The mix the industry has converged on, by his account: about a million hours of first-person human video, tens of thousands of hours from handheld demonstration grippers, and a few hours of teleoperation in the target setting; tactile and simulation data are still to be added. The field is in “millions of hours,” against Tesla’s decade of driving data for “two degrees of freedom”; a robot hand has dozens. On whether general frontier models will absorb robotics: their recent gains on robot tasks are “actually quite expected,” because robot data now sits in their pre-training mix, while specialised models may reach production reliability on particular tasks first.