What AI Can’t Do — And Why

“Humans manage to do so much with surprisingly little,” says Douglas Guilbeault, an assistant professor of organizational behavior at Stanford Graduate School of Business. “Whereas AI, by comparison, is doing relatively little, but with so much power, so much compute, so many resources, and by comparison, relatively fewer constraints.”

On a bonus episode of the If/Then podcast, Guilbeault describes the implications of his recent work… Guilbeault and his colleagues believe they have identified a key principle that distinguishes human intelligence from machine intelligence — and one which illuminates the limitations of machine thinking.

— Channel description, If/Then (Stanford GSB)

A ~29-minute If/Then podcast episode (Stanford Graduate School of Business YouTube, 25 June 2026), host Kevin Cool in conversation with Douglas Guilbeault (assistant professor of organizational behavior, Stanford GSB). The hook is unusual for the wiki’s corpus: not what AI can do, but what it can’t — and may never — argued from cognitive science rather than benchmarks. The peg is Guilbeault’s paper “A Simple Threshold Captures the Social Learning of Conventions” (which, he notes, “doesn’t say anything about AI” but has large implications for it).

TL;DR

• The core asymmetry. Humans “do so much with surprisingly little” — achieving robust, even universal understanding (mathematics, quantum physics) under heavy constraints (limited memory, attention, data). Current AI is “the exact opposite strategy”: doing relatively little in comparison to what humans can do, but via enormous compute, engineering, and data. “Data centers the size of Austin, Texas” to complete one next-word prediction.
• Optimization vs. satisficing. AI boosters frame both humans and machines as optimizers. Guilbeault leans on Herbert Simon’s satisficing: people act on “good enough” approximations under real constraints, then adapt in real time — “that’s just how we roll.” His paper shows satisficing predicts human behavior where the optimization framework fails (and, in forthcoming results, LLMs replicate the optimizer pattern, not the human one).
• Why LLMs learn differently. The LLM’s core task — mask a word (“I grab the leash to walk my ___”) and predict the most likely filler by scanning “every sentence ever created on the internet” — requires data no human encounters a fraction of. The inference: humans “must be adopting a qualitatively different kind of approach,” one cognitive science doesn’t yet have a full theory for. Therefore we are “in absolutely no position to conclude” LLMs solve problems the way humans do, “let alone that they’re in a position to replace humans.”
• The stakes / the hype critique. Guilbeault is speaking to “every person trying to understand their place… in the age of AI” against a “grandiose narrative” of imminent super-intelligence. Two San Francisco startup ads he uses with his MBAs: “predict anything” and “Humanity has had a good run.” His objection is both ethical and empirical: compressing intelligence into “it’s all just prediction and number-crunching” both misrepresents human capability and leaves people unable to “motivate or defend their relevance.”
• Conceptual leaps. Human learning is marked by intuition / insight / the aha-moment — a leap, not a smooth gradient step. Current LLM architectures are debated to be incapable of this because they “move step by step in a continuous way” within an already-understood space. Guilbeault reads his own paper’s finding — behavior that is random in early stages, then suddenly converges to a stable categorical understanding — as “a kind of conceptual leaping”: a jump from a random state to an ordered state that LLMs may be fundamentally limited in modeling.
• Beyond statistics — metaphor, vibes, reasoning. Humans reason through non-statistical mechanisms: metaphor and analogy, and “vibes” / an aesthetic sense (“you just kind of know something’s a good idea… there’s an ineffability to it”). The mathematician who holds an intuition for ten years against all evidence — “irrational” until the breakthrough — is the emblem. None of this “falls out of the current statistical framework.”
• Harnessing randomness. Humans “inhabit disorder… and somehow make meaning from that state” (he cites a PNAS study: mathematicians’ gaze and movement become more erratic right before an insight). LLMs are fed “highly structured data… perfectly designed to be learned from” — a crutch (“knock out a word and know for sure a word’s supposed to be there”) humans never had evolutionarily. The genuinely human problem is making order where you “don’t even know if there’s regularity and order in the world at all.”
• What optimization leaves out. Closing, near-poetic: an optimization/mechanization mindset discards the “fundamental strangeness” — Wittgenstein’s “the mystery is not what exists, but that it exists.” Biology (the giraffe, the squid) is the model: “we’re not this orderly machine as much as society and the incentives… might want that to be true.”

Why this matters for the wiki

This is the wiki’s first cognitive-science account of the AI capability ceiling — a theoretical “why” beneath the empirical jagged-frontier. Where the frontier sources describe where AI is jagged, Guilbeault offers a mechanism for the jaggedness: optimization-over-massive-data can match humans on bounded, well-structured prediction but may be structurally unable to reproduce the conceptual leap, the metaphor, the random→ordered convergence. It is the deepest grounding the wiki has for why durable-skills are durable — intuition, taste, analogical reasoning, and meaning-making-from-chaos are named as the human residual — and a second, distinctly academic anchor for analogical-reasoning (metaphor/analogy as a load-bearing non-statistical human mechanism).

Linked entities and concepts

• Concepts touched: jagged-frontier (a cognitive-science mechanism for the capability ceiling); durable-skills (the human residual — intuition, taste, conceptual leaps); analogical-reasoning (metaphor/analogy as non-statistical reasoning — second source); automation-vs-augmentation (the “replace human scientists” hype vs. the human edge); foundation-models (the next-word-prediction mechanism, described from the outside); responsible-ai (the AI-hype / “good run” narrative as a disempowerment risk).
• Entity: Stanford GSB (channel/publisher — promoted to an entity page on this ingest, its second author:-value appearance, which also resolved pre-existing broken [[Stanford GSB]] links).
• Dangling (single-source mention, deferred): Douglas Guilbeault (presenter), Kevin Cool (host), Herbert Simon (satisficing), Ludwig Wittgenstein (the closing quote), If/Then (the podcast).

Related sources

• supports Mollick — The AI Skills Nobody Is Teaching — the practitioner twin: taste / experience / point of view as the human edge; Guilbeault supplies the cognitive-science account of why (vibes, aesthetic sense, conceptual leaps are non-statistical).
• supports Argenti — Mindset, Not Skillset — both refuse to reduce humans to prediction machines; hold onto instincts/judgment/intuition.
• contradicts Csaszar, Ketkar & Kim — AI and Strategic Decision-Making — Csaszar finds LLMs reach parity with entrepreneurs/investors on realistic strategy tasks; Guilbeault argues optimization has a ceiling on the human conceptual leap. The tension turns on task scope (bounded strategy generation vs. open-ended social learning / insight).
• Adjacent (untyped): Benedict Evans is the corpus’s measured hype-skeptic at the market-analyst altitude; Guilbeault is the same temper from the cognitive-science altitude.

Notes on scope and provenance

• Full episode, ASR transcript fetched via the Playwright skill (JSON mode) and cleaned: inline “N minutes, N seconds” timestamp interjections removed, [music]/[laughter] markers dropped, the video’s own chapter headings restored with timestamps, proper nouns corrected against the description (Douglas Guilbeault, Kevin Cool, Wittgenstein, PNAS). Caption track is kind: asr (auto-generated) — transcript provenance does not feed confidence.
• The paper referenced is “A Simple Threshold Captures the Social Learning of Conventions” (Stanford GSB faculty publication); not separately ingested.
• No Warner & Wäger dynamic_capabilities: tags — the source is cognitive-science / AI-capability theory, outside the digital-transformation lens.