Stanford CS329A Self-Improving AI Agents | Part 1 | Course Overview

Want to dive deeper? This curriculum is covered in the following online courses:

• Agentic AI professional education program: https://learn.stanford.edu/agentic-ai-2026.html
• XCS329 graduate course: https://online.stanford.edu/courses/cs329a-self-improving-ai-agents

Follow along with the course schedule and syllabus: https://cs329a.stanford.edu/

TL;DR

The opening lecture of Stanford CS329A, Self-Improving AI Agents, taught by Aakanksha Chowdhery (adjunct professor at Stanford; research at Reflection AI) and Azalia Mirhoseini (assistant professor, Stanford CS), published on the Stanford Online channel 3 August 2026. Both taught at Google Brain and later Google DeepMind; Mirhoseini also worked on Claude at Anthropic and on Gemini. This is the course’s second run. Roughly the first two-thirds is a technical overview of how language models got here; the last third is course logistics, excluded below.

1. The three scaling axes, and where they stopped. Test loss falls as you scale compute, dataset size, and parameter count — the foundation from GPT-3 through ChatGPT, PaLM and Gemini. The trajectory is given concretely: BERT at 340M parameters, GPT-2 at 1.5B, GPT-3 at 175B, PaLM at 540B, GPT-4 “estimated to be… trillions.” But the framing is retrospective: this held “until last year, where this was starting to hit some kind of a saturation point.”
2. Why size mattered: few-shot learning and emergent behaviour. Larger models improve on benchmarks, acquire few-shot and zero-shot learning (“earlier you had to fine-tune the model for [a] specific domain, but just by giving a few examples the model can follow that template”), and exhibit emergent behaviour — “capabilities like reasoning only emerge in larger models.” Mirhoseini’s framing of why this matters epistemically: beyond “predictable scaling laws… there are new behaviours that appear in the model that they didn’t have before, and we never could predict that until we had these bigger models.” The canonical case is chain of thought, which small models cannot exploit — LaMDA and GPT at ~7–8B “can’t really benefit from chain of thought, it doesn’t do anything for them” — alongside modular arithmetic and word-unscrambling appearing abruptly at scale.
3. What actually made ChatGPT different from GPT-3. Not parameter count but a post-training stack, walked through in order: pre-training (next-token prediction, “the easiest step”); fine-tuning on much higher-quality data (“data that companies pay millions of dollars, or hundreds of millions of dollars, to buy”); instruction tuning on instruction/question/answer triples, increasingly synthetic; and RLHF, where “instead of just creating some supervised prompt and labels, we are creating a reward model out of human preferences” against which the LLM’s parameters are then steered. Reward types can be weighted per objective — correctness, helpfulness, specificity, harmlessness. Their candid framing of alignment: “we try to steer the AI models to follow the goals and preferences and values of humans. And this is still a big problem. It’s a big issue. We haven’t mastered it yet.”
4. The pivot the course is built on — inference became a frontier. “Pre-training and then fine-tuning were the big pieces. But since a year and a half ago, it turns out inference is also a frontier for making the models much more capable.”
5. Large Language Monkeys — Mirhoseini’s lab result, and the lecture’s most striking finding. Named for the infinite monkey theorem: sample the same problem repeatedly and use a verifier to select. Scaling samples per problem from 1 to 10,000 on math and coding benchmarks, coverage (fraction of problems solved by at least one sample) rose enough that smaller open models “in all of these cases… do better than the GPT-4o model” that beat them at one sample. Her interpretation: “it kind of seems like the models already know a whole lot more than what you get out of them when you just ask them once.” The sharp detail: “for some of these problems, out of these 10,000 solutions, maybe three or four of them were correct.” Crucially, no parameters change — “the model is fixed, and we are just at inference time.”
6. Why sampling beats search, in answer to a student. Asked how this differs from traversing the answer space, Mirhoseini’s answer is about the prior: “the space of all possible answers is way bigger. If we do a tree search or a random search… there’s no way to hit [it].” The model’s distribution makes the sampling “highly sample efficient — 10,000 is not a whole lot,” given some of the problems are “really hard IMO-level” ones solved by a small model.
7. Closing the loop — where “self-improving” comes from. With DeepSeek and the o1/Gemini-thinking generation, “we are bringing this fine-tuning and test-time scaling together. Because with this test-time scaling… we have this new engine that we can generate a whole lot of synthetic data” — for problems with known answers, keep the traces that reach the gold answer, and fine-tune on them. “There’s no boundary in how good the models can become with test-time scaling, and then bringing that back to the process of training the model. And that’s the self-improving piece that we are very excited about.”
8. What reasoning models actually do. OpenAI’s o1 showed a log-linear relationship between test-time compute and pass@1 accuracy on AIME — “for test time alone… without changing the parameter count.” The behaviours: problem analysis, task decomposition, self-evolution against feedback (running tests, using a calculator), self-correction and backtracking. The distinction Chowdhery draws from ordinary chain of thought is that “the model itself is producing this chain of thought.” Reasoning models outperform on math, data analysis and programming — “but not necessarily in personal writing or editing texts.”
9. Three student questions the instructors answer with visible uncertainty, which is the most useful part of the lecture for a wiki tracking what is actually known. (a) Is the gain from generating traces, or from being asked to decompose? — “the data basically has made the model become a generalised thinker,” with Chowdhery adding that repeated sampling shows a base model can already produce a correct chain, so much of the training gain “comes down to it learns which is correct,” moving pass@k into pass@1. (b) Can you use a different model for reasoning versus the final answer? — “models, surprisingly or not surprisingly, they like their own traces a lot more,” even when the traces come from a better model. (c) Why is the RL jump so large if pre-training already contains the capability? — “I think that whole loop is not completely well understood. It’s like the first signs of life and it starts to get commercialised, but I think there’s a lot more research still open.”
10. From LLMs to agents. Chatbots and reasoning models “are basically still single-turn… they’re fun to interact with, but they’re not necessarily accomplishing a task for you.” What changed “almost in the last few months” is that agents like Claude Code and Deep Research “have really enabled people to do real-world workflows.” Her definition: a goal, planned steps, interaction with an environment, correction from feedback, and deciding when to stop — “that’s what makes agents different from the chatbot types” — plus tool use and memory. Her honest scoping of the state of the art: “in most scenarios, you are still having very static workflows… today, what exists is closer to agentic workflows,” with the fully open-ended loop reserved for cases like coding and research where “we are starting to see signs of life.”
11. The workflow vocabulary the course will use. LLM calls, verifiers, critics/judges (LLM-as-a-judge), tool calls; composed via prompt chaining, routing, parallelisation (deep research as the example), and an orchestrator — “in simple words, an LLM manager… in Claude Code you actually start to see that, that there is some notion of a plan that it comes up with, and then based on that plan it will make subsequent LLM calls.”
12. Verification as the bottleneck — the thread that runs through the whole lecture. In verifiable domains “this verification is a good way to give feedback back to the model.” Outside them, “there’s this whole set of research on how to train LLM-as-judge.” The framing Chowdhery gives it is the generator–verifier gap: “it’s easy for models to generate a whole bunch of nonsense or sensible sets of reasoning traces… but at the end of the day, whether that’s useful or not, we need [a] feedback loop for that. And if you’re creative writing, how much feedback can you get? So human feedback ends up becoming a bottleneck. … Verification continues to be one of the bottlenecks in this space.”
13. Why coding agents work now when they didn’t a year ago. Asked directly, the answer is not architecture: “the general architecture is, I would say, not changed that much. The paradigm is very much the same. It’s mostly a matter of more powerful models and then better RL. RL with verifiable rewards is working.” Plus a compounding effect — “once the models start to get better, there’s this self-improvement loop that kicks in, because you can now generate tests and the tests become more reliable.” The tasks where this pays off are named as “extremely repetitive”: code migrations, version upgrades, codebase restructuring, data engineering, data-warehouse migration.
14. A requirement the lecture surfaces that is easy to miss: intent clarification. “Even when you give the model a goal, it has to clarify the user intent. … Oftentimes, the users will not specify the problem well enough, so clarifying the user intent — so that it knows what to go look for and how to verify — starts to become important.”

What was actually ingested

The full English caption track (613 segments, consistent with duration: 69:42 / length_seconds: 4182). The video carries both an auto-generated and a human-curated (“English - CC”) track; the panel scraped is whichever YouTube served by default, and this ingest did not verify which. No chapter markers. Slides are central to this lecture and are not captured — the scaling-law plots, the coverage-versus-samples curves from Large Language Monkeys, the o1 test-time-compute chart and the agentic-workflow diagrams are described from speech only, so numbers read off axes are not available. Student questions are frequently inaudible; the instructors repeat most of them, and the summary follows those repetitions. The final third of the lecture is course logistics (grading, homework schedule, project proposals) and is excluded.

Date note: the lecture was delivered 22 September 2025 and published 3 August 2026 — an eleven-month gap. Per the wiki’s convention date_published and the filename use the publication date. This matters for reading the content: statements like “the shift in the last few months” and “the models are just starting to get reliable” describe September 2025, not August 2026, and several are now dated. The lecture is also Part 1 of a series; the remaining parts are not ingested.

Dynamic-capabilities tagging

Deliberately untagged. This is a technical course lecture on LLM training and inference-time methods, with no firm-level digital-transformation claim to map onto a Warner & Wäger microfoundation. Per CLAUDE.md’s when-not-to-tag guidance, which names “pure LLM-internals papers” explicitly, the field is omitted rather than stretched.

Linked entities and concepts

• Stanford Online — publishing channel. Updated in this ingest.
• Anthropic, Google DeepMind, OpenAI, Claude Code — named in the instructors’ backgrounds and throughout the technical content.
• Hugging Face agentic-evaluations workshop — the practical counterpart to this lecture’s verification-bottleneck argument; see this source’s relationships:.
• Frey — the same verification-is-the-bottleneck claim at economy scale.
• Khattab et al. — the loop-around-the-model framing, and the claim that recent agent capability came from better models and RL rather than new architecture.
• foundation-models — the scaling-law trajectory, its stated saturation, and the post-training stack that separated ChatGPT from GPT-3.
• ai-agents — the goal/plan/act/feedback/stop definition, the static-workflow-versus-open-loop honesty, and the orchestrator pattern.
• agentic-engineering — prompt chaining, routing, parallelisation, orchestrators, judges and verifiers as the composition vocabulary.
• agent-harness — the coding-agent loop, and “the general architecture… has not changed that much.”
• ai-benchmarks — coverage/pass@k versus pass@1 as distinct measures, and the AIME log-linear test-time-compute result.
• ai-agents and responsible-ai — RLHF, weighted reward types, and the instructors’ statement that alignment “is still a big problem… we haven’t mastered it yet.”
• generative-ai — few-shot learning and emergent behaviour as the capabilities that made the current wave possible.

Dangling (single-source mention, deferred per author-entity promotion): Aakanksha Chowdhery, Azalia Mirhoseini, Reflection AI, Jeff Dean (referenced), the Large Language Monkeys and SWiRL papers.

Source quality note

University course lecture by two active researchers presenting partly their own published work (Large Language Monkeys, SWiRL) — an evidence class with no commercial incentive, and the instructors are notably willing to say what is not known (“that whole loop is not completely well understood,” “I don’t think there’s a published piece of work that does that,” “at this point in time both processes help” with “no single point of consensus”). That candour is the main reason this source earns its place alongside the practitioner material in this batch.

Three limits. The eleven-month publication lag means the “state of the art” framing is a year stale in places — most visibly the claim that coding-agent reliability is “just starting” to arrive. The ingest is slide-blind, so every quantitative result here is a verbal description of a plot; specific figures should be taken from the underlying papers, not this page. And this is Part 1 of a course: it is an overview that sets up questions (verifier design, process versus outcome reward models, multi-step reasoning) which later lectures address, so treating any position here as the course’s settled view would be premature.