Agent fleet management

Confidence 0.75 · 18 sources · last confirmed 2026-10-25

The practice of running many coding or operations agents concurrently and treating the human’s job as managing them rather than doing the work. The unit of analysis shifts from how good is the agent to how many can one person supervise, in what queue, with what gates — a span-of-control question borrowed from organisational design and applied to software.

The claim

Stated most directly by Ryan Carson in 2026-08-24-carson-vo-how-i-ai-manage-15-ai-agents-solo-founder:

“All of us have to uplevel our ability to manage agents. Like that is our job… How do you effectively manage 10, 20, 100, 500 employees? This is the whole reason that organizations were built as pyramids — humans can’t scale and have a thousand directs.”

The corollary is a claim about who is advantaged: management skill developed over a career — structuring, teaming, prioritising, delegating, deciding where to micromanage and where to let go — transfers directly. That is an unusual position in a discourse that mostly predicts seniority is a liability.

The four load-bearing mechanisms

1. Isolation, which is what makes parallelism possible at all. Concurrent agents editing one working tree collide. Cloud agents with a fresh VM per session do not. Carson’s position — “if you are working locally, I honestly think you are a caveman” (2026-07-25-darroman-profitable-founder-managing-ai-agents-25-prs-a-day) — is an argument about workspace isolation, not about hardware. 2026-03-23-geng-neubig-caid-asynchronous-software-engineering-agents validates the same primitive experimentally: isolated workspaces is one of CAID’s three coordination primitives, alongside centralized task delegation and asynchronous execution, worth +25.6% on PaperBench and +14.7% on Commit0 over single-agent baselines.

2. A queue the machine holds and priorities the human holds — deliberately in different media. Carson buckets 10–15 Devin threads into folders (bugs, P0, P1, P2, investors) and keeps a handwritten weekly priority list. The split is not eccentricity: the folder tree scales with the fleet, the paper list scales with human attention, and mixing them means the fleet’s volume sets the human’s agenda.

3. Decision load explodes before output does. 2026-07-25-darroman-profitable-founder-managing-ai-agents-25-prs-a-day is the clearest statement: 2–3 high-stakes decisions a day becomes 10–20 before lunch. The countermeasures are all attention-rationing devices — 3 of ~25 daily PRs pinned for focused attention, a 25-minute check cadence, work from a phone, and agents never holding production credentials. Note what these are: not better agents, but a bounded human interface to them.

4. Reusable procedure — playbooks and skills. A fleet is only manageable if instructions are written once. Carson’s named examples are Watchdog (iterate a per-customer skill across every account, then rank the top three problems and check them against open PRs), Land PR (fresh review, up to two loops, narrated video walkthrough, then merge), and an investor-update skill. 2026-06-11-abujadallah-rejection-of-agentic-pull-request-fixes gives the empirical form of the same lesson from 306 rejected PRs: hint the approach, state forbidden approaches, instruct on validation.

What the evidence does not support

More agents do not reason better. 2026-04-02-tran-kiela-single-agent-outperforms-multi-agent-under-equal-budgets shows that under matched reasoning-token budgets, a single agent matches or beats multi-agent systems on multi-hop reasoning — the Data Processing Inequality argument being that decomposition can only lose information. Fleets are justified by throughput across independent tasks and by isolation, not by collective intelligence. Reach for multiple agents on one problem only when a single context window stops being enough, which is Tran & Kiela’s own stated condition and the regime CAID measures in.

And the throughput is gross, not net. 2026-06-11-abujadallah-rejection-of-agentic-pull-request-fixes finds 46.41% of agent-proposed fixes rejected, consuming review, CI and tokens on discarded work. 2026-03-30-liu-debt-behind-the-ai-boom finds 22.7% of AI-introduced issues still present at the latest revision. A fleet’s PR count is an input measure.

The practitioner’s own limit

The most credible thing in the practitioner material is where it stops. Carson, spending $20k/month on agents, argues output is not the constraint:

“I really try to constrain my output — not quality, not on bugs — because I don’t think I get multiples of quality off of multiples of output.”

And on self-improving product loops: “we’re nowhere near any frontier model having the intelligence to know what to ship.” This is the boundary between agent fleet management (a real and improving discipline) and the claim that fleets produce products (which nobody in the corpus demonstrates). Compare 2026-08-14-blomfield-yc-building-structuring-ai-native-company, which argues the quality gate “probably should not be a human in all but the most extreme cases” — a direct disagreement about where irreducible judgement sits.

Economics

2026-07-25-darroman-profitable-founder-managing-ai-agents-25-prs-a-day is the only source in the corpus with unit economics attached: $20,000/month current spend, judged unsustainable, against ~$5,000/month per engineer-equivalent as the target. The route is model routing — frontier models for hard reasoning, fine-tuned models for repetitive work — enforced through parent–child agent patterns. Per-automation costs: ~$60 per end-to-end signup run, ~$5 per quality-grading session.

Squads and fleets are the same primitive under two names (added 2026-09-03)

The GitHub Podcast, S02E02 supplies a vocabulary note this page needs, because the corpus has been accumulating names for one thing. GitHub’s own Copilot CLI ships both — squads (a slash command that puts several models on the same task) and fleets (parallel sub-agents) — and the hosts, who work there, are visibly unsure which is which on air. Their resolution is the useful part:

“Most things in AI, there are no standardizations — but the goal of leveraging something like a squad or a fleet, or whatever they call it in whatever tool, is to have the work done in some type of parallelization.”

Their working definition of a squad is role specialisation inside a loop — “one agent that’s dedicated to the planning part, one dedicated to checking and testing, one dedicated to acting and implementing, and one for reviewing” — which is the same decomposition Osmani’s orchestra and Allen’s team structures describe. Three vendors, three nouns, one primitive. Treat squad, fleet and orchestra as tool-local labels for role-specialised parallelism, and resolve them to this page rather than proliferating sub-concepts.

Correction (2026-09-04). This note originally counted graph engineering as a fourth name for the same primitive, on the strength of Google Cloud’s escalation-from-loops framing. The follow-up video from the same team makes clear that graph does not belong in this cluster: a graph is engineer-specified control flow over history-free nodes, whose defining payoff is determinism, and it is explicitly contrasted with agent swarms — which is where a squad of role-specialised agents actually sits. Parallelism (fan-out) is one pattern inside a graph, not what a graph is. The three-name resolution above stands; graph now resolves to graph-engineering.

The episode adds no evidence and does not move this page’s confidence: it is three developer advocates describing their own product’s terminology, with no measurement of whether the parallelism pays.

Related concepts

agentic-engineering (the parent discipline), agent-harness (what the fleet runs on), agentic-pull-requests (the fleet’s output artifact), agent-oversight-and-delegation (the gating layer), multi-agent-failure-modes (what breaks), ai-agents.

Debates and supersession

• Does fleet parallelism buy quality or only throughput? Practitioners claim throughput and explicitly disclaim quality (Carson). CAID claims accuracy gains too, but in a regime (per Tran & Kiela) where a single context window is insufficient. Open: nobody has measured a fleet against a single agent at equal total spend on the same task set.
• Is the span-of-control limit ~4–5 or ~15? Carson cites “four or five concurrent tasks you can actually keep track of” as the reason for folder bucketing, then runs 10–15 threads. The reconciliation is presumably that folders raise the effective limit by chunking — but this is asserted, not measured.
• All evidence for the practitioner claims is n=1 and self-reported. Carson is the single practitioner behind three of this concept’s sources; 2026-04-13-branco-lgtm-auto-merged-llm-agentic-prs suggests the graduated-gate discipline he and Claire Vo describe is rare in the wild, where repositories mostly auto-merge all agentic PRs or none. Open: is disciplined fleet management a practice, or currently a proposal?

Delegating the span of control to a file (added 2026-09-16)

This page frames fleet management as a span-of-control problem: how many agents can one person supervise. [[2026-07-06-google-cloud-agent-factory-intent-driven-development|Anthropic’s Lydia Hallie (July 2026)]] describes the vendor’s answer, which is to stop supervising individually and write the span down as a program.

Claude Code’s dynamic workflows emits a JavaScript file describing an entire run — parallel build agents inside sequentially-gated phases — which the user can edit, save as a named command, and re-invoke. The fleet-management consequences are three:

• Scale is the design point, not a side effect. “It can go into hundreds of subagents because this is a feature for tasks that takes hours or days or these really long spanning tasks.” Against Carson’s hand-managed 15 and Darroman’s 25 PRs/day, this is two orders of magnitude, purchased by removing the human from the per-agent decision entirely.
• It answers the page’s own contradiction with the multi-agent literature more directly than the corpus has. The recorded reconciliation is that fleets parallelise independent tasks. The workflow file makes independence a declared property of the phase: build agents run concurrently because the author asserted they can, and integration, review and verify are serialised because they cannot. Whether the assertion is right is still the author’s problem — but it is now explicit and inspectable rather than implicit in an agent’s spawning behaviour.
• Per-subagent model selection becomes an edit. “Maybe I don’t want all subagents to use Fable… Maybe I want to use Sonnet or Opus.” Token economics (see §Economics above) moves from a global setting to a per-role one, in source the user controls.

The problem it was built for is stated as a defect in the previous behaviour, by the vendor: ad-hoc spawning “is very non-deterministic — sometimes it might spin up four subagents, sometimes it might not even use subagents. The other time it uses 10 subagents.” That is the condition Osmani’s orchestra pattern hand-rolls around; here the vendor ships the fix.

What the episode does not establish is whether any of it works at the claimed scale. The demo ran four build agents on a toy game, and it was too slow to complete on camera — she cut to a pre-baked tab, noting the large model’s latency. Hundreds of subagents over days is a design target with no reported instance behind it.

Two orders of magnitude past anything else on this page (added 2026-09-16)

The practitioner evidence here tops out at a solo founder running ~15 agents and teams managing dozens. a16z, August 2026 reports 100,000–200,000 agents instantiated per day, each in its own virtual machine, running a live business.

At that scale the page’s organising question — span of control — stops being about a human’s attention budget, because no human is in the loop per agent. What replaces it:

• The fleet is partitioned by customer, not by task. One long-running agent owns one customer relationship end to end, so the coordination problem the page usually treats (many agents on one goal) does not arise; the agents are independent by construction.
• Isolation is per agent and absolute — “its own virtual machine” — which matches the isolation primitive this page already records, applied at a scale where it is the only thing making the fleet tractable.
• Attention rationing is replaced by an inbound queue. Rather than a human sampling agent output, agents call for help and humans staff the other end (see agent-oversight-and-delegation). Human attention is spent on exceptions the fleet raises, not on a review cadence the human sets.
• Quality is held by evals rather than review — the same engineer time, tokens and money as building the agents, measured on business outcomes.
• Learning is fleet-wide. “If they make a mistake, they learn it, and the next day not just them but the other 200,000 agents will have learned from that mistake.” Mechanism unstated — this is the least substantiated claim in the episode and the wiki should not infer online learning from it.

Caveats that matter at this scale. The figures are self-reported on a venture-capital podcast. “Agents instantiated per day” is a provisioning count, not a concurrency or utilisation measure, and many of those lifetimes are minutes. None of the throughput-quality numbers this page tracks elsewhere — rejection rates, unfixed-defect rates — has an analogue here. See Kavak and agent-harness.

An in-house VM fleet invoked from a task comment (added 2026-09-18)

Notion’s internal “Boxy” system, described by Nystrom on How I AI (May 2026), is fleet infrastructure built into a company’s own tooling: VMs with Codex and Claude Code installed, invoked by @-mentioning Codex in a comment on a Notion task, returning a pull request with a preview URL and its own UI-verification screenshots. Nystrom manages the result the way Carson does — fire off several, then manage them “round-robin” rather than iterate beside one — and prefers Codex for it because it “can grind for like hours.” Host Claire Vo’s prescription is organisational: a large engineering org without “a like VM strategy and background agent strategy” should get one, and fleet throughput is capped by CI speed (see agentic-pull-requests).

One human, one agent, dozens of accounts (added 2026-09-28)

Tan’s “20x company” episode (YC, Feb 2026) describes GigaML’s internal agent Atlas working “in tandem with a human FTE to service dozens of accounts”: one person handles customer relationships and requests while Atlas does the browser work, policy edits and code. Phaseshift goes the other way, with one custom agent per employee built from each person’s written list of manual tasks. Founder claims, no detail on how the agents are supervised.

Leading agents measures leading people (added 2026-10-15)

This page treats directing agents as management. Weidmann, Xu & Deming (Harvard Kennedy School, 2025) give the first experimental evidence that it is the same skill. In a pre-registered lab study, 249 leaders each directed six teams of randomly assigned human followers and six teams of GPT-4o agents through information-pooling puzzles. A leader’s score with the agents correlated with their causal effect on human teams at 0.81 after correcting for measurement error (0.69 after also controlling for hard skills). The same behaviours predicted success with both kinds of followers: asking more questions, more turn-taking, and saying “we”, but not talking more. Emotional warmth mattered much less with agents. The authors stress that the test has not been validated against leadership outcomes in real organisations; the task is a text-chat puzzle with three followers, not a fleet of coding agents.

The HBS AI Institute’s Frontier Firm page draws the organisational consequence: agents need “more substantive onboarding (codifying tacit knowledge, adding a need for ‘agent management skills’)“. Because the skill is unevenly spread, they also create “a sharper ‘winners and losers’ dynamic, potentially increasing the performance gap.” Sadun makes the same point from management research: “even today, it’s not like everybody is a good manager.” Read together, the span-of-control question on this page is also a selection question: who should run the fleet. The lab result suggests that the people who lead human teams well are a reasonable first answer.

From many local agents to a factory, and to teams of agents (added 2026-10-25)

• Lloyd (Warp, Oct 2026): interactive agents on each developer’s laptop hit three limits: they are “bottlenecked by people,” their costs cannot be managed collectively, and there is “no central layer for governing them” (each developer runs their own MCPs, so a compromise acts as the logged-in user). Moving them into a cloud factory gives a live view of every agent run across the team, “almost like Google Docs, but for your team’s coding agents,” and fleet metrics: human interactions per pull request, time to ship, and cost per pull request by user, agent and model.
• Smith (Okta) & Cohen (NanoCo) at TechCrunch Disrupt (Oct 2026): Gavriel Cohen runs “probably 32 agents on a team” and expects hundreds within months, calling agent teams “probably a new scaling law” after training and test-time compute. His rule for the fleet: every agent has an owner who takes “ultimate responsibility for it.”