Branco, Canelas, Gamboa & Fonseca — LGTM! Characteristics of Auto-Merged LLM-based Agentic PRs
TL;DR
An MSR 2026 Mining Challenge paper on the AIDev dataset, opening with the sentence that names the problem this whole ingest circles:
“AI tools are generating code faster than humans can properly review it, leading repositories to skip review and auto-merge agentic PRs directly.”
Four findings:
- Auto-merged agentic PRs tend to be smaller and more focused than the alternatives. Encouraging on its face — the changes going through unreviewed are the low-risk ones — though it is equally consistent with size being the only signal anyone is filtering on.
- Maintainers are cautious toward PRs that delete existing code. Deletion is the one operation that reliably attracts a human. That is a crude but real proxy for Merge Mommy’s reversibility and blast radius dimensions, arrived at by maintainer instinct.
- Repositories tend to either auto-merge all agentic PRs or none. This is the paper’s most consequential result. Auto-merge is a repository-level governance posture, not a per-change judgement. The bimodality means most projects are not doing risk triage at all — they are choosing a stance and applying it uniformly, which is exactly the thing a scored gate is supposed to replace.
- Mature repositories more frequently reject auto-merge policies, and OpenAI Codex and Claude Code receive the highest auto-merge rates among agentic tools.
Why finding 3 matters most. Vo’s six-dimension risk score and Carson’s two-loop video-gated Land PR are both middle paths — auto-approve the low band, escalate the rest. This paper says the middle path is rare in the wild: the population has mostly sorted itself into trust everything and trust nothing. That reframes the practitioner material from “here is what people do” to “here is a discipline most repositories have not adopted” — which makes it more valuable as guidance, not less, but also means it is not yet validated at scale.
Finding 4’s second half deserves a note: the agents with the highest auto-merge rates are two of the three that EvilGenie caught in explicit reward hacking. Trust and trustworthiness are being set by different processes.
Two companions complete the picture. Abujadallah et al. work the opposite tail of the same dataset — the 46.41% that reviewers reject — so together the two describe a bimodal population in which repositories either wave everything through or discard nearly half of what reaches them. And DORA supplies the survey-level consequence of the first mode: AI adoption’s relationship with delivery stability stays negative precisely when change volume rises without control systems in place. Auto-merging all agentic PRs is those control systems being switched off wholesale.
Dynamic-capabilities reading
digital-transforming/redesigning-internal-structures— auto-merge policy is a structural change to the review function, adopted at repository level and visible in the data as a discrete posture rather than a gradient.contextual/internal-barriers— the all-or-none pattern is a barrier to graduated governance: without a risk-scoring layer, the only available policies are the two extremes.
Linked entities and concepts
- Entities: GitHub, OpenAI, Anthropic, Claude Code
- Concepts: agentic-pull-requests, agent-oversight-and-delegation, agentic-engineering
- Dangling (single-source mention, deferred): Ruben Branco, Paulo Canelas, Catarina Gamboa, Alcides Fonseca
Scope and reliability
Abstract and programme-page findings only — the preprint PDF was not converted, so no effect sizes, no definition of “mature,” and no share of repositories in each auto-merge mode were read. Those are exactly the numbers that would make finding 3 actionable, and they should be recovered before the claim is quoted quantitatively. A short Mining Challenge paper, not a full research track submission. All AIDev coverage limits carry over: public GitHub, five agents, self-identified attribution. Artifact available at Zenodo (10.5281/zenodo.18341679).