AI-Wiki

DIU's CTO on defense AI adoption: buy inference compute, assure the fallback model, tune open weights

12 min read

DIU’s CTO on defense AI adoption: buy inference compute, assure the fallback model, tune open weights

▶ Watch on YouTube · YC Root Access · 22:40

The defense industrial base of the future will be built by small, fast-moving teams working on some of the hardest problems in national security.

Live from Washington, D.C., Y Combinator brings together founders, senior government officials, and military leaders for a day of conversations about rebuilding America’s defense industrial base, working with the government, and closing critical capability gaps across autonomy, munitions, space, secure communications, manufacturing, and more.

— Channel description, YC Root Access (event description; this page covers one chapter)

A 23-minute fireside near the end of YC’s defense-industry livestream (the event page). Chris Murphy, Chief Technology Officer of the Defense Innovation Unit (DIU), is interviewed by Pete Koomen of Y Combinator. DIU is the Pentagon unit, based in Mountain View, that brings commercial and non-traditional vendors into defense acquisition. Murphy also runs its 10X portfolio, which looks for step-change technologies. Before DIU he worked on AI at INDOPACOM, the US command for the Indo-Pacific. The first half of the interview is about how DIU buys: competitive solicitations, a prize challenge, and a drone programme scored on “a public leaderboard.” The second half, from about 7:29, is about AI.

The AI half is why the talk has its own page. It is an account of AI adoption from a defense buyer, and it covers ground the enterprise sources do not. The constraint Murphy names is power and inference compute rather than token cost. He treats a model failover, from a cloud model to a local one during a disruption, as a problem for assurance. And he gives one number: a weekend of fine-tuning a US open-weights model raised a track-detection score from about 40% to 78%. Koomen’s questions steer the talk toward open-weight models and toward keeping evals current as the frontier moves.

TL;DR

  • Power and inference compute are the binding constraint. “Power is always going to be our limiting constraint. So buy the best compute you can, the most inference efficient compute you can.” (7:30:01). Demand in real operations did not match what exercises predicted: “the inferencing demand loads that we projected based on, you know, command post exercise versus what real conflict looks like is significantly different. And so this will continue to grow.” (7:30:16). He gives no figures (“without getting into numbers”).
  • Compute is run as one pool, whoever the vendor is. The next step is to “treat that compute as an aggregate resource: distributed compute orchestration, implementing quality of service standards for how we service the prompts that are coming out of our AI-enabled software, whatever that is, through whatever vendor.” (7:30:46).
  • Today’s confidence in outputs is qualitative, and a failover makes that worse. He calls for “AI assurance frameworks and benchmarking to those command critical workflows because today we build qualitative confidence in the outputs of these systems” (7:31:13). The case he gives is a failover: “in the event that we experience a disruption in service and it must fail over to a local model or a different model or a different cloud provider, are the outputs of that system still good? And the answer might be I don’t know unless I’ve managed a very deliberate development to deployment cycle where I can snapshot that” (7:31:34). The risk he names is a warfighter “receiving information believing it is good” when the model “didn’t have the context when required” (7:31:59).
  • Open weights, and the one number in the talk. Asked whether DIU uses open-weight models for failover: “absolutely, right, there’s been a big push about sovereign, owning your AI, owning your model, having confidence in that” (7:33:07). The example: “We took a well-known US open weights model … A weekend of tuning improved, you know, track detection from that model from something like 40-something% to 78%. And so that was just one weekend, just trying it out.” He adds a caveat: “we also need to be very clear about what we lose when we do that” (7:33:36). What the figure does and does not tell us is under Debates.
  • Evals on every new model, and no “rails-free” models. Koomen: “How do you maintain your sort of set of evals on every successive model in order to actually adopt what is available?” (7:34:32). Murphy names a DIU programme, captioned Mystic Depot, that asks “how do you look at models, test them, understand, you know, what do the rejections look like? What are the guard rails that are in place” (7:34:41), and adds “Rails free is not the answer. That creates an insider threat risk” (7:35:04).
  • Switching costs and fragmented apps slow adoption, AI included. “The challenge is when we overindex on point solutions when there are existing solutions that are good enough or just a little bit better” (7:27:49). In software, “point solutions for applications, even with AI integrations, and interoperability remains one of the main issues”: buying “100 postage stamp sized applications” for 100,000 users “is very hard to manage and maintain” (7:28:11).
  • A chatbot as market research, and DIU as the link to the frontier. To find buyers in the department, “a $20 subscription to ChatGPT will get you really close” (7:23:17). DIU stays “tightly coupled to those both on the frontier and leading specialty AI development so that we can fast follow” (7:32:44), because commercial firms face the same problem: “every company I talk to is struggling with the same thing because we’re living inside the exponential” (7:32:32).

Why this matters to the wiki

1. A failover is a change of model, and the evals have to cover it. ai-benchmarks mostly covers public benchmarks and their limits. Its newer sections deal with evaluations an organisation runs on its own workflows. Murphy adds a case that page does not have: the model under a workflow can be swapped during an incident, without anyone choosing a new model. The question is then whether the fallback model’s outputs are still good, and he says the honest answer is often “I don’t know.” This fits two arguments from the Hugging Face agentic evaluations workshop. Narayanan argues that reliability matters most where a system acts without a human checking it. Habib shows that a benchmark score moves when the scaffold around the model changes. A failover changes the model itself. Koomen’s question on keeping “a set of evals on every successive model” is the same problem in its peacetime form. Murphy’s “we build qualitative confidence” is also the starting point Alim argues against from inside a product-security team: specify ground truth, then measure output “at scale, not just spot check.”

2. A continuity reason for sovereignty. The reasons ai-sovereignty records for owning AI are economic, cultural, governance-related and about where queries go. Murphy’s reason is continuity: the service has to keep running when the cloud connection or provider is lost, and the local model has to be trusted when it takes over. Delangue’s closest example is a model taken down by its provider. Murphy uses the word “sovereign” himself, and his mix of sources resembles Huang’s per-capability ownership. He works with frontier labs and also with “people building specialty models, you know, tuning models specifically for the government … people who are distilling models for use cases on the edge” (7:33:13). That puts him on the small-language-models side as well.

3. The fine-tune lands in a dispute the corpus has not settled. open-source-ai records a disagreement. Huang says an owned, post-trained model can beat the frontier on your domain. The CFA Institute roundtable treats open models as a way to cut cost, not to gain capability. Allen & McDonald tested open models without tuning, found them lower, and noted that fine-tuning “could yield very different results.” Murphy’s weekend tune moves one open model on one task from about 40% to 78%. He compares it with no frontier model, so it does not settle the dispute. His caveat, “what we lose when we do that,” names something none of those sources measures: what a model gets worse at when it is tuned for one task.

4. A capacity limit next to the cost limit. In the enterprise-ai-adoption sources, compute is a cost: token budgets that cannot be forecast (Brynjolfsson) and AI spend that has to be justified (Lukic & Goydan). Murphy’s limit is capacity. Power caps the inference a command can run, and demand in real operations did not match what exercises predicted. His answer is to buy the most efficient compute available and pool it across vendors with quality-of-service rules. His adoption barriers are also of a different kind: switching costs, and many small apps that cannot talk to each other. The DIU solicitation captioned “class act” clears uncleared counter-drone startups so they can “compete on a level of technology and not on a level of bureaucracy” (7:37:33).

Dynamic capabilities. digital-transforming/navigating-innovation-ecosystems: DIU’s stated job is to “accelerate the integration of commercial and nontraditional vendors” (7:15:28). For AI that means working with frontier labs, with specialty and edge-model builders, and with firms from “finance, healthcare, other regulated industries” who face the same guardrail problems (7:35:20). digital-sensing/digital-scouting: the 10X portfolio looks for “technical surprise” (7:21:43), and talking to vendors tells DIU “where is the cutting edge, what should we be asking for, what’s realistic” (7:23:58). The AI half applies the same idea to AI: “if we aren’t respecting the trend lines we’re seeing in the commercial sector, we’re probably off base” (7:30:32). contextual/internal-barriers: the barriers Murphy names are inside the buyer: switching costs that block a better product, too many apps that do not work together, and clearances and bureaucracy that keep new vendors out. DIU sits in Mountain View “to get us out of the Pentagon, out of the bureaucracy” (7:24:36).

Linked entities and concepts

Debates and supersession

  • What the 40% to 78% rests on. It is one anecdote from the speaker. The model is “a well-known US open weights model,” not named. The partner is “a customer who’s delivering a lot of software to the department,” not named and explicitly “not a prime vendor.” The task is “track detection,” which he does not define. In military use a track is usually an object followed over time in sensor data, but he does not say what the input was, what counted as a hit, or what the test set was. He also does not say whether the score was measured on held-out data, what kind of tuning was done, or how a frontier model or a purpose-built detector would score on the same test. The baseline is approximate (“something like 40-something%”). The effort is “one weekend, just trying it out.” Murphy flags the losses himself and does not size them. The figure shows that a quick in-domain tune can move a score a lot. It does not show that the tuned model is good enough to use, or better than the alternatives.
  • The demand claim has no numbers. The statement that demand under real conflict differs “significantly” from what exercises predicted comes “without getting into numbers.” The captions name the operation as “enduring freedom,” which does not fit the period and is probably an ASR error, so the page does not identify it.
  • Programme names are unverified. Mystic Depot (model testing: refusals, guardrails, tailoring), Titan Core (compute purchasing, said to run with what is captioned as “the department’s AI arsenal”) and the “class act” solicitation are taken from the captions and were not checked. No method, scope or results are given for any of them.
  • Guardrails, two positions. Murphy holds that a model without guardrails creates an insider-threat risk, so the department needs tested, tailored guardrails. Delangue holds that API guardrails are “very shallow and quite ineffective” and that openness makes models safer. The two are about different things, a deployed model inside an organisation versus the open-or-closed release question, and the page records both without choosing.
  • Who is speaking to whom. A government official recruits vendors at a YC event, interviewed by a YC partner. The talk is DIU explaining itself to potential suppliers, not a report on outcomes. The claims about DIU’s reach (Secretary Hegseth’s references, the $100 million prize challenge) are the speaker’s and were not checked.

What was actually ingested

The chapter from 7:15:03 to 7:38:12 of the livestream’s raw auto-generated captions, about 579 segments. The conversation ends at about 7:37:44. The captions were not cleaned at acquire. Quotes above were checked against the raw text at the cited timestamps; fillers and repeated words were dropped, and these ASR errors were corrected: Hegsth → Hegseth, du.m mill → diu.mil, OTAA → OTA, sentcom → CENTCOM, Paycom → INDOPACOM, Al Paparo → Admiral Paparo, Aahu → Oahu, chat GPT → ChatGPT, DIOU → DIU, special a specialty → specialty, and with we say AI → when we say AI. The prize-challenge amount is captioned “$und00 million” and read as $100 million. Left as captioned and not relied on: Mystic Depot, Titan Core, AI arsenal, RNA (an intelligence-community partner), enduring freedom, class act, and an exchange at 7:35:14 captioned “you don’t want a hugging face inside of the military,” whose wording and speaker are unclear. Speaker turns in the captions are not always reliable. One passage on switching costs (7:27:59–7:28:11) may be partly Koomen’s, so only Murphy’s lines on either side of it are quoted. No stills were taken; this is a fireside with no slides. DIU’s solicitations, the Drone Dominance leaderboard, the programmes named above and the fine-tuning result were not checked against any outside source.

Relationships

Part of

Supports

  • How Companies Are Building Their Own Intelligence | Sonya Huang, Sequoia Capital — Both describe owning and post-training model weights for a narrow domain while still using frontier labs. Huang argues that open models can now outperform closed ones in-domain and that sovereignty is per capability; this talk describes DIU working with frontier labs and with makers of government-tuned and edge-distilled models, and reports one weekend of tuning a US open-weights model that raised track detection from 'something like 40-something percent' to 78%.
  • Hugging Face's CEO on why companies are done renting their AI | Equity Podcast — Both treat dependence on a model you rent through an API as an operational risk. Delangue's example is a model taken down by its provider, leaving API renters 'in the lurch'; this talk's is a service disruption in which a workflow fails over from a cloud model to a local or different model, and it names a 'push about sovereign, owning your AI, owning your model'.
  • Agentic Evaluations Workshop — Deep Dive on the Future of Evals for Agents — Both separate the evaluation needed before deployment from public benchmark scores. In the workshop, Narayanan separates capability from reliability and ties the reliability bar to tasks where the system acts without a human, and Habib shows scores shifting with scaffold and harness; this talk asks for assurance and benchmarking on 'command critical workflows', including after the model under a workflow is swapped, and Koomen asks how evals are kept current across 'every successive model'.
  • The AI Security Paradox: Asymmetric Threats and the Future of Defense - Ammar Alim — Both treat confidence in a model's output as something to measure rather than assume. Alim asks practitioners to specify ground truth and build evals that check output 'at scale, not just spot check'; this talk says that today command-critical workflows rest on 'qualitative confidence in the outputs' and need assurance frameworks and benchmarking.

Graph View

Backlinks