Responsible AI

Confidence 0.95 · 19 sources · last confirmed 2026-07-01

The umbrella discipline (and increasingly, the corporate function) addressing AI’s risks: bias, factuality, safety, transparency, accountability, security, regulatory compliance. As of 2024, organizational acknowledgment of RAI risks has outpaced mitigation effort — and standardized RAI evaluations remain scarce among major industrial developers.

Working definition

“Responsible AI” (RAI) refers to a constellation of principles and practices for developing and deploying AI systems that are fair, accountable, transparent, secure, and aligned with stakeholder values. Concretely, an org’s RAI work typically spans: bias evaluation, safety/red-teaming, transparency reporting, incident logging, regulatory compliance, governance structures, and content provenance.

Closely related but not identical: AI safety (often used to mean alignment / catastrophic-risk research), AI ethics (more philosophical), AI governance (more institutional / policy-facing).

Key claims

Risk acknowledgment > mitigation

• McKinsey RAI survey via AI Index 2025 §3 Highlight 3: organizations identify key risks but most are not taking active mitigating steps. Top risks cited: inaccuracy 64%, regulatory compliance 63%, cybersecurity 60% — all under 70%, and far fewer say they’re actively addressing them.

Incidents are rising

• AI incidents +56.4% YoY: 233 reported in 2024 (record high). Source: AI Incidents Database via AI Index 2025 §3.
• 2025 update: 362 documented incidents (AI Index 2026 Top Takeaway #6) — another ~55% YoY rise. Reporting on responsible-AI benchmarks remains spotty even as capability-benchmark reporting is near-universal among frontier labs.

RAI dimensions can trade off (AI Index 2026)

A new finding flagged in 2026: improving one responsible-AI dimension can degrade another. Example pattern: improving a model’s safety can degrade its accuracy. This makes the RAI evaluation problem multi-objective rather than additive — gains on one axis are not free.

• 2024 election misinformation case studies (selected from §3.10):
  • Pakistan: deepfake audio of Imran Khan calling for election boycott (Feb 2024); deepfake video of Yasmin Rashid alleging election commission was bought. Doctored, traced.
  • United States: Chinese-government-linked “spamouflage” influence campaign using AI-generated images on X/Facebook/YouTube/TikTok/Medium; particularly active around the 2024 election; identified by the Institute for Strategic Dialogue.
  • South Africa (May 2024): AI-generated images of Cape Town potholed roads attempting to discredit the Democratic Alliance’s municipal governance; traced by AFP Fact Check / Agence France-Presse.
  • Uruguay (Oct 2024): Morning show staged an “impossible debate” with an AI hologram of presidential candidate Yamandú Orsi.
• Spread broadly: AI-related election misinformation surfaced in >12 countries across >10 social media platforms in 2024. Measurable impact remained unclear — many expected larger effects than were observed.

Standardized evaluations are scarce, but emerging

• Major industrial model developers rarely publish standardized RAI benchmark results. (AI Index 2025 §3 Highlight 1.)
• New benchmark efforts in 2024:
  • HELM Safety — safety evaluation suite from Stanford CRFM
  • AIR-Bench — RAI benchmark
  • FACTS — factuality
  • SimpleQA — factuality
  • Hughes Hallucination Evaluation Model leaderboard (updated)
• Older factuality benchmarks (HaluEval, TruthfulQA) failed to gain widespread adoption within the AI community.

Transparency improving but uneven

• Foundation Model Transparency Index (CRFM, Stanford): average transparency score among major foundation-model developers rose from 37% (Oct 2023) to 58% (May 2024). Substantial progress, but ~42% gap remaining. See foundation-models.

Persistent implicit bias

• LLMs trained for explicit unbiasing continue to exhibit implicit bias on standard probes — including GPT-4 and Claude 3 Sonnet. They:
  • Disproportionately associate negative terms with Black individuals
  • More often associate women with humanities than STEM
  • Favor men for leadership roles
• Although bias metrics on standard benchmarks have improved, bias on implicit-association probes persists.

Governance frameworks proliferating

• 2024 saw simultaneous frameworks from OECD, European Union, United Nations, African Union focused on transparency, trustworthiness, and accountability.
• U.S. federal AI regulations: 59 introduced in 2024 — 2× the 2023 rate, from 2× as many agencies.
• Globally: legislative mentions of AI rose 21.3% across 75 countries since 2023; a 9-fold increase since 2016.

The data commons is shrinking

• A recent study cited in AI Index 2025 §3 Highlight 5 found that data-use restrictions on actively maintained domains in the C4 common-crawl dataset jumped from 5–7% (2023) to 20–33% (2024) — websites are aggressively blocking AI training scrapers.
• Implications: data diversity, model alignment, and scalability under data constraints — and possibly new approaches to training under data scarcity.

RAI is gaining academic attention

• RAI papers at major AI conferences: 992 (2023) → 1,278 (2024) = +28.8% YoY. Continuing a steady annual rise since 2019. Source: AI Index 2025 §3 Highlight 10.

RAI as operationalized stewardship (not just principles)

• MIT CISR’s Four S framework places Stewardship as one of the four organizational challenges to scale AI from pilots (Stage 2) to embedded ways of working (Stage 3) — alongside Strategy, Systems, and Synchronization. Concretely, this means RAI lives in architecture review boards, governance integrated with risk/legal/compliance, and “by design” rather than “by audit”.
• Worked example: Guardian Life Insurance (regulated US insurance) — embedded governance with risk, legal, and compliance teams; architecture reviews via formal and fast-track boards so privacy, security, and regulatory requirements are built into new AI solutions at design time. Source: MIT Sloan article.
• Worked example: Italgas — governance via a Chief People, Innovation & Transformation Officer + an AI Officer + a Group AI Office, overseeing integration and monitoring. Initiatives balance efficiency with new business opportunity (commercializing WorkOnSite generated €3M revenue in 2024). Source: MIT Sloan article.
• This framing complements the policy-level RAI literature (OECD/EU/UN/AU frameworks) by giving the organizational mechanism for how policy principles get translated into procurement and design decisions.

Intent validation as a Constraints-layer RAI control (Chatterjee 2026 + Kokane 2026)

A complementary engineering pattern to the policy and architectural frameworks above: in production agent systems, RAI controls are increasingly implemented at the Constraints layer of the agent harness — the pre-tool / post-tool middleware that gates every action the model proposes.

Chatterjee 2026’s Friday-in-March story is the wiki’s most concrete worked example of an RAI failure traceable to missing harness controls rather than missing model alignment:

“The user had asked it to ‘clean things up before the board review.’ The agent — competent, helpful, working exactly as designed — interpreted this as a request to archive stale documents, prune duplicate insights, and remove sources that hadn’t been touched in a while. Each individual action was defensible. The aggregate was a small disaster… The model was not the problem. The problem lived in the layer around the model — the layer that should have recognized ‘clean up’ as a destructive intent, paused before mass archiving, surfaced a preview, requested confirmation.”

The architectural principle: validate intent, not just output format. Kokane 2026 names this as one of the two genuinely novel kernels of agent-harness engineering: “The Claude Code permission pipeline confirms this: it doesn’t just check if a tool call is syntactically valid, it checks whether the model is authorized to want what it wants.” Specific harness-layer RAI controls flagged across both sources:

• Destructive-verb recognition — pre-tool hook routes the agent toward soft alternatives (preview + confirm) when intent is destructive.
• Workspace isolation — Customer A’s agent cannot reach Customer B’s data, enforced at the executor not the model.
• Loop detection — agent has called the same search 6+ times → surface exhaustion warning rather than burn another twenty cents in tokens.
• Output scoring — post-tool hooks score against citation coverage, source triangulation, severity distribution, density, thematic diversity → inject corrective context if below threshold.

Together with the Anthropic Managed Agents security as structural unreachability principle (below), this gives the wiki a three-layer RAI runtime taxonomy:

Layer	Pattern	Wiki source
Architectural (model unable to directly drive risky tools)	Brain / hands / session decomposition; executor decides whether to run model-emitted tool calls	Anthropic Managed Agents
Application-layer middleware (intent validation, isolation, loop detection)	Pre/post-tool hooks in the harness Constraints layer	Chatterjee 2026
Output evaluation (quality-floor enforcement)	Contracts: formal evaluable specifications of “successful output,” score breakdown injected as corrective context	Chatterjee 2026

The combination is what operationalizes RAI policy frameworks (OECD/EU/UN/AU) at runtime — moving RAI from documentation to deterministic enforcement. The full treatment lives in agent-harness.

Security as structural unreachability (Anthropic Managed Agents)

A complementary engineering pattern to the policy frameworks above: in production agent architectures, the most reliable security control is structural unreachability rather than model-level refusal. The Managed Agents engineering post describes the brain/hands/session decomposition such that the model (the brain) can only emit tool-call requests; a separate executor decides whether and how to run them. Three concrete design moves:

• Per-tool sandboxing. Each tool has its own blast-radius. The hands tier isolates them so a compromised tool call cannot reach others.
• Session ≠ context window. Long-running orchestration state lives outside any one model call, so context-window manipulation by an adversarial input cannot persistently corrupt agent state.
• Out-of-band oversight. Approval gates, allow-lists, and per-call telemetry sit on the executor, not the model — they cannot be reasoned around by the model.

This reframes the ai-agents safety story: rather than relying on the model to refuse dangerous actions (a probabilistic guarantee at best), the architecture makes most dangerous actions structurally unreachable (a deterministic guarantee). Worth pairing with the MITTRI/Cisco AI security taxonomy below — they answer different questions (“what’s the org’s security posture?” vs. “what’s the runtime architecture?”) but reinforce the same direction.

AI security as a discipline (MITTRI/Cisco)

Cisco elevates AI security as a distinct discipline within RAI, with its own fundamentals:

• Understand AI security and safety taxonomy
• Identify vulnerabilities in AI models, software, and hardware code (each is a different attack surface)
• Secure vector databases (an AI-specific concern not captured in traditional infosec)
• Apply established security best practices to AI training environments
• Establish AI security as an ongoing practice — identify unique needs of each phase of the system lifecycle
• Use reference architectures to reduce exposure in LLM training environments
• Select secure embedding models for content generation

Quoted: “Safety and security are fundamental, because they’re one of the big fears impeding adoption for AI technologies today. So if you don’t trust something, you’re not going to use it.” — Jeetu Patel, President & CPO, Cisco.

This framing complements the Anand-Wu guidance to focus on guarding most-critical risks (PII leakage, regulated data) rather than blanket risk minimization. The JPMorgan ChatGPT-block (2023) example illustrates the tradeoff: blocking 60,000 users from experimentation while security teams completed third-party reviews was a sensible precaution but not a sustainable posture.

AI-generated code as verbose and less secure — the earlier-review prescription (McKinsey panel, June 2026)

Palaniappan, Harrysson & Linderman add a code-specific claim to the wiki’s AI-security thread, distinct from the MITTRI/Cisco model/hardware/vector-database taxonomy above: AI-generated code has, in their consulting experience so far, tended to be more verbose and less secure than human-written code. Their prescription is procedural rather than technical — move security and risk review earlier, to design/product-definition time, rather than treating it as a downstream, sequential gate a separate risk team applies after code is written. They expect the quality/security gap to narrow as models improve, but treat what gets checked, and when as the variable organizations can control now.

This is a qualitative, unquantified claim (no cited benchmark or study) — it does not lift the page’s confidence, and should be read as one consulting panel’s practice-informed impression rather than a measured finding. Its value is naming a shift-left prescription for AI-code security specifically, complementing the MITTRI/Cisco taxonomy’s what to secure with a when to check it recommendation.

Risk-vs-experimentation tradeoff (Anand-Wu)

A specific tension that surfaces across organizations: how to permit AI experimentation while managing real risk. Anand-Wu’s framing:

• Don’t try to minimize all risks; focus on guarding the most-critical ones (PII leakage, regulated data, brand-reputation exposure).
• Don’t make IT the bottleneck. “If access stalls at the IT desk or hides behind compliance forms, you cede ground to rivals whose staff can experiment in real time.”
• Innovation moves only as fast as your slowest approval queue.

This is operationally concrete and worth pairing with the MIT CISR Stewardship pillar — which solves the same problem from the architecture-review-board angle.

”Learn responsibly” as a fifth RAI dimension (Ransbotham et al. 2024)

The MIT SMR × BCG 8th annual report identifies a set of RAI concerns specific to organizational learning with AI — distinct from the standard taxonomy (incidents, bias, factuality, governance, transparency, security). Each is illustrated with executive-interview anchors:

• Invasive monitoring as agency threat: workers may perceive AI-enabled knowledge capture as a threat to their agency, reducing engagement. The report cites Mark Surman (Mozilla Foundation) on this as the equity question — “Will vulnerable workers become even more economically insecure when AI eliminates the tasks they are most qualified to do?”
• Knowledge ownership in ecosystem partnerships: knowledge dissemination across firms (federated learning, ecosystem partnerships) carries risk of losing control of knowledge capital. Surman: “the core piece is, there’s just so many questions about copyright and what it means to own knowledge. Maybe the copyright law we have just needs to be reinterpreted for the AI era.”
• Data trust as load-bearing: knowledge dissemination without trust in the underlying data is “a well-known hurdle to data-driven decision-making.” The Expedia Group case study illustrates a nudge approach to dissemination — recommendations rather than consequences/incentives that “directly manipulate behaviors.”
• Worker disposition and learning styles: “Will learning capabilities be equally accessible to humans with a range of learning styles and needs?” — an inclusion concern adjacent to but distinct from the bias-in-output concern of standard RAI.

The report’s prescription: deliberately apply responsible AI practices to ensure knowledge capture and dissemination represent established learning principles and values. This positions “learn responsibly” as a fifth dimension alongside the standard four (fairness/transparency/safety/governance) — explicitly tied to organizational learning rather than to model behavior.

Labor-market disruption as an under-attended RAI concern

The standard RAI taxonomy (incidents, bias, factuality, governance, transparency, security) is missing a major real-world concern: AI’s measurable effects on labor markets. As of late 2025, the empirical evidence is no longer abstract — see ai-employment-effects and Brynjolfsson, Chandar & Chen (2025).

The headline finding worth integrating into RAI thinking:

• Early-career workers (ages 22–25) in the most AI-exposed occupations: ~13% relative decline in employment since late 2022.
• Decline is concentrated in automation uses (vs. augmentation) — see automation-vs-augmentation.
• The pattern is consistent with AI substituting for the codified knowledge that formal education provides; experienced workers’ tacit knowledge is harder to substitute.

For organizations, this is now an explicit RAI question: what is the policy on the entry-level pipeline? Concrete RAI-flavored questions raised by the data:

• If AI deployments shrink your entry-level hiring, who will be the senior workforce in 10–15 years?
• Are your AI deployments augmentative (which appear to leave employment intact) or automative (which appear to substitute for entry-level workers)?
• What reskilling/training/internal-mobility commitments accompany each AI deployment decision?

Most current RAI frameworks (AI Index 2025 §3, OECD/EU/UN/AU) under-weight this concern relative to bias, factuality, and security. Worth tracking when sources begin making this link explicitly.

The nine-German-women postpartum-blood-loss anchor + women’s-cardiac-health gap ( MIT SMR May 2026)

Peron (May 2026) supplies the wiki’s first clinical-domain worked examples of AI-as-bias-auditor in a clinical-protocol setting — distinct from the existing bias-in-LLM-training-data framing in §“Bias mitigation effectiveness” above.

The nine-German-women postpartum-blood-loss anchor. Peron’s WEF-panel anecdote: the global clinical standard for normal postpartum blood loss was established based on nine women in Germany and exported worldwide. India recalibrated to ~300mL given smaller average body sizes. Peron’s frame: “I think technology, that interconnectivity, that now AI can analyse such a big data set so quickly, can really improve the way we are practicing medicine.” Sam Ransbotham (host) extends the implication: “I can also imagine a very simple job for agents would be — hey, go through all of our clinical practices in every area and find the root study for that and assess how that plays out. Your nine-person-in-Germany sample should rise pretty quickly to the top of that list.” The wiki’s first articulation of AI-agent-as-clinical-protocol-bias-auditor at population scale.

Women’s cardiac health gap. Peron names a second worked example with the same structural shape: cardiac causes are the highest-mortality category in women; women experience longer waiting times for cardiac diagnostic because symptoms present differently; most cardiac protocols were designed based on male-only trial data. Peron names two specific physiological differences AI algorithms can incorporate — heart position (slightly different in female vs male chest, affecting MR-imaging capture parameters) and rhythm patterns (the female cardiac rhythm has a slightly-different pattern that algorithms need to calibrate against).

Both examples share an operational structure: AI’s strength at analysing large data sets quickly becomes an active bias-correction tool — auditing population-scale clinical-protocol data to identify samples that don’t match local body populations + retraining diagnostic algorithms with appropriate sub-population variability. The wiki had previously held the AI-bias-correction framing as a LLM-training-data concern (gender/race/class bias in language-model outputs); Peron extends it to the clinical-decision-substrate layer where the biases are upstream of any specific model and embedded in decades of medical-research-sampling decisions. A new RAI category: AI-as-audit-tool-against-pre-AI-embedded-biases. Open follow-up: are there analogous worked examples in finance (credit-scoring protocols based on historic-population samples) or law (sentencing-guidelines based on historic case-mix) where the same audit-pattern would apply?

The pairing with Peron’s Future Health Index trust-gap data (79% of clinicians optimistic about AI vs ~50% of patients worried about reduced face-to-face time) gives this concept page a second new framing: clinician trust is data-driven (bias, validation), patient trust is experiential (will-I-see-my-doctor-less). The reconciliation work in healthcare AI is therefore not a single-discipline trust-building exercise; it’s a two-population audit that has to address two distinct trust-deficits with distinct evidence types.

Vendor-pipeline + human-in-the-loop-as-default in developer tooling ( Microsoft Dec 2025)

The GitHub/Microsoft Agentic DevOps keynote supplies a worked example of responsible AI implemented as platform plumbing, not policy. Every Copilot request is routed through Microsoft’s responsible-AI pipeline, which validates both the input and the returned code (security vulnerabilities, NSFW content) — even when the chosen model is third-party (Anthropic/Google): enough context is sent across the wire, and the response is checked before it reaches the developer, in milliseconds. The load-bearing governance control is structural rather than advisory: by default the person who initiated a Copilot agent request cannot be the final approver to merge into the parent branch — the human-in-the-loop is enforced by the workflow, not left to discipline. “AI suggests, humans decide.”

The keynote also names the AI-code risk surface RAI has to govern (presenter-cited, attribution deferred to the deck): “322% more privilege-escalation paths in AI code,” “40% increase in secrets-exposure risk,” and AI-assisted commits “merged 4× faster, bypassing reviews.” Mechanism: frontier models are trained on the whole internet (including insecure code) and tend to offer the cheap/easy auth (passwords/PATs) over safer OAuth — so GitHub Advanced Security (secret + code scanning + Copilot autofix, “found means fixed”) is positioned as the compensating control. The wiki’s first developer-platform-vendor articulation of RAI-as-enforced-default — complementary to the declarative runtime-enforcement framing in §“Runtime enforcement as a declarative discipline” below (AgentSpec governs agent actions; the Microsoft pipeline governs model I/O; both move RAI from guideline to mechanism).

Singapore’s agentic-governance framework, in operational detail ( AWS Sydney 2026)

The wiki’s Allen page names Singapore’s framework as the only government model worth adopting; Brovich’s Sydney talk supplies the operational detail. Singapore IMDA’s Model AI Governance Framework for Agentic AI v1 — launched January 2026 at Davos by Minister Josephine Teo — is “the first state-backed governance framework specifically designed for autonomous AI agents” (extending IMDA’s 2019 general model). Four dimensions: (1) upfront risk assessment, (2) human accountability chains — “every agent action is traced to a named human,” (3) technical guardrails across the lifecycle, (4) end-user transparency (users must know they’re dealing with an agent and its bounds). Five distinguishing features: it’s the first to mandate agent identity management (a verifiable identity before an agent can act); concrete testing across five risk categories (building on AI Verify + the 2025 Global AI Assurance pilot); explicit multi-agent coordination risk (what happens when agents disagree/escalate/emerge — “no other framework has yet”); voluntary-but-directional (de facto standard for regulated/government work); and it addresses the deskilling trap head-on (must show you keep training the humans who’ll take over).

The architectural punchline — convergent with Amazon Bedrock AgentCore, which “launched on the same four” questions (who’s the agent / who authorised it / what may it do / can we audit it) — is policy enforcement outside the LLM loop: “you don’t ask the agent nicely not to do something; you stop it at the gateway before the LLM ever sees the request,” separating who-writes-policy (security) from who-writes-the-agent (engineering). This is RAI-as-infrastructure (cf. the Microsoft-pipeline and AgentSpec framings above): “running code enforcing rules, every request, every time” — the harness Constraints layer raised to a governance plane.

Agent 365 + asserts + social permission: the hyperscaler RAI-as-infrastructure view ( Possible, June 2026)

Nadella (Microsoft, June 2026) adds a platform-CEO instance of RAI-as-running-infrastructure (cf. the Microsoft-pipeline, AgentSpec, and Singapore framings above):

• Agent 365 — inventory, identity, sandbox, policy, observability. “There may be 20 million agents at Microsoft — I first need an inventory… what are they doing, what are their reasoning traces; they need to be fully inspectable, fully auditable.” Because agents “execute code with file-system and network access,” they get identities (extending Entra), security (Defender), and data labeling/protection (Purview), plus sandboxes and policy governance. This is the agent-identity-management requirement that Singapore’s IMDA framework also mandates, here productised — “security, containment, manageability, observability is how we have confidence around these agents.”
• Asserts vs guardrails-as-classifier. For long-running agents, “instead of guardrails as a classifier, you need the ability during execution to keep the execution path from going off the rails” — runtime boundary enforcement (see agent-harness §asserts), the deterministic-enforcement direction RAI has been moving (cf. AgentSpec).
• Social permission and child safety as first-class RAI. Nadella names child safety alongside cyber/bioweapon/alignment as a first-class AI-safety axis (children’s agency in interacting with chatbots, not being persuaded). And he frames social permission as the industry’s central RAI debt: the sector lost trust by saying “all economic opportunity will go away for knowledge workers” while being excited to build it; the repair is concrete and local — a data center must demonstrably help its community (tax base, schools, water, electricity prices), employment claims must name “the new jobs and wages I can train for,” and every firm must participate in the frontier ecosystem, not just feed data to a foundation model (“sovereignty and dignity both lost”). He ties this to Pope Leo’s encyclical on AI and human dignity. This extends the concept’s labor-market-disruption-as-under-attended-RAI-concern section with a vendor-CEO articulation of earning and keeping social permission as an RAI obligation.

RAI as political economy: extraction, power concentration, and data sovereignty ( Kaushik, June 2026)

The wiki’s RAI coverage has skewed toward governance-and-safety (bias, factuality, red-teaming, runtime enforcement, regulatory compliance). The FT documentary on India’s tech industry adds the under-attended political-economy face of responsible AI — the labor, power, and sovereignty questions that sit upstream of model behavior:

• Extractive supply chain + surveillance-consent. The film frames data-annotation and robot-training labor as “a supply chain which is extractive by design,” and names a concrete consent/surveillance harm: in some geographies workers “will be asked to wear these cameras by their employers with no extra money given to them, and all this data can be used against the workers.” This is a data-provenance-and-consent concern at the labor layer — distinct from the model-output bias/safety concerns the page otherwise tracks.
• Power concentration as a design property. “AI should be understood as a technology that by design concentrates power”, paired with the “whose knowledge was it that created these systems” question about LLMs trained on others’ data while “these companies are now profiting in the billions.” The structural worry: the mediating technology “companies run away with the lion’s share of the value” while the labor that trains them sits at “the bottom end of the tech job.”
• AI-hype as capital mechanism. “AI is fundamentally a marketing term… a way to unlock vast amounts of capital.” This is the material-economy sibling of Guilbeault’s cognitive-science hype critique — the same skepticism of the “grandiose narrative,” grounded in capital flows and labor rather than in what models can’t do.
• Data sovereignty as a national-RAI concern. “Offering up the scale of our population as a carrot to attract foreign tech companies is not a pathway to anything resembling sovereignty or resilience.” Sovereignty-and-resilience joins the wiki’s existing jurisdictional-governance thread (Singapore’s agentic-governance framework) but from the labor-and-development rather than the compliance angle.

The labor-disruption dimension here reinforces the page’s existing supports → ai-employment-effects edge: this source is cited on both pages, with ai-employment-effects carrying the displacement/training-labor mechanics and this page carrying the extraction/power/consent normative reading. Qualitative journalism — does not lift confidence; widens normative coverage.

Debates and supersession

• Open vs. closed model risk framing. Open-weight models are increasingly close to closed-weight in capability (foundation-models) — does that change the policy calculus on disclosure / weights release?
• Election misinformation impact. AI-generated misinformation spread widely in 2024 elections, but its measurable impact remained unclear; many expected larger effects than were observed. Open question: is the saturation of fact-checking infrastructure containing AI-generated content, or is the impact lagged?
• Bias mitigation effectiveness. Explicit-bias mitigation reliably improves bias metrics but not implicit-bias probes. Open question: are current benchmarks measuring the right thing, or is implicit bias structurally untouchable by current alignment methods?
• Risk acknowledgment vs. action gap. Why does the McKinsey-measured gap between “acknowledged risk” and “active mitigation” persist year over year? Worth exploring once another source examines it.

Runtime enforcement as a declarative discipline ( Wang, Poskitt, Sun — ICSE ‘26)

The wiki’s first peer-reviewed primary source specifically on LLM-agent runtime safety as a formal verification problem with declarative rule syntax. AgentSpec (Wang, Poskitt, Sun, ICSE ‘26 — Singapore Management University) introduces a domain-specific language for specifying and enforcing runtime constraints on LLM agents:

rule <Id>
  trigger <Event>
  check <Pred>
  enforce <Enforce>+
end

with enforcement actions: user_inspection / llm_self_examine / invoke_action(Params) / stop.

Empirical anchors across three domains: prevents >90% unsafe code executions (RedCode-Exec, 750 scenarios across 25 vulnerability types), eliminates all hazardous embodied actions (SafeAgentBench, 10 categories), and enforces 100% AV law compliance in 5/8 scenarios (FixDrive). LLM-generated rules (OpenAI o1, few-shot) achieve 87.26% / 95.56% / 5-of-8 precision across the three domains. Runtime overhead is millisecond-scale — effectively free at the scale of LLM-call latency.

The post is the formal-academic operationalisation of Chatterjee’s Constraints layer in the agent-harness taxonomy. The wiki holds RAI’s deployable safety enforcement and agent-harness Constraints layer as overlapping concept territory — AgentSpec is the strongest single artifact in that overlap.

The three named failure modes for LLM-generated rules (overfitting / over-broad rules / insufficient specification) are RAI-relevant in their own right: the rule-generation step is itself an alignment problem the wiki should track.

Related concepts

• enterprise-ai-adoption — the demand-side context that makes RAI urgent
• generative-ai — the technology driving most current incidents
• foundation-models — the locus of transparency and provenance discussions
• ai-benchmarks — methodological intersection (RAI benchmarks live here)