Responsible AI

Confidence 0.95 · 25 sources · last confirmed 2026-08-30

The umbrella discipline (and increasingly, the corporate function) addressing AI’s risks: bias, factuality, safety, transparency, accountability, security, regulatory compliance. As of 2024, organizational acknowledgment of RAI risks has outpaced mitigation effort — and standardized RAI evaluations remain scarce among major industrial developers.

Working definition

“Responsible AI” (RAI) refers to a constellation of principles and practices for developing and deploying AI systems that are fair, accountable, transparent, secure, and aligned with stakeholder values. Concretely, an org’s RAI work typically spans: bias evaluation, safety/red-teaming, transparency reporting, incident logging, regulatory compliance, governance structures, and content provenance.

Closely related but not identical: AI safety (often used to mean alignment / catastrophic-risk research), AI ethics (more philosophical), AI governance (more institutional / policy-facing).

Key claims

Risk acknowledgment > mitigation

• McKinsey RAI survey via AI Index 2025 §3 Highlight 3: organizations identify key risks but most are not taking active mitigating steps. Top risks cited: inaccuracy 64%, regulatory compliance 63%, cybersecurity 60% — all under 70%, and far fewer say they’re actively addressing them.

Incidents are rising

• AI incidents +56.4% YoY: 233 reported in 2024 (record high). Source: AI Incidents Database via AI Index 2025 §3.
• 2025 update: 362 documented incidents (AI Index 2026 Top Takeaway #6) — another ~55% YoY rise. Reporting on responsible-AI benchmarks remains spotty even as capability-benchmark reporting is near-universal among frontier labs.

RAI dimensions can trade off (AI Index 2026)

A new finding flagged in 2026: improving one responsible-AI dimension can degrade another. Example pattern: improving a model’s safety can degrade its accuracy. This makes the RAI evaluation problem multi-objective rather than additive — gains on one axis are not free.

• 2024 election misinformation case studies (selected from §3.10):
  • Pakistan: deepfake audio of Imran Khan calling for election boycott (Feb 2024); deepfake video of Yasmin Rashid alleging election commission was bought. Doctored, traced.
  • United States: Chinese-government-linked “spamouflage” influence campaign using AI-generated images on X/Facebook/YouTube/TikTok/Medium; particularly active around the 2024 election; identified by the Institute for Strategic Dialogue.
  • South Africa (May 2024): AI-generated images of Cape Town potholed roads attempting to discredit the Democratic Alliance’s municipal governance; traced by AFP Fact Check / Agence France-Presse.
  • Uruguay (Oct 2024): Morning show staged an “impossible debate” with an AI hologram of presidential candidate Yamandú Orsi.
• Spread broadly: AI-related election misinformation surfaced in >12 countries across >10 social media platforms in 2024. Measurable impact remained unclear — many expected larger effects than were observed.

Standardized evaluations are scarce, but emerging

• Major industrial model developers rarely publish standardized RAI benchmark results. (AI Index 2025 §3 Highlight 1.)
• New benchmark efforts in 2024:
  • HELM Safety — safety evaluation suite from Stanford CRFM
  • AIR-Bench — RAI benchmark
  • FACTS — factuality
  • SimpleQA — factuality
  • Hughes Hallucination Evaluation Model leaderboard (updated)
• Older factuality benchmarks (HaluEval, TruthfulQA) failed to gain widespread adoption within the AI community.

Transparency improving but uneven

• Foundation Model Transparency Index (CRFM, Stanford): average transparency score among major foundation-model developers rose from 37% (Oct 2023) to 58% (May 2024). Substantial progress, but ~42% gap remaining. See foundation-models.

Persistent implicit bias

• LLMs trained for explicit unbiasing continue to exhibit implicit bias on standard probes — including GPT-4 and Claude 3 Sonnet. They:
  • Disproportionately associate negative terms with Black individuals
  • More often associate women with humanities than STEM
  • Favor men for leadership roles
• Although bias metrics on standard benchmarks have improved, bias on implicit-association probes persists.

Governance frameworks proliferating

• 2024 saw simultaneous frameworks from OECD, European Union, United Nations, African Union focused on transparency, trustworthiness, and accountability.
• U.S. federal AI regulations: 59 introduced in 2024 — 2× the 2023 rate, from 2× as many agencies.
• Globally: legislative mentions of AI rose 21.3% across 75 countries since 2023; a 9-fold increase since 2016.

The data commons is shrinking

• A recent study cited in AI Index 2025 §3 Highlight 5 found that data-use restrictions on actively maintained domains in the C4 common-crawl dataset jumped from 5–7% (2023) to 20–33% (2024) — websites are aggressively blocking AI training scrapers.
• Implications: data diversity, model alignment, and scalability under data constraints — and possibly new approaches to training under data scarcity.

RAI is gaining academic attention

• RAI papers at major AI conferences: 992 (2023) → 1,278 (2024) = +28.8% YoY. Continuing a steady annual rise since 2019. Source: AI Index 2025 §3 Highlight 10.

RAI as operationalized stewardship (not just principles)

• MIT CISR’s Four S framework places Stewardship as one of the four organizational challenges to scale AI from pilots (Stage 2) to embedded ways of working (Stage 3) — alongside Strategy, Systems, and Synchronization. Concretely, this means RAI lives in architecture review boards, governance integrated with risk/legal/compliance, and “by design” rather than “by audit”.
• Worked example: Guardian Life Insurance (regulated US insurance) — embedded governance with risk, legal, and compliance teams; architecture reviews via formal and fast-track boards so privacy, security, and regulatory requirements are built into new AI solutions at design time. Source: MIT Sloan article.
• Worked example: Italgas — governance via a Chief People, Innovation & Transformation Officer + an AI Officer + a Group AI Office, overseeing integration and monitoring. Initiatives balance efficiency with new business opportunity (commercializing WorkOnSite generated €3M revenue in 2024). Source: MIT Sloan article.
• This framing complements the policy-level RAI literature (OECD/EU/UN/AU frameworks) by giving the organizational mechanism for how policy principles get translated into procurement and design decisions.

Intent validation as a Constraints-layer RAI control (Chatterjee 2026 + Kokane 2026)

A complementary engineering pattern to the policy and architectural frameworks above: in production agent systems, RAI controls are increasingly implemented at the Constraints layer of the agent harness — the pre-tool / post-tool middleware that gates every action the model proposes.

Chatterjee 2026’s Friday-in-March story is the wiki’s most concrete worked example of an RAI failure traceable to missing harness controls rather than missing model alignment:

“The user had asked it to ‘clean things up before the board review.’ The agent — competent, helpful, working exactly as designed — interpreted this as a request to archive stale documents, prune duplicate insights, and remove sources that hadn’t been touched in a while. Each individual action was defensible. The aggregate was a small disaster… The model was not the problem. The problem lived in the layer around the model — the layer that should have recognized ‘clean up’ as a destructive intent, paused before mass archiving, surfaced a preview, requested confirmation.”

The architectural principle: validate intent, not just output format. Kokane 2026 names this as one of the two genuinely novel kernels of agent-harness engineering: “The Claude Code permission pipeline confirms this: it doesn’t just check if a tool call is syntactically valid, it checks whether the model is authorized to want what it wants.” Specific harness-layer RAI controls flagged across both sources:

• Destructive-verb recognition — pre-tool hook routes the agent toward soft alternatives (preview + confirm) when intent is destructive.
• Workspace isolation — Customer A’s agent cannot reach Customer B’s data, enforced at the executor not the model.
• Loop detection — agent has called the same search 6+ times → surface exhaustion warning rather than burn another twenty cents in tokens.
• Output scoring — post-tool hooks score against citation coverage, source triangulation, severity distribution, density, thematic diversity → inject corrective context if below threshold.

Together with the Anthropic Managed Agents security as structural unreachability principle (below), this gives the wiki a three-layer RAI runtime taxonomy:

Layer	Pattern	Wiki source
Architectural (model unable to directly drive risky tools)	Brain / hands / session decomposition; executor decides whether to run model-emitted tool calls	Anthropic Managed Agents
Application-layer middleware (intent validation, isolation, loop detection)	Pre/post-tool hooks in the harness Constraints layer	Chatterjee 2026
Output evaluation (quality-floor enforcement)	Contracts: formal evaluable specifications of “successful output,” score breakdown injected as corrective context	Chatterjee 2026

The combination is what operationalizes RAI policy frameworks (OECD/EU/UN/AU) at runtime — moving RAI from documentation to deterministic enforcement. The full treatment lives in agent-harness.

Security as structural unreachability (Anthropic Managed Agents)

A complementary engineering pattern to the policy frameworks above: in production agent architectures, the most reliable security control is structural unreachability rather than model-level refusal. The Managed Agents engineering post describes the brain/hands/session decomposition such that the model (the brain) can only emit tool-call requests; a separate executor decides whether and how to run them. Three concrete design moves:

• Per-tool sandboxing. Each tool has its own blast-radius. The hands tier isolates them so a compromised tool call cannot reach others.
• Session ≠ context window. Long-running orchestration state lives outside any one model call, so context-window manipulation by an adversarial input cannot persistently corrupt agent state.
• Out-of-band oversight. Approval gates, allow-lists, and per-call telemetry sit on the executor, not the model — they cannot be reasoned around by the model.

This reframes the ai-agents safety story: rather than relying on the model to refuse dangerous actions (a probabilistic guarantee at best), the architecture makes most dangerous actions structurally unreachable (a deterministic guarantee). Worth pairing with the MITTRI/Cisco AI security taxonomy below — they answer different questions (“what’s the org’s security posture?” vs. “what’s the runtime architecture?”) but reinforce the same direction.

AI security as a discipline (MITTRI/Cisco)

Cisco elevates AI security as a distinct discipline within RAI, with its own fundamentals:

• Understand AI security and safety taxonomy
• Identify vulnerabilities in AI models, software, and hardware code (each is a different attack surface)
• Secure vector databases (an AI-specific concern not captured in traditional infosec)
• Apply established security best practices to AI training environments
• Establish AI security as an ongoing practice — identify unique needs of each phase of the system lifecycle
• Use reference architectures to reduce exposure in LLM training environments
• Select secure embedding models for content generation

Quoted: “Safety and security are fundamental, because they’re one of the big fears impeding adoption for AI technologies today. So if you don’t trust something, you’re not going to use it.” — Jeetu Patel, President & CPO, Cisco.

This framing complements the Anand-Wu guidance to focus on guarding most-critical risks (PII leakage, regulated data) rather than blanket risk minimization. The JPMorgan ChatGPT-block (2023) example illustrates the tradeoff: blocking 60,000 users from experimentation while security teams completed third-party reviews was a sensible precaution but not a sustainable posture.

AI-generated code as verbose and less secure — the earlier-review prescription (McKinsey panel, June 2026)

Palaniappan, Harrysson & Linderman add a code-specific claim to the wiki’s AI-security thread, distinct from the MITTRI/Cisco model/hardware/vector-database taxonomy above: AI-generated code has, in their consulting experience so far, tended to be more verbose and less secure than human-written code. Their prescription is procedural rather than technical — move security and risk review earlier, to design/product-definition time, rather than treating it as a downstream, sequential gate a separate risk team applies after code is written. They expect the quality/security gap to narrow as models improve, but treat what gets checked, and when as the variable organizations can control now.

This is a qualitative, unquantified claim (no cited benchmark or study) — it does not lift the page’s confidence, and should be read as one consulting panel’s practice-informed impression rather than a measured finding. Its value is naming a shift-left prescription for AI-code security specifically, complementing the MITTRI/Cisco taxonomy’s what to secure with a when to check it recommendation.

Risk-vs-experimentation tradeoff (Anand-Wu)

A specific tension that surfaces across organizations: how to permit AI experimentation while managing real risk. Anand-Wu’s framing:

• Don’t try to minimize all risks; focus on guarding the most-critical ones (PII leakage, regulated data, brand-reputation exposure).
• Don’t make IT the bottleneck. “If access stalls at the IT desk or hides behind compliance forms, you cede ground to rivals whose staff can experiment in real time.”
• Innovation moves only as fast as your slowest approval queue.

This is operationally concrete and worth pairing with the MIT CISR Stewardship pillar — which solves the same problem from the architecture-review-board angle.

”Learn responsibly” as a fifth RAI dimension (Ransbotham et al. 2024)

The MIT SMR × BCG 8th annual report identifies a set of RAI concerns specific to organizational learning with AI — distinct from the standard taxonomy (incidents, bias, factuality, governance, transparency, security). Each is illustrated with executive-interview anchors:

• Invasive monitoring as agency threat: workers may perceive AI-enabled knowledge capture as a threat to their agency, reducing engagement. The report cites Mark Surman (Mozilla Foundation) on this as the equity question — “Will vulnerable workers become even more economically insecure when AI eliminates the tasks they are most qualified to do?”
• Knowledge ownership in ecosystem partnerships: knowledge dissemination across firms (federated learning, ecosystem partnerships) carries risk of losing control of knowledge capital. Surman: “the core piece is, there’s just so many questions about copyright and what it means to own knowledge. Maybe the copyright law we have just needs to be reinterpreted for the AI era.”
• Data trust as load-bearing: knowledge dissemination without trust in the underlying data is “a well-known hurdle to data-driven decision-making.” The Expedia Group case study illustrates a nudge approach to dissemination — recommendations rather than consequences/incentives that “directly manipulate behaviors.”
• Worker disposition and learning styles: “Will learning capabilities be equally accessible to humans with a range of learning styles and needs?” — an inclusion concern adjacent to but distinct from the bias-in-output concern of standard RAI.

The report’s prescription: deliberately apply responsible AI practices to ensure knowledge capture and dissemination represent established learning principles and values. This positions “learn responsibly” as a fifth dimension alongside the standard four (fairness/transparency/safety/governance) — explicitly tied to organizational learning rather than to model behavior.

Labor-market disruption as an under-attended RAI concern

The standard RAI taxonomy (incidents, bias, factuality, governance, transparency, security) is missing a major real-world concern: AI’s measurable effects on labor markets. As of late 2025, the empirical evidence is no longer abstract — see ai-employment-effects and Brynjolfsson, Chandar & Chen (2025).

The headline finding worth integrating into RAI thinking:

• Early-career workers (ages 22–25) in the most AI-exposed occupations: ~13% relative decline in employment since late 2022.
• Decline is concentrated in automation uses (vs. augmentation) — see automation-vs-augmentation.
• The pattern is consistent with AI substituting for the codified knowledge that formal education provides; experienced workers’ tacit knowledge is harder to substitute.

For organizations, this is now an explicit RAI question: what is the policy on the entry-level pipeline? Concrete RAI-flavored questions raised by the data:

• If AI deployments shrink your entry-level hiring, who will be the senior workforce in 10–15 years?
• Are your AI deployments augmentative (which appear to leave employment intact) or automative (which appear to substitute for entry-level workers)?
• What reskilling/training/internal-mobility commitments accompany each AI deployment decision?

Most current RAI frameworks (AI Index 2025 §3, OECD/EU/UN/AU) under-weight this concern relative to bias, factuality, and security. Worth tracking when sources begin making this link explicitly.

The nine-German-women postpartum-blood-loss anchor + women’s-cardiac-health gap ( MIT SMR May 2026)

Peron (May 2026) supplies the wiki’s first clinical-domain worked examples of AI-as-bias-auditor in a clinical-protocol setting — distinct from the existing bias-in-LLM-training-data framing in §“Bias mitigation effectiveness” above.

The nine-German-women postpartum-blood-loss anchor. Peron’s WEF-panel anecdote: the global clinical standard for normal postpartum blood loss was established based on nine women in Germany and exported worldwide. India recalibrated to ~300mL given smaller average body sizes. Peron’s frame: “I think technology, that interconnectivity, that now AI can analyse such a big data set so quickly, can really improve the way we are practicing medicine.” Sam Ransbotham (host) extends the implication: “I can also imagine a very simple job for agents would be — hey, go through all of our clinical practices in every area and find the root study for that and assess how that plays out. Your nine-person-in-Germany sample should rise pretty quickly to the top of that list.” The wiki’s first articulation of AI-agent-as-clinical-protocol-bias-auditor at population scale.

Women’s cardiac health gap. Peron names a second worked example with the same structural shape: cardiac causes are the highest-mortality category in women; women experience longer waiting times for cardiac diagnostic because symptoms present differently; most cardiac protocols were designed based on male-only trial data. Peron names two specific physiological differences AI algorithms can incorporate — heart position (slightly different in female vs male chest, affecting MR-imaging capture parameters) and rhythm patterns (the female cardiac rhythm has a slightly-different pattern that algorithms need to calibrate against).

Both examples share an operational structure: AI’s strength at analysing large data sets quickly becomes an active bias-correction tool — auditing population-scale clinical-protocol data to identify samples that don’t match local body populations + retraining diagnostic algorithms with appropriate sub-population variability. The wiki had previously held the AI-bias-correction framing as a LLM-training-data concern (gender/race/class bias in language-model outputs); Peron extends it to the clinical-decision-substrate layer where the biases are upstream of any specific model and embedded in decades of medical-research-sampling decisions. A new RAI category: AI-as-audit-tool-against-pre-AI-embedded-biases. Open follow-up: are there analogous worked examples in finance (credit-scoring protocols based on historic-population samples) or law (sentencing-guidelines based on historic case-mix) where the same audit-pattern would apply?

The pairing with Peron’s Future Health Index trust-gap data (79% of clinicians optimistic about AI vs ~50% of patients worried about reduced face-to-face time) gives this concept page a second new framing: clinician trust is data-driven (bias, validation), patient trust is experiential (will-I-see-my-doctor-less). The reconciliation work in healthcare AI is therefore not a single-discipline trust-building exercise; it’s a two-population audit that has to address two distinct trust-deficits with distinct evidence types.

Vendor-pipeline + human-in-the-loop-as-default in developer tooling ( Microsoft Dec 2025)

The GitHub/Microsoft Agentic DevOps keynote supplies a worked example of responsible AI implemented as platform plumbing, not policy. Every Copilot request is routed through Microsoft’s responsible-AI pipeline, which validates both the input and the returned code (security vulnerabilities, NSFW content) — even when the chosen model is third-party (Anthropic/Google): enough context is sent across the wire, and the response is checked before it reaches the developer, in milliseconds. The load-bearing governance control is structural rather than advisory: by default the person who initiated a Copilot agent request cannot be the final approver to merge into the parent branch — the human-in-the-loop is enforced by the workflow, not left to discipline. “AI suggests, humans decide.”

The keynote also names the AI-code risk surface RAI has to govern (presenter-cited, attribution deferred to the deck): “322% more privilege-escalation paths in AI code,” “40% increase in secrets-exposure risk,” and AI-assisted commits “merged 4× faster, bypassing reviews.” Mechanism: frontier models are trained on the whole internet (including insecure code) and tend to offer the cheap/easy auth (passwords/PATs) over safer OAuth — so GitHub Advanced Security (secret + code scanning + Copilot autofix, “found means fixed”) is positioned as the compensating control. The wiki’s first developer-platform-vendor articulation of RAI-as-enforced-default — complementary to the declarative runtime-enforcement framing in §“Runtime enforcement as a declarative discipline” below (AgentSpec governs agent actions; the Microsoft pipeline governs model I/O; both move RAI from guideline to mechanism).

Singapore’s agentic-governance framework, in operational detail ( AWS Sydney 2026)

The wiki’s Allen page names Singapore’s framework as the only government model worth adopting; Brovich’s Sydney talk supplies the operational detail. Singapore IMDA’s Model AI Governance Framework for Agentic AI v1 — launched January 2026 at Davos by Minister Josephine Teo — is “the first state-backed governance framework specifically designed for autonomous AI agents” (extending IMDA’s 2019 general model). Four dimensions: (1) upfront risk assessment, (2) human accountability chains — “every agent action is traced to a named human,” (3) technical guardrails across the lifecycle, (4) end-user transparency (users must know they’re dealing with an agent and its bounds). Five distinguishing features: it’s the first to mandate agent identity management (a verifiable identity before an agent can act); concrete testing across five risk categories (building on AI Verify + the 2025 Global AI Assurance pilot); explicit multi-agent coordination risk (what happens when agents disagree/escalate/emerge — “no other framework has yet”); voluntary-but-directional (de facto standard for regulated/government work); and it addresses the deskilling trap head-on (must show you keep training the humans who’ll take over).

The architectural punchline — convergent with Amazon Bedrock AgentCore, which “launched on the same four” questions (who’s the agent / who authorised it / what may it do / can we audit it) — is policy enforcement outside the LLM loop: “you don’t ask the agent nicely not to do something; you stop it at the gateway before the LLM ever sees the request,” separating who-writes-policy (security) from who-writes-the-agent (engineering). This is RAI-as-infrastructure (cf. the Microsoft-pipeline and AgentSpec framings above): “running code enforcing rules, every request, every time” — the harness Constraints layer raised to a governance plane.

Agent 365 + asserts + social permission: the hyperscaler RAI-as-infrastructure view ( Possible, June 2026)

Nadella (Microsoft, June 2026) adds a platform-CEO instance of RAI-as-running-infrastructure (cf. the Microsoft-pipeline, AgentSpec, and Singapore framings above):

• Agent 365 — inventory, identity, sandbox, policy, observability. “There may be 20 million agents at Microsoft — I first need an inventory… what are they doing, what are their reasoning traces; they need to be fully inspectable, fully auditable.” Because agents “execute code with file-system and network access,” they get identities (extending Entra), security (Defender), and data labeling/protection (Purview), plus sandboxes and policy governance. This is the agent-identity-management requirement that Singapore’s IMDA framework also mandates, here productised — “security, containment, manageability, observability is how we have confidence around these agents.”
• Asserts vs guardrails-as-classifier. For long-running agents, “instead of guardrails as a classifier, you need the ability during execution to keep the execution path from going off the rails” — runtime boundary enforcement (see agent-harness §asserts), the deterministic-enforcement direction RAI has been moving (cf. AgentSpec).
• Social permission and child safety as first-class RAI. Nadella names child safety alongside cyber/bioweapon/alignment as a first-class AI-safety axis (children’s agency in interacting with chatbots, not being persuaded). And he frames social permission as the industry’s central RAI debt: the sector lost trust by saying “all economic opportunity will go away for knowledge workers” while being excited to build it; the repair is concrete and local — a data center must demonstrably help its community (tax base, schools, water, electricity prices), employment claims must name “the new jobs and wages I can train for,” and every firm must participate in the frontier ecosystem, not just feed data to a foundation model (“sovereignty and dignity both lost”). He ties this to Pope Leo’s encyclical on AI and human dignity. This extends the concept’s labor-market-disruption-as-under-attended-RAI-concern section with a vendor-CEO articulation of earning and keeping social permission as an RAI obligation.

RAI as political economy: extraction, power concentration, and data sovereignty ( Kaushik, June 2026)

The wiki’s RAI coverage has skewed toward governance-and-safety (bias, factuality, red-teaming, runtime enforcement, regulatory compliance). The FT documentary on India’s tech industry adds the under-attended political-economy face of responsible AI — the labor, power, and sovereignty questions that sit upstream of model behavior:

• Extractive supply chain + surveillance-consent. The film frames data-annotation and robot-training labor as “a supply chain which is extractive by design,” and names a concrete consent/surveillance harm: in some geographies workers “will be asked to wear these cameras by their employers with no extra money given to them, and all this data can be used against the workers.” This is a data-provenance-and-consent concern at the labor layer — distinct from the model-output bias/safety concerns the page otherwise tracks.
• Power concentration as a design property. “AI should be understood as a technology that by design concentrates power”, paired with the “whose knowledge was it that created these systems” question about LLMs trained on others’ data while “these companies are now profiting in the billions.” The structural worry: the mediating technology “companies run away with the lion’s share of the value” while the labor that trains them sits at “the bottom end of the tech job.”
• AI-hype as capital mechanism. “AI is fundamentally a marketing term… a way to unlock vast amounts of capital.” This is the material-economy sibling of Guilbeault’s cognitive-science hype critique — the same skepticism of the “grandiose narrative,” grounded in capital flows and labor rather than in what models can’t do.
• Data sovereignty as a national-RAI concern. “Offering up the scale of our population as a carrot to attract foreign tech companies is not a pathway to anything resembling sovereignty or resilience.” Sovereignty-and-resilience joins the wiki’s existing jurisdictional-governance thread (Singapore’s agentic-governance framework) but from the labor-and-development rather than the compliance angle.

The labor-disruption dimension here reinforces the page’s existing supports → ai-employment-effects edge: this source is cited on both pages, with ai-employment-effects carrying the displacement/training-labor mechanics and this page carrying the extraction/power/consent normative reading. Qualitative journalism — does not lift confidence; widens normative coverage.

”Say I don’t know” — refusal as a design requirement ( MotherDuck, July 2026)

Asked at a panel what she wished her agent could do, Bev Turnbaugh’s answer was “Say I don’t know.” — “I don’t want it to guess. I don’t want it to hallucinate. I don’t want it to make me look bad if I take its answer and give it to somebody.” Her operational form is a rule in the context layer rather than a model property: “if you don’t know what I’m talking about, do not infer — come back and say [you don’t know].” She pairs it with a detection practice most users skip: rather than tabbing away while an agent works, “I watch exactly who it’s talking to, exactly where it’s pulling data, and then I’ll go — oh yeah, he has no clue, look at all these sources he’s looking at and he can’t find an answer,” treating that as the signal to go fix the context rather than to distrust the tool. The framing worth carrying: the dangerous failure is not refusal, it is a fluent answer built on an inferred definition — and the countermeasure sits in the harness, not the weights.

Ontological assumptions as a governance surface diversity fixes don’t reach ( Stanford CS547, July 2026)

This page’s material on bias and fairness largely concerns representation — who appears in outputs, and how. Nava Haghighi argues that leaves the deeper layer untouched. Her claim: “what ontological assumptions get built into systems — encoded into algorithms and models — they risk becoming everyone’s reality,” and as AI systems proliferate “they will reinforce a set of boundaries, making other boundaries and the worlds in which they exist difficult to imagine.”

The line that makes this a distinct critique rather than a restatement is her account of what a diversity fix achieves: “we will keep generating variations of the tree — at best, different species or different styles of rendering, more equitable representations of all trees — without questioning what the boundaries of the tree were to begin with, without any of them ever having roots.” Her demonstration is small and concrete: repeatedly prompting a model for a tree, including by stating her nationality (which produced stereotyped ornament and a desert landscape rather than a different conception of what a tree is), until only “everything in the world is connected” produced one with roots.

She also names an empirical finding from analysing four commercial chatbots and an agent architecture: “even when diverse ontological perspectives exist in training data, specific ontological orientations are surfaced” over others — i.e. corpus diversity does not imply output diversity. Her proposed governance instrument is a four-orientation analytical framework (multiplicity, groundedness, liveliness, enactments), each a way of asking “where is the boundary being drawn, and what is it leaving outside?” — with the stated ambition of moving critique “from reaction to prevention,” i.e. applying it during design rather than after deployment. The framework’s details are in a paper this ingest has not read; treat the framework as a pointer.

A related governance datapoint from the CS329A lecture, stated by two researchers who have worked on frontier models at Google DeepMind and Anthropic: on alignment via RLHF, “this is still a big problem. It’s a big issue. We haven’t mastered it yet.”

Risk appetite as something a board must state before deploying ([[2026-08-16-hill-bloomberg-leaders-ceo-skills-age-of-ai|Hill / Bloomberg Leaders, August 2026]])

Most of this page’s governance material addresses mechanisms — runtime enforcement, refusal behaviour, oversight structures. Linda Hill points at a precondition upstream of all of them: whether the authorising body has ever articulated what risk it is willing to take.

“Very senior people don’t necessarily talk about risk really. They can have very different risk appetites… but they don’t ever really address the fact that they have different risk appetites.”

On boards, where the divergence matters most: “I don’t think board members like to have conflict. Conflict is complicated. So sometimes — do we bring it up or not? It’s there whether you bring it up or not, and it’s going to play out in different ways. But CEOs or board chairs or lead chairs understand that. Nowadays, given what’s going on, we need to talk. We need to say what is our risk appetite as we begin to, for instance, roll out AI solutions. And that conversation is a really hard one to have.”

Her account of why it stays unheld is specific and is the governance-relevant part: the reputational cost of appearing “too conservative, or resistant to the future.” A director who raises a risk concern about an AI deployment is positioned as an obstacle to progress, which suppresses exactly the dissent that responsible-AI governance depends on. This is the same dynamic her creative abrasion work addresses in innovation contexts, applied to the risk conversation.

Two implications for this page. First, a runtime enforcement policy, a refusal specification or an oversight structure all encode a risk posture — and if the posture was never agreed, the encoding is made by whoever wrote the policy rather than by the body accountable for it. Second, Hill’s complementary prescription (decision rights made explicit: “being real clear who has decision-making rights for what, and what do you want to hear before they make a decision”) is the mechanism she offers for surfacing the divergence, since decision-rights work forces the question of what an authoriser needs to see before authorising.

The claim is unmeasured — an experienced adviser’s observation from board programmes, delivered in a podcast interview. It is recorded here as a named failure mode rather than as a finding, and it is the only governance-layer blocker in the corpus that sits above the technical controls rather than among them.

Debates and supersession

Debates and supersession

• Open vs. closed model risk framing. Open-weight models are increasingly close to closed-weight in capability (foundation-models) — does that change the policy calculus on disclosure / weights release?
• Election misinformation impact. AI-generated misinformation spread widely in 2024 elections, but its measurable impact remained unclear; many expected larger effects than were observed. Open question: is the saturation of fact-checking infrastructure containing AI-generated content, or is the impact lagged?
• Bias mitigation effectiveness. Explicit-bias mitigation reliably improves bias metrics but not implicit-bias probes. Open question: are current benchmarks measuring the right thing, or is implicit bias structurally untouchable by current alignment methods?
• Risk acknowledgment vs. action gap. Why does the McKinsey-measured gap between “acknowledged risk” and “active mitigation” persist year over year? Worth exploring once another source examines it.

Runtime enforcement as a declarative discipline ( Wang, Poskitt, Sun — ICSE ‘26)

The wiki’s first peer-reviewed primary source specifically on LLM-agent runtime safety as a formal verification problem with declarative rule syntax. AgentSpec (Wang, Poskitt, Sun, ICSE ‘26 — Singapore Management University) introduces a domain-specific language for specifying and enforcing runtime constraints on LLM agents:

rule <Id>
  trigger <Event>
  check <Pred>
  enforce <Enforce>+
end

with enforcement actions: user_inspection / llm_self_examine / invoke_action(Params) / stop.

Empirical anchors across three domains: prevents >90% unsafe code executions (RedCode-Exec, 750 scenarios across 25 vulnerability types), eliminates all hazardous embodied actions (SafeAgentBench, 10 categories), and enforces 100% AV law compliance in 5/8 scenarios (FixDrive). LLM-generated rules (OpenAI o1, few-shot) achieve 87.26% / 95.56% / 5-of-8 precision across the three domains. Runtime overhead is millisecond-scale — effectively free at the scale of LLM-call latency.

The post is the formal-academic operationalisation of Chatterjee’s Constraints layer in the agent-harness taxonomy. The wiki holds RAI’s deployable safety enforcement and agent-harness Constraints layer as overlapping concept territory — AgentSpec is the strongest single artifact in that overlap.

The three named failure modes for LLM-generated rules (overfitting / over-broad rules / insufficient specification) are RAI-relevant in their own right: the rule-generation step is itself an alignment problem the wiki should track.

Related concepts

• enterprise-ai-adoption — the demand-side context that makes RAI urgent
• generative-ai — the technology driving most current incidents
• foundation-models — the locus of transparency and provenance discussions
• ai-benchmarks — methodological intersection (RAI benchmarks live here)

Agentic AI governance (added 2026-08-30)

Governance for systems that take actions is a different problem from governance for systems that produce outputs, and 2026 is when the frameworks caught up. Full treatment at agent-oversight-and-delegation.

The first dedicated framework. Singapore’s IMDA published what it presents as the world’s first governance framework specifically for agentic AI (Davos, 22 Jan 2026; updated 20 May 2026). Four pillars: bound risks upfront through use-case selection and limits on agents’ powers; make humans meaningfully accountable by defining checkpoints at which approval is required; technical controls across the agent lifecycle, including baseline testing and access restricted to whitelisted services; and end-user responsibility via transparency and training. The structural move is pillar 1’s ex ante posture — risk bounded before deployment rather than detected in outputs afterwards, which is the right shape when failures are actions.

Where the checkpoints go, empirically. Pochampally et al. name delegation regret — “users regret not that the agent erred, but that it acted beyond what they would have authorized” — and find that trust is calibrated per task, not per agent, that the trigger for withdrawing it is irreversibility combined with external visibility rather than stakes, and that regret appears whenever an agent acts without preview, even on successful output. A correct outcome does not repair an unauthorised action.

Convergence is the notable part. A regulator, a solo founder keeping production credentials out of agent hands, a product executive escalating medium-risk PRs to Slack, and a human-factors study arrived at the same control set without citing each other.

What responsible oversight cannot rest on. Models disavow cheating and cheat anyway; prompts requesting honesty do not help. Monitoring the reasoning trace works until you optimise it, after which the misbehaviour continues and the evidence stops. And human self-report about AI’s effects is wrong by ~39 points in a known direction.