The announcement sounded reassuring. A major AI laboratory would establish an independent safety board. Experts from academia, civil society, and government would scrutinize the company’s most powerful models before deployment. The public could trust that someone qualified was checking the work.
Then, quietly, the board’s actual powers were examined more closely. It could offer recommendations. It could ask questions. It could not delay a product launch, compel document disclosure, or impose any consequence on the company that had created it. The board was independent in name. In practice, it was advisory, funded by the very organization it was meant to scrutinize.
This dynamic plays out repeatedly across the AI industry, and it sits at the heart of one of the most consequential debates in technology today. As AI systems become embedded in hiring decisions, medical diagnoses, financial services, and national security infrastructure, the question of who is actually watching, and with what authority, has moved from academic seminar to urgent public concern.
What “Independent” Really Means in This Context
Independence, in the context of AI oversight, is a term that carries significant definitional weight. A body can be independent in its personnel, meaning its members have no financial relationship with the company it reviews. It can be independent in its funding, drawing on public money or philanthropic sources rather than industry revenue. It can be independent in its authority, possessing legal powers that exist outside any company’s consent.
Most existing AI oversight mechanisms achieve some of these conditions. Almost none achieve all three simultaneously.
The closest analogs from other industries are instructive. Aviation safety boards in the United States and Europe have statutory authority, access to accident data, and the power to ground aircraft. Financial regulators can examine internal records and impose fines. Drug regulators can block product approval. These bodies are imperfect, and regulatory capture is a persistent risk in each sector, but they operate with a legal foundation that most AI oversight lacks entirely.
The AI field, by contrast, has largely produced what researchers and policy observers have described as a patchwork of voluntary commitments, self-assessments, and advisory panels. Companies publish safety cards, system cards, and model cards. They release responsible scaling policies. They invite third-party auditors to conduct evaluations. Each of these practices has genuine value. None of them constitutes binding oversight in the traditional regulatory sense.
The Voluntary Commitment Era and Its Limits
The period between roughly 2023 and 2025 produced a wave of voluntary commitments from leading AI developers. Governments in the United States, the United Kingdom, and the European Union convened summits at which companies pledged to share safety information with authorities, conduct pre-deployment testing for dangerous capabilities, and not undermine democratic institutions with their systems.
These commitments were meaningful signals. They established norms, created reputational stakes, and in some cases led to real internal changes at companies. Anthropic, OpenAI, Google DeepMind, Meta, and others each developed internal safety teams and published documentation about their evaluation processes. Anthropic’s responsible scaling policy, made public in 2023, articulated conditions under which the company would slow or halt development of certain capabilities.
But voluntary commitments have a structural problem that critics, including some who designed them, have acknowledged openly: they depend entirely on the committing party to define compliance. When a company says it will test for dangerous capabilities before deployment, the question of what counts as a dangerous capability, what testing methodology is adequate, and what result would actually halt deployment remains within the company’s discretion. An external observer can read the published policy. They cannot easily verify whether it was followed.
Red-teaming exercises illustrate this tension well. Pre-deployment red-teaming, in which a team of researchers attempts to elicit harmful outputs from a model before it is released, has become a broadly adopted practice. The concept is sound. But red-team exercises conducted solely by internal employees, or by external contractors hired and briefed by the company, face inherent limitations. The scope of what gets tested, the severity thresholds that would trigger concern, and whether any finding actually delays release are all decisions made inside the organization being evaluated.
Government Bodies and Their Early Efforts
Governments have begun constructing more formal oversight infrastructure, though the results vary considerably in scope and authority.
The United Kingdom’s AI Safety Institute, established in late 2023 (and renamed the AI Security Institute in February 2025, with a sharper focus on security risks), represented an early attempt at a government-run technical evaluation body. It was designed to test frontier AI models for dangerous capabilities before and after deployment, with particular focus on potential threats related to weapons of mass destruction, cyberattacks, and societal manipulation. The institute operated with a staff of technical researchers rather than relying solely on policy generalists, which gave it credibility in the research community.
The United States established a parallel body within the National Institute of Standards and Technology, which was likewise renamed in June 2025 as the Center for AI Standards and Innovation. The European Union’s AI Act, which entered into force in 2024, created a tiered regulatory framework based on risk level, with the most stringent requirements applied to systems deemed to pose unacceptable or high risk. That framework assigned compliance responsibilities to both developers and deployers, and it included requirements for conformity assessments and post-market monitoring.
Each of these developments marked genuine progress. Critics have nonetheless identified gaps. Governments face significant asymmetries relative to frontier AI companies: the companies have more compute, more proprietary data, more highly compensated researchers, and far greater speed. A government body that takes eighteen months to hire a technical team and design an evaluation framework is working in a different temporal world than a lab that ships a new model version every few months.
There is also the question of what information regulators can actually access. Internal training data, model weights, evaluation results, and red-team findings are all commercially sensitive. Companies have competitive incentives not to share them, and in most jurisdictions, absent specific statutory authority, they are not required to.
Third-Party Auditors: Promise and Problems
Between the self-assessment model and formal government regulation sits a third approach: independent third-party auditing. Several organizations have begun offering AI auditing services, and some AI developers have voluntarily submitted systems for external evaluation.
The concept draws on precedents from financial auditing and cybersecurity assessment. An external team with technical expertise examines a system against defined criteria and issues a report. In the best cases, the auditor has genuine access to model internals, not just the public-facing product.
In practice, several constraints limit what current AI audits can accomplish. Access remains a central problem: auditors typically evaluate what the company chooses to show them. The criteria for evaluation are often negotiated rather than standardized. There is no universal accreditation body establishing what qualifies as a legitimate AI auditor, no equivalent to the standards that govern financial auditing in major economies.
Some researchers have argued that behavioral evaluations, testing what a model does in response to specific prompts, are inherently incomplete because they cannot reveal what a model might do in conditions the auditor did not test. Interpretability research, which aims to understand what is actually happening inside a model’s computation rather than just what outputs it produces, remains an active area of study rather than a mature auditing tool.
There are also incentive questions. Third-party auditors are generally paid by the companies they audit. This creates pressures that are familiar from the financial sector’s pre-2008 experience with credit rating agencies, in which institutions paid to be rated had leverage over the rating process. No AI auditing scandal of comparable scale has emerged, but the structural concern is regularly raised by governance researchers.
Anthropic’s Internal Architecture as a Case Study
Anthropic occupies an unusual position in this landscape. The company was founded in 2021 by former members of OpenAI who, among other stated motivations, wanted to prioritize safety research. Its organizational structure is a public benefit corporation, and it has published detailed documentation about its approach to model evaluation and deployment decisions.
Anthropic’s original responsible scaling policy articulated a framework in which the company committed to halting or slowing development if its models reached capability thresholds associated with serious risks, including potential contributions to biological or chemical weapons or the ability to conduct sophisticated autonomous cyberattacks. In February 2026 the company revised the policy (version 3.0), dropping that central commitment not to proceed when adequate mitigations could not be guaranteed in advance and shifting toward published risk reports and a safety roadmap. The company uses what it calls AI Safety Levels, modeled loosely on biosafety level frameworks, to categorize risk and specify what safety measures must be in place before deployment.
This structure represents a more detailed public commitment than most companies have made. It has also attracted scrutiny. External researchers have noted that the thresholds themselves are defined internally, the evaluations are conducted or commissioned internally, and the decision to conclude that a threshold has or has not been crossed is made internally. The policy is transparent about its architecture; it is not externally verified.
Anthropic has also established a Long-Term Benefit Trust, a body designed to have oversight authority over the company’s actions in extreme scenarios. The details of that body’s actual powers and the conditions under which they would be invoked remain subjects of ongoing discussion in the governance community.
The company’s situation illustrates a broader pattern: organizations that take safety seriously can build sophisticated internal accountability structures, and those structures are better than nothing. They are not, however, the same as external accountability. The gap between internal good faith and verified compliance is where most of the serious governance proposals of recent years have concentrated.
Where Oversight Goes From Here
The trajectory of AI oversight over the past several years has been toward greater formalization, though the pace has been uneven and the mechanisms remain fragmented. Several directions are actively being pursued simultaneously.
Compute governance has attracted significant attention. The argument is that very large AI training runs require specialized semiconductor hardware, and that hardware supply chains are geographically concentrated and commercially trackable. Requiring disclosure of large training runs to a government body, or building monitoring into the infrastructure itself, would provide earlier warning of capability development than post-hoc model evaluation. Proposals along these lines have been discussed in policy circles in multiple countries.
International coordination is another area of active work. The national AI institutes in different countries have begun sharing information, and forums have emerged for technical experts across governments to compare evaluation methodologies. Progress has been slower than some proponents hoped, partly because governments have competing interests in AI development and are reluctant to constrain their own national champions.
Structured access regimes, in which independent researchers can query or examine frontier models under controlled conditions, have also been proposed as a middle path between full transparency and complete opacity. This would allow external researchers to conduct evaluations without requiring companies to release model weights publicly.
None of these approaches is sufficient on its own. Researchers who study AI governance broadly share the view that effective oversight will ultimately require a combination of mandatory disclosure requirements, technically competent government bodies, accredited third-party evaluation, and ongoing interpretability research that makes models more legible to outside observers. The voluntary commitment era established important norms and demonstrated that AI companies will engage with oversight processes when they believe the stakes justify it. The next period will test whether that engagement can be formalized into something with real accountability teeth.
The core challenge has not changed since those early advisory board announcements. Oversight that cannot compel disclosure, cannot delay deployment, and cannot impose consequences for noncompliance is, in a meaningful sense, not oversight. It is consultation. Understanding that distinction, and designing institutions that can bridge it, may be the most consequential governance problem of the decade.