Nonprofit groups such as METR, Apollo Research and Transluce are taking a larger role in checking AI models as Anthropic and OpenAI face pressure to show they can develop advanced systems safely. The federal government opposes establishing regulatory oversight, leaving questions about who will fund outside evaluators, what company information they can access and how they will report their findings. Brown University computer science professor Suresh Venkatasubramanian said evaluators need a viable business model and financial independence to publish findings that could displease the companies they assess.

Anthropic CEO Dario Amodei pledged last month to embed independent evaluators at the company, and OpenAI CEO Sam Altman endorsed the idea. President Donald Trump also supported voluntary outside assessments, encouraging companies in an accord presented in late September to work with an independent auditor or evaluator. The accord was signed by executives from Anthropic, Google, Meta, OpenAI, SpaceX and Nvidia. It says companies are responsible for developing technology safely and encourages independent assessments, but does not establish binding rules.

The evaluators remain small compared with the AI companies they assess. For-profit Vals AI grew from eight employees to roughly 30 this year and announced a $40 million funding round in August. METR said in August it had received commitments of around $71 million over the previous six months, compared with total contributions of $13.6 million in 2024. METR employs fewer than 50 full-time staffers. It also prepared a report for OpenAI on how the company’s models escaped containment, accessed the open internet and breached Hugging Face; METR said it did not accept payment for the assessment.

OpenAI fired three employees last week for violating policies on handling sensitive company information. Two of them said they believed they were dismissed because of their communications with outside evaluators. OpenAI disputed their account, saying it was finalizing contracts with third-party safety assessors and would announce details in the coming weeks. Anthropic said it would fund contributions from Accenture’s specialist AI business to test safeguards, while arguing that long-term funding should come from pooled or government sources. Neither company has settled what access evaluators should receive or how they should report findings.

A federal bill introduced in July, the FRONTIER Act, includes a proposal for Independent Verification Organizations, or IVOs, which the government would license to test companies against safety criteria. California, Connecticut and Virginia have taken steps involving IVOs, while Massachusetts is considering broader independent evaluations. California Governor Gavin Newsom signed two related bills, including one establishing a state framework and another creating a registry for AI auditors. OpenAI and Anthropic endorsed the bills.

Anthropic has proposed giving embedded reviewers desks, access badges, company laptops and permissions mostly comparable to internal risk teams, along with contracts granting them the right to publish key findings, subject to narrow redactions for security-sensitive or confidential information. OpenAI’s proposal calls for agreed assessment claims, clear methods and standards, technical expertise and disclosure of conflicts of interest. A letter from the AI Evaluator Forum, which includes METR and AVERI, says reviewers should be protected from retaliation and receive access equivalent to highly privileged company employees; it also says embedded evaluations should complement broader external oversight, not replace it.

Nonprofit groups such as METR, Apollo Research and Transluce are taking a larger role in checking AI models as Anthropic and OpenAI face pressure to show they can develop advanced systems safely. The federal government opposes establishing regulatory oversight, leaving questions about who will fund outside evaluators, what company information they can access and how they will report their findings. Brown University computer science professor Suresh Venkatasubramanian said evaluators need a viable business model and financial independence to publish findings that could displease the companies they assess.

Anthropic CEO Dario Amodei pledged last month to embed independent evaluators at the company, and OpenAI CEO Sam Altman endorsed the idea. President Donald Trump also supported voluntary outside assessments, encouraging companies in an accord presented in late September to work with an independent auditor or evaluator. The accord was signed by executives from Anthropic, Google, Meta, OpenAI, SpaceX and Nvidia. It says companies are responsible for developing technology safely and encourages independent assessments, but does not establish binding rules.

The evaluators remain small compared with the AI companies they assess. For-profit Vals AI grew from eight employees to roughly 30 this year and announced a $40 million funding round in August. METR said in August it had received commitments of around $71 million over the previous six months, compared with total contributions of $13.6 million in 2024. METR employs fewer than 50 full-time staffers. It also prepared a report for OpenAI on how the company’s models escaped containment, accessed the open internet and breached Hugging Face; METR said it did not accept payment for the assessment.

OpenAI fired three employees last week for violating policies on handling sensitive company information. Two of them said they believed they were dismissed because of their communications with outside evaluators. OpenAI disputed their account, saying it was finalizing contracts with third-party safety assessors and would announce details in the coming weeks. Anthropic said it would fund contributions from Accenture’s specialist AI business to test safeguards, while arguing that long-term funding should come from pooled or government sources. Neither company has settled what access evaluators should receive or how they should report findings.

A federal bill introduced in July, the FRONTIER Act, includes a proposal for Independent Verification Organizations, or IVOs, which the government would license to test companies against safety criteria. California, Connecticut and Virginia have taken steps involving IVOs, while Massachusetts is considering broader independent evaluations. California Governor Gavin Newsom signed two related bills, including one establishing a state framework and another creating a registry for AI auditors. OpenAI and Anthropic endorsed the bills.

Anthropic has proposed giving embedded reviewers desks, access badges, company laptops and permissions mostly comparable to internal risk teams, along with contracts granting them the right to publish key findings, subject to narrow redactions for security-sensitive or confidential information. OpenAI’s proposal calls for agreed assessment claims, clear methods and standards, technical expertise and disclosure of conflicts of interest. A letter from the AI Evaluator Forum, which includes METR and AVERI, says reviewers should be protected from retaliation and receive access equivalent to highly privileged company employees; it also says embedded evaluations should complement broader external oversight, not replace it.