White House AI audit accord leaves Big Tech testing control with companies

Traders Union
All News
Crypto
White House AI audit accord

Highlights

Nvidia, OpenAI, Anthropic, Google, Meta, and xAI signed a Sept. 29 accord committing to external AI audits but retain control over auditor selection and audit scope.
Stanford, Harvard, and Brown researchers found AI companies averaged 53% compliance with 2023 White House commitments, with half scoring zero on external issue reporting.
Google DeepMind's August double-blind AI test with Singapore’s AI Safety Institute showed technical progress, but hardware-based trust remains vulnerable and external oversight is still proposed.

Audit design and access limits

As reported by CoinDesk, the accord signed on Sept. 29 by Nvidia, OpenAI, Anthropic, Google, Meta and xAI commits the companies to work with independent external auditors or evaluators, while independent board members review the resulting reports and track whether problems are addressed.

The article argues that the pledge does not specify who selects the auditors, what parts of a model they may examine, whether findings become public or what penalties apply if a company ignores the process. In practice, that means each company can choose its own auditor and define the boundaries of the review.

The comparison drawn is to the Enron era, when Arthur Andersen both audited and advised the company before both collapsed within two years. The subsequent Sarbanes-Oxley Act removed auditor selection from management, restricted most non-audit services and created a public body to inspect auditors themselves.

The piece also points to the AI sector's uneven record on voluntary safeguards. A review by researchers at Stanford, Harvard and Brown of 16 companies' performance against 2023 White House AI commitments found an average score of 53%, while eight companies scored zero on helping outsiders identify and report problems.

Technical options and policy implications

Even when outside testing occurs, developers still often set the terms. METR, a nonprofit that evaluates AI models, said its prerelease testing of OpenAI's o3 model was conducted over only three weeks, with limited information access and no visibility into the model's step-by-step reasoning.

The article says newer technical methods may reduce the trade-off between security and scrutiny. In August, Google DeepMind, working with Singapore's AI Safety Institute and three outside groups, ran what it described as the first double-blind test of an AI model using sealed computer hardware so testers did not see the model and Google did not see the questions.

That approach still shifts trust toward the hardware, which researchers breached last year with less than $1,000 of equipment after gaining physical server access. Even so, the author contends that the risk is narrower and more manageable than barring outsiders from meaningful testing altogether.

Drawing on a Decentralization Research Center report published this week, based on interviews with 36 people and nearly 30 mapped alternatives to concentrated AI control, the article concludes that outside parties can now scrutinize advanced AI without taking possession of the model or its data. It argues that auditors should be free to choose what they test, companies should disclose who conducts the audits and what areas are covered, and Congress should consider creating an external oversight mechanism for the auditors themselves.

Our earlier coverage of ByteDance’s Seed unit focused on the launch of an Alignment & Preparedness team and a global hiring push to strengthen frontier-model safety, automated testing, and preparedness for emerging risks. The piece highlighted how rising scrutiny of increasingly autonomous AI systems is driving developers to formalize safety and independent evaluation as an operational requirement for deployment at scale.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.