AVERI released definitions for key AI assessment terms including audit, evaluation, and verification
The AI Evaluator Forum's AEF-1 standard
Latest: AEF-1 adopted globally as AVERI publishes AI audit terminology
AEF-1 is shaping third-party AI evaluation requirements across jurisdictions, with the EU AI Office among those citing it. Inconsistent terminology across audit, evaluation, and verification work is a practical barrier to coordination that AVERI's definitions aim to address.
When stories like this move, I write about them in an email each morning.
One free email each morning, around 8 AM US Eastern.
Check your email.
I have sent you a confirmation link. One click and you are in.
Not in your inbox? Look in spam or promotions. If it's there, mark it "Not spam" so future editions land in your inbox.
The full picture
AVERI has released definitions for key AI assessment terms including audit, evaluation, and verification, citing terminological chaos as a barrier to collaboration in the field. Separately, the AI Evaluator Forum's AEF-1 standard is in active global use: Brundage said it is 'quickly becoming the default source of standards for third-party AI evaluation, not just in the US but globally,' with the EU AI Office citing it and Fangcun AI applying the AEF-1 checklist in a DeepSeek assessment. Six organizations joined the AEF, including Apollo Research, the Center for Democracy and Technology, and CommonSense's Youth AI Safety Institute. AEF-1 sets minimum operating conditions for independent third-party AI evaluations, covering access, conflicts of interest, funding relationships, recusal requirements, and transparency of evaluation terms. The AEF's stated position is that no single evaluator can perform frontier AI oversight alone and that a collaborative ecosystem is required.
How it developed
CASRAI's comparison of AEF-1 against peer frameworks found a concrete definitional difference: AEF-1 and the EU Code of Practice both require an external organization as evaluator, while the Frontier Model Forum permits a sufficiently independent internal team, creating a documented gap in what counts as independent evaluation across standards regimes.
CASRAI published comparison showing AEF-1 defines independence more narrowly than Frontier Model Forum
xAI, OpenAI, and Anthropic cosigned AEF-1, a baseline standard for independent AI evaluations covering independence, access, and transparency, as the AI Evaluator Forum added six members including Apollo Research and the Center for Democracy and Technology.
Miles Brundage wrote AEF is "quickly becoming the default source of standards for third-party AI evaluation, not just in the US but globally," with the EU AI Office citing AEF-1. Analysis then identified a structural gap: AAM Cyber found AEF-1 treats evaluation as a discrete event rather than an ongoing process, leaving dynamic AI systems inadequately covered after initial assessment.
MaxxBench findings published: models diverged 23 points on modified tasks, over a quarter of failures attributed to grader-pleasing behavior
AAM Cyber analysis identifies AEF-1's five operating conditions and critiques its one-time-event scope as insufficient for dynamic AI systems
Latent Space reports AEF-1 with xAI, OpenAI, and Anthropic as cosignatories; Kevin Bass posts thread alleging financial ties between Anthropic and the evaluator ecosystem
AEF published AEF-1, establishing minimum operating conditions for independent third-party AI evaluations
AI Risk Management Newsletter describes AEF-1 as a standard for minimum operating conditions covering independence, access, and transparency
Sources
- Google DeepMind's double-blind AI evaluation pilot
- OpenAI Embedded Safety Audits Find Gaps; Disclosure Limits Disputed
- Massachusetts Senate AI lab audit bill
- New York's RAISE Act AI registration
BackgroundWhen AI doesn't do what we meant
Get the daily email
When stories like this move, I write about them in an email each morning.
One free email each morning, around 8 AM US Eastern.
Check your email.
I have sent you a confirmation link. One click and you are in.
Not in your inbox? Look in spam or promotions. If it's there, mark it "Not spam" so future editions land in your inbox.