The Information Machine
Updated todayDay 3473 sources

The AI Evaluator Forum's AEF-1 standard

The gist

Latest: AEF-1 adopted globally as AVERI publishes AI audit terminology

AEF-1 is shaping third-party AI evaluation requirements across jurisdictions, with the EU AI Office among those citing it. Inconsistent terminology across audit, evaluation, and verification work is a practical barrier to coordination that AVERI's definitions aim to address.

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

The full picture

AVERI has released definitions for key AI assessment terms including audit, evaluation, and verification, citing terminological chaos as a barrier to collaboration in the field. Separately, the AI Evaluator Forum's AEF-1 standard is in active global use: Brundage said it is 'quickly becoming the default source of standards for third-party AI evaluation, not just in the US but globally,' with the EU AI Office citing it and Fangcun AI applying the AEF-1 checklist in a DeepSeek assessment. Six organizations joined the AEF, including Apollo Research, the Center for Democracy and Technology, and CommonSense's Youth AI Safety Institute. AEF-1 sets minimum operating conditions for independent third-party AI evaluations, covering access, conflicts of interest, funding relationships, recusal requirements, and transparency of evaluation terms. The AEF's stated position is that no single evaluator can perform frontier AI oversight alone and that a collaborative ecosystem is required.

How it developed
16 November 2026

AVERI released definitions for key AI assessment terms including audit, evaluation, and verification

15 November 2026

CASRAI's comparison of AEF-1 against peer frameworks found a concrete definitional difference: AEF-1 and the EU Code of Practice both require an external organization as evaluator, while the Frontier Model Forum permits a sufficiently independent internal team, creating a documented gap in what counts as independent evaluation across standards regimes.

First citedx.com
14 November 2026

CASRAI published comparison showing AEF-1 defines independence more narrowly than Frontier Model Forum

13 November 2026added 14 Nov

xAI, OpenAI, and Anthropic cosigned AEF-1, a baseline standard for independent AI evaluations covering independence, access, and transparency, as the AI Evaluator Forum added six members including Apollo Research and the Center for Democracy and Technology.

Miles Brundage wrote AEF is "quickly becoming the default source of standards for third-party AI evaluation, not just in the US but globally," with the EU AI Office citing AEF-1. Analysis then identified a structural gap: AAM Cyber found AEF-1 treats evaluation as a discrete event rather than an ongoing process, leaving dynamic AI systems inadequately covered after initial assessment.

12 November 2026

MaxxBench findings published: models diverged 23 points on modified tasks, over a quarter of failures attributed to grader-pleasing behavior

6 October 2026

AAM Cyber analysis identifies AEF-1's five operating conditions and critiques its one-time-event scope as insufficient for dynamic AI systems

15 September 2026

Latent Space reports AEF-1 with xAI, OpenAI, and Anthropic as cosignatories; Kevin Bass posts thread alleging financial ties between Anthropic and the evaluator ecosystem

14 September 2026

AEF published AEF-1, establishing minimum operating conditions for independent third-party AI evaluations

5 December 2025

AI Risk Management Newsletter describes AEF-1 as a standard for minimum operating conditions covering independence, access, and transparency

Sources

Get the daily email

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

See the latest edition · Privacy