What evaluation frameworks exist for AI, and what's their rationale?

​This session is part of the How to Think about Tech? The Case of 'AI Safety' study group initiated by some of the fellow candidates of the 2025/2026 Introduction to Political Technology course. It is open to faculty and fellowship candidates only.

​Evaluations (or "evals") have become central to AI governance - companies use them to justify model releases, regulators require third-party assessments, and researchers design benchmarks for dangerous capabilities.

​But who decides what gets measured? Whose values are embedded in evaluation design? And how do evals function in practice?

​​This study group session examines AI evaluation as both a technical practice and a political process, analyzing how "safety" gets operationalized through benchmarks, who holds power in defining risk, and what systematically gets excluded from evaluation frameworks.

​---

​Key Discussion Questions:

• ​Who decides what capabilities or risks to evaluate?

• ​How do evaluation frameworks shape what gets built and deployed?

• ​What's the relationship between evals and actual safety?

• ​Can we evaluate "societal impact"? What would that require?

• ​How do evals function in governance?

• ​What gets optimized when benchmarks become targets?

• ​What's the gap between evaluation results and deployment decisions?

​Recommended Readings (from the study group):

​From the News

• ​The Guardian: Experts find flaws in hundreds of AI safety and effectiveness test.

• ​Computerworld: Testing can’t keep up with rapidly advancing AI systems (International AI Safety Report coverage

• ​Cambridge University: Most AI agents lack basic safety disclosures

​Introduction to AI Evals

• ​ Anthrophic: Demystifying evals for AI agents

• ​Curious Beginner's Guide to AI Evaluations (technical)

• ​Beyond vibe checks: A PM’s complete guide to evals

​Listen/Watch

• ​80,000 Hours - #217 – Beth Barnes on the most important graph in AI right now — and the 7-month rule that governs its progress 

​Critical Perspectives

• ​The AI Safety Washing problem

• ​Ada Lovelace Institute — Participatory AI

• ​International AI Safety Report (Executive Summary)

• ​EU AI Watch: AI benchmarking: nine challenges and a way forward

• ​Paper: AI Auditing: The Broken Bus on the Road to AI Accountability

​Technical Approaches (for reference)

• ​METR: Model Evaluation & Threat Research

• ​Anthropic: Constitutional AI (paper) - Constitutional AI: Harmlessness from AI Feedback

• ​OpenAI: Preparedness Framework

• ​AI Security Institute – Frontier AI Trends report factsheet

​Technical Deep Dives (optional)

• ​Paper: Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation (EU JRC / AIES 2025) 

• ​Paper: AI Sandbagging: Language Models Can Strategically Underperform on Evaluations

​Please read at least one of the linked items to ensure a good study group discussion, thank you!