AI Governance Reading Group – The State of Reward Hacking in AI
AI Governance Reading Group
Each month, we gather to discuss an important AI governance paper. We recommend reading the paper beforehand and coming with your comments and insights. Bilingual discussion.
Groupe de lecture sur la gouvernance de l'IA
Chaque mois, nous nous réunissons pour discuter d’un article important sur la gouvernance de l’IA. Nous vous recommandons de lire l’article à l’avance et de venir avec vos commentaires et vos réflexions. Discussion bilingue.
The paper this time / L'article de ce mois-ci :
The State of Reward Hacking in AI (Aaron Scher and Joseph Rogero, MIRI Technical Governance Team, September 2026): https://techgov.intelligence.org/research/the-state-of-reward-hacking-in-ai
En bref : le rapport fait le point sur le « reward hacking », quand un système d'IA obtient un bon score sur une mesure de substitution sans adopter le comportement voulu, sur les cas où cela se produit et sur les moyens d'y remédier.
From the report's abstract:
Modern machine learning reinforces outputs and behaviors that score well on certain measures. Most of these measures are proxies for things we really care about: User ratings are proxies for helpful conversation, benchmark scores are proxies for task competence, passing unit tests are proxies for working code. Because these measures incompletely track the complexity of human aims, AIs learn to "reward hack", or pursue an unexpected and undesirable strategy that scores highly but is not the intended behavior. This report summarizes the state of reward hacking: when it happens and what mitigations are available.
—
Où / Where:
Ω Labs
Parking spot available on requestOnline / En ligne: https://zoom.us/j/99005830598?pwd=3e21GRcTVAPTbvGOJv4RawurTc5ZXx.1
Contact host if you need assistance or for questions
—
AI Safety Montréal and Ω Labs run on member contributions. Your donation keeps the project going and the snacks coming.