Bliss Reading Group - November 23
BLISS Reading Group – AI Control: Improving Safety Despite Intentional Subversion
Most AI safety work asks how to make a model trustworthy. This week’s paper asks a different question: what if we can’t be sure it is? Can we still get useful work out of a powerful model while making sure it can’t cause harm, even if it’s actively trying to?
We’ll read AI Control: Improving Safety Despite Intentional Subversion (Greenblatt, Shlegeris, Sachan & Roger, 2023), in a session led by Eduard Hübner. This is the paper that started “AI control” as its own research agenda. The authors treat safety as a game between a blue team and a red team. The blue team designs a safety protocol. The red team plays a misaligned model trying to get past it.
The testbed is a programming task. A capable but untrusted model (GPT-4) solves coding problems. A weaker trusted model (GPT-3.5) and a small budget of human review are available to check its work. The red team’s goal is to slip in a backdoor: code that passes every test but fails on an input only the attacker knows. The blue team has to stop that without throwing away GPT-4’s usefulness. The paper compares several protocols:
Trusted monitoring: GPT-3.5 rates how suspicious the code looks, and humans review the most suspicious cases.
Trusted editing: GPT-3.5 rewrites the suspicious parts of the code before it’s submitted.
Untrusted monitoring: a second copy of GPT-4 checks the first, with countermeasures to stop the two copies from colluding.
These protocols do substantially better than simple baselines. The paper’s broader claim may matter even more: that safety can be engineered and measured around a model we don’t trust.
We’ll start with a short walkthrough of the setup and protocols, then open it up for discussion. Some questions to think about:
How realistic is the red team as a stand-in for a genuinely misaligned model? Is it too strong or too weak?
How far can this go? Does control still work when the gap between the trusted and untrusted models gets much larger?
Untrusted monitoring relies on stopping two copies of the same model from coordinating. How robust is that?
Does a control approach complement alignment research, or does it risk becoming a substitute for it?
No need to read the whole paper. The abstract, introduction and discussion are enough to join in.
📍 MAR 0.015, TU Berlin (Marchstraße 23, 10587 Berlin)
🕡 Monday, 23 November, 18:45