Daniel Benarroch | Never trust an agent: accountability for agentic AI
Foresight Institute’s Computation Group
Never trust an agent: accountability for agentic AI
Abstract: Most of what we call trust in business and public life is really machinery for not having to trust. Contracts, audits, registries, permissions, insurance, liability. None of it makes anyone honest. It reduces how much honesty you need, and puts what's left somewhere you can check. That has worked for a long time, because at the end of every one of those chains sat a person who could be held to it.
Agents don't fit. An agent is not a moral actor, and it is not really a tool either. It is closer to a young child holding your passwords: it will do the thing you told it not to, no matter how many times you told it. So "do you trust this agent?" is a question with no good answer, and it is pulling the safety conversation in the wrong direction. Ask companies what they actually do and you find they never answer it. They decide what to expose instead. Call recordings, fine. Bank statements, no.
There is an older idea underneath this that I think matters more. You can delegate a task, and you can delegate authority, but you cannot delegate responsibility. That stays where it started. And an agent has no intentions of its own. Whatever it is trying to do began as something a person wanted. So responsibility and intent both stay with people, and the systems we build have to hold on to that link. Today they lose it almost immediately. A few steps into an agentic process, nothing anywhere knows who asked for this, or why.
The talk is about what has to exist for that link to hold. Identity for something that is not a person. Authority that is granted rather than inherited. A record of what happened, written by something other than the thing that did it. And a way to carry the original intent through a long chain of delegated work. I'll move between the philosophy and the engineering, and I'll be straight about where the engineering runs out. Intentionality and alignment are real problems, and no permissions system has ever solved one.
Speaker Bio: Daniel Benarroch: CEO and Co-Founder of Inversed Tech. With over a decade in cryptography, Daniel founded zkproof.org and the Crypto Lounge Experience. He combines technical vision with conscious leadership.
Inversed is an R&D studio specialized in secure and verifiable computation. Our team has delivered production-grade systems spanning privacy-preserving biometrics, where we designed and scaled a large-scale identity database secured end-to-end with multi-party computation, to advanced cryptographic infrastructure including zero-knowledge virtual machines, private blockchain asset transfers, and differential privacy frameworks for sensitive consumer data. Across these efforts, we combine deep cryptographic research with practical systems engineering to turn frontier techniques—MPC, ZKPs, and distributed algorithms—into deployable infrastructure that enables accountability, privacy, and control in increasingly autonomous and data-driven environments. https://www.inversed.tech/
A group of scientists, engineers, and entrepreneurs in computer science, ML, cryptography, and related fields who leverage those technologies to improve voluntary cooperation across humans, and ultimately AIs.
Subscribe to our newsletter
Nominate a seminar presenter/topic
Share this application with colleagues who’d like to join
Our book Gaming the Future: Technologies for Intelligent Voluntary Cooperation is now online on Substack.
Zoom link: https://us02web.zoom.us/j/81623367983