Scaling Rational Cooperative Intelligence for Pluralistic AI Futures

​Tan Zhi Xuan (NUS / A*STAR, CoSI Lab) will give a talk on steering multi-agent AI worlds toward mutual flourishing. She introduces rational cooperative AI: agents that are cooperative-by-design, learn to follow and enforce norms, and hold up where enforcement and information are decentralized.

Talk abstract: One plausible extrapolation of current AI trends is that we are headed towards pluralistic AI futures: scenarios where a plethora of AI agents, systems and services are deployed towards various ends by various actors. How can we influence this process so that it occurs not just safely — avoiding AI-powered conflict and anti-human equilibria — but also in the direction of mutual flourishing? In order to address this challenge, I argue that we need to scale cooperative intelligence, advancing the cooperative capacities and dispositions of the agents we create, while creating institutions and incentives that promote AI and human cooperation at scale. To develop these capacities, I will introduce recent research on rational cooperative AI — a paradigm for building coherent AI agents that are cooperative-by-design. Across domains like assistive planning and natural language contracting, rational cooperative AI avoids the jaggedness and unreliability of current generative AI. Taking a rational approach also allows us to build "self-governing" AI agents that learn to follow and enforce societal norms, and to design institutional rules and mechanisms that operate well when both enforcement power and information flow is decentralized. I will close by discussing the deeper connections between reasoning, rationality, and cooperation, and how a cooperative theory of reasoning could ultimately lead us to more human-aligned reasoners.

Speaker bio: Tan Zhi Xuan is an Assistant Professor in the National University of Singapore, Department of Computer Science, with a joint appointment at the A*STAR Institute of Advanced Intelligence and Computing. Xuan's research focuses on scaling cooperative intelligence via rational AI engineering, spanning the areas of AI alignment, probabilistic programming, and computational cognitive science. Together with their research group, the Cooperative Systems & Intelligence (CoSI) lab, Xuan aims to reverse engineer the computational foundations of human cooperation and normativity, thereby enabling the development of human-level AI cooperators and the design of cooperative infrastructure for an increasingly automated future. Previously, Xuan completed their PhD with the MIT Probabilistic Computing Project and Computational Cognitive Science lab.