Reading Group (+🧋): Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration

​Join the Snorkel AI Reading Group, a recurring forum to explore the latest frontier developments in AI while building meaningful connections within the community.

In this afternoon session, Yijia Shao, a PhD candidate at Stanford NLP, will cover work she recently presented at ICLR: Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration.

Agenda:

​3pm - doors open
3:30pm - talk begins

​🧋🧋🧋 Boba tea and other refreshments will be provided ! 🧋🧋🧋

​Among other things, you'll learn:

• ​How Co-Gym formalizes human–agent collaboration as a POMDP with dual control over a shared workspace.

• ​Why enforced turn-taking breaks down for real tasks, and what replaces it: two collaboration acts plus a notification protocol.

• ​How an evaluation suite scores both outcomes and process, including an entropy-based measure of shared initiative.

• ​Why a Collaborative Agent with Situational Planning beats fully autonomous baselines across three tasks.

• ​How real users rate the best collaborative agent: 86% 74% 66% win rates over autonomous on Travel, Tabular, and Related Work.

• ​Why communication and situational awareness remain the dominant failure modes (65% and 40% of real trajectories).