Human-like autonomy emerges from self-play and a pinch of human data

​About Event

​🔬 AI4Science on alphaXiv
🗓 Friday July 31st 2026 · 11 AM PT
🎙 Featuring Daphne Cornelisse
💬 Casual Talk + Open Discussion

​🎥 Zoom: Upon Registration

Description: Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play can learn effective but alien driving conventions incompatible with people. Previous works attempt to mitigate such behavioral misalignments through extensive reward engineering and domain randomization, which are brittle and labor-intensive. Instead of completely discarding human demonstrations, our method treats them as a regularization objective on top of a minimal safe goal-reaching reward. Like the spice in a good stew, we find that a little human data goes a long way: our method uses only 30 minutes of human demonstrations, 2500× fewer than comparable imitation learning approaches. Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU. Videos and full source code are available at https://spiced-self-play.com/.

Check out the full paper here: https://www.alphaxiv.org/abs/2606.19370

​Whether you’re working on the frontier of LLMs or just curious about anything AI4Science, we’d love to have you there.

​Hosted by alphaXiv