LLM Simulations and Research: Promise & Pitfalls

​LLMs have created new, exciting avenues for social science research through the ability to simulate human behavior at scale. With AI agents that interact, communicate, and respond to one another, research can study complex social phenomena and offer a stage to test theories about how platform design shapes human behavior. 

​Our speaker, Jacy Reese Anthis, studies LLM social simulations and how they can be used to learn about online behaviors. Even with the promise of LLM-based social simulations to test prosocial design interventions before they even touch real users, just as with any emerging research method, these simulations carry important limitations that deserve attention. Questions about validity, representational bias, and whether we are studying human behavior or how LLMs model it are important to confront before adopting these tools. This workshop will explore both the promises and the pitfalls of using LLM social simulations as a research method to equip us with the grounding of whether and how LLM social simulation is appropriate to answer research questions. 

​​​PDN Prosocial Design Research Workshops are an informal meeting space where we will gather for a one-hour, interactive online session to learn more about new work in this area. Each workshop session begins with a presentation, followed by a Q&A that also encourages constructive feedback. We’ll end with breakout groups to dig deeper and build connections around our session’s topic.