Refactor Arena Deep Dive
Refactor Arena is a configurable, extensible AI control setting for exploring how agents might inject vulnerabilities into software applications while performing complex tasks such as refactoring code.
With it, you can run the built-in AI control evals or extend it to create your own.
In this deep dive, you'll learn how to:
Install the package & run the built-in evals.
Use configuration files + CLI options to vary evals.
Extend the setting to run your own evals.
Add custom command-line parameters via the config.
In addition, Preeti Ravindra will be showcasing how she extended Refactor Arena for her own research during her Pivotal Research fellowship, and we'll be answering questions on how you can do the same.
To learn more about Refactor Arena:
Read the LessWrong introduction post.
Explore the documentation.
Browse the code repository.