Vals AI Hackathon: Benchmaxxing
Vals AI Hackathon:
Vals AI is hosting its first hackathon. Model capabilities have jumped in recent months, and a growing share of public benchmarks no longer tell frontier models apart. Come spend a day on both sides of that problem.
Who is Vals AI?
Vals AI builds independent benchmarks that measure how well AI models perform on real professional work: law, finance, healthcare, coding, and frontier risks like cybersecurity and recursive self-improvement. Public benchmarks keep breaking as models ace them, so we test capabilities the way they actually get used, and we keep our test sets private to keep the scores honest. We run every evaluation ourselves. This hackathon puts that work in your hands for a day.
Two Tracks:
Post-Training: Sponsors can provide access to their platforms for this track. Participants will post-train smaller models, such as Inkling Small, on synthetic data they create. We will provide topics and a test set for evaluating these models. Models can be post-trained on any topic, and we will also have an overall category to determine which models perform best across all benchmarks.
Build Your Own Eval: Participants will receive access to the internal harness we use to run agentic evaluations. They can create an evaluation of their choice and use it to test model capabilities.
Who should apply?
We're inviting students and recent graduates (18 to 30) working in evaluations, post-training, and model capabilities, and anyone eager to build the future of AI.
Speakers:
Agenda:
Prizes:
In-person, San Francisco. Approved applicants only, no door entry. Teams up to five. 18+.