Frontier Build Night | Self-Improving Agents
One night. Five hidden challenges. Build an AI agent that improves itself.
On August 25, approximately 100 engineers, researchers, and technical founders will take over AWS Builder Loft for a judged build night using SIA—Hexo Labs’ open-source framework for iteratively improving AI agents.
At 5:30 PM, we reveal five previously unseen challenge environments. Each comes with a baseline agent, a fixed evaluation, and a recorded starting score. Teams will have under two hours to make the agent more diagnosable, adaptable, and capable—then prove the improvement through reproducible runs.
The people who built SIA will be in the room. A live leaderboard will run throughout the evening. Finalists will demonstrate their systems before a technical judging panel.
Come alone or with a team of up to four. Dinner, compute, and admission are covered. Capacity is limited and approval is required.
IMPORTANT: TWO REGISTRATIONS ARE REQUIRED
Step 1: Request approval on this page.
Step 2: Once approved, immediately complete the required AWS venue registration.
AWS will admit only guests who appear on its registration list. Register using the exact name shown on your government-issued ID, and bring that physical ID with you.
Luma approval alone does not provide entry to the venue.
THE CHALLENGE
SIA runs an agent against a fixed evaluation, observes where it fails, diagnoses the failure, modifies the system, and runs it again—producing a scored sequence of increasingly capable versions.
Published results from Hebbar et al. include a 502% improvement in single-cell RNA denoising, a 91.9% reduction in GPU-kernel runtime, and a 56.6% improvement on LawBench.
Build Night turns that research into a live competition.
Five challenge environments will span areas such as code, scientific reasoning, and professional knowledge. Each challenge includes:
A shared baseline agent
A fixed evaluation harness
A verified starting score
A controlled compute environment
A complete SIA improvement loop
Every team begins from the same baseline within its selected challenge. The challenges remain hidden until they are revealed simultaneously at 5:30 PM.
HOW THE COMPETITION WORKS
Teams may improve the scaffold that determines how effectively SIA can understand and modify the agent: its logging, structure, prompts, memory, tool interfaces, and guidance.
You may not manually implement a task-specific solution and submit it as an autonomous improvement. The scored capability gain must be produced and documented through the SIA loop.
The objective is not merely to generate a larger number. It is to discover what makes an AI system improvable:
What evidence helps the system diagnose its own failures?
What architecture allows precise changes without breaking unrelated behavior?
What information should persist between iterations?
What forms of guidance help the improvement agent reason effectively?
What can the experiment history teach us about the system itself?
Complete competition rules will be included in the setup kit and reviewed before the sprint begins.
WHAT YOU’LL LEAVE WITH
A working self-improving agent built on a real evaluation
A scored improvement curve from baseline through successive versions
A reproducible experiment history showing what changed and why
A project repository you can continue developing
Direct experience running closed-loop agent improvement
Feedback from the engineers who built SIA
Past Frontier Syndicate rooms have included researchers, engineers, and founders from DeepMind, Meta, Google, Stanford, UC Berkeley, and many of the Bay Area’s most ambitious AI companies.
Frontier Build Night brings that community together around a shared technical problem.
AWARDS
🏆 Best Self-Improvement System
The evening’s grand prize. Judges will select the team that demonstrates the most insightful, original, and reproducible approach to making an agent improve itself.
🏆 Highest Validated Improvement
Awarded to the team producing the strongest verified improvement over its challenge’s baseline.
Judges will consider:
Measured improvement against the fixed evaluation
Quality and reproducibility of the experiment history
Insight into the agent’s failure modes
Originality and technical elegance
Clarity of the final demonstration
Integrity of the baseline and submitted results
A modest but well-understood improvement can outperform a larger unexplained result. We are interested in what worked—but even more interested in why.
AGENDA
5:00 PM — Doors, dinner, setup, and team formation
Come solo or with a pre-formed team. Solo participants can join teams at the venue. Maximum team size: four.
5:30 PM — SIA briefing and challenge reveal
A rapid technical introduction from the team that built SIA, followed by rules, compute orientation, and the simultaneous reveal of all five challenges.
5:50 PM — Build sprint begins
Teams select a challenge, establish their baseline, and begin running improvement experiments. SIA engineers and challenge leads will be available throughout the room.
7:40 PM — Submissions close
Each submission must include the final agent, experiment history, verified baseline and best scores, and a short explanation of the team’s central insight.
7:45 PM — Finalist demonstrations
Selected teams receive three minutes to demonstrate their systems, followed by live questions from the judges.
8:15 PM — Awards
8:25 PM — Group photo
8:30 PM — Doors close
SETUP
Every approved participant will receive a preparation kit on Monday morning containing:
Installation instructions
Required credentials
A one-command environment setup
Pre-download instructions
A small practice challenge
The complete competition rules
The practice run is designed to verify your environment before you arrive. Setup should take approximately 15 minutes.
If you need technical assistance, arrive at 5:00 PM with your laptop. A staffed setup desk will be available before the challenge briefing.
WHO SHOULD APPLY
This is a hands-on technical event for:
AI and machine-learning engineers
Agent and evaluation researchers
Technical founders
Research engineers
Advanced students working directly with AI systems
You do not need previous experience with SIA. If you can write Python, work with a repository, and have used an LLM API, you are equipped to participate.
Applications are reviewed individually. Priority will be given to people who intend to build for the full evening.
FAQ
Do I need to know SIA already?
No. The preparation kit and opening technical briefing provide the necessary onboarding.
Do I need a team?
No. Most participants may arrive solo, and we will facilitate team formation at the venue. You may also bring a pre-formed team of up to four people. Solo submissions are also welcome.
What should I bring?
A laptop, charger, and physical government-issued ID matching the name on your AWS registration.
What does it cost?
Nothing. Admission, dinner, drinks, and event compute are provided.
Where exactly do I go?
AWS Builder Loft, 525 Market St. The entrance is up the stairs on the southwest side of the building (next to Chipotle); elevator access is through the main lobby. Doors open at 5:00 PM — budget a few minutes for building security and ID check.
HOSTS
The Frontier Syndicate
A private research and venture network convening frontier-technology researchers, builders, founders, and investors across the Bay Area.
Hexo Labs
Creators of SIA and AIE-Bench, building open-source infrastructure for AI systems that improve AI systems.
AWS Builder Loft
Hosting Build Night at its San Francisco space.
Photography and short-form video will be captured during the evening for event documentation and promotion. By attending, you acknowledge that you may appear in this material.
READY TO BUILD?
Seats are limited and applications are reviewed individually. Two steps, both required:
Step 1 — Request to join at the top of this page.
Step 2 — The moment you're approved, complete the AWS venue registration: https://events.builder.aws.com/zV1dm9