

The Harness Engineering Challenge: Fix the Agent, Not the Model | Hack Hour by Failproof AI
Part of Hack Summit 2026. See the full summit: https://magicball.dev/hack-summit-2026
Your agent works in the demo and fails in production. We have spent our time on why, and the fix is not a better prompt. We show our method, then you use it to raise a failing agent’s success rate with runtime policies only, on a live leaderboard that shows whose harness gains the most.
You’ll use: The Failproof CLI, a prepared challenge repo and one fixed model for everyone.
It skips a step, calls the wrong tool, loops, or declares success on a task it did not finish. Rewriting the prompt fixes one failure and breaks another, and a new model brings new failures. We treat reliability as a harness problem. We leave the model and prompt alone and write runtime policies that intercept tool calls to allow, deny or instruct the agent back on course. Each failure becomes a rule that survives prompt edits and model upgrades. This is backed by our research on runtime policies.
What we’ll do together
You start with a prepared agent, tasks it partly fails and a scoring script. The model and prompt files are locked, so you only change the policy file, then rerun the script. Your score is the pass rate across the tasks.
Set up and baseline. Clone the repo, run the agent and record your score.
Write the first policy with us. We take one failure as a group: read the trace, name the failure, write the policy, rerun.
Take the open challenge. Fix the remaining failures on your own and submit scores to the live leaderboard. Your policies may beat ours.
See the results. The top harnesses go on screen and the winners explain their best policy, so the whole room learns from the sharpest ones.
You leave with
A working policy file that runs with the Failproof CLI against the agents you already use.
A repeatable method: read the trace, name the failure, write the policy, verify the score.
The challenge repo, to rerun or adapt as a reliability benchmark for your own agent.
Your own before and after score, showing how much reliability you can gain at the harness layer without changing the prompt or model.
Who it’s for
Engineers who build or run LLM agents and have seen them fail in ways a prompt edit did not fix: AI, backend and platform engineers, and anyone responsible for an agent in production. You should be comfortable in a terminal and with git, and know how an agent calls tools. No machine learning experience is needed.
Before you arrive
Bring a laptop (macOS, Linux, or Windows with WSL) with a terminal, git and a code editor, and a GitHub account.
Install the Failproof CLI and the runtime the challenge repo needs, and clone the repo. If you can, run the baseline so the hour starts at the first failure. We email approved guests the links.
We provide the repo, scoring script, leaderboard and model access for the session, so you do not need your own paid API keys. No private data is needed.
After the session
The challenge repo stays public, the Failproof CLI keeps running on your own agents, and the leaderboard stays open after the summit for late submissions.
Our engineers are available afterwards to help you apply the method to your own agents.
About Hack Summit 2026
Magicball's Hack Summit is on Sat 12 Dec in HSR Layout, Bengaluru: 20 Hack Hours, 50+ booths and 1,000+ AI engineers in one building. Approval for this Hack Hour includes entry to Hack Summit 2026. Each Hack Hour has its own approval, so request a seat for every Hack Hour you want to join.
Hack Summit 2026 is in partnership with Grayscale Ventures and Testsigma.
See the full summit: https://magicball.dev/hack-summit-2026
More Hack Hours
Explore every Hack Hour: https://www.magicball.dev/hack-summit-2026#hack-hours