Private Event
Unit Test Benchmarks
Registration
About Event
Hosted by Isaac Flath and Joe Barrow.
Unit tests can make strong benchmarks. Joe and I will discuss why small, concrete checks are useful, when they fit the task, and how to design them well. We’ll use ParseBench, olmOCR-bench, and OmniParseBench as examples.
We’ll discuss:
Why testing specific behaviors can give useful feedback.
What passing tests tells you, and what remains unmeasured.
How to choose documents, write checks, and define correctness.
How to combine focused tests with broader evaluation.
Wednesday, November 18, 2026, 7:30–8:30 pm Eastern. Free to join on Riverside. Bring questions.
Joe’s notes on ParseBench · ParseBench paper · olmOCR-bench · OmniParseBench