Friday Hacks #297: How We Evaluate AI Agents Before and After Release & LiquidJS: Building a template engine & growing an OS project
Friday Hacks, by NUS Hackers, is a weekly event where invited speakers share their technical experiences and interests! In this week's session, we have:
Talk 1: How We Evaluate AI Agents Before and After Release
An AI agent can pass a demo and still fail in the real world. It may call the wrong tool, misunderstand a user’s intent, produce a convincing but unhelpful answer, or break after a seemingly harmless prompt or model change. So how do you know an agent is genuinely ready to ship?
This sharing details the evaluation loop behind Kai: before release, we run curated “golden” scenarios to catch regressions in a controlled environment; after release, we evaluate real interactions to discover the failures that no test set anticipated. We will unpack how deterministic checks, self improving LLM-as-a-judge, production traces, and human feedback work together—not as a one-off accuracy score, but as a continuous system for finding and reducing agent failures.
Speaker Profile:
James Li is an Engineering Manager in AI at Airwallex, leading the team behind Kai, Airwallex's AI agent product. He and his team are pushing the frontier of agentic finance, building production AI that turns complex financial workflows into simple, intelligent experiences. He is also joined by Weikun and Hao-Tong, AI engineers at Airwallex who work on AI Agents and LLM Modeling respectively.
Talk 2: LiquidJS-Building a template engine and growing an open source project
What happens when a small open-source project becomes software that thousands of projects depend on? This talk explores how a template engine works, how we can create a widely used library, and what building, maintaining and growing open source actually looks like.
Speaker Profile:
Jun Yang holds degrees in Physics and Computer Science from Peking University and has worked as a software engineer and architect at Microsoft, TikTok, and other tech companies. He is the creator of LiquidJS, a widely used JavaScript template engine.