AI SYD
Sydney is home to some of the best AI engineers in the world, from early-stage startups to global enterprises. SYD AI Meet Up is where they come together. Whether you're shipping AI features in production, experimenting with AI SDK, or figuring out how AI fits into your stack - this is your room.
The headline: Collaboration with Clickhouse
What to Expect
6:00 – 6:30 PM: Pizza & Networking Grab a slice, meet fellow engineers, and connect with the community.
6:30 – 7 PM: Lightning Talks
"We planted real CVEs in 16 repos to find out which AI model actually catches vulnerabilities. The best model is a surprise." Patrick Collins CEO, Dam Secure
From Traces to Trust: Shipping Reliable LLM Apps with Langfuse by Muhammad Ali, Senior Solution Architect @ ClickHouse
7 PM – 9:00 PM: Networking & Wrap Up
More details on the lightning talks:
Patrick Collins CEO, Dam Secure
We planted real CVEs in 16 repos to find out which AI model actually catches vulnerabilities. The best model is a surprise."
Every new model is benchmaxxing and also test against the same public vulnerable repos, like OWASP Juice Shop. That makes the standard benchmarks essentially useless.
So we built our own eval suite and benchmark. We mine CVEs from open-source commit history and replay the reverted fix back onto the codebase, across 16 repos.
I'll show the results and give a sneak peak into how we built that eval suite and the results. The leading model, which we ran through Vercel will surprise you!
This is a talk for developers interested in behind-the-scenes of eval suites and benchmarks.
Traces to Trust: Shipping Reliable LLM Apps with Langfuse
Building an LLM demo is easy; running one in production is not. This deep dive walks through how Langfuse helps teams move from "it works on my prompt" to measurable, debuggable, cost-aware AI systems. We'll explore tracing and spans for complex agent pipelines, prompt management and versioning, evaluation workflows to catch regressions before users do, and dashboards for cost and latency monitoring. Along the way, we'll look at real traces from a live application and discuss how observability data feeds back into improving prompts, models, and architecture. If you're building with LLMs or agents, you'll leave with a concrete playbook for instrumenting your own stack.
Speaker: Muhammad Ali, Senior Solution Architect @ ClickHouse
Muhammad Ali is a Solutions Architect at ClickHouse, helping teams design high-performance analytics systems and make deliberate choices about data formats, engines, and lock-in trade-offs in the agentic era.
Who Should Attend
Whether you're at a startup or an enterprise - if you're building with AI, this is for you.
AI engineers building intelligent features and applications
Frontend and backend developers integrating AI into real products
Founders and CTOs exploring AI-native architectures
Anyone curious about AI SDK and modern AI tooling