​Description

​AI demos are easy. Building AI systems that actually work in production - in real time, under pressure, with messy data and real users - is where things get interesting.

At this OSA Community meetup, engineers and practitioners will share how they’re building real-time analytics and AI systems at production scale: streaming pipelines, real-time databases, feature stores, event-driven architectures, and the open-source technologies making it all possible.

If you work on data infrastructure, AI applications, observability, streaming systems, or just like hearing how modern systems are really built, you’ll fit right in.

​Food and drink are provided!

​Speakers

• ​Robert Hodges, CEO @ Altinity

• ​Coleman Rollins, Staff Software Engineer @ Grafana

• ​Brandon Wilcox, Director of Engineering @ Gamebeast

• ​Jessica Wang, DevRel @ Braintrust

​​​Agenda

• ​6 pm - Networking

• ​6:15 - 8:00 pm - Talks

• ​8:00 - 9:00 pm - Networking

​​​​​Description of the Talks

​Is It Safe? Letting AI Touch Production Data Without The Tears

​Speaker: Robert Hodges, CEO @ Altinity

​Abstract: Let's talk about AI safety. On June 2nd our first support case arrived in which a user's AI agent deleted a production table. It's a moment we've long anticipated. This talk presents practical ways to give AI access to production data while avoiding the dangers. We'll focus on three specific problems: ensuring only authorized agents access data, preventing accidental loss, and keeping proprietary information out of hosted LLMs. The solutions range from here-and-now approaches like OAuth and MCP to long-range opportunities like local models. Letting AI agents range free over data is the dream. Let's do the engineering to make it real.

​

​AI Observability in Grafana: Instrument, Explore, and Evaluate Your AI Agents

​Speaker: Coleman Rollins, Staff Software Engineer @ Grafana

​Abstract: AI agents are shipping to production faster than ever — but how do you know they're working correctly? In this session, we'll walk through AI Observability in Grafana: how to instrument your agents with a few lines of code, explore conversations and multi-agent pipelines, track cost and performance, set up online evaluations to continuously monitor quality, and configure guards to catch issues before they reach the model.

​

​Bringing Game Developers Closer to their Players with AI and Analytics

​Speaker: Brandon Wilcox, Director of Engineering @ Gamebeast

​Abstract: Players generate more insights than developers can ever discover. We'll show how our LLM pipeline surfaces the pain points, complaints, and praise hidden across millions of player conversations - wherever they happen - and grounds all of it in hard data.

​

​Agentic vs. Vector Search: An Eval-Driven Approach to Coding Agent Performance

​Speaker: Jessica Wang, DevRel @ Braintrust

​Abstract: Evals let you replace gut feelings with quantifiable decisions. This talk breaks the basic concepts of evals, including the four core components: datasets, tasks, scoring, and experiments. Then, to solidify the concept, we’ll walk through a real eval comparing agentic search versus vector search for coding agents. We'll also cover practical challenges like tracing Claude Code subprocess calls and why a single eval run is never enough. You'll leave with a concrete framework for building evals that actually inform your ship decisions.

​Follow the OSA Community

​Make OSA COM your community.

​Website: www.osacom.io

​LinkedIn: https://www.linkedin.com/company/osa-con/

​Slack: https://go.osacom.io/slack