Retry, Reindex, Recover: Elasticsearch in Production + AI Runtime Security
Join us for a night of real-world Elasticsearch stories from engineers running production systems at Recurly, Intuit, and Asymptote Labs. You'll hear three talks tackling very different challenges: keeping async pipelines reliable when production gets complicated (late events, retries, traffic spikes, zero-downtime migrations), and hunting risky AI agent behavior by shipping runtime telemetry into Elastic for security investigations.
Expect practical lessons, architecture patterns, and war stories from devs in the field across reliability, observability, and AI runtime security.
Plus: pizza, drinks, and time to hang with other engineers building in the Bay.
Agenda
5:30 - 6:00: Doors open + food and drinks
6:00 - 6:35: Talk #1 + Q&A
6:40 - 7:15: Talk #2 + Q&A
7:20 - 7:50: Talk #3 + Q&A
7:50 - 8:00: Networking
Talk Abstracts
🎤 Talk 1: Sharding, Partitioning, and Indexing Pipelines: Powering Search at Recurly - Chris Barton, Lead Platform Architect at Recurly
Recurly runs app search across billing entities (accounts, invoices, transactions, subscriptions) in a large multi-tenant SaaS environment — 16+ indices, hundreds of shards, many TBs of data on hot-warm nodes.
This talk walks through the custom integration points that make it work:
Three-dimensional index shaping: tenant bucketing via routing, year-based time partitioning with ILM tier preferences, and an old/current/new triad enabling zero-downtime reindexing through dual-writes
A dedicated long-lived indexer instead of inline indexing: model callbacks enqueue real-time changes, bulk backfills push into the same queue, and the indexer batches via the bulk API while managing backpressure and graceful shutdown
How all three dimensions are configured declaratively and resolved transparently at query time
🎤 Talk 2: Building Event-Driven Apps That Work in the Real World - Garvit Kataria, Senior Software Engineer at Intuit
Event-driven architectures promise resilience, but real-world failures — late messages, retries, out-of-order events — often derail it. This talk shows how EWOK, Intuit's event-driven workflow engine, coordinates disaster recovery using declarative steps for traffic switches, capacity changes, and DB role shifts.
Covers idempotent workflows, safe retries, and chaos validation. EWOK events and metadata stream into Elasticsearch to correlate outcomes and detect anomalies, making recovery practical, auditable, and repeatable.
🎤 Talk 3: Hunting Risky AI Agent Behavior with Agent Beacon & Elastic - Justin D’Souza, Co-founder & CEO at Asymptote Labs
AI agents are beginning to behave like users, automation, and applications simultaneously: reading files, invoking tools, executing commands, using browsers, and interacting with internal systems. Traditional endpoint telemetry can show process and network activity, but it often misses the agent-level context: which agent action triggered it, what tool was called, what file was accessed, and how that behavior fits into the agent’s workflow.
Agent Beacon closes that gap by emitting structured AI runtime telemetry that can be shipped into Elastic for investigation. This talk shows how to turn Agent Beacon telemetry into Elastic-native security investigations. We’ll cover how Beacon events are forwarded with Filebeat or Elastic Agent, normalized through ingest pipelines and index templates, and explored in Kibana alongside existing endpoint, identity, and infrastructure data. From there, we’ll walk through practical hunts for risky agent behavior: unexpected tool use, sensitive file access, shell execution, and workflow drift.