Observing GenAI and Agentic Workflows on OpenShift AI
About the session
Many GenAI projects stall somewhere between a promising pilot and a service people can depend on, and the gap is widest in regulated environments. Models need to run where the data is allowed to live, spending has to be accounted for, and when an AI agent takes an action, someone will eventually ask what it did and why.
In this live session, Red Hat and Randoli walk through what it takes to close that gap. Andrew Pitt from Red Hat will show how OpenShift AI lets teams build and serve models with vLLM on their own infrastructure, whether on-premises, disconnected or across clouds, with OpenTelemetry, Prometheus and logging built into the platform.
Rajith Attapattu, Co-founder and CTO of Randoli, will then show how to monitor and observe those workloads in production with a live demo of the Randoli Observability Platform. Randoli is a Red Hat OpenShift certified operator, available in the OpenShift operator catalog, and runs either as SaaS or fully on-premises, including air-gapped environments, so your telemetry can stay inside your own boundary.
What you’ll see
Serving a model with vLLM and deploying a GenAI application on OpenShift AI
vLLM performance and token usage
GenAI cost by model and workflow
End-to-end tracing of LLM and tool call chains with OpenTelemetry
Monitors that flag performance and cost issues as they happen
Agenda (all times ET)
1:00 PM · Welcome and introductions
1:05 PM · Introducing OpenShift AI
Andrew Pitt, Red Hat
Andrew explains how OpenShift AI supports building, serving and running models across on-premises, disconnected and public cloud environments, with built-in security, vLLM model serving, and OpenTelemetry, Prometheus and logging included in the platform.
1:20 PM · What to monitor and observe when GenAI runs in production
Rajith Attapattu, Randoli
Rajith covers the signals that matter once models and agents are live: token usage and cost, latency and throughput, and call chains across LLMs and tools.
1:35 PM · Live demo: Serving a model and deploying a GenAI application on OpenShift AI
Andrew Pitt, Red Hat
Andrew serves a model with vLLM and deploys an application that uses it.
1:55 PM · Live demo: Observing the model and the application with Randoli
Rajith Attapattu, Randoli
Using the same deployment, Rajith shows vLLM performance and token usage, GenAI cost by model and workflow, application performance, end-to-end tracing of LLM and tool call chains, and the monitors watching it all.
2:15 PM · Live Q&A
2:30 PM · Close
Can’t attend live? Register and we’ll send you the recording.