Optimizing AI Infrastructure: Systematically, Agentically, Collectively

​Hosted by KRAI, as part of the Cambridge Computer Science and Technology Supporters' Club Technical Talks series.

​AI infrastructure is the hardware and software that AI runs on: specialized servers with accelerator chips, memories, networks, and frameworks, runtimes, compilers, kernels that turn models into computation. AI Infrastructure is essential to both training and deploying AI, and it largely determines how fast, how expensive, and how energy-hungry the result is.

​Making AI software fit AI hardware like a glove is hard. There are never enough experts or enough time to achieve the most ambitious and effective optimization across diverse workloads, target hardware, and user-specific performance, quality, and cost requirements.

​At KRAI, we are working with leading companies to automate low-level optimization through systematic agentic engineering. In one striking case, we accelerated a reference implementation of a frontier open-source model by over 1,000 times in under a week. In this talk, Anton will share several examples, and what they taught us: the good, the bad, and the ugly of putting AI agents to work on real optimization problems.

​These experiences underpin The Collective, our framework of cooperating specialized agents that tirelessly work together to optimize AI infrastructure automatically. He will describe how it is designed, where agents already help, where they still fall short, and which open problems are worth tackling. Attendees will leave with a practical sense of what agentic optimization can and cannot do today, and where they could contribute.

​Lunch will be served.

​About the speaker

​Dr Anton Lokhmotov has spent over 20 years designing and optimizing computer systems as an entrepreneur, engineer, and researcher. In 2020, he founded KRAI to build ultra-efficient, cost-effective systems for Artificial Intelligence applications. Previously, he co-founded dividiti (dv/dt) and led GPU Compute compiler development at Arm.

​KRAI leverages the team’s decades of expertise in automated optimization to reduce the total cost of ownership (TCO) and mitigate risks of deploying AI solutions in enterprise.
As a Founding Member of MLCommons' MLPerf benchmarking competitions, the KRAI team has acted as an "Olympic coach" to industry leaders such as Qualcomm, Google, Nebius, HPE and Dell, contributing to some of the fastest and most energy-efficient results in MLPerf history.

​Dr Lokhmotov holds a PhD in Computer Science from the University of Cambridge, where he is an External Member of the Faculty Board.