​Description

​This session examines the practical engineering challenges of deploying AI to constrained physical systems and edge devices. By breaking down the on-device stack into models, memory, and retrieval, the event covers the economics of local LLM inference, the design of agentic memory harnesses for continuous state management, and the implementation of embedded vector search for offline autonomy. Designed for engineers working in robotics, applied ML, and embedded systems, the talks focus on building resilient agentic behavior that functions reliably without cloud dependencies.

​​Agenda

​We begin at the model layer by analyzing the hardware mechanics and economics of running LLMs directly on edge devices. Next, we examine agentic memory, exploring how long-running agents retain state, rank sub-goals, and manage context limits without cloud infrastructure. We conclude with the retrieval layer, demonstrating how embedded vector engines enable sub-millisecond, offline hybrid search directly inside IoT hardware and autonomous robotics.

​18:00 Doors open

​18:30 - 19:00 LLM Inference Economics: From Datacenters to the Edge (Paul Willot, Senior MLE @ Liquid AI)

​19:00 - 19:30 Agentic memory for on-edge devices (Stefania Druga, Staff Research Scientist @ Sakana AI RSI Lab)

​19:30 - 20:00 Agentic retrieval for IoT hardware and autonomous systems (Ewa Szyszka, DevRel Engineer @ Qdrant)

​20:00 - 21:00 Networking

​21:00 Doors close

​Talks

​Talk 1 - LLM Inference Economics: From Datacenters to the Edge

​Speaker: Paul Willot (Senior MLE, Liquid AI)

​Abstract: As language models move beyond the datacenter and onto laptops, phones, robots, and other edge devices, a fundamental question emerges: where does it make economic sense to run intelligence? Answering it requires looking beneath model benchmarks at the mechanics of inference itself: how memory bandwidth, model size, hardware utilization, and concurrency determine the cost and speed of producing a token.

Datacenters benefit from scale, sharing the cost of large models across many concurrent users. Edge devices face a different regime, where resources are more constrained and there is far less opportunity to amortize inference through large batches. This makes model and system design increasingly important: quantization, sparsity, hybrid architectures, caching, and other techniques can materially shift the trade-off between cost, throughput, and interactivity.

Drawing on Liquid AI's work on efficient foundation models, the talk will explore how these trade-offs shape the emerging role of edge intelligence. Recent techniques such as speculative decoding (including Liquid AI's DSpark work) illustrate how rethinking the inference stack can move the performance frontier and make increasingly capable models practical on a much wider range of hardware.

​Bio: Paul Willot is a senior engineer at Liquid AI Japan, where he leads work on efficient AI models spanning vision, audio, and language. He has over a decade of experience in machine learning research and production, with previous roles at Mercari, Elix, and AlpacaJapan. His work has ranged from large-scale search and low-latency ML systems to computer vision and models deployed on low-power edge hardware. He holds a Master's degree in Data Science and Machine Learning from Sorbonne Université, and is particularly interested in making advanced AI models practical and efficient in real-world systems.

​Talk 2 - Agentic Memory for Edge Devices: Lessons from Long-Running Research Agents

​Speaker: Stefania Druga (Staff Research Scientist, Sakana.ai RSI Lab)

​Abstract: At Sakana AI, we build agents that run for hundreds of turns to read literature, conduct experiments, and draft papers. The model is rarely the only constraint. The surrounding harness forgets earlier decisions, repeats completed work, retrieves the wrong state, or drifts from the original research question.

​Edge and physically embodied agents face the same memory problem under even tighter limits on context, compute, storage, and connectivity. Using our existing experiments with long-running research agents, I will show when memory helps, when it becomes expensive noise, and what happens once relevant state falls out of context. I will compare memory-off, deployed recall, gating, active-subgoal ranking, and oracle conditions, showing that the main improvement came from ranking prior state by the current sub-goal.

​This is a practical field guide to memory harnesses: context eviction, progressive disclosure, recall-first compaction, structured decision memory, and trajectory-level evaluation. The central argument is that agentic memory is not merely storage; it is a policy for deciding which past information should have authority over the present action.

​Bio: Stefania Druga is a Staff Research Scientist at Sakana AI’s RSI Lab, where she studies memory, continual learning, and self-improvement in agentic systems. Previously, she was an AI Research Scientist at Google DeepMind and conducted research at the MIT Media Lab. She holds a PhD from the University of Washington and an MS from MIT. Her work spans agent evaluation, multimodal interaction, and embodied AI, with a focus on systems that learn from experience and remain reliable beyond a single session.

​Talk 3 - Qdrant Edge: agentic retrieval for IoT hardware and autonomous systems

​Speaker: Ewa Szyszka (DevRel Engineer, Qdrant)

​Abstract: Agents running on hardware can't wait on a network. A robot deciding where to step next, or a drone in a building with no uplink, needs retrieval that happens where the data is created.

​Qdrant Edge is the Qdrant vector search engine as an embedded library: you open a shard on local disk inside your own process, write embeddings to it, and query them offline, with an install footprint around 11 MB and no server, no Docker, no background threads.

​This talk walks through the on-device agentic loop (capture, embed, search, decide) using a live robot demo where hybrid search over dense vision vectors and BM25 captions returns results in under a millisecond.

​Bio: Ewa Szyszka is a DevRel Engineer at Qdrant, working on applied vector search from how embeddings from models like ESM-2 get indexed, screened, and inspected at scale to optimizing agentic retrieval latency on edge devices. She previously worked across computer vision R&D, benchmarks, observability and applications of ai in underwater navigation devices for marine applications at Keio University.

​​Organizers

​Ilya Kulyatin is an entrepreneur with work and academic experience in the US, Netherlands, Singapore, UK, and Japan. He holds a BA in Economics, an MA in Finance, and an MSc in Machine Learning. He's a 3x founder, now helping Japan grow the local AI ecosystem through a not-for-profit community, Tokyo AI (TAI), while building an AI-native system integrator and solutions provider, Foundry Labs株式会社.

​Ewa Szyszka is a DevRel Engineer at Qdrant, the open-source vector database, where she works on retrieval for scientific and multimodal data. She co-organizes Tokyo AI (TAI) events at the intersection of AI and the life sciences.

​​Supporters

​Foundry Labs K.K. is a Tokyo-based AI systems integrator and solutions provider, delivering end-to-end support for enterprises: from strategy design through implementation, deployment, and operations. They tailor AI to each client's operational, regulatory, and security requirements, with hands-on experience across finance, government, and industry, and a track record of shipping production systems in secure and regulated environments.

​Qdrant is an open-source vector search engine built to handle high-dimensional data at scale. It powers the retrieval layer behind some of the most demanding AI applications in production today, from RAG pipelines to recommendation systems to AI agents, giving developers fast, accurate similarity search wherever it's needed.

​About TAI

​Tokyo AI (TAI) is the largest international AI community in Japan, with 5,000+ members mainly based in Tokyo: engineers, researchers, investors, product managers, and corporate innovation leaders. Through 80+ events a year and 300+ speakers spanning startups, enterprises, and academia, TAI connects the people building AI in Japan with the global ecosystem, working to transform Tokyo into a global AI hub.

​​​Privacy Policy

​We will process your email address for the purposes of event-related communications and ongoing newsletter communications. You may unsubscribe from the newsletter at any time. Further details on how we process personal data are available in our Privacy Policy.