Inside Vercel's Agent Desktop, Plus Decision Models: AI Collective Webinar (w/ The AI Collective)
AI agents are getting good at code. The next frontier is the rest of your computer.
Part 1: Inside Agent Desktop
Agent Desktop is an open-source CLI from Vercel Labs that lets AI agents see and operate real desktop apps on macOS, Windows and Linux. It reads the operating system's accessibility tree, not screenshots, and gives your agent a reference for every button and field it can act on. I'll show how it works today, as Vercel built it.
Part 2: Adding decision models
Then I'll share my own experiment: adding decision models on top. Decision models answer typed questions with calibrated probabilities instead of writing text. That can mean confidence-gated actions, faster and cheaper next-step choices, and an agent that knows when to ask a human. I'll walk through Jev from TypeSafe AI, and test it head to head against Nace.AI's Drex. This layer is my work, not part of Agent Desktop.
What you'll learn
How Agent Desktop works: snapshot, act on a ref, verify
What decision models are and how they differ from LLMs
How Jev and Drex compare on real desktop tasks
How to gate agent actions on confidence, so risky steps wait for a human
Where this setup works today, and where it still breaks
Who it's for
Builders, engineers and operators automating work across desktop apps.
Format
A live demo of both parts, then open Q&A. Streaming live on LinkedIn, YouTube and X. The join link is coming soon.
Hosted by AJ Green, President & Executive Director of The AI Collective.
The AI Collective is a global non-profit community of 500K members building the human layer for the AI era.
All attendees and organizers at events affiliated with The AI Collective agree to our privacy policy and are subject to our code of conduct.