​🦄 ai that works

​A weekly conversation about how we can all get the most juice out of todays models with @vaibcode & @dexhorthy

​https://github.com/ai-that-works/ai-that-works

​

​This episode is all about latency. How do we stop users from twiddling their thumbs when LLM apis are getting faster, but still too slow? The answer shouldn't be "LLMs will eventually get faster".

​We'll talk about:

• ​why time-to-first-token is not time-to-useful-content

• ​why streaming partially-complete JSON data is hard from a tech perspective

• ​balancing perceived performance with actual utility with semantic streaming

• ​designing to keep users engaged during longer operations

​Pre-reading

​To prevent repeating the basics, we recommend you come in having already understanding some of the tooling we will be using:

• ​Discord

• ​Cursor or VS Code

• ​Programming languages

  • ​Application Logic: Python or Typescript or Go

  • ​Prompting: BAML (recommend video)

​Meet the Speakers 🧑‍💻

​​​Meet Vaibhav Gupta, one of the creators of BAML and YC alum. He spent 10 years in AI performance optimization at places like Google, Microsoft, and D. E. Shaw. He loves diving deep and chatting about anything related to Gen AI and Computer Vision! 

​Meet Dex Horthy, founder at HumanLayer and coiner of the term Context Engineering. He spent 10+ years building devops tools at Replicated, Sprout Social and JPL. DevOps junkie turned AI Engineer.