

Backchannel Berlin (with Inworld)
A research discussion on building frontier AI models, for people working on speech, audio and language models.
End-to-end speech-to-speech models can now listen while they talk, handle interruptions and keep a consistent voice. Getting them to frontier quality is still largely open research: data, architecture, training objectives and evaluation. Running them for consumer apps, in every language and within a few hundred milliseconds, adds constraints of its own. The afternoon covers both.
Open questions: speech-to-speech and conversation
What should a full-duplex model predict, and at what granularity, to handle turn-taking, interruptions and backchannels?
Cascaded STT, LLM and TTS versus end-to-end speech-to-speech: where does each break down outside benchmarks, and what closes the gap?
How do you keep an LLM's reasoning and instruction-following when you train it to listen and speak?
What data and speech representations does full-duplex training need, and how far can synthetic dialogue go?
How do you train a model to follow plain-language direction on delivery while keeping a speaker's identity stable?
Open questions: at consumer scale
How far can cross-lingual transfer take these models in languages with little transcribed speech?
Which of batching, caching, quantization and speculative decoding matter most when a conversation has to respond within a few hundred milliseconds?
MOS, WER, speaker similarity and preference arenas each miss something. What should we measure for conversations, and how do we evaluate languages with few native raters?
Format
2 Knowledge sharing sessions with tech-talks and the panel session closing them:
15:00 Doors open, make yourself at home
15:30 Welcome
15:45 – 17:05 · Tech talks, part 1 (four talks, 20 min each)
17:05 Break 1
17:15 – 18:35 · Tech talks, part 2 (four talks, 20 min each)
18:35 Break 2
18:45 – 19:15 · Fireside chat
19:15 – ... Food, drinks and hanging out
Who it's for
Researchers and engineers working on speech-to-speech and full-duplex models, TTS, ASR, audio codecs, conversational modelling, LLM serving or evaluation, in academia or industry. PhD students are welcome.
If you’re traveling from outside Berlin to join us, we may be able to help cover your travel costs. Please reach out to alesya@inworld.ai, as support is approved on a case-by-case basis.
About Inworld
Inworld AI is a research lab building realtime speech models and inference for consumer applications. We're working on full-duplex speech-to-speech and recently acquired Ultravox to accelerate that research. Our models team also trains our TTS and speech recognition models. TTS-2, our current TTS model, takes direction on delivery in plain language. Our inference research team works on backends, kernels, quantization and decoding. In production we serve trillions of LLM tokens and tens of billions of characters of speech a month.