Nordlys Open Research
Nordlys Labs Open Research
Session 1: Mixture of Models Architecture Search
We are hosting the first Nordlys Labs Open Research session, a weekly one-hour discussion focused on open and collaborative AI research.
Open Research takes place every Saturday and is discussion-driven. Each session focuses on a specific research topic, with shared context and open conversation.
The first session will present Mixture of Models (MoM), a framework developed at Nordlys Labs based on the observation that no single large language model performs best across all tasks. Analysis of software engineering benchmarks shows that models have complementary strengths, and tasks failed by one model are often solved by others. This creates a higher performance ceiling for systems that select the best model per task rather than relying on a single model.
The session will cover how MoM works in practice, including semantic clustering of tasks, learning per-cluster model performance, and inference-time routing. Problems are embedded into dense representations that group tasks by underlying structure, allowing new tasks to be routed to the model with the strongest historical performance for that cluster.
We will discuss evaluation results on SWE-bench Verified, showing that this lightweight, training-free routing approach exceeds single-model baselines and recovers a significant portion of the oracle performance gap. The session will also cover evidence of model complementarity and how unsupervised clustering yields interpretable task categories without manual labeling.
The session will be presented by Mohamed El Amine Atoui, Founder of Nordlys Labs.
LinkedIn: https://uk.linkedin.com/in/mohamed-el-amine-atoui-3a39a5299
The Mixture of Models framework is fully open source and available at:
https://github.com/Nordlys-Labs/nordlys
The event will take place on our Discord server. Some future sessions may be held in person.