«Hack Apertus» - Apertus 1.5 under test: academic and real business tasks
Hoi and welcome to «Hack Apertus»!
This session is part of the Online Hackathon (1.–16.10.2026) and presents what Apertus 1.5-8B can and cannot do, based on two independent evaluations by the DS-NLP Lab at the University of St. Gallen. The aim is practical: to help you decide where the model is a good fit for your hackathon project, when to switch on thinking mode, and where you might find some challenges.
Part 1 Standard benchmarks. How much did Apertus 1.5 improve over 1.0, and what does thinking mode actually contribute? We ran both generations under identical settings on one H100 across core text benchmarks (MMLU-Pro, GSM8K, IFEval, LongBench, GPQA), multilingual suites in 14 languages, and a reasoning suite (MATH-500, AIME, MGSM). The short version: large gains almost everywhere, thinking mode is a targeted tool for multi-step maths, and there is one regression we report rather than average away.
Part 2 Real business tasks. Benchmarks are one thing; support tickets with typos are another. We tested Apertus 1.5 on four everyday business tasks with real, public data (ticket routing, urgency triage, receipt extraction, contract clause detection) against seven open 7–9B models and two frontier models via API. Apertus 1.5 came out as the strongest open model of its size and within a few points of the frontier systems. We also show why synthetic test data made the same model look 30 points better than it is.
Further reading: https://blog.nlp-lab.ai
Speakers:
Prof. Dr. Siegfried Handschuh, Chair of Data Science and Natural Language Processing, University of St. Gallen (introduction, business benchmark)
Götz-Henrik Wiegand, DS-NLP Lab, University of St. Gallen (academic benchmarks)
📅 Date & Time: 02.10.2026, 10:15-11:00
⏱️ Duration: 45 minutes
🌐 Language: English
🗣️ Format: 35 min. presentation + 10 min. Q&A
📍 Location: Live on Zoom, YT
👥 Target group: Hackers
«Hack Apertus» is supported by its Main Partners Phoeniqs, CSCS, Canton St.Gallen, Innosuisse, Supertext, Canton Bern, and BaselTech.