The Model Picks the Code, the Rules Decide: Guardrails for an LLM-Driven Medical Interview

​What should an LLM be allowed to decide in a medical interview, and what must stay in code? A 40-minute engineering talk built on measurements from MedBot, followed by 20 minutes of Q&A.

​MedBot is a research prototype that conducts structured medical interviews with synthetic patients in simulated cases, then hands the resulting report to a physician evaluator. It never gives advice to patients.

​What you'll see

• ​Closed catalogs instead of free-text safety decisions. The model can select only red-flag codes from a YAML catalog. Urgency levels and escalation language come from deterministic rule files; out-of-catalog values are discarded as invalid.

• ​Structured output generated from the rules. JSON Schema enum values are generated from the same rule files used during runtime validation.

• ​Detect, then audit. One LLM call selects red-flag codes; a second call audits each code against its formal condition. Giving the condition to the detector itself dropped sensitivity from 1.00 to 0.91, so the two jobs stay separate. If either call returns no verdict, the turn is rolled back.

• ​Measure, don't guess. A phrase matcher achieved 0.81 sensitivity on the tuning set but only 0.60 on a sealed holdout set. Across a 220-case evaluation, an open-weight model detected every labeled alarm for approximately $0.05 per run, although it produced more false positives. The tested Sonnet configuration missed five alarms.

• ​Safety gates we removed. A dosage filter could not reliably distinguish medication history ("the patient takes aspirin") from treatment advice ("take aspirin"). A fact-grounding gate rejected 25 of 40 first-pass outputs.

• ​Prompt caching in a multi-turn interview. Why the framework's default caching strategy yields zero cache reads, and costs more than running without caching.

​Who it's for

​Engineers building LLM features in regulated or high-stakes domains: healthcare, finance, legal. The implementation uses Java and Spring AI with Anthropic and open-weight models; the patterns are stack-agnostic.

​Logistics

• ​CIC Cambridge, One Broadway, 11th floor (Singapore room)

• ​Doors 8:45 AM; we start at 9:00 sharp and end at 10:00 AM

• ​16 seats. If your plans change, please cancel your RSVP so someone on the waitlist can take your seat.

​About the speaker

​Piotr Obolewicz is a senior Java engineer building production AI systems, currently for healthcare triage. Based in Fukuoka and Warsaw. More at obolewicz.com