26/10/2026
The most powerful AI model isn't necessarily the best one for healthcare.
That's one of the key findings from CliniScribe AI's recent research collaboration with five postgraduate Data Science students at the University of Melbourne.
Our goal was to explore a critical question:
How can we make AI-generated clinical documentation more accurate, reliable and efficient?
Over several months, the researchers conducted more than 10,000 model evaluations, examining clinical accuracy, hallucinations, omissions, speed and prompt design.
Three findings stood out:
1. No single AI model performed best across every measure. Bigger and faster doesn't always mean better clinical documentation.
2. All eight models showed higher hallucination rates in challenging scenarios. Clinical reliability remains a critical consideration.
3. Prompt design matters enormously. Even small changes produced dramatic differences in performance for some models.
The challenge? AI is evolving faster than traditional research cycles.
The study used GPT-4o as its frontier-model benchmark. By the time the report was completed, newer generations of GPT and Claude models had already emerged!
This reinforces why continuous evaluation matters more than simply adopting the latest AI model.
While the findings are based on synthetic clinical data and require further real-world validation, they provide valuable direction for CliniScribe's ongoing development.
A sincere thank you to the postgraduate researchers and their academic supervisors for their excellent work.
We're proud to collaborate with academia to advance responsible healthcare AI.
Our mission remains simple: less time on documentation, more time for patient care, without compromising clinical accuracy and trust.
