AI diagnostic tool outperforms doctors in complex case study

Microsoft’s new AI diagnostic system correctly identified complex medical conditions in simulated scenarios 83 percent of the time, a figure that more than doubled the performance of human generalist physicians. The study, published this Thursday in the journal Science, found that primary care doctors and general internists working individually with reference materials were correct in only 36 percent of these difficult cases. This stark difference highlights the potential for artificial intelligence to handle medical reasoning that traditionally requires specialized expertise.

The system, known as MAI-DxO, functions by coordinating a panel of distinct AI agents. Rather than relying on a single model to generate an answer, these agents work together to question a virtual patient, order necessary tests, and debate the findings to reach a final diagnosis. This collaborative approach mimics the way a team of specialists might consult on a particularly tricky case, aiming to reduce the errors that can occur when a single provider makes a high-stakes decision alone.

Dominic King, the vice president of Health at Microsoft AI, described the findings as substantial and exciting. He noted that the healthcare sector currently faces a growing mismatch between the demand for medical expertise and the available supply. By automating parts of the diagnostic process, the technology aims to bridge this gap. Microsoft joins other major technology firms, including Google and OpenAI, in claiming that its AI systems can match or even outperform human doctors in specific clinical tasks.

The underlying data from the study suggests that much of the performance improvement stems from the use of more powerful foundational AI models. Microsoft tested its orchestration system with models from several competitors, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The best results were achieved when using OpenAI’s GPT-5 and GPT-5.5 models, indicating that the choice of the base model is critical to the system's overall effectiveness.

Cost efficiency was also a key metric in the analysis. When using GPT-5 with a basic prompt, the system achieved a 78.9 percent accuracy rate at an estimated cost of $8,400 per case. The addition of the MAI-DxO orchestration layer reduced that cost by approximately $3,000 while improving accuracy by only four points. For emergency department cases, the improvement in both accuracy and cost was minimal, suggesting that the system adds the most value in complex scenarios rather than routine triage.

With the newer GPT-5.5 model, released in April 2026, the results were mixed. For complex cases, the orchestration layer boosted accuracy from 72 percent to nearly 87 percent at a slightly lower cost. However, this gain was not replicated in emergency department scenarios, where both the standard and orchestrated versions scored around 85 percent. The system proved most beneficial for less capable models, such as Gemini 2.5 Flash, which saw its accuracy rise from 46.4 percent to nearly 69 percent with the assistance of the diagnostic tool.

Mustafa Suleyman, the CEO of Microsoft AI, stated that the company is working with the Mayo Clinic to train a foundation model specifically for healthcare, with a release planned for next year. He believes this model will enable significant leaps in medical AI. Suleyman, who previously co-founded DeepMind before it was acquired by Google, has long focused on using AI to solve medical issues. He joined Microsoft in 2024 to lead its AI unit, bringing his expertise in clinical applications to the company.

Suleyman predicted that by 2027, the technology will be on the verge of a breakthrough in handling medical records. He envisions a system that can stream highly accurate predictions about how a patient’s condition might unfold, providing clinicians with valuable foresight. He expects healthcare models to become an order of magnitude more capable within the next 24 to 36 months, driven by a flywheel of real-world usage that identifies gaps and allows for rapid iteration and improvement.

To validate these findings in a real-world setting, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. The company will evaluate the clinical reasoning capabilities of the system across various hospital environments. Patients will receive standard care, while those who consent will also interact with the AI tool. The recommendations from the system will be compared against the decisions made by human doctors. King described this as the largest study of its kind, aiming to provide compelling evidence for the future deployment of such systems at scale.

The study coincides with a broader manifesto from Microsoft on the path to medical superintelligence. This term is defined as the capacity to surpass the strongest human-led health systems in medical reasoning and coordination. The company aims to move beyond simple benchmarking tasks and integrate directly with clinicians and healthcare systems. Microsoft already sells dictation and radiology tools and views this new diagnostic capability as a major expansion of its offerings, with Suleyman insisting that the company must maintain the top position in clinical decision support products in the coming years.