Microsoft AI diagnostic system outperforms physicians in benchmark study

Microsoft says its artificial intelligence diagnostic system, MAI-DxO, achieved higher diagnostic accuracy than generalist physicians across two medical case benchmarks, while identifying opportunities to reduce the cost of diagnostic investigations.

The findings, published in Science on 22 October 2026, report an accuracy rate of 83% on complex cases drawn from The New England Journal of Medicine (NEJM), compared with 36% for physicians using reference resources and 22% for physicians without them.

On a separate set of emergency department cases from the MIMIC-IV-ED database, MAI-DxO achieved 80% accuracy, against 47% for physicians, according to Microsoft.

The research examines whether AI can reproduce the sequential reasoning involved in clinical diagnosis, including deciding which questions to ask, which tests to request and how to revise a diagnosis as new evidence becomes available.

The results are from structured benchmark evaluations, rather than evidence that the system improves patient outcomes in routine clinical practice.

How Microsoft’s MAI-DxO diagnostic system works

Microsoft developed the Microsoft AI Diagnostic Orchestrator (MAI-DxO) to coordinate multiple AI-driven diagnostic perspectives instead of relying on a single model to provide an immediate answer.

The system operates through successive rounds in which virtual diagnostic agents propose possible conditions, request additional information, select investigations and assess competing explanations. The orchestrator coordinates this process and weighs the potential diagnostic value of further tests against their estimated cost.

This approach reflects a key distinction between medical examinations and clinical practice. Conventional medical knowledge tests typically provide a fixed set of information and ask candidates to select an answer. Doctors, by contrast, must often develop a diagnosis incrementally as they gather evidence.

Microsoft introduced the Sequential Diagnosis Benchmark (SDBench) to assess this process. It evaluates diagnostic accuracy alongside the information requested and the estimated cost of tests and procedures.

AI accuracy results across two medical benchmarks

The evaluation covered two different types of medical cases.

The first comprised 304 complex cases drawn from NEJM clinical-pathological case proceedings. These cases present challenging diagnostic problems and are intended to examine clinical reasoning.

Microsoft reported 83% diagnostic accuracy for MAI-DxO. Generalist physicians achieved 22% without external reference resources and 36% when they could consult such resources.

The second evaluation used more typical emergency department presentations from MIMIC-IV-ED, a database of hospital records. MAI-DxO achieved 80% accuracy, compared with 47% for physicians.

The results suggest that an AI system designed to gather and assess evidence sequentially can perform differently from models that are prompted to produce a diagnosis from information supplied upfront.

However, the comparisons should be interpreted within the conditions of the benchmark. The findings do not establish that MAI-DxO is more effective than specialist physicians or multidisciplinary clinical teams working with patients in real healthcare settings.

Microsoft examines the cost of medical investigations

The study also assessed the estimated cost of reaching a diagnosis, an important consideration for healthcare providers managing diagnostic resources.

Microsoft reported that configurations optimised for lower costs maintained relatively high diagnostic accuracy. On the NEJM cases, one configuration achieved 81% accuracy at less than half the cost of the comparative systems evaluated.

This indicates that diagnostic AI could potentially help healthcare organisations evaluate the trade-off between the accuracy of a diagnosis and the resources required to reach it.

The results do not, however, demonstrate equivalent savings in hospital budgets. The benchmark uses estimated costs for investigations, and actual healthcare expenditure varies according to the health system, local test prices, staffing, follow-up requirements and the consequences of an incorrect diagnosis.

Further evaluation would be needed to establish whether the approach reduces total costs without compromising patient safety or the quality of care.

What the research means for healthcare technology

The study adds to efforts to develop AI systems that support clinical reasoning rather than simply answer medical questions. Its sequential approach could be relevant to clinical decision-support software, diagnostic workflow tools and research into more efficient use of medical investigations.

For hospitals and healthcare technology providers, the central question is whether benchmark performance can translate into reliable assistance for clinicians under real operating conditions.

That would require assessment across diverse patient populations, clinical settings and diagnostic pathways, including cases where the available information is incomplete or a condition is difficult to identify.

Microsoft also reported that the approach continued to perform well when tested with newer AI models, including GPT-5.5, without changing the orchestration method. The company said it was making selected SDBench cases and evaluation components available to researchers to support further testing.

Clinical validation remains the next step

The results provide evidence of MAI-DxO’s performance under the study’s evaluation conditions, but they do not establish that the system is ready to replace physicians or independently direct patient care.

A key next step is to determine how the technology performs in clinical environments, where diagnosis involves patient interaction, physical examinations, medical histories, imaging and other evidence that may not be represented in a text-based benchmark.

For healthcare organisations considering AI diagnostic tools, clinical validation, safety, integration with existing workflows and demonstrable effects on patient outcomes will be central to assessing their practical value.

The research therefore represents progress in evaluating AI-based diagnostic reasoning, while leaving questions about clinical effectiveness and real-world cost savings to be resolved.

Follow B2B Market Signal on social media for the latest business news, industry developments, market intelligence and analysis. Connect with us on LinkedIn and follow us on Instagram at @b2bmarketsignal for regular updates.