AI Industry

Microsoft's MAI-DxO Beats Generalist Doctors on Hard Cases

Oct 23, 2026·4 min read·By Rahish Saifi

A clinician's hands typing on a laptop on a mobile ward-rounds cart in a hospital emergency department corridor, the screen showing a branching diagnostic flowchart beside a panel of typed case notes, with a stethoscope and a patient case file on the cart shelf.

On October 22, 2026, researchers at Microsoft AI published in Science a study of sequential diagnosis with frontier language models—how an AI system gathers information over multiple turns, orders tests, and revises a differential diagnosis the way clinicians do in real workups. The paper introduces the Sequential Diagnosis Benchmark (SDBench) and Diagnostic Orchestrator (DxO), marketed as MAI-DxO, a model-agnostic orchestrator that simulates a panel of virtual physicians debating across rounds. On hard New England Journal of Medicine clinicopathological conference (CPC) cases and emergency-department admissions drawn from MIMIC-IV-ED, DxO with GPT-5 substantially outscored participating generalist physicians on diagnostic accuracy while often landing on a better cost-accuracy trade-off in the benchmark's text-only setting. Gains held—and improved—when the team re-ran evaluations with GPT-5.5 without changing orchestration prompts (Science; Microsoft AI). The authors and Microsoft leadership stress that these are controlled benchmark results, not evidence of superiority over specialists, multidisciplinary teams, or real-world bedside care—and the system is not FDA-cleared or deployed as a clinical product from this publication alone.

Key facts

Item	Detail
Publication	Science, “Sequential diagnosis with language models” (Nori et al.); print date 22 October 2026
Microsoft announcement	October 22, 2026 — “Building Medical Superintelligence” (Microsoft AI blog)
Benchmark	SDBench: 304 NEJM CPC cases and 325 MIMIC-IV-ED hospital admissions after ED evaluation, each turned into an interactive, stepwise text encounter (Science)
AI system	DxO / MAI-DxO — orchestrates simulated clinical experts, cost-conscious test stewardship, and aggregated runs; evaluated with GPT-5 (Science; Microsoft AI)
CPC accuracy (open-book physicians)	DxO: 83.2% at ~$5,431 mean test cost (unconstrained); 80.9% at ~$2,241; 79.6% at ~$1,303; up to 84.5% at ~$8,945 (Science). Physicians: 36.1% open-book; 22.3% closed-book (Science). Microsoft blog rounds to 83%, 36%, and 22% and cites ~3.7× and 2.3× accuracy vs physicians with and without external resources on NEJM cases
ED accuracy	DxO: 80.3% at ~$4,979 mean cost (unconstrained). Physicians: 46.8% average accuracy, ~$3,400 mean test cost (Science). Microsoft blog: 80% vs 47% (~1.7×). ED cases come from Beth Israel Deaconess Medical Center admissions data via MIMIC-IV-ED (The Verge)
Physician cohort	Active primary-care and hospital/acute-care physicians (including internal medicine, acute-care, and emergency physicians) on held-out CPC cases; open-book arm could use search and references (Science)
Model updates	DxO gains persisted with GPT-5.5 without changing prompts or orchestration (Science; Microsoft AI)
Open source	Microsoft AI is open-sourcing SDBench for select NEJM cases—benchmark, Gatekeeper, Judge, and cost-estimation code (Microsoft AI)
Real-world path	Prospective evaluation with Beth Israel Deaconess Medical Center planned; Microsoft AI CEO Mustafa Suleyman told The Verge the work “needs clinical validation” (Microsoft AI; The Verge)

How SDBench and MAI-DxO work

Most medical AI tests still look like licensing exams: every fact is in the prompt and the model picks one label. SDBench instead starts each case with a brief presentation. A Diagnostic agent asks for history or exam findings, orders tests, or commits to a diagnosis. A Gatekeeper returns only obtainable information (or neutral synthetic results when data are missing). A Judge scores the final answer against physician-authored rubrics, while cumulative test cost is estimated from CPT-based chargemaster pricing (Science).

DxO does not rely on a single-shot answer. It simulates a panel of virtual physicians proposing diagnoses, picking questions or tests, and challenging the leading view over multiple rounds—with cost-aware test stewardship (Microsoft AI). The orchestration is model-agnostic; on CPC cases DxO with GPT-5 asked about 25.5 questions before the first test versus 5.6 for physicians on the same held-out set, favoring structured history before expensive workups (Science). Under budget caps, DxO traded modest accuracy for lower estimated costs while staying on a favorable Pareto frontier versus physicians and baseline LMs (Science; Microsoft AI).

Results on NEJM and emergency cases

On NEJM CPC cases—deliberately hard teaching cases—DxO with GPT-5 hit 83.2% accuracy at ~$5,431 mean test cost in the unconstrained setting, versus 36.1% for open-book physicians and 22.3% closed-book (Science). The best individual physician reached 51.8% open-book (Science). On MIMIC-IV-ED admissions drawn from routine ED workflow, DxO reached 80.3% versus 46.8% for physicians; the top human scored 62.5% across 40 cases (Science). Microsoft's blog reports no other evaluated human or AI approach beat DxO on both accuracy and cost on these datasets.

Comparisons are against individual generalist physicians inside a text oracle—not subspecialist panels or real multidisciplinary rounds. The Science paper states the comparisons do not establish superiority over specialists, multidisciplinary teams, or real-world clinical practice (Science).

What it means—and hard limits

This is benchmark evidence, not a deployed clinical product. SDBench is text-only: no live EHR, imaging pixels, waveforms, or slides. Cases skew pathological—especially NEJM CPC material—so prevalence does not match routine care (Science; Medical Economics). Cost numbers are U.S. estimates and omit time-to-result, invasiveness, and logistics. The Judge grades diagnostic labels, not treatment safety or outcomes (Science).

Microsoft AI frames diagnosis as one block in a longer medical superintelligence roadmap and plans prospective work with Beth Israel Deaconess Medical Center. CEO Mustafa Suleyman told The Verge the work is “a major milestone along that journey” but “needs clinical validation” (The Verge). Nothing in the Oct. 22 paper or posts indicates FDA clearance or regulatory authorization. Microsoft AI will open-source SDBench for select NEJM cases (Gatekeeper, Judge, cost code) and highlighted Mayo Clinic collaboration on multimodal, longitudinal data (Microsoft AI).

Bottom line

Microsoft AI's MAI-DxO, evaluated in Science as DxO on SDBench, reached 83.2% accuracy on NEJM CPC cases and 80.3% on MIMIC-IV-ED admissions—well above 36.1% and 46.8% for participating generalist physicians in the same controlled, text-only benchmark—while exploring cost-accuracy trade-offs down to roughly $1,303 mean testing cost on CPC cases. The work is a serious step toward evaluating AI the way clinicians actually diagnose, but authors and executives are aligned that it remains benchmark evidence, not proof of real-world superiority, specialist-level performance, or regulatory-ready deployment.

For builders, the lesson is methodological: multi-agent orchestration with explicit cost trade-offs can change measured diagnostic paths, not just final labels. For clinicians and patients, the Oct. 22 headlines mark progress on a long validation path—not a license to bypass human judgment in the ED.