Microsoft says AI system diagnosed complex medical cases more accurately in a simulation

Getting through test after test without a clear explanation for your symptoms can be draining. Microsoft says a new AI system may help doctors work through some of medicine’s more difficult diagnostic cases, though its early results come with important caveats.

In a study published October 22 in the journal Science, the system correctly identified 83% of a set of complex medical cases. Generalist physicians, who could consult reference materials, reached 36% accuracy in the study’s comparison. That sounds like a striking gap. But the research was conducted in a simulation, not in hospitals treating patients.

Microsoft calls the technology the MAI Diagnostic Orchestrator, or MAI-DxO. It uses several AI agents that consider a case together, rather than producing an answer immediately. The system can ask follow-up questions, weigh possible tests and revisit its conclusions as new information comes in. The agents can also challenge one another’s reasoning.

That process could matter when symptoms point in several directions. For example, unexplained fatigue and unusual lab results might have multiple possible causes. The system can explore different explanations and adjust its view as evidence accumulates—a useful idea, at least in theory, when diseases have overlapping symptoms.

Researchers tested MAI-DxO on 304 challenging cases from the New England Journal of Medicine. Generalist doctors working alone scored 36% when they had reference materials and 22% without them. In a separate test using historical emergency department cases, the AI reached about 80% accuracy, compared with 47% for the physician group.

Those figures need context. The doctors in the comparison were generalists, including primary care physicians and general internists. They assessed cases individually, rather than working in teams. In real hospitals, difficult cases may be referred to specialists who review evidence together. Microsoft did not compare MAI-DxO with teams of medical specialists.

The groups also worked through different numbers of cases: the AI was tested on 304, while doctors assessed 56 cases set aside from the system’s development. The study relied on existing written case reports, not real-time patient visits. Actual patients may have trouble describing symptoms, records can be incomplete, and physical examinations can reveal details that never appear in a case summary.

There are other limits. The system used AI to provide information drawn from the case reports and could generate plausible responses when details were missing. That made it possible to simulate a diagnostic conversation, but it cannot fully reproduce the uncertainty of an actual clinical encounter. And finding the right diagnosis is only one part of care: doctors must also choose treatment and take each patient’s circumstances into account.

The study’s authors work for Microsoft, so independent evaluations will be important. For now, the results show how the system performed under controlled conditions. Whether it can help patients in everyday clinical settings is still unknown.

How much does the system add to existing AI?

The researchers also tested models from several AI companies, including OpenAI, Google and Anthropic. The results suggest that the underlying model already accounts for much of the system’s diagnostic ability.

OpenAI’s GPT-5 solved about 79% of the complex cases with a basic prompt. MAI-DxO raised that figure to roughly 83%. With GPT-5.5, the orchestration system increased accuracy from around 72% to nearly 87% on complex cases. But it added little on emergency department cases when paired with the strongest models. The benefit, it seems, can depend on both the model and the type of case.

Microsoft CEO of AI Mustafa Suleyman has described a broader goal of creating “medical superintelligence”—AI that could eventually outperform today’s leading medical teams in clinical reasoning and coordination. The company is also working with Mayo Clinic on AI intended to help clinicians anticipate how a patient’s condition might change. Those are ambitions, not capabilities demonstrated by this study.

Could AI reduce testing costs?

Microsoft also examined whether the system could reach diagnoses while limiting unnecessary tests. In one configuration designed to be cost-conscious, MAI-DxO achieved about 81% accuracy on complex cases, with estimated diagnostic costs of $2,241 per case. Physicians using reference materials averaged about $4,725.

Other configurations aimed at maximizing accuracy produced higher testing costs, so the potential savings depended on how the system was set up. The dollar amounts are estimates based on diagnostic workups and standardized visit costs. They are not actual patient bills, and they do not include the cost of running the AI. Whether similar savings are possible in hospitals remains to be tested.

Testing the system with patients

Microsoft has announced a partnership with Beth Israel Deaconess Medical Center in Boston to study MAI-DxO in a clinical setting. Consenting patients will continue to receive standard care while researchers compare the system’s recommendations with physicians’ assessments. Doctors will remain responsible for treatment.

The evaluation should offer a clearer picture of how the technology handles real patients and the unpredictable pace of hospital care. Other companies are exploring similar approaches. In a study published in The Lancet, Google tested its conversational medical AI, AMIE, with 98 patients ahead of urgent primary care appointments. Physicians monitored the conversations, and none required a safety-related interruption under the study’s criteria. Clinicians also said the AI’s summaries often helped them prepare.

That research was small, too, but it offers an early look at how AI might support doctors without replacing them. Microsoft’s clinical study will add another test of whether these systems can work beyond carefully controlled simulations.

For now, MAI-DxO is a research tool, not a substitute for seeing a doctor. If systems like it prove reliable, they could give clinicians another way to review difficult cases, notice possibilities that might have been missed and perhaps reduce unnecessary testing. But accuracy by itself is not enough reason to put AI into patient care. Doctors need to know where a system can fail, and patients should understand how their medical information is used and who checks AI-generated recommendations.

People are increasingly turning to chatbots for health advice, which makes caution worthwhile. AI responses can help you prepare questions for a doctor, but they should not be treated as a confirmed diagnosis. Before sharing medical records with any AI service, check how it handles sensitive information. The most useful role for this technology may be helping physicians make better-informed decisions while keeping patients involved.