Microsoft Says AI Tool Outperforms Physicians on Complex Medical Cases

Microsoft AI said Monday (June 30) that the Microsoft AI Diagnostic Orchestrator (MAI-DxO) correctly diagnoses 85% of diagnostically complex cases published by the New England Journal of Medicine (NEJM), compared to a mean accuracy of 20% among the practicing physicians the company included in a study.

MAI-DxO also achieved correct diagnoses more cost effectively than physicians, the company said in a Monday blog post.

“For AI to make a difference, clinicians and patients alike must be able to trust its performance,” the post said. “That’s where our new benchmarks and AI orchestrator come in.”

Earlier benchmarks used to evaluate AI systems in medicine were based on the United States Medical Licensing Examination (USMLE), which is based on multiple-choice questions, favors memorization and therefore overstates the apparent competence of AI systems, according to the post.

To overcome the limitations of that test, Microsoft AI developed a new one that requires sequential diagnosis and uses 304 recent cases published by NEJM, the post said. This test requires AI models and human physicians to ask questions, order tests and work toward a final diagnosis.

Microsoft AI’s test also includes a virtual cost that reflects real-world healthcare expenditures, per the post.

The MAI-DxO, being an orchestrator, accesses multiple language models and integrates diverse data sources, according to the report. It is also configurable so that it can be told to operate within defined cost constraints.

“Together with our partners, we strongly believe that the future of healthcare will be shaped by augmenting human expertise and empathy with the power of machine intelligence,” the post said. “We are excited to take the next steps in making that vision a reality.”

The PYMNTS Intelligence and AI-ID collaboration “Generative AI Tracker®: Generative AI Can Elevate Health and Revolutionize Healthcare” found that while Americans are enthusiastic about the potential benefits of AI in healthcare, they still feel uncomfortable with the idea of healthcare providers relying on AI or replacing their medical professionals with this technology.

Sixty percent of Americans said they are uncomfortable with a provider relying on AI in their healthcare, while 57% believe using AI to diagnose diseases and suggest treatments would harm the patient-provider relationship, according to the report.

The post Microsoft Says AI Tool Outperforms Physicians on Complex Medical Cases appeared first on PYMNTS.com.