Medical AI and the cost of obtaining the next answer

On October 22, 2026, Microsoft described MAI-DxO and a planned prospective study with Beth Israel Deaconess Medical Center. The study is a next step, not a completed clinical result. Microsoft announcement

The important question for a hospital considering such a system is not only whether it can suggest the right diagnosis. It is whether the next piece of information it requests is worth obtaining in that setting. An answer acquired immediately in a simulated consultation and an answer requiring another appointment are different inputs to a care pathway, even if their text is identical. This article develops an evaluation framework, not instructions for diagnosing or treating an individual.

Information has a route to the decision

The SDBench documentation describes a gatekeeper that supplies requested case information and can generate a synthetic result when the case text lacks it. Benchmark documentation That makes the boundary between a simulated information environment and a working service particularly important. A benchmark can explore a sequence of questions without establishing that every corresponding examination is available, appropriate or timely for a particular patient.

Consider a hypothetical comparison between two hospitals. Both receive the same AI suggestion to request further information. One can obtain it during the current visit; the other would require a return journey and a later appointment. Assume no difference in the model or in the eventual answer. The useful output may nevertheless differ because the information reaches the decision at a different time. This is a proposed evaluation scenario, not a description of the announced study or a claim about either hospital.

A purchasing team could therefore ask for a local map from each requested information item to the work required to produce it. Who must arrange it? Which equipment and staff are involved? Can the result arrive before the decision it is meant to inform? Where the answer is unavailable, the evaluation should retain that state rather than quietly replacing it with a fully observed case. Otherwise the deployment is being credited with a service the institution has not supplied.

A smaller test bill is only one component

For an original assessment framework, distinguish the invoice for a test from the burden of obtaining its result. Relevant questions might include travel, scheduling, staff time, whether a procedure is invasive, and whether repeating it is appropriate. These are matters for clinicians and study designers to assess in context. They cannot be converted into a universal penalty per test, and this article assigns no such weights.

There is also a cost to withholding useful information. An evaluation designed only to reward fewer requests could favour stopping too soon. One designed only to reward diagnostic certainty could favour asking indefinitely. The interesting comparison is between complete decision pathways: which information changed a clinically meaningful action, which requests added little, and what happened when the system could not obtain what it asked for. Such a comparison must include possible benefit as well as burden; it is not an argument for minimising testing.

NICE's digital health evidence framework says diagnostic accuracy alone does not establish clinical usefulness. NICE digital health framework Applying that distinction here suggests a practical question: if two sequences reach the same conclusion, does one allow an appropriate decision earlier, with less burden, without creating a different problem later? Conversely, if the conclusions differ, a lower immediate bill cannot by itself decide which sequence was preferable.

Study the alternatives that were actually available

The comparator matters. Comparing an AI pathway with an idealised clinician who has unlimited time would answer one question; comparing it with the local service as it operates would answer another. Neither should silently stand in for the other. A useful protocol would describe the available appointments, staffing and information at the point each decision was made, as well as the rule for involving clinical judgement when the system's request cannot be followed.

This need not make an early study impossibly large. A bounded study could first examine whether its suggested next steps are feasible and whether the information arrives in time. A later investigation could test effects on decisions and patient outcomes. Separating those questions avoids presenting workflow feasibility as established clinical benefit, while still recognising feasibility as useful evidence. The announced prospective work should be judged by its actual protocol and reported findings, not by outcomes that outside observers hope it will measure.

A further distinction concerns mistakes. Missing a condition and initiating an unnecessary investigation do not necessarily have equivalent consequences; a BMJ methods paper explains why clinical context matters when comparing diagnostic performance. Methods paper Our implication is that a hospital should not conceal these consequences inside one average score or one money figure. Reporting them separately lets clinical reviewers see what trade-off a claimed saving contains.

What would change the adoption decision?

An encouraging result would connect the model's information requests with a service that can fulfil them, a clinician's subsequent decisions, and a relevant observation period. An inconclusive result might show good reasoning but insufficient access to the requested information. A negative result could reveal that a seemingly economical sequence simply displaced work to another team. These are possible interpretations to investigate, not predictions about MAI-DxO.

The strongest case for sequential diagnostic AI is that it may help organise uncertainty rather than merely produce another answer. Its value would then depend partly on how well it recognises the information it cannot obtain. For developers and hospital buyers, this changes the product question: beyond asking what the system knows, ask what its proposed next step demands from the surrounding institution. The next evidence should show whether those demands produce useful care, not merely a tidy simulated conversation.