Hospitals Are Deploying AI Like Crazy. We Still Don’t Know If It Helps Patients

I don’t need to tell you that AI is everywhere. Or that it’s creeping into hospitals at a pace that makes my head spin.

Doctors are using AI scribes to take notes during appointments. Algorithms are scanning patient records to flag people who might need extra support. Others are reading x-rays and lab results. The list gets longer every quarter.

A pile of studies shows these tools can be accurate. But accuracy isn’t the same as actually helping patients live longer, suffer less, or recover faster. And a new paper in Nature Medicine from Jenna Wiens at the University of Michigan and Anna Goldenberg at the University of Toronto argues we’re failing to ask the hard question: does any of this stuff make a real difference?

Wiens has been trying to sell clinicians on AI for over a decade. For most of that time, nobody was buying. Then, a few years ago, the switch flipped. Suddenly hospitals are adopting these tools like there’s no tomorrow. The problem, she says, is that most of them aren’t bothering to check if the tools actually work in the real world.

Take ambient AI scribes. These things listen to conversations between doctors and patients, then spit out a summary. Multiple vendors sell them, and adoption has been rapid. A staffer at a major New York medical center told me a few months ago that doctors are “overjoyed” — they can finally look at patients instead of screens, and the paperwork monster is tamed. Early studies back up the happiness, showing reduced burnout.

Great. But what about patient health? “Researchers have evaluated provider or clinician and patient satisfaction, but not really how these tools are affecting clinical decision-making,” Wiens told me. “We just don’t know.”

And it’s not just scribes. Predictive tools that guess a patient’s trajectory, recommend treatments, or flag deterioration are all being rolled out. Even accurate ones might not improve outcomes. An AI might read a chest x-ray faster, but how much does the doctor trust it? Does it change how they interact with the patient? Does it lead to different treatment decisions? The answers probably vary by hospital, department, and even the experience level of the doctor.

Here’s something that bothers me: research on AI in education suggests these tools can change how people process information. Could AI scribes make a resident think differently about a patient’s story? Could they subtly alter how medical students learn to reason through cases? Wiens thinks we need to look into this. “We like things that save us time, but we have to think about the unintended consequences,” she said.

Paige Nong at the University of Minnesota published a study in January 2025 that found roughly 65% of US hospitals were using AI-assisted predictive tools. Only two-thirds of those bothered to check accuracy. Even fewer tested for bias. That number has probably gone up since then, but the evaluation gap hasn’t closed.

Wiens isn’t anti-AI. She believes it has real potential. She just wants hospitals and independent researchers to actually measure whether these tools help or just add noise. The worst case is that some patients end up worse off, but she thinks it’s more likely the tools just aren’t delivering the benefits everyone assumes they are.

“I do believe in the potential of AI to really improve clinical care,” she said. “I have to believe that in the future it’s not all AI or no AI. It’s somewhere in between.”

That somewhere in between requires data we don’t have yet. And until we get it, hospitals are flying blind.