Dango Daily Byte-sized AI and tech news for developers.
AI Read My MRI and Said No Tear. The Doctor Glanced at It and Said Over 50% Torn

AI Read My MRI and Said No Tear. The Doctor Glanced at It and Said Over 50% Torn

AIHealthcareMRIDiagnosis

Sources：HN + web research · HN

AI Read My MRI and Said No Tear. The Doctor Glanced at It and Said Over 50% Torn

The day Antoine got his MRI results, he sat in the clinic listening to the doctor tell him: right shoulder, infraspinatus tendon — “Grade III partial tear (greater than 50% width), at the terminal insertion.” Before he could fully process the diagnosis, treatment had already begun — a shockwave therapy device pressed directly against his shoulder, with the clinic recommending three sessions of this.

Everything happened too fast. Walking out of the clinic, Antoine had a nagging feeling: was the doctor’s judgment too hasty?

He did what any programmer in that position would do — he fed the 266MB of raw MRI data to AI. Claude Code + Opus 4.8 model. He let the AI install medical imaging packages itself and analyze hundreds of DICOM slices frame by frame. An hour later, the AI delivered a diagnostic report.

The doctor said “torn over 50%.” The AI said “tendon fully intact.”

Two completely opposite conclusions. Who do you trust?

This story hit the Hacker News front page a few days ago — 300+ points, 403 comments. And the best part isn’t in the original post. It’s in the comment section.

A Radiologist Said One Sentence, and Everyone Went Quiet

The top-voted comment on HN came from a radiologist, username sxg. His first sentence cut straight to the heart of it:

“I’m a radiologist, but without seeing the full 3D MRI data, I can’t give a real judgment.”

Then he pivoted and pointed out something Antoine had completely failed to realize in his original post.

Antoine had complained that the clinic did an ultrasound, said there was “no calcification,” and then gave him shockwave therapy. He looked up clinical guidelines using ChatGPT and found that shockwave therapy is not recommended for tendinopathy without calcification — so he started doubting the clinic’s competence.

The radiologist sxg’s reply woke everyone up:

“Ultrasound is not a good tool for assessing calcifications. It can catch large ones but easily misses small ones. Plain film (X-ray) would be more useful, though MRI can also pick them up. The key is: when a radiology report says a finding is ‘absent,’ there is always an implicit qualifier: absent within the limits of this examination modality and the images obtained in this study.”

In other words: an ultrasound report saying “no calcification” and an X-ray report saying “calcification present” — these two statements do not contradict each other. Ultrasound uses sound waves, X-ray uses radiation. They’re good at seeing fundamentally different things — like using a telescope to judge whether your food is salty enough.

And this is precisely the core flaw that AI exposed in this incident.

AI’s Problem: It Doesn’t Know What It’s Looking At

To understand why AI stumbles on medical imaging, you need to know one fact upfront: current general-purpose large language models (LLMs) are not designed for reading medical images.

They are trained to understand text and generate text. Even when frontier models like Claude and GPT-5.5 have “vision” multimodal capabilities, their understanding of images differs fundamentally from a radiologist’s.

When a radiologist reads an MRI, their brain is running a comprehensive reasoning process: What does the subtle grayscale variation between this frame and the next mean? Is the signal intensity in this region normal or abnormal for this particular scan sequence? How clinically significant is this finding given the patient’s age, sex, and symptom profile? — By contrast, when an LLM processes a medical image, it’s essentially doing pattern matching between pixel patterns and “image-text” pairs it saw in its training data.

In a position statement released by the Radiological Society of North America (RSNA) in July 2025, experts listed several core barriers LLMs face in radiology: a tendency to “hallucinate” (fabricate non-existent information), opaque training data making bias untraceable, and — most critically — a lack of genuine spatial understanding of images themselves.

A large-scale stress test study published in Nature Medicine in June 2026 confirmed this. Eric Topol’s team tested multiple frontier models — including GPT-5.5 Pro, Claude 3.5, and Gemini 2.5 Pro — on multimodal medical reasoning tasks. The conclusion was bluntly uncomfortable:

“GPT-5.5 Pro scored 79 out of 100, an improvement over the previous generation’s 69, but far from sufficient to be considered reliable for medical use. These models exhibit reasoning errors, inappropriate shortcut thinking, and hallucination problems.”

A 79, in an exam, might be a B+. But in a medical context, every point lost could be a missed diagnosis or a misdiagnosis.

AI’s “Overconfidence” Is a Real-World Risk

In medicine, there’s a repeatedly validated phenomenon: AI diagnostic models can perform near or even above human level on data within their training distribution, but the moment they encounter data outside that distribution — whether from different hospitals’ different scanning equipment, different patient populations, or different countries’ clinical guidelines — accuracy plummets.

A 2024 MIT study revealed an even more insidious problem: the AI models that were best at inferring a patient’s race and sex from X-ray images were also the ones showing the largest “fairness gap” — the greatest disparities in diagnostic accuracy across different demographic groups. This means AI can “see” features invisible to the human eye (like inferring race from an X-ray), but those features can become shortcuts to misdiagnosis.

Returning to Antoine’s case, there’s another detail many people missed: he gave the AI far less clinical information than he gave the doctor. In his original post, he writes that he only gave Claude Code “right shoulder pain, 2–3 weeks” as context. The doctor got a full patient intake history.

Later, he asked the AI to act as “arbitrator” — re-reading the two contradictory diagnostic reports and adding his chat history with ChatGPT about shoulder test maneuvers. This time the AI leaned toward “no tear.” But an HN user cut through it precisely:

“I subscribe to multiple large models. Ask the same medical question in different conversations and you can get completely contradictory answers — and every answer sounds supremely confident. The scariest part is, you can very easily steer each model toward whatever answer you want — when you keep mentioning a direction another model gave during your follow-ups, the conversation quietly drifts toward that direction.”

This is the essence of AI overconfidence: it’s trained to “sound good to humans.” A/B testing has repeatedly shown that when human users rate AI responses, scores correlate more with “how pleasant the tone is” than “how factually correct it is” — like how a hospital room’s view doesn’t change medical outcomes but significantly affects patient satisfaction scores.

The Difference Between Doctors and AI Isn’t Technical — It’s Knowing What Not to Answer

Another comment in the HN thread from a cardiac sonographer hit hard:

“I’m a cardiac sonographer. Watching people discuss AI coming for radiologists’ jobs — all I can say is: asking AI to tell you how to operate an ultrasound probe to acquire images is like pushing someone who’s never touched a musical instrument onto a stage and telling them ‘don’t worry, AI will teach you how to play.’”

This captures both the boundary of AI’s potential and the irreplaceability of human physicians in one sentence.

AI is excellent for certain things: helping you understand the numbers on a blood test, alerting you to drug combinations that may be problematic, even — as in Antoine’s case — providing a different perspective when you feel uneasy about a diagnosis, pushing you to seek a second opinion. In these scenarios, AI is an “information magnifier,” not a “decision-maker.”

But when you ask AI “is my tendon torn,” you think it’s looking at your MRI. What it’s actually doing is: taking the images you provided and running probabilistic matching against the vast set of “MRI-like images + labels” it has seen, then telling you the answer in the smoothest, most confident tone available.

It doesn’t know what it missed. It doesn’t know whether this MRI machine’s scan sequence parameters match those of other hospitals. It doesn’t know that certain rare tendon pathologies are only visible at specific angles. And most critically — it doesn’t know when to say “I’m not sure.”

The radiologist sxg’s very first sentence on HN was: “Without seeing the full data, I can’t give a real judgment.”

That trained restraint is itself part of medical professionalism.

Medical Diagnosis Is Just Incredibly Complex

A point that’s easy to misunderstand needs clarifying: this incident is not saying “AI is useless.”

On certain specific medical imaging tasks — automated lung nodule detection, retinal病变 screening from fundus photographs — AI has already demonstrated near-human or even super-human single-point accuracy. But these are all under highly constrained conditions: fixed equipment, standardized scanning protocols, clear binary classification tasks, rigorously labeled and validated training data.

Antoine’s scenario was completely different: non-standard DICOM export, no labels, a general-purpose LLM rather than a specialized medical AI, an open-ended diagnostic question, and minimal clinical context. Any single link in the chain going wrong could derail the conclusion.

A radiologist’s “modality-level expertise” — knowing what ultrasound, X-ray, CT, and MRI are each good at seeing, where each has blind spots, and when to switch to a different exam — this kind of full-chain clinical judgment is something current AI completely lacks. AI just gave a “plausible-looking” answer on some ambiguous block of pixels.

Closing

The goal of this article is not to pronounce a death sentence on AI, nor to manufacture panic for readers. What I want to say is: the speed and manner in which AI changes medicine may be very different from what many people imagine.

It won’t be a sudden announcement one day that “AI has replaced radiologists.” It will start with the most tedious, most verifiable tasks — flagging suspicious regions, comparing historical imaging changes, reducing repetitive labor. When these tools are truly mature, you won’t see it in a headline; you’ll feel it in a doctor’s daily workflow.

As for now — when you feed your MRI to AI and ask “am I okay,” remember the HN user’s summary:

“The key thing is better information, and AI cannot yet reliably provide that.”

The next time you’re holding a medical report you can’t make sense of, before asking AI, the better choice might be to first ask your doctor: Is this imaging modality the right one to answer my question? Is there an additional exam I should consider? The answers to those questions are worth trusting far more than any AI-generated diagnosis.

Reference Links:

References（2）