Signals: Stop Saying Medical AI Can’t Be Held Accountable
STAT First Opinion just published a letter I wrote responding to a piece by Arya Rao and Marc Succi on how people are using AI self-care apps to do things the healthcare system used to do for them. They call it a "shadow medical system" and, re the accountability they think is lacking, they end with an IBM training slide from 1979 asserting that “a computer can never be held accountable, therefore a computer must never make a management decision.”
That's a pretty definitive "never": no AI making management decisions. Ever.
The problem is that, as I point out in the letter, that “never” ended in 2018, when the FDA approved LumineticsCore's no-doc-in-the-loop system for diagnosing diabetic retinopathy. With LumineticsCore bearing responsibility for its AI’s medical judgment and carrying appropriate malpractice insurance.
Same for Doctronic's current prescription renewal pilot in Utah: no doc for most decisions means Doctronic had to get malpractice insurance.
Unfortunately, as Rao and Succi’s article suggests, much of the medical community still has not registered these developments — including the one from 2018 — and keeps treating AI accountability as a problem with no solution.
The solution for unaccountable practice of medicine is there, we just need to fully implement it — exactly as we do with human doctors.