Note

Notes: I Just Enrolled in Apple's Health AI Study. Here's What I Learned.

Consumer HealthFive MigrationsHealth System StrategyAI
part of consent form



Last week I wrote that we're guessing about why people ask AI their health questions, and that we need data on what they do next: skip the doctor visit, delay it, or book it.

As far as I know, the only data we have on that question comes from a Gallup and West Health survey released in April. It estimated that about 14 million US adults had skipped a provider visit in the previous 30 days because of what an AI told them. I wrote about that number when it came out and again the following week.

Now it looks like Apple, working with OpenEvidence (OE) — the leading AI tool used by 700,000 US clinicians a month — is trying to answer the question. Yesterday at HLTH in Vegas they launched a study that will ask up to 200,000 people that question, and others, for a year.

It's called the Apple Health AI Usage Study, and you enroll and participate via an iPhone app. The app answers health questions, using OE technology. It took me about five minutes to enroll this morning.

For maybe the first time in my life, I actually read the consent form, all 14 pages. Here's what I found.

What Apple will collect

For 12 months, Apple will ask me to report any new symptoms, doctor visits, diagnoses, medications, procedures and hospitalizations. I'll get a survey at least once a month. If I want, the app's "prepare for a visit" feature can export a summary for my provider, and then I'll get a follow-up survey after the appointment.

According to the consent form the study is designed to look at:

  • how AI use "shapes interaction with the healthcare system"

  • "where handoff to a clinician may be most valuable"

This, especially the "shapes interaction" part, is pretty much what I asked for last week — though I didn't expect anyone to start collecting it six days later.

Apple's data will be self-reported, like Gallup's. But Gallup was a one-time survey, and Apple will follow the same people for 12 months — and it can compare what they report with summaries of what they actually asked the app.

The form says results may be published, and that the study will be listed on ClinicalTrials.gov — though I looked and it's not there as of this morning.

Who's running it

Both named investigators work for Apple. The PI, Rajiv Kumar, is a pediatric endocrinologist Apple hired from Stanford's Children's Hospital in 2016. The sub-investigator is Asha Chesnutt, a pulmonary and critical care doc who spent 20 years at The Oregon Clinic before going to Apple. The form discloses that members of the study team "may have a financial interest in Apple," which should be obvious because they're both employees whose compensation includes stock.

A commercial review board, Advarra, oversees the study, and a contractor called UST Global helps staff it. No university affiliation is mentioned anywhere in the form, which says something, I think, about how much research is shifting away from the universities.

In July I wrote about a study that OE designed and paid for, and in which OpenEvidence scored highest. As with that study, I'd have a little more confidence in this one with an outside investigator involved — though importantly that was an evaluation of OE itself, and this is an evaluation of consumer health behavior (not, say, of the iPhone Health app).

This won't tell us anything about how accurate OE is

The consent form, not surprisingly, admits that AI "can sometimes be wrong or missing information."

Maybe more surprisingly, I couldn't find anything about checking the validity of the health answers generated by OE.

Questions go to OE, which generates the answers and has to delete both within 24 hours. Apple's study team gets a summary of each conversation, produced by "an automated evaluation process" (emphasis on automated: that process "does not involve human review of your individual conversations").

Apple's press release lists "the quality of responses" among the things the study will examine but the only measure of quality I could see was participants (i.e. consumers) rating the answers. But while participants can tell you whether an answer was clear, most of them aren't doctors and can't say if the answer was correct (that's why they're asking!).

So the study can show what 200,000 people did after getting an answer, but it doesn't seem to do anything to measure if the answers were right. To me what Apple will (hopefully) shed light on here is not the accuracy of AI models for health, but how AI models for health change our interactions with healthcare.

Who is writing the answers?

The app says its answers are "generated using OpenEvidence," and the consent form says it's "a version made for everyday people, not doctors" (so it's not the one I use in the clinic).

The form also says OE "may use third party LLM providers" to answer ... but it doesn't name them. Those providers are held to the same 24-hour limit, after which they have to delete the data. This is interesting because I thought (please correct me, readers, if you know) that OE didn't use outside LLMs for its clinician version. Wondering why they would need to for this one.

I'd like to know which LLMs those are so I've sent Apple and OE that question and a few others, and I'll update this post if they answer.

Liability: just wellness, people, no medical advice going on here

The app is using the well-known "we're just wellness, not medical advice" approach (see Whoop, Oura, et al) to avoid FDA approval. The consent form says the app is "only meant to give you general health and wellness information," that it is not FDA approved, and that guardrails are supposed to keep it from giving medical advice.

Even though that is pretty much what people will think it is doing.

As far as liability, the consent form and the software license say that:

  • There's no doctor-patient relationship. The license says using the app doesn't create one.

  • I'm responsible for what I do with the answers. The license says that I, and not Apple, am responsible for any action I take based on the AI's output.

  • BUT .... if I get hurt. While the license limits what Apple owes me to $50 (typical for software), it ALSO says the cap doesn't apply to my rights as a research participant, and that the consent form decides those. And then under the consent form, if I follow the study's directions and am hurt as a direct result, Apple will pay for medically necessary care, though not lost wages or other costs. And I keep the right to bring a negligence claim.

The fine print

ItemWhat the documents say
How long Apple keeps the dataUp to 10 years after the study ends.
AI trainingThe press release says study data won't be used to train AI models. The consent lets Apple use it to develop products and algorithms, "which may include machine learning and experiences around generative models." I've asked Apple how those two statements could be compatible.
Off-limits topicsSuicide, self-harm, violence and disordered eating get a refusal and a referral. Nobody monitors chats in real time.

Why this matters

I think that from Apple's perspective this is the trial run for a health assistant inside the Apple Health app. If you remember, last February Bloomberg said that Apple was scaling back its plans for a health coach. Maybe they're shifting from scaling back to scaling up — or maybe that depends on the results of this study.

From the healthcare perspective, to me this is about nailing down more data about what I call Interpretation Migration: when the task of explaining your health data moves "from the exam room to the living room"; from a doctor to an app on your phone.

Gallup gives us some idea that AI is moving some health work away from healthcare (14 million lost visits per month!). But as I pointed out in June, current projections of a physician shortage from AAMC and HRSA assume that none of this new tech exists, and that people in the future will use healthcare exactly like they do now.

I think they're wrong about that, and Gallup is the first evidence for how wrong. But a year from now Apple will know more than anyone about how asking an AI changes who goes to the doctor, and how often. Health system executives should want to see that data as much as I do. HRSA officials should want to see it too, to update their models.

I often say that AI and other technology are "making healthcare better, and smaller." This study is going to give us a clearer idea of how much smaller.

Sources

I give keynotes on these ideas. Explore keynotes →

← All thoughts