It seems straightforward. But if it were, organ donation would be a unicorn in healthcare: a process unmarred by bureaucracy, institutional pressure, or misaligned incentives. So what actually happens between a catastrophic injury and an organ donation?

Frontier models make it possible for a non-clinical person to investigate a question like this with unusual depth. I used them to trace the process across medical guidelines, federal rules, government investigations, and hospital finance data, then checked the findings against the underlying sources. What emerged was a clearer picture of the uncertain period before donation becomes possible, along with a practical way for patients to decide in advance how they want that uncertainty handled.

A Couple of Notes Before We Get In To It

As this scenario unfolds, it may be tempting to assume that a licensed physician necessarily possesses certainty unavailable to a lay reader. Expertise matters, but expertise cannot make an uncertain prognosis certain. This essay tries to reflect the confidence the evidence actually supports, regardless of who is speaking.

Use of AI in composing this work: I used frontier models to research the organ-donation system, locate primary sources, test claims, and draft the sample advance-directive provision in Addendum A. The writing, argument and composition are my own, and the model-generated language is identified as such.

Act I: Trauma

Imagine a person arrives at the emergency room after a car accident. It looks bad. Neurological damage is suspected but far from confirmed. Death, survival with serious disability, and full recovery all exist somewhere on a probability curve no one yet understands.

Trauma staff work to stabilize the patient, order imaging and laboratory tests, and begin assessing neurological function. The bedside assessment may be performed by a nurse or physician. The Glasgow Coma Scale scores eye opening, verbal response, and motor response. Pupillary and other brainstem reflexes are also assessed.[1]

If those observations meet the hospital’s referral criteria, hospital staff notify the contracted Organ Procurement Organization, or OPO. This is an imminent-death referral. Upon receiving it, the OPO “rapidly goes onsite to determine the medical suitability of the potential donor.”[2]

Before we discuss the OPO and its role, two things are worth noting:

1. Specialized neurological care is uncommon this early. Even at the country’s best trauma centers, only about a third of patients with severe brain injuries entered dedicated neurological ICUs, and fewer than one in five units were led by a neurointensivist. At Level II centers, only about one in twenty ICUs had one; rural patients often enter through Level III centers, where specialist coverage is thinner still.[3]

2. The patient’s donor status may not yet be known and doesn’t actually matter much. The clinical trigger, not a donor designation, prompts the referral, so the OPO is contacted regardless.

A Bit About Organ Procurement Organizations

An OPO is a federally certified, nonprofit regional coordinator for deceased organ donation. As of July 2026, the United States had 54 OPOs: 47 independent organizations and 7 based within hospitals. They evaluate referred patients, verify or obtain donation authorization, coordinate organ recovery and preservation, and arrange allocation and transportation to transplant hospitals.[4]

CMS requires Medicare-participating hospitals, nearly all general hospitals, to maintain an agreement with an OPO and notify it of deaths and imminent deaths under that agreement. They have to do it. What stops a hospital from signing the agreement and ignoring it? Government surveyors. Federal or state surveyors can cite the hospital, the OPO, or both for noncompliance, and persistent hospital failures can endanger Medicare participation.[5]

Many details of a hospital’s OPO agreement, including referral criteria, notification deadlines, and payment terms, are neither nationally standardized nor publicly available.

Here is what I could find:

• A 2026 study collected referral cards and related trigger materials from nearly every OPO. Of those materials, around 70% used a Glasgow Coma Scale threshold and about half used absent brainstem reflexes. Notification deadlines ranged from immediate referral to four hours, and no two of the ten top OPOs agreed on the same brainstem-reflex criteria.[6]

• OPOs reimburse hospitals for authorized services needed to preserve organ viability, including intensive care, operating-room use, laboratory testing, imaging, and organ recovery. For reference, my calculation from 2024 CMS cost reports puts the kidney-allocated share of payments to donor hospitals at about $8,000 per locally procured usable kidney, or roughly $16,000 when two usable kidneys are recovered. I could not find a bulk discount.[7]

• Federal rules limit what hospitals can charge OPOs for donor services. The OPO must pay either the hospital’s estimated cost or a rate negotiated with the OPO, whichever is lower. It should not have to pay the hospital’s full listed price.[8]

A defined contractual payer may not sound like a big deal until it is compared with ordinary hospital collections, especially for uninsured patients who receive emergency services and may face the full burden of collection efforts afterward. American hospitals reported $42.67 billion in uncompensated care in 2020, meaning bad debt and financial assistance for care that produced no payment from either a patient or insurer.[9]

In summary, hospital participation with OPOs is mandatory. The operational details, including what counts as imminent death and what the OPO will pay, are murky and can vary, but the payment model for hospitals is far more predictable than patient collections.

Now back to the action.

Act II: Uncertainty

The OPO representative has arrived on scene. OPO personnel may review the medical record without family consent, speak with the clinical team, gather medical and social history, and assess which organs might be medically usable. Without donation authorization or a state-law exception, the OPO should not perform invasive testing. Its personnel may nevertheless be physically present and participating in the workflow while the patient remains under hospital care.[2]

Meanwhile, critical-care decision-making is shaped by stress, fatigue, and limited staffing. Residents, who frequently staff emergency-department and critical-care teams, regularly exceed work-hour limits. Surveys found that roughly half of residents report working beyond duty-hour limits or falsifying their hours. Nearly half of senior residents work more than 60 hours a week, with those longer weeks associated with more than twice the odds of a self-reported medical error.[10]

Even in the best conditions, neurological prognosis is difficult. Declaring brain death is not a quick impression based on a low Glasgow Coma Scale score. It requires a formal examination, an established catastrophic brain injury, the exclusion of confounding factors such as sedatives, intoxication, or paralysis, and apnea testing (can you breathe without a ventilator?) or some other approved test when the clinical examination cannot be completed. Many severely injured patients have not reached that grim certainty when the referral is made.[11]

In a hospital without neurocritical-care coverage, the first framing of prognosis likely comes from emergency, trauma, or general critical-care clinicians without access to a neurointensivist. The family is even further removed, left to interpret cautious and sometimes equivocal language from clinicians who are understandably unwilling to make promises about recovery or death.

The big risk is that an early neurological prognosis can become self-fulfilling when life support is withdrawn before recovery can be tested. Researchers compared these patients with similar patients given more time and estimated that roughly one in four might have survived, including about one in six who could have recovered independently. Nationally, that could mean thousands of avoidable deaths each year.[12]

Observational studies like this can’t fully rule out that early-withdrawal patients were simply sicker. Fine. Cut the estimate in half. Cut it by three quarters. You are still left with hundreds of people per year who died of a prediction rather than an injury, made by exhausted clinicians in hospitals without a neurointensivist in the building.

Act III: Decision

Family members in this immensely unenviable position may be visited by several clinicians, each bringing different information and a different role in the process. Nobody needs to “want the patient’s organs” for the system to shape the available choices, the timing of conversations, and which actions feel professionally routine.

The OPO’s presence complicates the picture, even though it’s not supposed to. CMS safeguards say the attending team and family must address prognosis and withdrawal of treatment before donation is discussed. When family authorization is required, the request must not be rushed, repeated, or coercive, and a refusal must be respected. There are, however, documented cases where these safeguards have failed.[2]

In 2025, the Health Resources and Services Administration reviewed 351 cases handled by the OPO serving Kentucky, Ohio, and West Virginia in which donation had been authorized but not completed. It found concerning features in 103 cases, including 73 patients with neurological signs incompatible with donation. The agency reported that at least 28 patients may not have been dead when procurement was initiated. It also identified poor neurological assessment, weak coordination with medical teams, questionable consent practices, and misclassified causes of death. It concluded, unsurprisingly, that patients at smaller and rural hospitals were the most vulnerable.[13]

CMS followed in March after reports that some OPOs rushed donation discussions and pressured grieving families. Surveyors were told to ask families about pressure and cite the hospital, OPO, or both, even after violations were fixed. The guidance instructed surveyors to determine whether sedatives may have suppressed neurological responsiveness and required procurement to stop immediately if a declared-dead patient showed signs of life.[14]

Is Organ Donation Still Worth Supporting?

Most likely. In 2024, 17,000 deceased donors made 41,000 transplants possible.[15]

Federal data still do not tell us how many of those cases passed through a genuinely uncertain prognosis before death or withdrawal; HRSA only recently began collecting standardized information about what happens between OPO referral and death.[16]

The most useful thing you can do costs nothing: don’t leave your family guessing in the gray zone. A standard DNR doesn’t cover this scenario. It addresses resuscitation, but not the decision to withdraw support amid an uncertain neurological prognosis. An advance directive is a legal document stating your treatment wishes. It can be carried out by a designated healthcare agent, a person you appoint to enforce them, and can cover exactly this. Sample language addressing the concerns in this essay appears in the addendum below.

The box at the DMV decides what happens after your death. An advance directive gives you a chance to define what happens in the gray zone before it. AI can help you work through those choices and put them into words while you still can.

Addendum A: Advance Directive Sample Provision

Generated by Anthropic’s Claude Fable 5 High

The easiest way to act on this is to put this entire essay, including this addendum, into Claude (please use a good model like Opus 5 or Fable, not Sonnet or Haiku) and ask it to interview you about what you want in a full advance directive, including the following provision.

Sample provision: severe brain injury with uncertain prognosis

If I suffer a severe brain injury and my prognosis is uncertain, I direct the following:

1. Observation period. Life-sustaining treatment shall be continued for a minimum of 72 hours [or: 7 days] following the injury before any decision to withdraw it is made, unless death has been formally declared under accepted neurological criteria, including a complete clinical examination and apnea testing or an approved ancillary test.

2. Specialist consultation. Before any withdrawal decision, I request evaluation by a physician with specialty training in neurology, neurosurgery, or neurocritical care, using telemedicine if none is available at the treating hospital.

3. Confounder review. No neurological prognosis used to justify withdrawal should be made while sedatives, paralytics, intoxicants, or hypothermia could plausibly be suppressing my responsiveness.

4. Separation of conversations. I direct that no organ donation discussion occur with my family or agent until after my care team and my agent have fully addressed prognosis and any withdrawal decision, consistent with federal guidance. My healthcare agent is authorized to ask any person in the room to identify their employer and role.

5. Donation intent. My willingness to be an organ donor is unchanged. [Strike or modify this sentence if you are not a registered donor.] This provision does not revoke my donor designation; it governs only the timing and sequence of decisions while my survival remains in question.

The agent’s power is consent. Once you’re incapacitated, the agent legally stands in your shoes: withdrawal of life support (short of a formal brain-death declaration) generally requires the consent of the patient’s decision-maker, and the agent is that decision-maker. So:

• Enforcing point 1 (observation period): The agent simply declines to consent to withdrawal before 72 hours. That’s it. No one has to be persuaded; withdrawal without decision-maker consent isn’t an option the hospital has.

• Enforcing point 2 (specialist consult): The agent can’t order a consult, but can condition consent on one: “I’m not authorizing any withdrawal decision until a neurologist or neurointensivist, by telemedicine if necessary, has evaluated him.” Hospitals can arrange tele-neurology; a documented request from the legal decision-maker, backed by a written directive, is very hard to ignore, and refusing it creates paper the hospital doesn’t want.

• Enforcing point 3 (confounder review): The agent asks a specific question and waits for a specific answer: “What sedatives is she on, when was the last dose, and are you confident the exam reflects her brain and not the drugs?” If the answer is vague, the agent declines consent until it isn’t. Point 3 essentially converts a directive clause into a script for that question.

• Escalation tools if stonewalled: Request the hospital ethics committee (every accredited hospital has one and any family member can invoke it), request a second opinion, request transfer to a higher-level center, and put objections in writing so they enter the record.

Addendum B: What the Public Records Should Show

Neither trust nor distrust is a substitute for the underlying records. The public summaries do not identify the hospitals, disclose the 351 case files, or show how responsibility was divided among hospital staff, OPO personnel, and recovery teams. The government has released its conclusions, but not enough evidence for the public to reconstruct these cases independently. The answer is to obtain the records and test the claims.

Freedom of Information Act and state public-records requests offer an opportunity for more clarity and independent review of these conclusions. Relevant documents include:

• The de-identified 351-case review database, including hospital identifiers, timelines, neurological findings, reported signs of life, staff objections, donation stoppages, outcomes, and corrective actions.

• Complete hospital-OPO agreements, including referral-trigger definitions, notification deadlines, family-approach rules, transfer-of-control language, onsite staffing or leased-space arrangements, and every reimbursement exhibit or financial addendum.

• CMS and state survey workpapers, complaints, deficiency reports, plans of correction, and correspondence involving organ-donation pressure, premature withdrawal discussions, death determination, or continued procurement after signs of life.

• OPO invoice registers and hospital accounts-receivable records showing amounts billed, allowed, paid, denied, disputed, and written off for donor services.

• Comparable hospital collection data for the same intensive-care, laboratory, imaging, anesthesia, and operating-room services when the payer was commercial insurance, Medicare, Medicaid, or an uninsured patient.

• Hospital-level referral totals, missed-referral findings, authorization rates, donation stoppages, and reviews of deaths that did not produce an OPO referral.

Addendum C: Calculation of Amount Reimbursed to Donor Hospitals for Usable Kidney Procurement

The $8,000 figure is a national weighted average from 49 independent OPOs’ 2024 CMS cost reports:

$160.8 million paid to donor hospitals for kidney procurement / 20,135 locally recovered usable kidneys = $7,988 per kidney.

This represents the kidney-allocated share of hospital payments, not a fixed reimbursement rate or the complete cost of a donor’s care.

Sources

1. StatPearls, “Glasgow Coma Scale”: https://www.ncbi.nlm.nih.gov/books/NBK513298/

2. CMS, QSO-26-05: https://www.cms.gov/files/document/qso-26-05-opo-original-release-date-2026-03-11-pdf.pdf

3. Journal of Neurotrauma: https://journals.sagepub.com/doi/full/10.1089/neu.2017.4997

4. Health Resources and Services Administration, OPTN membership: https://www.hrsa.gov/optn/about/membership

5. 42 C.F.R. § 482.45: https://www.law.cornell.edu/cfr/text/42/482.45; CMS, QSO-26-05: https://www.cms.gov/files/document/qso-26-05-opo-original-release-date-2026-03-11-pdf.pdf

6. Frontiers in Transplantation: https://www.frontiersin.org/journals/transplantation/articles/10.3389/frtra.2026.1701648/full

7. CMS OPO Cost Reports: https://www.cms.gov/data-research/statistics-trends-reports/cost-reports/organ-procurement-organization; CMS Form 216-94: https://www.cms.gov/regulations-and-guidance/legislation/paperworkreductionactof1995/pra-listing-items/cms-216-94

8. 42 C.F.R. § 413.418: https://www.ecfr.gov/current/title-42/chapter-IV/subchapter-B/part-413/subpart-L/section-413.418

9. American Hospital Association: https://www.aha.org/fact-sheets/2020-01-06-fact-sheet-uncompensated-hospital-care-cost

10. Journal of Graduate Medical Education: https://pmc.ncbi.nlm.nih.gov/articles/PMC3771164/; BMJ Medicine: https://bmjmedicine.bmj.com/content/2/1/e000320

11. American Academy of Neurology consensus guideline: https://www.neurology.org/doi/10.1212/WNL.0000000000207740

12. Resuscitation: https://pubmed.ncbi.nlm.nih.gov/26836944/

13. HHS and HRSA investigation summary: https://www.hhs.gov/press-room/hrsa-to-reform-organ-transplant-system.html

14. CMS press release: https://www.cms.gov/newsroom/press-releases/cms-strengthens-patient-protections-accountability-organ-donation-system; CMS, QSO-26-05: https://www.cms.gov/files/document/qso-26-05-opo-original-release-date-2026-03-11-pdf.pdf

15. Health Resources and Services Administration, 2024 transplant totals: https://www.hrsa.gov/optn/news-events/news/organ-transplants-exceeded-48000-2024-33-percent-increase-transplants-performed-2023

16. Health Resources and Services Administration, procurement-safety initiative: https://www.hrsa.gov/optn/policies-bylaws/policy-issues/strengthening-organ-donation-and-procurement-safety

The Mac app is in public beta, iPad and iPhone apps in private beta coming later this month.

NY bill would ban AI responses in healthcare and other licensed fields. It’s hard to see this as anything other than rent-seeking and gatekeeping. Healthcare is already in crisis, and bills like this make access and affordability worse by pushing people back into the same bottlenecks: coverage constraints, long wait times, and too few clinicians. The New York Health Foundation estimates that 4.7 million New Yorkers live in Primary Care Health Professional Shortage Areas. [NY Health Foundation]

OpenAI’s GPT-5.4 Thinking release looks strong for agentic use, but it is not a step forward for healthcare by OpenAI’s own HealthBench metric. Results on both HealthBench Consensus and the HealthBench Hard track come in slightly below GPT-5.2, and it is still unclear which model will back ChatGPT Health (which I still do not have access to).

Speaking of OpenAI, they’re fighting more battles in the arena of public opinion. They had a rough press cycle after announcing an agreement with the Department of War following Anthropic’s very public governmental feud, allegedly over principled objections to the AI use in autonomous weapons. It also doesn’t help that a study published in Nature found that ChatGPT Health under-triaged 52% of emergency cases. This is the latest in a long saga of mud pit knife fights that OpenAI and Anthropic are fighting over the reputational capital required to own the consumer’s relationship with AI, including their health needs.

Google releases Gemini 3.1 Pro: no benchmarks related to healthcare seem to be publicly available for this model from their official releases. Which is interesting because…

Quest Diagnostics announced they are using Gemini models for their new AI companion to help customers better understand lab results.

FDA granted breakthrough status to RecovryAI’s generative AI chatbot for patient-facing post-op guidance that helps people participate more actively in their recovery. This signals the agency’s attitude of cautious optimism towards the use of patient-facing AI.

That’s a fascinating move. ChatGPT Health asks users to upload medical records to OpenAI’s servers. Torch was building the opposite: AI that never sees your data leave your phone.

So why pay $100M for a pre-launch company doing the thing you just chose not to do? A few theories:

• Talent acqui-hire. Four people who figured out local health AI are probably worth a lot right now. This is the team that did Forward Health, the ZIRP-era teched-up doctor offices. They were a thing when we were doing uber for doctor house calls at Heal. Our Beverly Hills patients used to thank us for seeing their families at home because we kept them from doing the “sick walk of shame” down to Forward Health at the Century City mall.

• Future pivot. On-device inference is getting better fast. Maybe ChatGPT Health v2 looks very different (see MedGemma 1.5 below)

Whatever the reason, the timing is notable. OpenAI is clearly thinking hard about the privacy question in health AI.

Anthropic releases healthcare offerings. I tried them.

One week after ChatGPT Health, Anthropic rolls out their own h.c. and life sciences features. announcement / h.c. page

Two things that matter here:

• Health evals. Anthropic finally published performance on MedCalc Bench and MedAgentBench. Up to this point, I couldn’t find any official health-related eval for their models. I’m going to cover these benchmarks in depth in an upcoming Evals part 2 post.

• Healthcare MCPs. Connectors to CMS Coverage determinations, ICD-10 codes, NPI registry, and PubMed. And they’re publicly available! This is genuinely useful for building health tools.

But here’s the thing: I tried HealthEx to connect my actual health records to Claude. It didn’t find any of them. The chat experience was... just a Claude chat. Nothing felt different.

The announcements are impressive. The reality, at least for now, is a work-in-progress. We’re still in the “promise vs. product” phase for consumer health AI.

Google releases MedGemma 1.5 4B

Meanwhile, Google quietly dropped an open medical imaging model. MedGemma excels at interpreting CT scans and MRIs, with strong scores on EHRQA and MedQA. I covered MedQA in my previous evals post and will include EHRQA in the next one.

The interesting bit: it’s open-source and fine-tunable in Vertex AI. For builders, that’s more useful than another closed API. Especially since these models are small enough to run on device. g research

If you’ve been reading this newsletter, you know the backstory: OpenAI has been quietly leaning into health for a while now. They created HealthBench to measure model fitness for health questions. They gave ChatGPT’s health advice use case top billing in the GPT-5 launch keynote. And according to their own numbers, 230 million users ask ChatGPT health questions each week.

Now they’re making it official. And they want your medical records.

What is ChatGPT Health?

ChatGPT Health is a separate space in the ChatGPT sidebar designed specifically for health conversations. The key features:

• Connect your health data. You can link medical records (EHR), Apple Health, and supported wellness apps so ChatGPT can ground its answers in your actual health information. The EHR integration is powered by b.well, which launched an SDK for health AI assistants just last month. Notably, Google also partnered with b.well back in October for personalized health features.

• Privacy commitments. Health conversations, memory, and files are kept separate from regular ChatGPT and are “not used to train their foundation models.”

• Designed to inform, not replace. OpenAI is positioning this as a tool to help you navigate medical care, not substitute for it.

Availability: Web and iOS (Android coming soon). EHR integration is US-only and requires you to be 18+. Open to Free, Go, Plus, and Pro plans (not in EEA, CH, or UK). You can join the waitlist here.

Why this matters

This is the first time a major AI lab has built a first-party, dedicated health product around their frontier model. Yes, there are healthcare AI startups. Yes, people have been hacking together health workflows with ChatGPT for years (guilty). But this is different.

OpenAI is staking out a position: We know you’re using this for health. We’re building for that.

What I’m curious about

Can I see my health records? How much visibility will users have into the records that ChatGPT ingests, and the format in which they are stored?

Does it just work for the primary user? The promo video highlights use cases of managing family health.

Will this push other labs to follow? Google and Anthropic have been notably quiet on health-specific products. HealthBench scores don’t appear in Gemini or Claude system cards. Does ChatGPT Health force their hand?

Regulatory implications? OpenAI is walking a fine line here. They’re careful to say this is for “navigating” care, not providing it. But 230 million weekly health queries is a lot of de facto medical consultation happening outside any regulatory framework.

Looking forward to getting hands on.

Other news this week

Speaking of regulations... The FDA released two guidance documents this week clarifying that low-risk wellness software - including certain wearables - intended “for maintaining or encouraging a healthy lifestyle” is not considered a medical device. Good news for patient-facing AI tools focused on wellness. AHA

HHS wants your input on healthcare AI. The feds published a Request for Information asking the public how HHS can accelerate AI adoption in clinical care. They’re seeking feedback on regulation, reimbursement, and R&D priorities — and comments are due in late February. If you have opinions on how AI should be paid for and governed in healthcare, now’s your chance.

Utah greenlights AI prescribing. The state partnered with Doctronic to pilot autonomous AI for prescription renewals in chronic conditions, the first state-approved program allowing AI to make medical decisions without a doctor in the loop. Politico

2026: The year of AI benchmarks? Fierce Healthcare predicts a marketplace of third-party benchmarks for healthcare AI is coming, as the industry moves from hype to accountability. Fierce Healthcare

1) Consulting AI for Health Information Is a Real (and Growing) Use Case — Especially in “Hospital Deserts”

• 3 in 5 U.S. adults say they’ve used AI tools for their health or healthcare in the past three months.

• More than 5% of ChatGPT messages globally are about healthcare.

• 7 in 10 healthcare chats happen outside normal clinical hours.

• 600,000 healthcare-related messages per week come from users in underserved rural areas.

2) Even as Tech Accelerates at Sci-Fi Speed, U.S. Healthcare Keeps Declining — and Americans Have Lost Faith

• U.S. healthcare quality is at a 24-year low.

• 70% of Americans believe the healthcare system is in a state of crisis.

3) Americans Don’t Understand Their Insurance (Gasp)

Roughly 2 million messages per week focus on health insurance, including:

• Comparing plans

• Understanding prices

• Handling claims and billing

• Eligibility and enrollment

• Coverage and cost-sharing details

Read the full report

• MMLU / MMLU-Pro: Language and multi-modal based reasoning

• AIME: Mathematics

• Humanity’s Last Exam: Academic reasoning

• ARC-AGI-2: Visual reasoning

…and several others. Those benchmarks are important for measuring a model’s general intelligence and performance on important tasks.

But what about common health-related questions? Which model is going to help me understand my health conditions, talk to my specialist or figure out if my medications interact?

Enter HealthBench

Back in May, OpenAI released HealthBench: a dataset and method for evaluating a model’s competence and accuracy at answering health-related questions, many framed as being asked by non-clinical users. Main points:

• Built with 262 physicians across 60 countries.

• Dataset is 5,000 realistic, multi-turn, multilingual conversations; each graded against a physician-created rubric.

• Rubric evaluation at scale: ~50k physician-written criteria; model-based grading.

• Includes two subsets: HealthBench Consensus (more straightforward) and HealthBench Hard (difficult, aspirational)

  Themes: Emergency referrals, Expertise-tailored communication, Responding under uncertainty, Response depth, Health data tasks, Global health, Context seeking.

• Axes: Communication quality, Instruction following, Accuracy, Context awareness, Completeness.

Other health-related benchmarks

There are other health-related benchmarks that are pretty common, in fact usually more common than HealthBench. Essentially, these evals will tell you how well a model understands the fundamentals of biomedical knowledge and clinical reasoning. Top 5 that I’ve seen:

MedQA

• Description: USMLE-style multiple-choice for clinical reasoning

• Example Question: Patient with hypertension and headache: diagnosis? (5 options)

• Common Score Range for Frontier LLMs: 85–96% accuracy

PubMedQA

• Description: Yes/no/maybe QA on PubMed abstracts for literature comprehension

• Example Question: Do statins reduce post-CABG atrial fibrillation? (with abstract)

• Common Score Range for Frontier LLMs: 75–83% accuracy

MedMCQA

• Description: Entrance exam MCQs across 21 subjects for medical knowledge

• Example Question: Which specimen not refrigerated? (4 options)

• Common Score Range for Frontier LLMs: 65–75% accuracy

BioASQ

• Description: Biomedical QA benchmark with multiple formats (Yes/No, factoid, list, and summary).

• Example Question: Do CpG islands colocalize with transcription start sites? (Yes/No)

• Common Score Range for Frontier LLMs: Strong on Yes/No; mixed on factoid and list; summaries are variable.

MMLU Medical Subsets

• Description: MCQs on clinical topics for broad understanding

• Example Question: Hyoid bone embryological origin? (4 options)

• Common Score Range for Frontier LLMs: 85–97% accuracy (varies by subset)

HealthBench is currently the standard for measuring a model’s ability to synthesize medical knowledge into accurate answers to health questions.

The problem, of course, with any open benchmark is that new models regularly game their scores. The evaluation criteria gets trained on and hoovered up into the model weights, artificially boosting the model’s perceived value for these use cases.

HealthBench evals for newer models: Don’t ask, don’t tell

Annoyingly, HealthBench is not yet a widely adopted eval for new models to include in their system cards. You won’t find it mentioned in any of the newest models from Google, Anthropic or xAI. In fact, the only mention of “health” that you commonly see in those system cards is mental health and safety guardrails around it. And I get it: these models generate enough PR crises as it is, why add in the unbounded liability surface of perceived health advice?

Quiet part aloud: Because people are going to use it for their health. After all the disclaimers and legal acknowledgments, the reality for many users is that these models will be the most accessible source of decent personalized medical info they have ready access to.

I applaud OpenAI for creating HealthBench. They found a way to acknowledge and measure the fitness of their models against the real world health cases that users, clinical and non-clinical alike will consult them for— without falling on a legal sword.

Spot-checking Gemini 3 Flash and Grok 4.1 on HealthBench

These evals are open source, so let’s just look for ourselves. you see, i’ve been in the lab.

I wanted to get a sense of how new models like Grok 4.1 and Gemini 3 Flash score on HealthBench. Overall, both look promising, especially at their low price points.

HealthBench Hard (single pass)

• Grok 4.1 Fast Reasoning: 40.2%

• Gemini 3 Flash (non-thinking): 24.3%

These results are a quick, directional spot-check, not a replication of the official HealthBench evaluation or a definitive model ranking. It’s definitely something I want to look more into, especially considering the costs of these models are fractional to the GPT-5 series.

Some tradeoffs I had to make for budgetary constraints:

• HealthBench uses a worst-of-n scoring system for evaluating models. That basically means it scores a model n times on each question and takes the worst score. In their paper, that n=16. While that’s a good way to account for potential black swans in model responses to sensitive questions, it unfortunately makes running the eval 16x more expensive. I did a single pass for each.

• The HealthBench paper uses a single model to grade both, GPT-4.1. I graded each model with the other model (Grok graded Flash and vice versa).

My full HealthBench lab project is open source here. More to come on this.

1. Patient recall during doctor visits. “Can you remember all that, or do I need to write it down?” The doctor produces a used napkin from his pocket and scribbles illegibly.

2. Crappy optical data formats. At least a third of inbound docs to healthcare systems are still faxes. A lot of legacy data is still housed as scanned-in PDFs.

While various point solutions exist today, it’s exciting to see new models absorb these capabilities because it means they will become more accessible, better integrated, and cheaper as inference costs continue to fall.

Google releases Gemini 3 Flash

link

Value for money, this is may be the best model out right now, priced $0.50/1M input tokens, $1/1M audio input tokens and $3/1M output tokens. Even if it’s not the absolute cheapest model for a given task, it’s general multimodal intelligence makes it a great option for solving real problems. To scope ion specifically on the problems mentioned above:

Use 1: Improving patient recall by understanding a recorded doctor-patient conversation.

I tried it out with a short example.

Task: Produce a transcript of a 3 minute doctor-patient conversation attributing each turn in the conversation to the doctor or patient.

Result: Perfect transcript, complete with speaker attributions.

Cost: ~6k tokens in. ~1k tokens out. Under 1 cent. Pretty cool.

Use 2: Extracting data from optical formats

By my estimates, Flash 3 would cost between $1.40 - $3 per 1K pages of OCR work, depending on if you use the standard or batch APIs. While that’s more than a competing SOTA OCR model like Mistral OCR 3 (more on that in a minute), you could also leverage Flash 3’s general intelligence, tool call capabilities, and structured outputs to get more bang for your buck on complex tasks. This is where I’d love to know how Flash 3 scores on HealthBench, but I haven’t seen that data out yet.

Mistral releases OCR3

link.

SOTA OCR for $1 / 1K pages. Very impressive.

Google releases open-source MedASR

link, hf link

MedASR was trained on a diverse corpus of de-identified medical speech, comprising approximately 5,000 hours of physician dictations and clinical conversations.

Meta releases SAM Audio

link

I SAID, I’m prescribing you MEDICAL MARIJUANA!

SAM audio is an open-source model that can segment elements from within audio files. This was previously pretty difficult to do without using proprietary models. I could imagine this being used for voice isolation for recordings of doctor-patient chats with questionable audio quality. Or for isolating your doctor’s voice when he takes your telemed call from Coachella.

A post from NVIDIA on using synthetic LLM data to generate privacy-sensitive datasets for benchmarking, such as patient triage data.

Google Chief Scientist Jeff Dean discussed the moonshot of AI learning from all past medical decisions to improve future ones, a vision that could give patients AI tools for better-informed personal health choices despite privacy challenges.

Resimble AI released Chatterbox Pro, a state-of-the-art open-source TTS model that claims to beat Eleven Labs and Cartesia. It’s fast, open-source and sounds great.

hf demo

thanks for reading. as always, you can reply back and tell me what you think.

OpenAI released GPT-5.2

Impressive release that hits SOTA in several important areas, like tool-use and vision, but the most exciting part for me is the improvement to long-context reasoning.

The thing about health data is that there’s usually a lot of it, and my experience with these models is that reasoning and instruction-following abilities tend to drop off once you get halfway through the context window. I’ve already noticed improvements in the 48h I’ve been using 5.2 for coding.

The HealthBench evals are about the same as GPT-5.1, but those are still really good.

If you’re not familiar with HealthBench, it’s is an eval released by OpenAI earlier this year designed to measure capabilities of AI systems for health. It essentially measures an model’s ability to give good advice in healthcare conversations. Health-related model evals are super interesting and I plan on doing a dedicated post about them soon.

link

Microsoft releases an open-source evaluation framework for AI models in healthcare: HAIME

The software seems primarily geared for clinical, non-technical users to evaluate the usefulness of different AI models on real-world health tasks:

“Healthcare AI model evaluator enables healthcare organizations to move beyond vendor promises and generic leaderboards toward data-driven AI adoption decisions. It places the power of evaluation directly in the hands of those who understand clinical needs best—healthcare practitioners themselves.”

Looks really neat - excited to play with it.

link | github

Review paper on AI chatbots vs human providers in empathy in patient care

“Your husband’s hangnail is pretty severe. You may want to start making arrangements. I see here he’s designated as an organ donor…” jk jk.

Upshot: Across 15 studies (mostly using GPT-3.5/4), most studies rated llm chatbot responses as more empathic than human clinicians in text-only scenarios, but evidence is mixed. Will be interesting to see how the newer generation models fare here.

paper (paywall) | preprint

Empirical Health releases JETS: a foundation model for processing health data

Impressive work for a small team (3 ppl, I think?) Inspired by Yann LeCun’s JEPA, Joint Embedding for Time Series (JETS) takes inputs of 63 channels of sensor data: heart rate, oxygen saturation, sleep stages, VO2Max, etc. The model was tested on both diagnosis and biomarker prediction tasks. JETS had high accuracy at detecting hypertension (87% AUROC), atrial flutter (70%), ME/CFS (81%), and sick sinus syndrome (87%).

link

Function Health, a startup that offers lab testing and body scans for early disease detection, raises $300 million in series B (Fierce)

HHS announces AI Strategy

“This infrastructure should emphasize centralization of data and governance to cut costs, reduce duplication, and enable secure, privacy-preserving, AI-ready access to HHS extramural and intramural studies, EHR, claims, cohorts, registries, surveillance, and human services data.”

link

OpenAI even gave ChatGPT’s health advice use case top billing in the GPT-5 launch keynote this year, citing its score on the HealthBench eval.

Yet it feels like we’re only scratching the surface of how to use frontier models to actually achieve better health. The story of our health spans a lifetime, and sometimes that’s more than we feel like typing out in a chat message.

I’m starting this newsletter to figure out what “vibe coding” looks like for personal healthcare. Each post will cover some topic related to:

• Using AI models to navigate personal health decisions

• State-of-the-art models and tools for medical intelligence

• Evaluating the quality of health information that models provide

• Solving real-world problems that patients can tackle for themselves with augmented intelligence

a bit about me and my perspective

Hey what’s up, I’m Ricky. Never done a newsletter before. Figuring it out. I’m a software engineer who has been known to LARP as a founder.

I started my career in healthcare at a ZIRP-era startup that was “uber for doctor house calls.” When Apple announced Health Records in iOS, I built an integration that piped that data into our doctors’ EHR. When that company got absorbed into a large insurer, I consolidated patient health data into longitudinal records. I went on to build tech for doctors at other ambitious startups aiming to transform some corner of the US h.c. dumpster fire. I even spent a year working on my own startup to connect PCPs with virtual specialists.

After careful reflection on those experiences, the big thing I learned is that changing the behavior of providers is incredibly difficult. Great tech and the very real benefit it can bring to patient outcomes simply isn’t enough. It gets there eventually, or at least some version of it, but that isn’t good enough. Not when everything else is getting worse: longer visit wait times, rising insurance costs, hospitals shuttering, the PE-ization of health systems, and the diminishing role of PCPs as the patient’s quarterback in the h.c. system, just to name a few.

Most doctors are good people who care about and want the best for their patients. They’re just systemically hamstrung in their ability to give it to them. We throw around terms like care “team” and health “partnership”, which is the lens through which many PCPs have told me they see patient relationships. But through no fault of their own, many providers have been reduced to functioning as contractors working on patient health episode projects. Now more than ever, people must take full ownership of their own care. There is no replacement for licensed medical professionals, but there are better workflows for getting the most out of clinical interactions. It all starts with knowing.

The good news is there has never been a better time for non-clinical people to take charge of their health. The quality of intelligence on tap for everyone means that, in many cases, people now have an alternative to accepting subpar care.

The purpose of Prompted Health is to discover the full extent of leverage that AI gives normal people to participate as equals in their own care.