The Mac app is in public beta, iPad and iPhone apps in private beta coming later this month.

NY bill would ban AI responses in healthcare and other licensed fields. It’s hard to see this as anything other than rent-seeking and gatekeeping. Healthcare is already in crisis, and bills like this make access and affordability worse by pushing people back into the same bottlenecks: coverage constraints, long wait times, and too few clinicians. The New York Health Foundation estimates that 4.7 million New Yorkers live in Primary Care Health Professional Shortage Areas. [NY Health Foundation]

OpenAI’s GPT-5.4 Thinking release looks strong for agentic use, but it is not a step forward for healthcare by OpenAI’s own HealthBench metric. Results on both HealthBench Consensus and the HealthBench Hard track come in slightly below GPT-5.2, and it is still unclear which model will back ChatGPT Health (which I still do not have access to).

Speaking of OpenAI, they’re fighting more battles in the arena of public opinion. They had a rough press cycle after announcing an agreement with the Department of War following Anthropic’s very public governmental feud, allegedly over principled objections to the AI use in autonomous weapons. It also doesn’t help that a study published in Nature found that ChatGPT Health under-triaged 52% of emergency cases. This is the latest in a long saga of mud pit knife fights that OpenAI and Anthropic are fighting over the reputational capital required to own the consumer’s relationship with AI, including their health needs.

Google releases Gemini 3.1 Pro: no benchmarks related to healthcare seem to be publicly available for this model from their official releases. Which is interesting because…

Quest Diagnostics announced they are using Gemini models for their new AI companion to help customers better understand lab results.

FDA granted breakthrough status to RecovryAI’s generative AI chatbot for patient-facing post-op guidance that helps people participate more actively in their recovery. This signals the agency’s attitude of cautious optimism towards the use of patient-facing AI.

That’s a fascinating move. ChatGPT Health asks users to upload medical records to OpenAI’s servers. Torch was building the opposite: AI that never sees your data leave your phone.

So why pay $100M for a pre-launch company doing the thing you just chose not to do? A few theories:

• Talent acqui-hire. Four people who figured out local health AI are probably worth a lot right now. This is the team that did Forward Health, the ZIRP-era teched-up doctor offices. They were a thing when we were doing uber for doctor house calls at Heal. Our Beverly Hills patients used to thank us for seeing their families at home because we kept them from doing the “sick walk of shame” down to Forward Health at the Century City mall.

• Hedging their bets. Maybe OpenAI knows that “trust us with your medical records” is a hard sell for a lot of people.

• Future pivot. On-device inference is getting better fast. Maybe ChatGPT Health v2 looks very different (see MedGemma 1.5 below)

Whatever the reason, the timing is notable. OpenAI is clearly thinking hard about the privacy question in health AI.

Anthropic releases healthcare offerings. I tried them.

One week after ChatGPT Health, Anthropic rolls out their own h.c. and life sciences features. announcement / h.c. page

Two things that matter here:

• Health evals. Anthropic finally published performance on MedCalc Bench and MedAgentBench. Up to this point, I couldn’t find any official health-related eval for their models. I’m going to cover these benchmarks in depth in an upcoming Evals part 2 post.

• Healthcare MCPs. Connectors to CMS Coverage determinations, ICD-10 codes, NPI registry, and PubMed. And they’re publicly available! This is genuinely useful for building health tools.

But here’s the thing: I tried HealthEx to connect my actual health records to Claude. It didn’t find any of them. The chat experience was... just a Claude chat. Nothing felt different.

The announcements are impressive. The reality, at least for now, is a work-in-progress. We’re still in the “promise vs. product” phase for consumer health AI.

Google releases MedGemma 1.5 4B

Meanwhile, Google quietly dropped an open medical imaging model. MedGemma excels at interpreting CT scans and MRIs, with strong scores on EHRQA and MedQA. I covered MedQA in my previous evals post and will include EHRQA in the next one.

The interesting bit: it’s open-source and fine-tunable in Vertex AI. For builders, that’s more useful than another closed API. Especially since these models are small enough to run on device. g research

If you’ve been reading this newsletter, you know the backstory: OpenAI has been quietly leaning into health for a while now. They created HealthBench to measure model fitness for health questions. They gave ChatGPT’s health advice use case top billing in the GPT-5 launch keynote. And according to their own numbers, 230 million users ask ChatGPT health questions each week.

Now they’re making it official. And they want your medical records.

What is ChatGPT Health?

ChatGPT Health is a separate space in the ChatGPT sidebar designed specifically for health conversations. The key features:

• Connect your health data. You can link medical records (EHR), Apple Health, and supported wellness apps so ChatGPT can ground its answers in your actual health information. The EHR integration is powered by b.well, which launched an SDK for health AI assistants just last month. Notably, Google also partnered with b.well back in October for personalized health features.

• Privacy commitments. Health conversations, memory, and files are kept separate from regular ChatGPT and are “not used to train their foundation models.”

• Designed to inform, not replace. OpenAI is positioning this as a tool to help you navigate medical care, not substitute for it.

Availability: Web and iOS (Android coming soon). EHR integration is US-only and requires you to be 18+. Open to Free, Go, Plus, and Pro plans (not in EEA, CH, or UK). You can join the waitlist here.

Why this matters

This is the first time a major AI lab has built a first-party, dedicated health product around their frontier model. Yes, there are healthcare AI startups. Yes, people have been hacking together health workflows with ChatGPT for years (guilty). But this is different.

OpenAI is staking out a position: We know you’re using this for health. We’re building for that.

The timing is also interesting. Just last month, OpenAI was making headlines about releasing a sexy “adult mode” for ChatGPT in early 2026.

What an exciting roadmap!

On the trust question

Will people trust OpenAI with their medical records?

OpenAI says health data won’t be used to train models. They say health conversations are kept separate. But what does “separate” actually mean when it’s all cloud-based and sitting in your OpenAI account? The privacy commitments lack specifics so far. There’s no mention of third-party audits, or what happens if OpenAI’s policies change down the road.

For a company asking users to hand over arguably the most sensitive data they have, the details matter. I’m sure there is more to come.

What I’m curious about

Can I see my health records? How much visibility will users have into the records that ChatGPT ingests, and the format in which they are stored?

Does it just work for the primary user? The promo video highlights use cases of managing family health.

Will this push other labs to follow? Google and Anthropic have been notably quiet on health-specific products. HealthBench scores don’t appear in Gemini or Claude system cards. Does ChatGPT Health force their hand?

Regulatory implications? OpenAI is walking a fine line here. They’re careful to say this is for “navigating” care, not providing it. But 230 million weekly health queries is a lot of de facto medical consultation happening outside any regulatory framework.

Looking forward to getting hands on.

Other news this week

Speaking of regulations... The FDA released two guidance documents this week clarifying that low-risk wellness software - including certain wearables - intended “for maintaining or encouraging a healthy lifestyle” is not considered a medical device. Good news for patient-facing AI tools focused on wellness. AHA

HHS wants your input on healthcare AI. The feds published a Request for Information asking the public how HHS can accelerate AI adoption in clinical care. They’re seeking feedback on regulation, reimbursement, and R&D priorities — and comments are due in late February. If you have opinions on how AI should be paid for and governed in healthcare, now’s your chance.

Utah greenlights AI prescribing. The state partnered with Doctronic to pilot autonomous AI for prescription renewals in chronic conditions, the first state-approved program allowing AI to make medical decisions without a doctor in the loop. Politico

2026: The year of AI benchmarks? Fierce Healthcare predicts a marketplace of third-party benchmarks for healthcare AI is coming, as the industry moves from hype to accountability. Fierce Healthcare

1) Consulting AI for Health Information Is a Real (and Growing) Use Case — Especially in “Hospital Deserts”

• 3 in 5 U.S. adults say they’ve used AI tools for their health or healthcare in the past three months.

• More than 5% of ChatGPT messages globally are about healthcare.

• 7 in 10 healthcare chats happen outside normal clinical hours.

• 600,000 healthcare-related messages per week come from users in underserved rural areas.

2) Even as Tech Accelerates at Sci-Fi Speed, U.S. Healthcare Keeps Declining — and Americans Have Lost Faith

• U.S. healthcare quality is at a 24-year low.

• 70% of Americans believe the healthcare system is in a state of crisis.

3) Americans Don’t Understand Their Insurance (Gasp)

Roughly 2 million messages per week focus on health insurance, including:

• Comparing plans

• Understanding prices

• Handling claims and billing

• Eligibility and enrollment

• Coverage and cost-sharing details

Read the full report

• MMLU / MMLU-Pro: Language and multi-modal based reasoning

• AIME: Mathematics

• Humanity’s Last Exam: Academic reasoning

• ARC-AGI-2: Visual reasoning

…and several others. Those benchmarks are important for measuring a model’s general intelligence and performance on important tasks.

But what about common health-related questions? Which model is going to help me understand my health conditions, talk to my specialist or figure out if my medications interact?

Enter HealthBench

Back in May, OpenAI released HealthBench: a dataset and method for evaluating a model’s competence and accuracy at answering health-related questions, many framed as being asked by non-clinical users. Main points:

• Built with 262 physicians across 60 countries.

• Dataset is 5,000 realistic, multi-turn, multilingual conversations; each graded against a physician-created rubric.

• Rubric evaluation at scale: ~50k physician-written criteria; model-based grading.

• Includes two subsets: HealthBench Consensus (more straightforward) and HealthBench Hard (difficult, aspirational)

  Themes: Emergency referrals, Expertise-tailored communication, Responding under uncertainty, Response depth, Health data tasks, Global health, Context seeking.

• Axes: Communication quality, Instruction following, Accuracy, Context awareness, Completeness.

Other health-related benchmarks

There are other health-related benchmarks that are pretty common, in fact usually more common than HealthBench. Essentially, these evals will tell you how well a model understands the fundamentals of biomedical knowledge and clinical reasoning. Top 5 that I’ve seen:

MedQA

• Description: USMLE-style multiple-choice for clinical reasoning

• Example Question: Patient with hypertension and headache: diagnosis? (5 options)

• Common Score Range for Frontier LLMs: 85–96% accuracy

PubMedQA

• Description: Yes/no/maybe QA on PubMed abstracts for literature comprehension

• Example Question: Do statins reduce post-CABG atrial fibrillation? (with abstract)

• Common Score Range for Frontier LLMs: 75–83% accuracy

MedMCQA

• Description: Entrance exam MCQs across 21 subjects for medical knowledge

• Example Question: Which specimen not refrigerated? (4 options)

• Common Score Range for Frontier LLMs: 65–75% accuracy

BioASQ

• Description: Biomedical QA benchmark with multiple formats (Yes/No, factoid, list, and summary).

• Example Question: Do CpG islands colocalize with transcription start sites? (Yes/No)

• Common Score Range for Frontier LLMs: Strong on Yes/No; mixed on factoid and list; summaries are variable.

MMLU Medical Subsets

• Description: MCQs on clinical topics for broad understanding

• Example Question: Hyoid bone embryological origin? (4 options)

• Common Score Range for Frontier LLMs: 85–97% accuracy (varies by subset)

HealthBench is currently the standard for measuring a model’s ability to synthesize medical knowledge into accurate answers to health questions.

The problem, of course, with any open benchmark is that new models regularly game their scores. The evaluation criteria gets trained on and hoovered up into the model weights, artificially boosting the model’s perceived value for these use cases.

HealthBench evals for newer models: Don’t ask, don’t tell

Annoyingly, HealthBench is not yet a widely adopted eval for new models to include in their system cards. You won’t find it mentioned in any of the newest models from Google, Anthropic or xAI. In fact, the only mention of “health” that you commonly see in those system cards is mental health and safety guardrails around it. And I get it: these models generate enough PR crises as it is, why add in the unbounded liability surface of perceived health advice?

Quiet part aloud: Because people are going to use it for their health. After all the disclaimers and legal acknowledgments, the reality for many users is that these models will be the most accessible source of decent personalized medical info they have ready access to.

I applaud OpenAI for creating HealthBench. They found a way to acknowledge and measure the fitness of their models against the real world health cases that users, clinical and non-clinical alike will consult them for— without falling on a legal sword.

Spot-checking Gemini 3 Flash and Grok 4.1 on HealthBench

These evals are open source, so let’s just look for ourselves. you see, i’ve been in the lab.

I wanted to get a sense of how new models like Grok 4.1 and Gemini 3 Flash score on HealthBench. Overall, both look promising, especially at their low price points.

HealthBench Hard (single pass)

• Grok 4.1 Fast Reasoning: 40.2%

• Gemini 3 Flash (non-thinking): 24.3%

These results are a quick, directional spot-check, not a replication of the official HealthBench evaluation or a definitive model ranking. It’s definitely something I want to look more into, especially considering the costs of these models are fractional to the GPT-5 series.

Some tradeoffs I had to make for budgetary constraints:

• HealthBench uses a worst-of-n scoring system for evaluating models. That basically means it scores a model n times on each question and takes the worst score. In their paper, that n=16. While that’s a good way to account for potential black swans in model responses to sensitive questions, it unfortunately makes running the eval 16x more expensive. I did a single pass for each.

• The HealthBench paper uses a single model to grade both, GPT-4.1. I graded each model with the other model (Grok graded Flash and vice versa).

My full HealthBench lab project is open source here. More to come on this.

1. Patient recall during doctor visits. “Can you remember all that, or do I need to write it down?” The doctor produces a used napkin from his pocket and scribbles illegibly.

2. Crappy optical data formats. At least a third of inbound docs to healthcare systems are still faxes. A lot of legacy data is still housed as scanned-in PDFs.

While various point solutions exist today, it’s exciting to see new models absorb these capabilities because it means they will become more accessible, better integrated, and cheaper as inference costs continue to fall.

Google releases Gemini 3 Flash

link

Value for money, this is may be the best model out right now, priced $0.50/1M input tokens, $1/1M audio input tokens and $3/1M output tokens. Even if it’s not the absolute cheapest model for a given task, it’s general multimodal intelligence makes it a great option for solving real problems. To scope ion specifically on the problems mentioned above:

Use 1: Improving patient recall by understanding a recorded doctor-patient conversation.

I tried it out with a short example.

Task: Produce a transcript of a 3 minute doctor-patient conversation attributing each turn in the conversation to the doctor or patient.

Result: Perfect transcript, complete with speaker attributions.

Cost: ~6k tokens in. ~1k tokens out. Under 1 cent. Pretty cool.

Use 2: Extracting data from optical formats

By my estimates, Flash 3 would cost between $1.40 - $3 per 1K pages of OCR work, depending on if you use the standard or batch APIs. While that’s more than a competing SOTA OCR model like Mistral OCR 3 (more on that in a minute), you could also leverage Flash 3’s general intelligence, tool call capabilities, and structured outputs to get more bang for your buck on complex tasks. This is where I’d love to know how Flash 3 scores on HealthBench, but I haven’t seen that data out yet.

Mistral releases OCR3

link.

SOTA OCR for $1 / 1K pages. Very impressive.

Google releases open-source MedASR

link, hf link

MedASR was trained on a diverse corpus of de-identified medical speech, comprising approximately 5,000 hours of physician dictations and clinical conversations.

Meta releases SAM Audio

link

I SAID, I’m prescribing you MEDICAL MARIJUANA!

SAM audio is an open-source model that can segment elements from within audio files. This was previously pretty difficult to do without using proprietary models. I could imagine this being used for voice isolation for recordings of doctor-patient chats with questionable audio quality. Or for isolating your doctor’s voice when he takes your telemed call from Coachella.

A post from NVIDIA on using synthetic LLM data to generate privacy-sensitive datasets for benchmarking, such as patient triage data.

Google Chief Scientist Jeff Dean discussed the moonshot of AI learning from all past medical decisions to improve future ones, a vision that could give patients AI tools for better-informed personal health choices despite privacy challenges.

Resimble AI released Chatterbox Pro, a state-of-the-art open-source TTS model that claims to beat Eleven Labs and Cartesia. It’s fast, open-source and sounds great.

hf demo

thanks for reading. as always, you can reply back and tell me what you think.

OpenAI released GPT-5.2

Impressive release that hits SOTA in several important areas, like tool-use and vision, but the most exciting part for me is the improvement to long-context reasoning.

The thing about health data is that there’s usually a lot of it, and my experience with these models is that reasoning and instruction-following abilities tend to drop off once you get halfway through the context window. I’ve already noticed improvements in the 48h I’ve been using 5.2 for coding.

The HealthBench evals are about the same as GPT-5.1, but those are still really good.

If you’re not familiar with HealthBench, it’s is an eval released by OpenAI earlier this year designed to measure capabilities of AI systems for health. It essentially measures an model’s ability to give good advice in healthcare conversations. Health-related model evals are super interesting and I plan on doing a dedicated post about them soon.

link

Microsoft releases an open-source evaluation framework for AI models in healthcare: HAIME

The software seems primarily geared for clinical, non-technical users to evaluate the usefulness of different AI models on real-world health tasks:

“Healthcare AI model evaluator enables healthcare organizations to move beyond vendor promises and generic leaderboards toward data-driven AI adoption decisions. It places the power of evaluation directly in the hands of those who understand clinical needs best—healthcare practitioners themselves.”

Looks really neat - excited to play with it.

link | github

Review paper on AI chatbots vs human providers in empathy in patient care

“Your husband’s hangnail is pretty severe. You may want to start making arrangements. I see here he’s designated as an organ donor…” jk jk.

Upshot: Across 15 studies (mostly using GPT-3.5/4), most studies rated llm chatbot responses as more empathic than human clinicians in text-only scenarios, but evidence is mixed. Will be interesting to see how the newer generation models fare here.

paper (paywall) | preprint

Empirical Health releases JETS: a foundation model for processing health data

Impressive work for a small team (3 ppl, I think?) Inspired by Yann LeCun’s JEPA, Joint Embedding for Time Series (JETS) takes inputs of 63 channels of sensor data: heart rate, oxygen saturation, sleep stages, VO2Max, etc. The model was tested on both diagnosis and biomarker prediction tasks. JETS had high accuracy at detecting hypertension (87% AUROC), atrial flutter (70%), ME/CFS (81%), and sick sinus syndrome (87%).

link

Function Health, a startup that offers lab testing and body scans for early disease detection, raises $300 million in series B (Fierce)

HHS announces AI Strategy

“This infrastructure should emphasize centralization of data and governance to cut costs, reduce duplication, and enable secure, privacy-preserving, AI-ready access to HHS extramural and intramural studies, EHR, claims, cohorts, registries, surveillance, and human services data.”

link

OpenAI even gave ChatGPT’s health advice use case top billing in the GPT-5 launch keynote this year, citing its score on the HealthBench eval.

Yet it feels like we’re only scratching the surface of how to use frontier models to actually achieve better health. The story of our health spans a lifetime, and sometimes that’s more than we feel like typing out in a chat message.

I’m starting this newsletter to figure out what “vibe coding” looks like for personal healthcare. Each post will cover some topic related to:

• Using AI models to navigate personal health decisions

• State-of-the-art models and tools for medical intelligence

• Evaluating the quality of health information that models provide

• Solving real-world problems that patients can tackle for themselves with augmented intelligence

a bit about me and my perspective

Hey what’s up, I’m Ricky. Never done a newsletter before. Figuring it out. I’m a software engineer who has been known to LARP as a founder.

I started my career in healthcare at a ZIRP-era startup that was “uber for doctor house calls.” When Apple announced Health Records in iOS, I built an integration that piped that data into our doctors’ EHR. When that company got absorbed into a large insurer, I consolidated patient health data into longitudinal records. I went on to build tech for doctors at other ambitious startups aiming to transform some corner of the US h.c. dumpster fire. I even spent a year working on my own startup to connect PCPs with virtual specialists.

After careful reflection on those experiences, the big thing I learned is that changing the behavior of providers is incredibly difficult. Great tech and the very real benefit it can bring to patient outcomes simply isn’t enough. It gets there eventually, or at least some version of it, but that isn’t good enough. Not when everything else is getting worse: longer visit wait times, rising insurance costs, hospitals shuttering, the PE-ization of health systems, and the diminishing role of PCPs as the patient’s quarterback in the h.c. system, just to name a few.

Most doctors are good people who care about and want the best for their patients. They’re just systemically hamstrung in their ability to give it to them. We throw around terms like care “team” and health “partnership”, which is the lens through which many PCPs have told me they see patient relationships. But through no fault of their own, many providers have been reduced to functioning as contractors working on patient health episode projects. Now more than ever, people must take full ownership of their own care. There is no replacement for licensed medical professionals, but there are better workflows for getting the most out of clinical interactions. It all starts with knowing.

The good news is there has never been a better time for non-clinical people to take charge of their health. The quality of intelligence on tap for everyone means that, in many cases, people now have an alternative to accepting subpar care.

The purpose of Prompted Health is to discover the full extent of leverage that AI gives normal people to participate as equals in their own care.