License the Intelligence, Not the Device: David Blumenthal on Governing Generative AI in Healthcare

David Blumenthal spent 35 years practicing primary care before most physicians had electronic health records. He worked on Senator Edward Kennedy's health policy staff in the 1970s, advised three presidential campaigns on health policy, and under Obama ran the Office of the National Coordinator for Health Information Technology, overseeing the $25 billion federal push that digitized American medicine. He is now at the Harvard T.H. Chan School of Public Health, co-authoring research on AI governance in healthcare, including a paper published in JAMA Health Forum proposing a new framework for overseeing generative AI as a medical practitioner rather than a medical device.

He came on Inside MedTech Innovation to work through a question Shannon has been asking since 2018: if generative AI can function as a clinician, should it be regulated like one?

This episode covers Operation Warp Speed as a model for speed-without-recklessness, the "humanware" lesson from EHR adoption, why generative AI is closer to a form of intelligence than a tool, why clinical trials cannot scale to general-purpose AI physicians, and what the physician licensing model would actually need to look like applied to AI.

Who Is David Blumenthal? [04:38]

Blumenthal's path to health policy began with the Vietnam War draft. As a Harvard undergraduate studying government, he had completed only the minimum science requirements for medical school. When the draft lottery made his call-up certain, medical school offered a deferral, and Blumenthal saw a way to pursue public policy from the standpoint of a physician. He didn't know at the time that he would end up practicing primary care for 35 years.

He describes a specific moment during his residency when that shifted: examining a patient and thinking, "I can do this." The competence felt natural rather than effortful. That experience, of knowledge becoming automatic through repetition and supervised practice, turned out to be directly relevant to how he thinks about AI oversight decades later.

After practicing and researching at Massachusetts General and Harvard, he joined Kennedy's Senate staff during the saccharin controversy in the late 1970s, managing a politically volatile moment in FDA regulation. He describes sitting in high-stakes policy hearings noticeably calmer than everyone else around him. When people asked why, he said, "Nobody's gonna die." The clinical training had recalibrated his sense of what actually constitutes a crisis.

He later led the Commonwealth Fund, one of the oldest health policy foundations in the country, where his team produced research that shaped the Affordable Care Act, including the provision requiring employer coverage of dependents through age 26. He is also co-author of the recent book "Whiplash: From the Battle of Obamacare to the War on Science."

Operation Warp Speed: Speed and Safety as Partners [13:43]

Blumenthal describes Operation Warp Speed as possibly "the most important scientific accomplishment in the history of the human species, bar none." His reasoning: since single-celled organisms first appeared in the primordial oceans, life on earth has been subject to microbial attack, and our species was largely defenseless against it. COVID appeared in December 2019. Within a week its RNA sequence was deciphered. Within two weeks, Tony Fauci's lab had identified the vaccine target. Within 11 months, a 95% effective vaccine existed. That had never happened before.

The science required luck, specifically that mRNA technology had matured enough to be applied. But Blumenthal is careful to note that scientific breakthroughs frequently go unrealized, and Operation Warp Speed was about humans bringing what they had created to application in an extraordinarily compressed timeline. He credits President Trump and the Trump administration team with deserving "enormous credit" for the outcome, and notes that Alex Azar's background as a skilled pharmaceutical executive and his knowledge of the Manhattan Project model (which Operation Warp Speed was explicitly modeled on) were decisive.

The episode then gets into a point Shannon draws out about FDA's market function: pharmaceutical companies cannot maximize sales of drugs and vaccines without credible third-party certification, so FDA's rigor serves the industry's commercial interests as much as patients. Blumenthal illustrates this with what happened when pharma pressure threatened the vaccine trials: nine biopharmaceutical companies took out full-page newspaper ads in support of FDA, then joined FDA in extending the trial length to ensure a safety signal would be detected. The trials weren't completed until after the November 2020 election, which President Trump publicly interpreted as a political act. Blumenthal connects that experience directly to what he describes as the "aftermath" visible in the current administration's behavior toward federal science agencies.

Meaningful Use: How the Federal Government Got Medicine to Go Digital [24:33]

The HITECH Act put more than $25 billion into getting doctors and hospitals to adopt electronic health records. Blumenthal's job as National Coordinator was not to understand the technology but to understand physicians. He describes it as figuring out what would cause people like himself to take the risk of bringing this technology into their offices and wards, knowing the federal subsidy would not fully cover the cost of implementation.

His approach was an escalator analogy: "We're gonna get you on an escalator, but all you have to do is take that little first step and then just stay on it, and it'll get more and more complicated." Meaningful Use was consciously modest in its first phase precisely because he needed doctors to believe it was doable. He also made the case that medicine was not going to stay in the paper world forever, and that the federal government was offering to pay for a transition they would eventually have to make anyway.

The "humanware" insight he took from the experience is the one Shannon considers most relevant to AI adoption: the technology was never the hard part. Getting humans to change was. He later reflected that if he could go back and tell his former self one thing, it would be that interoperability was not a technical problem. "It was a political and economic problem."

Generative AI Is Not a Medical Device [32:49]

Blumenthal and co-author Bakul Patel published a paper in JAMA Health Forum proposing a novel oversight framework for generative AI in clinical settings. The paper's starting premise is a distinction that Shannon argues is the most important conceptual move in the whole debate.

A traditional machine learning model, like a sepsis prediction algorithm, is trained on a curated dataset and produces deterministic outputs. It behaves like a tool or a device: predictable, bounded, evaluable against a specific set of inputs and outputs. Generative AI does not. Large language models are trained on vast, uncurated corpora, produce probabilistic responses, and generate different outputs to the same prompt. They change over time as they retrain. They cannot be fully explained even by their developers.

Blumenthal's framing: "the one we typically interact with is more like a form of intelligence than a medical device. Now, we don't understand exactly how it comes up with every answer, but you don't understand exactly how I come up with every answer I give you."

That last clause is the key move. Blumenthal argues that human intelligence has always been regulated through an indirect mechanism: training, credentialing, and oversight of practice, not direct inspection of the cognitive process producing each clinical decision. If generative AI is more like intelligence than a device, that's the model that applies.

Licensing AI Like a Clinician [36:08]

Physician training follows a specific arc: undergraduate prerequisites, medical school didactics, clinical rotations under close supervision, then residency where real autonomous responsibility escalates rapidly. Blumenthal describes that escalation as having an "imprinting" quality: "The sociology, the psychology of suddenly having real responsibility for another human and their welfare is an awesome change in your status as a professional."

The competence that emerges from this process is not exam performance. It is pattern recognition built through volume. You have to see enough chest pain cases, enough stroke presentations, enough abdominal pain workups, until the common ones become nearly automatic and you have enough reserve to handle the atypical ones. "An exam, passing exam is not the same thing. The licensing process requires and assumes a training process prior to the granting of a license."

His proposal is that a trained clinician could work with an AI model in practice and tune it, with the help of developers, to instill that level of competence. Shannon asks whether a new version of meaningful use incentives would be needed to get doctors to participate. His answer: it would need to be a compensation program, and it would require a new kind of practitioner, someone with clinical expertise and enough technical literacy to work with a computer scientist partner to identify and fill gaps in the AI's performance, the way supervising physicians filled gaps in his own training.

He is candid that this framework exists as a proposal, not a proven path. The standards around error rates, the accreditation processes, the testing regimes: none of them exist yet. What exists is the analogy, and Blumenthal believes it points in the right direction.

Why Clinical Trials Don't Scale to General AI [01:01:00]

Shannon arrives at what she calls the "aha moment" of the conversation. The FDA approval model is indication-by-indication. Thalidomide is approved for multiple myeloma and contraindicated in pregnancy. GLP-1s are approved for diabetes management, then separately for cardiovascular prevention, clinical trial by clinical trial. That model works for tools with defined, bounded applications.

A generative AI functioning as a primary care physician cannot be bounded that way. When a patient walks in, they may present with any of hundreds of possible problems. The breadth of clinical situations that primary care must handle is "far greater than present to any other kind of physician."

Blumenthal draws the distinction sharply: you could absolutely run a clinical trial on an agent designed specifically to control blood pressure in a defined patient population, or to manage diabetes, or to control rheumatoid arthritis. Those are achievable. But an agent designed to substitute for a generalist clinician would need to be competent at too many indications to individually trial. "To put it another way, the traditional way of regulating doesn't apply."

This is why the physician licensing model becomes relevant rather than optional. Not because clinical trials are impossible in healthcare AI, but because they are insufficient for the category of AI that most closely resembles what we actually need: a general-purpose clinical intelligence.

The Humanware Problem, Again [01:06:50]

Blumenthal's final major argument echoes what he learned from EHR adoption. The technology is not the constraint. What constrains AI adoption in healthcare is the economics of the humans who decide what to deploy.

"This is back to humanware," he says. A health system with a thousand possible AI applications will not start with population health management. It will start with whatever makes its clinicians' lives easier and its billing more efficient. In a fee-for-service environment, that means prior authorization workarounds and coding optimization. In an organization paid to keep populations healthy, like Kaiser or an accountable care organization, the calculus looks different: blood pressure control, early diabetes management, prevention of expensive downstream hospitalizations.

"If you're paid to make people healthy rather than to increase your revenue, then you might pay attention to getting blood pressure controlled or diabetes controlled."

The same AI tool deployed in these two environments will be used differently. The technology does not suspend the laws of economics. CMS Access, the new Medicare model creating performance-based payments for chronic disease management, is Shannon's read on the federal government starting to shift those incentives.

What This Means for MedTech

Shannon's thesis going into this episode was that the governance gap around generative AI in healthcare is real and closing the wrong way. Blumenthal's argument is that the right framework already exists in the structure of physician oversight. The gap is that nobody has built the equivalent path for AI: the supervised training, the clinical tuning with developer partners, the error rate standards, the accreditation process.

Blumenthal's single wish for the future: remove barriers to interoperability. Everything downstream, AI performance, training data quality, outcome measurement, depends on data that can actually be read across systems. That problem is not technical. It is political and economic.

His one piece of advice for every regulator and policymaker: "it's the humans that matter in the use of technology."

The regulatory framework for AI in healthcare will not be built from scratch. It will be borrowed from the best analog we have, the centuries-long experiment in licensing human intelligence to provide care. The open question is whether anyone moves quickly enough to build it before the technology outpaces it.

Listen to the full episode: https://creators.spotify.com/pod/profile/shannon-lantzy

Inside MedTech Innovation is hosted by Shannon Lantzy. This post was created with AI assistance from the full episode transcript.