The next frontier in health AI

Issue 78 l Eka’s Weekly Roundup (7 June 2024)

Jun 07, 2024

AI Agents in Healthcare🗞️

We were inspired by the Future of Health’s article on the Emerging AI Health Landscape published earlier this week, and wanted to comment on trends which we had seen on the ground since publishing our Health x AI piece a year ago (time flies!).

Wait, what’s an AI Agent?

Borrowing a lot from the FoH newsletter, AI agents help make autonomous or semi-autonomous decisions based on general instructions. There are three key pieces we would draw out here (and yes, we will be talking about regulation later on).

Large language model component for understanding: The agent uses some form of LLMs to understand the user’s intent and formulate an action plan based on the core objectives.
Tools and actioning the request: Once the LLM is able to understand the request, it then needs to action these into the digital world such as web search, document retrieval, code execution, and data base integration. It could also need to call on other AI models in order to check & reference itself, for example in complex diagnostic cases using chat-interface or other forms of mental health therapies.
Ability to remember, reflect, and self-critique: The agent needs to be able to recall what the user said in the past with high accuracy, both in the short and long term. It also needs to be able to be self-critical and understand when it has made mistakes & hallucinated.

Generalist versus specialist approaches

We haven’t seen many complex AI agents yet, but can speak to broader AI developments we have seen within health. Agents as described above are similar to Toqan or 11x in the generalist fields, but health (as far as we know!) hasn’t seen many of these ‘phase 3’ LLMs as of yet.

Our early sense is that AI agents within health will be very verticalised given the higher levels of regulation related to software-as-a-medical-device. Software must be regulated according to intended levels of specific use, so our policy frameworks and mental models may need to change as we enter this new wave of AI agents.

Bringing regulation back in

Here’s a quick refresher on how the software-as-a-medical-device regulation works (shout out to Hardian Health for writing some of the most thoughtful stuff here). Their article on classification is definitely worth looking at it if you haven’t already gone through it, from the original EU taxonomy here.

Essentially, the current framework doesn’t account for physical actioning of the diagnosis or medical device by a digital agent under our reading. For the geeks out there, this is the full definition put out by the European Union on SaMD (software as a medical device):

Software intended to provide information which is used to take decisions with diagnosis or therapeutic purposes is classified as class IIa, except if such decisions have an impact that may cause:
death or an irreversible deterioration of a person's state of health, in which case it is in class III; or
a serious deterioration of a person's state of health or a surgical intervention, in which case it is classified as class IIb.
Software intended to monitor physiological processes is classified as class IIa, except if it is intended for monitoring of vital physiological parameters, where the nature of variations of those parameters is such that it could result in immediate danger to the patient, in which case it is classified as class IIb.
All other software is classified as class I.

We’ll wait and see how the MHRA thinks about agents in the months to come.

Themes we are thinking about

With all of that said, existing Health x AI startups will have a huge distribution advantage when it comes to 1) co-creating and 2) selling these new AI agents. Some to note (again, thank you to Future of Health for this list) include: