Eka’s Weekly Roundup (19 May 2023)
Issue 32 l GPT vs. Clinicians
The University of California San Diego published a paper showing Chat GPT’s performance against doctors when giving medical advice. As reported widely (CNN, Guardian, Daily Mail), Chat GPT showed better scores on patient empathy compared to physician responses when replying to simple medical questions. While this is very early days for large language models (LLMs) in healthcare, this could become a tool for clinicians to reduce workloads, improve patient outcomes, and create efficiencies in a strained healthcare system.
The AI will see you now 🗞️
Comparing medical responses to social media posts…
In late April of this year, academics from UC San Diego compared the quality of responses from physicians and AI chatbots to medical questions. The study evaluated the effectiveness of ChatGPT in providing quality and empathetic responses to patient questions. The researchers used a public database of questions from Reddit's r/AskDocs forum. They randomly selected 195 question-answer exchanges where a verified physician had responded. Chatbot responses were generated by inputting the original question into a fresh session without prior questions asked. A team of licensed healthcare professionals evaluated the responses.
… with one example of how GPT can outperform physicians.
In one example highlighted in the study, a patient asked a question on a social media forum around the risk of losing vision after getting bleach in the eye. ChatGPT began its response by expressing apologies for causing concern. Then, it provided seven additional sentences offering advice and encouragement, reassuring the patient about the low likelihood of going blind in such a situation. On the other hand, a physician responded with a brief comment stating, "sounds like you will be fine," along with the phone number for Poison Control.
The results showed positive support for GPT…
The results showed that the evaluators preferred chatbot responses over physician responses in 78.6% of the evaluations. Chatbot responses were rated higher in quality and empathy compared to physician responses. The chatbot responses were longer in length and provided more detailed information. The prevalence of good or very good quality responses was 3.6 times higher for the chatbot compared to physicians, and the prevalence of empathetic or very empathetic responses was 9.8 times higher for the chatbot.
… but the road ahead will require further research and regulation.
The study suggests that AI chatbot assistants have the potential to provide quality and empathetic responses to patient questions in an online forum. As The Guardian reported, Prof James Davenport form the University of Bath commented: “The paper does not say that ChatGPT can replace doctors, but does, quite legitimately, call for further research into whether and how ChatGPT can assist physicians in response generation.” There are well documented risks of GPT hallucinating (Alkaissi and McFarlane, 2023) (Beutel Geerits and Kielstein, 2023) (Zapier).
Further research is needed to explore the use of chatbots in clinical settings, such as drafting responses for physicians to review. Randomized trials could assess whether using AI assistants can improve responses, reduce clinician burnout, and enhance patient outcomes.
Week in Impact Articles ✍🏽
Monday: Babylon Health goes private
Tuesday: Epic and Microsoft integrate AI into EHRs
Wednesday: Climate tech needs a lesson in circularity
Thursday: Green energy valuation multiples
Friday: Chatbots in the voice of god
3 Key Charts 📊
1. Electric vehicle sales in Southeast Asia are booming
2. The UK’s cow milk consumption is declining
3. Displaced oil with the EV boom
Deal Capture 💰
Deals in the impact space across the UK and Europe
Pharmacy enablement platform Fak raised E5m. Included Connect Ventures, Cocoa, and Seedcamp.
Climate tech company Mitiga Solutions raised E13m in a Series A. Led by Kibo Ventures.
Organ transplant company ScubaTx raised £1.5m. Led by Praetura.
Climate tech startup Pledge raised a £8m Series A. Included existing investors LowerCarbon.
Plant biotech company Resurrect Bio raised $2m. Included AgFunder.
Getting in Touch 👋.
If you’re looking for funding, you can get in touch here.
Don’t be shy, get in touch on LinkedIn or on our Website 🎉.
We are open to feedback: let us know what more you’d like to hear about 💪.
Thanks for reading Eka Ventures Newsletter! Subscribe for free to receive new posts and support my work.