Tag Archives: artificial intelligence

Answering Patient Questions: AI Does Better Than Doctors

Posted on by

Eric Topol (4/28/23): When Patient Questions Are Answered With Higher Quality and Empathy by ChatGPT than Physicians

RE: JW Ayers et al. JAMA Intern Med 2023; doi:10.1001/jamainternmed.2023.1838 Open Access! Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum

Thanks to Jeff Lewis for sharing Eric Topol’s article. Here are some excerpts:

In JAMA Internal Medicine, a new report compared the quality and empathy of responses to patient questions for doctors vs ChatGPT, a generative AI model that has already been superseded by GPT4…

Researchers at UCSD used the Reddit social media platform (Reddit’s/AskDocs) to randomly selected 195 patient questions that had been answered by verified, volunteer physicians and also posed them to ChatGPT in an identical fashion. The answers were reviewed by a panel of 3 health care professionals blinded to whether the response was from a doctor or the chatbot…

The results were pretty striking, as shown in the graph below. For quality, the evaluators preferred the ChatGPT response 79% of the time; the quality score of good and very good was 79% for the chatbot vs 22% for physicians, a near 4-fold difference. That gap was even greater for empathy rating of the responses, nearly 10-fold higher favoring the chatbot, for proportion of empathetic or very empathic responses (45.1 vs 4.6%). As would be expected, the length of responses by doctors was significantly reduced (average of 52 vs 211 words, respectively). Of course, doctors have less time available and machines can generate such long-form output in seconds…

Several examples were presented, and I show one below that is representative. This actually happened to my mother-in-law several years ago and it was quite a frightening incident with extensive corneal injury.There were definite limitations of the study, since the questions were derived from a social medial platform, not from physicians who were actually involved in the care of the patient asking questions. And, of course, this was not comparing the additivity of a chatbot plus a physician..

My take: This study indicates that physicians/patients may benefit from leveraging chatbots to improve communication.  Artificial intelligence assistants can aid in drafting responses to patient questions. Though physicians, at this time, are more capable of providing patient-specific information. For more general questions, chatbots appear to do quite well. My personal office-related experience with ChatGPT includes drafting an appeal letter, providing educational material on encopresis, helping with a condolence letter, and researching a rare condition. Just two days ago (at the time of drafting this post), I asked ChatGPT to explain ulcerative colitis and treatments, including mesalamine and steroids, at an 8th grade reading level in Spanish. I found this was a good way to start a conversation with a family.

This study was covered in many news outlets including WSJ, CNN, Yahoo, Daily Mall…

Related blog posts:

Have you tried out ChatGPT?

Posted on by

There has been a lot written about ChatGPT (see some NY Times links below). Earlier this week, one of my sons placed the following prompt:

“Write me a funny poem about a doctor from Atlanta who deals with diarrhea all day long”

This is the ChatGPT response:

There once was a doctor from Atlanta
Whose patients all suffered from diarrhea
They came in by the dozens
Leaving behind a stench that was quite humbling
But this doctor was quite unflappable

He had a cure for every case
And a smile on his face
As he worked tirelessly
To help each patient feel less queasy

His office was filled with the sounds of relief
As patients left feeling much less grief
They praised his skill and his bedside manner
And promised to spread the word about this Atlanta diarrhea-banner

So if you’re in Atlanta and feeling quite ill
And your stomach is acting quite shrill
Don’t hesitate, don’t be shy
Just make your way to this doctor’s side
And say goodbye to that diarrhea for good!

NY Times Articles:

“It can extrapolate and take ideas from different contexts and merge them together.” The new chatbots do this with what seems like complete confidence. But they do not always tell the truth. Sometimes, they even fail at simple arithmetic. They blend fact with fiction. And as they continue to improve, people could use them to generate and spread untruths.

An example cited in this article:

My take: ChatGPT (& similar programs) is a huge technological advance. In educational settings, it could help explain complicated topics. Of course, there is a concern that it could undermine education if students do not go through the process of doing their own work. In healthcare, it is possible that these programs could advance patient education and help in the diagnostic process.

Another example (on 12/16/22) -this took less than 30 seconds:

#NASPGHAN19 Impact of New Technologies on Patient Health

Posted on by

Along with Ragh Varier, I had the privilege of moderating a session on new technologies on patient health.  Below I’ve included a few slides and some notes; my notes may have errors of omission or transcription.

Chicago

 

Dr. Mehta’s lecture focused on wearable health technologies. Key points:

  • It is already in use in some areas (eg. continuous glucose monitoring for diabetes, ECG sensors).
  • She noted that wearable technology dates back to the 1600s with the abacus ring
  • Challenges: Accuracy, Actionability/outcome improvement, Reaching at-risk populations (not just the ‘worried well’ populations), regulation, sustainability (users may abandon quickly), and ethical/privacy concerns
  • Some families taking technology into their own hands, so to speak. #WeAreNotWaiting.  Example: artificial pancreas device system

Dr. Syed’s lecture focused on artificial intelligence in medical-decision making. Key points:

  • AI is already in use in areas like facial recognition
  • AI may be able to increase polyp detection rate in colonoscopy and improve histology reading
  • Her team has been working on using AI to help distinguishing enviromental enteropathy histology from other etiologies
  • Other potential uses: AI to help predict Crohn’s disease progression based on histology

Related study (not discussed in talk): Z Deng, H Shi et al. Gastroenterology 2019; 157: 1044-54. The authors collected more than 113 million images from 6970.  With a deep-learning algorithm, they found that video capsule endoscopy could have higher detection rates and improved reading time with a “CNN-based” reading system (CNN=convolutional neural network).  The mean reading time was reduced from 97 minutes with conventional reading to 6 minutes with CNN-based reading system.  The later had 99.88% sensitivity in per-patient analysis (vs. 74.57% with conventional reading).

The oral abstract presentation, by Sonja Swenson, detailed how machine learning was applied to try to improve transplantation selection/PELD scores.

  • The authors of this abstract (437) used data from 6273 patients with PELD scores and added additional variables to try to identify a more accurate model.
  • Link: All NASPGHAN 2019 Abstracts

Dr. Li, known by some as the ’emperor of emesis,’ presented a lecture on telemedicine. His full slides: Telemedicine NASPGHAN Updated 2019 (B Li)

Key points:

  • When surveyed, patients/families prefer telemedicine over conventional medicine.  Key reason is convenience
  • Lots of issues from health care provider viewpoint: reimbursement, licensing (improving), increased time
  • Many examples of telemedicine/telemonitoring that are ongoing

Disclaimer: NASPGHAN/gutsandgrowth assumes no responsibility for any use or operation of any method, product, instruction, concept or idea contained in the material herein or for any injury or damage to persons or property (whether products liability, negligence or otherwise) resulting from such use or operation. The discussion, views, and recommendations as to medical procedures, choice of drugs and drug dosages herein are the sole responsibility of the authors. Because of rapid advances in the medical sciences, the Society cautions that independent verification should be made of diagnosis and drug dosages. The reader is solely responsible for the conduct of any suggested test or procedure. Some of the slides reproduced in this syllabus contain animation in the power point version. This cannot be seen in the printed version.

 

What Doctors Could Do Together (Organized)

Posted on by

A recent commentary (recommended by one of my sons) by Eric Topol discusses how doctors could be organized to advance the practice of medicine, address the deterioration in doctor-patient relationships, and focus on the needs of patients, whereas current medical organizations are mainly focused on the business interests of medical practice.

An excerpt from Why Doctors Should Organize:

“It’s possible to imagine a new organization of doctors that has nothing to do with the business of medicine and everything to do with promoting the health of patients and adroitly confronting the transformational challenges that lie ahead for the medical profession. Such an organization wouldn’t be a trade guild protecting the interests of doctors. It would be a doctors’ organization devoted to patients. Its top priority might be restoring the human factor—the essence of medicine—which has slipped away, taking with it the patient-doctor relationship. It might oppose anti-vaxxers; challenge drug pricing and direct-to-consumer advertisements; denounce predatory, unregulated stem-cell clinics; promote awareness of the health hazards of climate change; and call out the false health claims for products advocated by celebrities such as Gwyneth Paltrow and Mehmet Oz. This partial list provides a sense of how many momentous matters have been left unaddressed by the medical profession as a whole…

Because of the unique technological moment at which we live, we may not see an opportunity like this one for generations to come. We have a chance to affect the future of medicine; to advocate for patient interests; to restore the time doctors need to think, to listen, to establish trust, and build bonds, one encounter at a time. For these purposes, and in these times, an organization of all doctors is necessary. Rebuilding our relationships with our patients: that is our lane.”

“Pistol Butt” Pine. Tree takes on this shape due to heavy snowfall leaning on tree at early stage. Crater Lake, Oregon.