ChatGPT for Colonoscopy Questions Plus One

T-C Lee et al. Gastroenterol 2023; 165: 509-511. Open Access! ChatGPT Answers Common Patient Questions About Colonoscopy

In this study, ChatGPT answers to questions about colonoscopy were compared to publicly available webpages of 3 randomly selected hospitals from the top-20 list of the US News & World Report Best Hospitals for Gastroenterology and GI Surgery.

Methods: To objectively interpret the quality of ChatGPT-generated answers, 4 gastroenterologists (2 senior gastroenterologists and 2 fellows) rated 36 pairs of CQs and answers, randomly displayed, for the following quality indicators on a 7-point Likert scale: (1) ease of understanding, (2) scientific adequacy, and (3) satisfaction with the answer (Table 1) Raters were also requested to interpret whether the answers were AI generated or not.

Key findings:

  • ChatGPT answers were similar to non-AI answers, but had higher mean scores with regard to ease of understanding, scientific adequacy, and satisfaction.
  • The physician raters demonstrated only 48% accuracy in identifying ChatGPT generated answers

My take:  This is yet another study, this time focused on gastroenterology, that show how physicians/patients may benefit from leveraging chatbots to improve communication.

Related blog posts:

Also this:

The Bigger Picture -Mammography as an Example

This week, a commentary makes a strong case for eliminating mammography (N Engl J Med 2014; 370:1965-1967):  “Abolishing Mammography Screening Programs? A View from the Swiss Medical Board”

Here’s a link from the NEJM: nej.md/1hV8q0L

What is fascinating is how ingrained mammography has become in our medical culture and how most individuals believe that mammography is so beneficial.  Take a look at the figure in the link to get a better perspective.  While women think that mammography may save 80 lives out of a thousand screened, according to the commentary, the data suggest that 1 woman may be saved.  The main problem: “for every breast-cancer death prevented in U.S. women over a 10-year course of annual screening beginning at 50 years of age, 490 to 670 women are likely to have a false positive mammogram with repeat examination; 70 to 100, an unnecessary biopsy; and 3 to 14, an overdiagnosed breast cancer that would never have become clinically apparent.”

If a well-established screening measure like mammography is not so beneficial, what else could be on the chopping block?  As noted in a previous blog post (Do you know about the “Choosing Wisely gutsandgrowth), even the annual physical exam has been deemed a low-value service.

Another related blog post:

There is More to Life Than Death” | gutsandgrowth