Skip to main content
AAO-HNSF 2023 Annual Meeting & OTO Experience Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Pediatric Otolaryngology

Utility of ChatGPT Following Tonsillectomy: A Pilot Study

Monday, October 2, 2023
5:42 PM - 5:48 PM East Coast USA Time
Location: 207CD

    Presenting Author(s)

  • Dhruv S. Kothari, MD

    Otolaryngology Resident
    Cedars-Sinai Medical Center
    Los Angeles, California, United States

    • Co-Author(s)

  • Travis J. Clarke, MD

    Resident
    University of Tennessee Health Science Center
    Memphis, TN, United States

  • Andrew J. Maroda, MD

    Resident Physician
    University of Tennessee Health Science Center
    Memphis, Tennessee, United States

  • Wade G. McClain, DO

    Assistant Professor
    University of Tennessee Health Science Center
    Memphis, TN, United States

  • Anthony M. Sheyn, MD, FACS

    Division Chief, Pediatric Otolaryngology
    LeBonheur Children's Hospital
    Memphis, Tennessee, United States

  • RT

    Robert M. Tuliszewski, Jr., MD

    Assistant Professor
    University of Tennessee Health Science Center
    Memphis, TN, United States

    • Senior Author(s)

  • Sanjeet V. Rangarajan, MD, MEng

    Associate Professor of Otolaryngology - Head and Neck Surgery
    University of Tennessee Health Science Center
    Memphis, Tennessee, United States

Disclosure(s):
Dhruv S. Kothari, MD: No relevant relationships to disclose.
Andrew J. Maroda, MD: No relevant relationships to disclose.
Anthony M. Sheyn, MD, FACS: No relevant relationships to disclose.
Sanjeet V. Rangarajan, MD, MEng: No relevant relationships to disclose.
Introduction: This pilot study was designed to investigate the accuracy of ChatGPT, a large language model, to common questions parents have for their children following tonsillectomy.

Methods: 20 otolaryngology residents were invited to anonymously submit common questions they are asked by parents of pediatric patients following tonsillectomy. After identifying the most common questions via a consensus-based approach, we asked ChatGPT to generate responses to these queries. Satisfaction with the AI-generated answers were rated from 1 (Worst) to 5 (Best) by an expert panel of 3 pediatric otolaryngologists.

Results: Of the 35 questions initially submitted, 16 were included via a consensus-based process. The distribution of questions across the five most common domains and their mean satisfaction scores were: [6, (3.67)] Pain management, [4, (3.58)] Complications, [3, (4.33)] Diet, [2, (4.33)] Physical Activity, [1, (2.67)] Follow-up. The panel noted that while answers for diet, bleeding complications, and return to school were thorough, the pain management and follow-up recommendations were inaccurate, including a recommendation to prescribe codeine to children at large, and a suggested follow-up of 1 week following surgery, rather than 2-4 weeks more customary for our panel.

Conclusions: This pilot study demonstrates that though ChatGPT often provides accurate answers for common patient questions following tonsillectomy, it sometimes provides eloquently written inaccurate information. This can potentially lead to patients using AI-generated medical advice contrary to what a physician may advise. In addition, re-asking ChatGPT the same question can result in a different answer, which may further confuse patients. The current inaccuracy in pain management answers likely reflects regional and national practice variability. If trained appropriately, ChatGPT can be an excellent resource for otolaryngologists and patients to answer questions and reduce inefficiencies in the post-operative period. Future research should investigate if an otolaryngologist trained model can increase the accuracy of responses.