ChatGPT fails self-assessment exam

[George K]

https://journals.lww.com/ajg/Abstract/9900/ChatGPT_Fails_the_Multiple_Choice_American_College.751.aspx

Introduction:
Chat Generative Pre-trained Transformer (ChatGPT) is a natural language processing model that generates human-like text.

Methods:
ChatGPT-3 and ChatGPT-4 were used to answer the 2022 and 2021 American College of Gastroenterology (ACG) Self-Assessment Tests. The exact questions were inputted in both versions of ChatGPT. A 70% or higher was required to pass the assessment.

Results:
Overall ChatGPT-3 scored a 65.1% on 455 included questions, and GPT-4 scored 62.4%.

Conclusion:
ChatGPT did not pass the ACG self-assessment test. We don’t recommend its use for medical education in gastroenterology in its current form.