ChatGPT is a new artificial intelligence (AI) system designed to generate human-like writing by predicting upcoming word sequences. According to a new research, ChatGPT may pass the United States Medical Licensing Exam (USMLE) at or around the 60% passing mark if its replies were internaly consistent, insightful, and frequent.The USMLE is a highly standardized and... Read More
According to a new research, ChatGPT may pass the United States Medical Licensing Exam (USMLE) at or around the 60% passing mark if its replies were internaly consistent, insightful, and frequent.
The USMLE is a highly standardized and controlled series of three tests, comprising Steps 1, 2CK, and 3, necessary for medical licensing in the US, according to the study by Tiffany Kung and colleagues at AnsibleHealth in California.
The USMLE is a test that medical students and doctors-in-training take to gauge their understanding of the majority of medical specialties, from biochemistry to diagnostic reasoning to bioethics.
The authors evaluated the program on 350 of the 376 public questions accessible from the USMLE release in June 2022 after screening to exclude image-based questions from the USMLE, according to the paper.
The study, which was published in the journal PLOS Digital Health, revealed that when indeterminate replies were eliminated, ChatGPT had scores ranging from 52.4 to 75 percent on each of the three USMLE tests.
Every year, over 60% of students pass the test.
A large language model (LLM), or new artificial intelligence (AI) system, called ChatGPT is intended to produce writing that resembles that of a person by anticipating future word sequences.
According to the report, ChatGPT cannot do online searches, unlike the majority of chatbots.
The study found that instead, it creates text using word associations that are anticipated by internal processes.
The research also found that ChatGPT provided at least one major insight, something that was novel, non-obvious, and clinically valid, for 88.9 percent of its replies, and that it showed 94.6 percent concordance across all of its responses.
According to the study, ChatGPT outperformed PubMedGPT, a rival model that was trained just on biomedical domain literature, and scored 50.8 percent on an earlier dataset of USMLE-style questions.
Although the depth and breadth of analysis were limited by the relatively small input size, the authors observed that their findings gave a peek of ChatGPT’s potential to improve medical education and, eventually, clinical practice.
They continued by citing the usage of ChatGPT by AnsibleHealth practitioners to rework jargon-heavy reports for simpler patient comprehension.
According to the authors, “achieving the passing score for this infamously challenging expert test, and doing it without any human reinforcement, marks a noteworthy milestone in clinical AI development.”
ChatGPT’s participation in this study, according to Kung, extended beyond just serving as the research subject.
“ChatGPT made a significant contribution to the creation of [our] paper. We communicated with ChatGPT like a colleague, asking it to summarize, delineate, and provide opposition to documents that were being worked on. The opinions of ChatGPT were respected by each co-author.”
