The NBOME recognizes that this score reporting modification for COMLEX-USA Level 1 (pass/fail only) will create new questions for osteopathic medical students, faculty and deans, state licensing board members, the GME community, and other stakeholders. More information and further specifics will be made available by July 2021 on the particulars and logistics of the new score reporting construct for candidates, and that for schools and other secondary users. Input was considered from stakeholder groups across the continuum, including the licensure community, osteopathic medical students and residents, those involved in undergraduate and graduate medical education, accreditation authorities, and numerous other professional organizations across the profession. Ultimately, the NBOME Board took this action in support of wellness across the continuum and in the interest of its mission to protect the public.
These FAQs serve as a complement to the December 2020 announcement of the transition of score reporting for COMLEX-USA Level 1 examinations from reporting numeric scores to only reporting Pass/Fail results starting May 1, 2022. Note that additional information will be provided by July 2021 on the particulars and logistics of the new score reporting construct for candidates, schools and other secondary users.
How long have you been thinking about this decision? What went into making this decision? Did you just do this because USMLE announced the same in 2020?
While there had been considerable discussion and consideration of reporting only pass or fail rather than numeric scores for COMLEX-USA examinations with NBOME’s testing committees and Board of Directors since the first COMLEX-USA redesign in 1995, the concept of the elimination of numeric scores from COMLEX-USA Level 1 garnered considerable attention and analysis by NBOME’s Blue Ribbon Panel for Enhancing COMLEX-USA, which was commissioned in 2009 and published an initial report in March 2012.
At the time, with the secondary use of licensing examination scores by Program Directors and the beginning of discussions regarding a potential transition to a Single Accreditation System for Graduate Medical Education, the decision was to continue to report numeric scores. NBOME continued to engage the medical licensure, regulatory, assessment and education communities on this topic, including inviting input from osteopathic medical students and residents. NBOME has presented on this topic at national meetings including those within the licensure community, but many expressed concerns about change. Students, residents and their advisors in particular cautioned against the unintended consequences of discontinuing reporting of COMLEX-USA Level 1 numeric scores, primarily related to concerns about application for residency programs. While other concerns were cited, the principle apprehension was that DO candidates with a COMLEX-USA score that reported as pass/fail only would not be able to be compared to applicants from MD-granting schools and international medical graduates who applied producing numeric scores. Upon USMLE’s announcement earlier in 2020 that they would discontinue reporting “Step 1” numeric scores beginning in 2022, this concern was mitigated, and stakeholder input changed.
This December 2020 decision was ultimately made after several years of analysis and considering input from across the education, training, and licensure continuum, and in the interest of wellness across the continuum and supporting NBOME’s mission of protecting the public through assessment. The comprehensive analysis included research, surveys of individuals, and solicitation of stakeholder position statements, including those from organizations representing the medical licensure community, undergraduate and graduate medical education, accreditation authorities, and students and residents.
Why May 2022 and not January 2022? Why doesn’t NBOME make this transition sooner?
Our COMLEX-USA test cycles align with the cohorts of students who take the examination and their anticipated graduation year and are established with input from colleges of osteopathic medicine and other stakeholders. This change to the score reporting for COMLEX-USA Level 1 examinations needs to align with the test cycle change, therefore we will make this change in May 2022 (a time when the Class of 2024 begins to take this examination). To make this change earlier would compromise both the psychometric integrity of the examination results of the 2021-2022 test cycle, and would potentially result in half of a graduating cohort having numeric scores, and half having only Pass/Fail results. In addition, this provides stakeholders appropriate notice to prepare for the transition. USMLE has reported that their “Step 1” exam, which is often compared to COMLEX-USA Level 1, will make the change to Pass/Fail reporting as early as January 2022, which is the start of their test cycle. Historically, students at some MD-granting schools and international graduates take Step 1 of USMLE earlier in the calendar year than DO students begin taking COMLEX-USA Level 1, who do not start taking Level 1 until May.
Will candidates or COMs get additional information on candidate performance as part of the COMLEX-USA Level 1 score reports after May 1, 2022?
It is anticipated that comprehensive score reports provided to candidates and available to schools for COMLEX-USA Level 1 examinations taken on or after May 1, 2022 will still include individual candidate performance profiles as they currently do. These demonstrate individual areas of strength and weakness within the examination blueprint for the benefit of continuous professional development, and, when relevant, to use as part of remediation and to enhance potential for success. These reports will likely continue to be generated for candidates and available to COM deans in their secure NBOME Portal accounts. However, it is anticipated that this information will NOT be reported in examination transcripts provided for residency program applications in ERAS or to state medical and osteopathic medical licensing boards. Particulars and logistics of the new score reporting construct for candidates as well and schools and other secondary users will be made available by July 2021.
Will score reports for COMLEX-USA Level 1 examinations administered prior to this transition look any different?
No. If you take/took your COMLEX-USA Level 1 examination on or prior to April 30, 2022, NBOME will continue to report numeric scores AND pass/fail result. Your numeric score will remain on your COMLEX-USA transcript and will be reported in ERAS and to state licensing boards as such. It will not change or update with this transition. Reports for Level 1 examinations starting May 1, 2022 will no longer report numeric scores.
