A GUIDE FOR RESIDENCY PROGRAM DIRECTORS
A GUIDE FOR RESIDENCY PROGRAM DIRECTORS
COMLEX-USA is an evidence-based assessment series designed specifically for osteopathic medical students and residents that measures competencies required to provide safe and effective osteopathic medical care to the public.
This guide is intended to help residency and fellowship program directors develop a better understanding of COMLEX-USA, the Comprehensive Osteopathic Medical Licensing Examination of the United States, what the examination series measures, how standards are set, what candidate performance on COMLEX-USA means, and how that translates to residency performance.
The primary and intended use of COMLEX-USA is for licensure, used by all medical licensing authorities to make licensing decisions for osteopathic physicians.
All osteopathic medical students must pass COMLEX-USA Level 1 and Level 2-CE to graduate with a DO degree1.
COMLEX-USA is accepted in all 50 United States and recognized by a number of international jurisdictions2, and is designed to assess osteopathic medical knowledge, fundamental clinical skills, and other foundational competencies considered essential for the practice of osteopathic medicine.
COMLEX-USA is based on a contemporary, two decision-point, competency-based blueprint and evidence-based design informed by extensive research on osteopathic physician practice, expert consensus, and stakeholder surveys3,. The enhanced COMLEX-USA blueprint assesses measurable outcomes of seven Fundamental Osteopathic Medical Competency Domains and focuses on high-frequency, high-impact health issues and clinical presentations that affect patients4,5.
DO candidates are not required to pass the United States Medical Licensing Examination (USMLE®) to be eligible to apply to ACGME-accredited residency programs. The American Medical Association (AMA)6, the Accreditation Council for Graduate Medical Education (ACGME), and the Federation of State Medical Boards (FSMB) all recognize equivalent uses for COMLEX-USA and USMLE.
“The ACGME does not specify which licensing board exam(s) (i.e., COMLEX-USA, USMLE) applicants must take to be eligible for appointment in ACGME-accredited residency programs.” — Frequently Asked Questions: Single Accreditation System Accreditation Council for Graduate Medical Education, 20207
COMLEX-USA is developed through a rigorous scientific process to ensure it is of the highest quality. The FSMB conducted a comprehensive review of COMLEX-USA and USMLE and found both were valid and reliable for their intended respective purposes. The FSMB concluded that support for the validity of COMLEX-USA is exemplary.
A 2020 study published by the NBOME and FSMB found the predictive validity of COMLEX-USA demonstrates a strong correlation between successful completion of COMLEX-USA and a lower likelihood of state licensing board disciplinary action. Further, multiple studies have reported a strong association between COMLEX-USA and USMLE scores, which can be used to predict performance on one examination from the other8,9.
COMLEX-USA and USMLE scores have been increasingly used to screen and evaluate residency applicants. In addition to use by residency program directors, other secondary uses of COMLEX-USA scores include student and resident promotion and assessment.
The NBOME works with residency programs and osteopathic organizations on education and outreach to ensure that DO applicants are considered fairly in residency selection. The NBOME also advocates for a holistic residency screening process—one that sees exam performance as just one factor of many in determining whether an applicant will perform well in residency.
The medical education community recognizes the need to invest in student wellness, reducing stressors and fostering inclusion for all residency applicants. To that end, a holistic admission review process in medical education considers the whole applicant, avoiding disproportionate focus on a single factor. This approach helps increase diversity in a physician population that is prepared to address varied needs across health care populations.
Further, the Coalition for Physician Accountability’s Undergraduate Medical Education to Graduate Medical Education Transition Review Committee (UGRC)—of which the NBOME is a member—published 34 recommendations for improvement along the UME to GME transition10.
These include recommendations aimed at reducing the perceived need for DO students to take a redundant exam and creating a combined score field in ERAS that allows the two exams to be easily compared, thereby mitigating structural bias11.
Since its implementation in 2020, the ACGME Single Accreditation System for GME has aligned accreditation standards for all residency programs in the U.S., providing all graduates of medical schools (DO and MD) the ability to complete residency and fellowship training in all programs, regardless of which licensing exam(s) the individual has pursued.
It is based on mutual recognition and understanding of the important and unique qualifications and philosophies of both DO and MD graduates. Acceptance of COMLEX-USA by all ACGME programs may reduce stress, support wellness in residency applicants, and diminish barriers for DO applicants.
OVER 90% OF ACGME ACCREDITED RESIDENCY PROGRAM DIRECTORS CONSIDER DO SENIOR CANDIDATES FOR INTERVIEW, FACTORING OSTEOPATHIC QUALIFICATIONS INTO THEIR ASSESSMENT OF APPLICANTS12.
“Students and their abilities as a physician are not defined solely by a number. Program directors need to get comfortable with DO applicants and using COMLEX-USA as just one tool to review which applicants have the skills necessary to thrive in a GME program — just like they learn about other things in their program, they need to learn to adapt to this change.”
– Kenneth B. Simons, MD, Senior Associate Dean for GME and Accreditation, Medical College of Wisconsin, Chair, Federation of State Medical Boards, 2021-2022
COMLEX-USA examinations use a criterion-referenced, standard-setting methodology. The minimum passing score for COMLEX-USA Level 1 or Level 2-CE is 400, while Level 3 minimum passing score is 350.
Passing Levels 1 and 2-CE means a candidate has demonstrated competence to enter into supervised clinical practice settings, enter graduate medical education, and prepare for lifelong learning.
Passing Level 3 means the candidate has demonstrated competence in foundational competency domains required for generalist physicians to deliver safe and effective osteopathic medical care of patients as required for entry into the unsupervised practice of osteopathic medicine and to continue lifelong learning and practice-based learning and improvement.
A percentile score conversion tool is accessible here as well as on the ERAS® Program Director Workstation. This tool converts an applicant’s 3-digit COMLEX-USA score to a percentile rank to help programs compare an applicant’s relative performance to other applicants within a testing cycle, and correctly interpret assessment scores when filtering applications.
Note: The Percentile Score Conversion Tool is available for Level 2-CE for the 2021-2022 testing cycle. With Level 1 transitioning to pass/fail reporting as of May 10, 2022, the percentile scores for Level 1 will not be available for test cycles beginning in 2022-2023. For candidates who took Level 1 prior to May 2022 the percentile score converter provides a percentile rank for the 3-digit scores.
CONVERSION VARIES BY TESTING CYCLE
Residents must pass COMLEX-USA Levels 1 and 2-CE in order to take COMLEX-USA Level 3 and continue on their pathway to licensure. In addition, residents must have attestations by a dean* and residency program director** that they are in good standing and are approved to take Level 3. The attestation process for COMLEX-USA Level 3 helps to fulfill the NBOME mission to protect the public, and adds value and entrustability to state licensing boards and patients. A Level 3 score report is provided to a residency program director who attests to a resident’s good academic and professional standing.
After the suspension of COMLEX-USA Level 2-PE, DO graduates in the classes of 2020-2023 must receive an attestation by their dean that indicates the resident has demonstrated the fundamental osteopathic clinical skills necessary for graduation.
The NBOME recommends that COMLEX-USA Level 3 be taken after a minimum of six months in residency. For more on COMLEX-USA Level 3 eligibility, visit the COMLEX-USA Bulletin of Information.
“The COMLEX-USA Level 3 examination blueprint, designed for osteopathic physicians, provides a good example of how medical examiners can update their assessment systems to stay ahead of the curve of change… focuses on ‘knowledge, skills, experiences, attitudes, values, and/or behaviors that are observable and measurable and can be directly assessed in a reliable manner’— all factors that can help medical regulators make sound, balanced decisions as they continue their work in protecting the public and ensuring quality in medical practice, keeping our patients safe.”
– Heidi M. Koenig, MD, Editor-In-Chief, Journal of Medical Regulation, October 2018
As part of its research program, NBOME conducts studies designed to help residency program directors understand COMLEX-USA examinations with the validity evidence for using these candidate performance results or scores in holistic residency application decisions.
Most recently, NBOME published research provided concordance of scores among DO students who took both COMLEX-USA Level 1 and Level 2-CE, as well as USMLE Step 1 and Step 2-CK. The NBOME created a helpful Concordance Table for residency proram directors to better understand the context of COMLEX-USA scores.
Learn more about properly interpreting score concordance.
A STRONG ASSOCIATION EXISTS BETWEEN THE SCORES ON THE COMLEX-USA AND USMLE EXAMINATIONS, ALLOWING PREDICTION OF PERFORMANCE ON USMLE FROM COMLEX-USA8.
-A Concordance Study of COMLEX-USA and USMLE Scores, Journal of Graduate Medical Education, February 2022
With growing support for equivalent uses of COMLEX-USA and USMLE for residency applications, NBOME provides program directors with insight into the uses of COMLEX-USA and how COMLEX-USA scores can be an asset in a holistic resident screening and selection processes.
FOR MORE INFORMATION ABOUT IMPORTANT RESEARCH ON COMLEX-USA, CLICK HERE