For Program Directors - September, 2010
COMLEX-USA Computer-Based Testing Frequently Asked Questions

COMLEX-USA Level 1, Level 2 Cognitive Evaluation, and Level 3

The National Board of Osteopathic Medical Examiners, Inc. (NBOME) is the creator and sole administrator of the COMLEX-USA series of examinations, the recognized pathway to licensure for osteopathic physicians in the United States and a number of international jurisdictions. The COMLEX-USA examination series consists of both computer-based cognitive examinations and a clinical skills examination. In addition to providing an overview of COMLEX-USA, this section addresses the most frequently asked questions regarding the computer-based testing (CBT) format of the COMLEX-USA cognitive examinations. Please see the COMLEX Level 2-Performance Evaluation Fact Sheet (http://www.nbome.org/docs/PEFactSheet.pdf) for information on the clinical skills assessment component of the COMLEX-USA examination series.

 

Introduction and Overview

 

What is COMLEX-USA?

COMLEX-USA stands for the Comprehensive Osteopathic Medical Licensing Examination of the United States. COMLEX-USA is a medical licensing examination series for osteopathic physicians.

 

COMLEX-USA is a quality, high-stakes three-level examination series. COMLEX Level 1, COMLEX Level 2-Cognitive Evaluation and COMLEX Level 3 each contain 400 test items in a CBT format. COMLEX-USA Level 2-Performance Evaluation is a clinical skills examination containing 12 clinical encounters, all using standardized patients. The COMLEX-USA is a comprehensive licensing examination series emphasizing medical problem-solving knowledge and cognitive skills.  It is the only licensing examination that covers osteopathic principles, practices, and the application of osteopathic manipulative treatment.

 

COMLEX-USA is accepted by all fifty states in the U.S. for the licensure of osteopathic physicians. Internationally, many countries accept COMLEX-USA scores as a component of their licensing requirements. For graduate education in the United States, all osteopathic residency programs accept COMLEX-USA scores. Most Accreditation Council for Graduate Medical Education-accredited residency programs accept COMLEX-USA scores for osteopathic physician applicants.

What does COMLEX-USA measure?

COMLEX-USA assesses the osteopathic medical knowledge and clinical skills considered essential for osteopathic generalist physicians to practice medicine without supervision. To more directly and effectively achieve its measurement objectives, COMLEX-USA is constructed in the context of medical problem-solving that involves clinical presentations and physician tasks. The NBOME’s Bulletin of Information on www.nbome.org details the COMLEX-USA design and content coverage.

 

COMLEX has three Levels. COMLEX Level 1 is typically given to second-year osteopathic medical students. It emphasizes the scientific concepts and principles necessary for understanding the mechanisms of health, medical problems and disease processes.

 

COMLEX Level 2 has two separate examinations: Level 2-Cognitive Evaluation (CE) and Level 2-Performance Evaluation (PE). COMLEX Level 2-CE is typically given to fourth-year osteopathic medical students. It emphasizes the medical concepts and principles necessary for making appropriate medical diagnoses through patient history and physical examination findings.

 

The COMLEX Level 2-PE is also given to fourth-year osteopathic medical students. It provides an assessment of fundamental clinical skills. These clinical skills are doctor-patient communication, interpersonal skills and professionalism, medical history-taking and physical examination skills, osteopathic principles and osteopathic manipulative treatment, and written communication skills (including synthesis of clinical findings, integrated differential diagnosis and formulation of a diagnostic and treatment plan).

 

COMLEX Level 3 is the final component to the COMLEX-USA series and is designed for osteopathic postgraduate candidates prior to completion of their residency training. It emphasizes the medical concepts and principles required to make appropriate patient management decisions.

 

COMLEX Level 1, Level 2-CE and Level 3 are computer-based cognitive examinations, delivered at more than 320 Prometric professional test centers in the US and in Canada. COMLEX Level 2-PE is administered at the NBOME’s National Center for Clinical Skills Testing in Conshohocken, Pennsylvania.

 

COMLEX-USA Computer-based examinations

 

What is the format of the computer-based COMLEX-USA examination?

There are two types of computer-based testing: computerized adaptive testing and computerized non-adaptive testing. In adaptive testing, the computer selects questions based on an examinee’s responses to the previous questions. The COMLEX-USA examinations are computerized non-adaptive tests. The questions selected for each COMLEX-USA examination are predetermined. The number of questions and time limit for the examinations are also predetermined and, under normal administration conditions, are the same regardless of when and where the examinations are administered.

 

The total number of questions for a computer-based COMLEX-USA examination (COMLEX-USA Level 1, Level 2-CE, and Level 3) is 400, and the total testing time is eight (8) hours. The examination is divided into two sessions consisting of four sections. Candidates have four hours to complete each session, and within a session, the individual sections are not timed.

 

All computer-based COMLEX-USA questions are multiple-choice questions. Most of the questions are one-best-answer questions, while a small number of questions are matching-type questions. Most of the one-best-answer questions are single, stand-alone items, while the remainder uses two or more one-best-answer items following a case history. 

 

 

Is there a limit on how many times a candidate can take a particular level of COMLEX-USA examination?

The NBOME does not limit the number of times a candidate may retake an examination that he/she did not pass; however, a candidate may take the COMLEX-USA Level 1, Level 2-CE or Level 3 a maximum of four times in any 12-month period. (NBOME guidelines state that the COMLEX Level 2-PE may only be taken three times in a 12-month period.) Candidates must check the policies of their academic institution for its retake guidelines, which may differ from the NBOME’s. A candidate may not retake any level of COMLEX-USA to improve his or her score, or for any purpose other than to pass a previously failed examination.

 

 

 

 

Score Reporting

 

What types of score reports will candidates, schools and others receive from their computer-based COMLEX-USA cognitive examinations?

For the computer-based COMLEX-USA cognitive examinations, candidates will receive an official printed copy of their score reports from the NBOME by mail. The student score report will provide 3-digit standard scores, 2-digit standard scores, and a pass/fail designation. The 2-digit scores are linear transformations of the 3-digit scores so that 75 is designated as the minimum 2-digit passing score. Percentile rank will not be reported due to the fact that candidates will be taking the examination throughout the year. (Following completion of the annual testing cycle, a candidate may look up his or her percentile rank in relation to others in the same cohort. This tool is available on the NBOME website – www.nbome.org.)  In addition, candidates will also be provided with a graphical performance profile according to the COMLEX-USA content categories. 

Since computer-based COMLEX-USA examinations are administered continuously throughout the year, osteopathic medical schools will receive summary reports at the end of the annual testing cycle. The school summary reports include school information compared with the national average performance.

Scores for computer-based COMLEX-USA examinations will also be available via secured student and school web accounts. Every COMLEX-USA candidate will be able to establish a secured web account provided by the NBOME when they register for the examinations. When scoring is finished, the NBOME will post a 3-digit score, a 2-digit score, and a pass/fail designation to each individual candidate account. Candidates will be able to access the test results immediately after they are available along with their examination history. Similarly, osteopathic schools will have secure web accounts. The NBOME will post test results to school accounts immediately after the results are available, and schools may go to their individual accounts to obtain up-to-date information about their students’ test-taking activities and test results at any time.

 

When and how will scores be reported?

Candidates can expect to receive test results within 4 – 6 weeks after the examination is taken. Score reports for computer-based COMLEX-USA examinations will be mailed within 5 business days after the score is released. Scores will also be available on a candidate’s online account immediately after it is released.

 

What are 3-digit COMLEX-USA scores?

COMLEX-USA 3-digit scores are standard scores. The mean standard scores of all three Levels of COMLEX-USA computer-based cognitive examinations, regardless of when the examinations were given, are 500. The minimal passing standard scores for Level 1 and Level 2-CE are 400, and for Level 3 is 350, regardless of when the examinations were taken. Standard deviations, however, are Level-specific and time-specific. Please go to Score Interpretations at http://www.nbome.org/score-interpretation.asp for the standard deviations of COMLEX-USA by Level and year in the unit of standard score.

COMLEX-USA 3-digit scores are NOT raw scores or the number of items answered correctly. Similarly, they are NOT the percentage of items candidates answered correctly.

 

What are 2-digit COMLEX-USA scores?

COMLEX-USA 2-digit scores are standard scores too. For all three COMLEX-USA Levels, the 2-digit scores are constructed such that the minimum passing score is 75 and mean is 82, regardless of when the examinations were given. Standard deviations for the 2-digit scores, however, are Level-specific and time-specific.

COMLEX-USA 2-digit scores are NOT percentile ranks. Similarly, they are not the percentage of items candidates answered correctly.

COMLEX-USA 2-digit scores and 3-digit scores are equal measures of a candidate’s performance on a COMLEX-UA cognitive examination. By definition, 2-digit scores can be directly converted from 3-digit scores. For example, if 3-digit score is 400 or the minimum passing score, its corresponding 2-digit score will be 75, the minimum passing score in 2-digit scale. Similarly, if 3-digit score is 500, the mean score, the corresponding 2-digit score will be 82, the mean score of 2-digit scale. On the NBOME website you will find a score conversion tool that allows you to convert 3-digit scores to 2-digit scores, and also see percentiles after the testing of a year-long cycle is completed. The tool is on NBOME website at www.nbome.org.

 

Why does the NBOME report COMLEX-USA 2-digit scores if they do not provide more information than 3-digit scores?
COMLEX-USA 3-digit scores provide all necessary performance information for COMLEX-USA examinations. Psychometrically, there is no need for 2-digit scores. However, some state legislatures require 2-digit scores. In addition, many residency programs are accustomed to 2-digit scores. It is for these reasons the NBOME reports 2-digit COMLEX-USA scores.

 

Why doesn’t the NBOME report COMLEX-USA percentile ranks?

COMLEX-USA is an examination series designed primarily for the licensure of osteopathic physicians. It helps fulfill the NBOME’s mission to protect the public by screening candidates according to its definition of competence for practicing osteopathic medicine. While the NBOME acknowledges that COMLEX-USA examination scores are sometimes used for secondary purposes, such as ranking candidates (e.g. those applying for residency programs), and while there is some evidence to support that practice, this is not the primary purpose of the COMLEX-USA examination series.

A percentile score is an examinee’s performance rank among a defined peer group that takes the same examination. For COMLEX-USA examinees, this peer group is considered to be the examinees who take an examination in the same testing cycle. Currently, the testing cycle for COMLEX Level 1 is from May to next April, for COMLEX Level 2-CE, it is from June to next May, and for COMLEX Level 3, it is from February to next January. To accurately compute a percentile score, scores of the whole cohort, or at least most of the cohort, need to be available. COMLEX-USA cognitive examinations are computer-delivered, and each Level is given 40 or more times per year. Candidates decide their own test date from a regularly web-posted testing schedule. As a result, not all candidates take the same examination at the same time. Therefore, when some candidates have finished their examinations and received their scores, a large portion of their peers have not yet taken the same examination. In this situation, COMLEX cannot calculate percentile scores for those who finished the examinations earlier without knowing the scores of their peers who have yet to take the examination.

 

How can the NBOME help those who would like to know percentiles?

The NBOME provides a 3-digit score-to-percentile conversion after a full-year testing cycle is completed. Percentile scores in that context provide a candidate’s rank among all the first-time takers who take the same examination in a defined year-long testing cycle. The testing cycle-based 3-digit scores, 2-digit scores, and percentile conversions can be found on the NBOME home page at www.nbome.org.

 

Can a percentile score be estimated from a COMLEX-USA 3-digit score?

Once a full-year testing cycle is complete, a 3-digit score can be converted to percentile using the conversion tool on the home page of the NBOME web site (www.nbome.org). Prior to that, the formula and table below can be used to estimate a percentile score from a 3-digit COMLEX-USA score.

In normal situations, a score’s distance from mean in terms of standard deviation implies a percentile rank. A COMLEX 3-digit score’s distance from the mean in terms of standard deviation can be computed in the following way:

 

(3 digit score - Mean)/Standard Deviation = distance from mean

 

For example, the mean for COMLEX Level 1 is 500 and the current standard deviation is 81. A score of 480 is -.247 deviations from the mean ((480-500)/81= -.247) or approximately .25 standard deviation below the mean.

The table below converts a few 3-digit scores to percentiles. From this table, it can be seen that the percentile for a score of 480 for COMLEX Level 1 is between 30 and 50. It would be a reasonable estimate that the percentile of 480 for COMLEX Level 1 is approximately 40.

 

 

The method described above provides a rough estimate of the percentiles as it assumes that the mean of a particular testing cycle is exactly 500 and standard deviation is exactly that of the 3-digit score scale. However, the differences between the estimates and actual percentiles are typically anticipated to be small enough to be tolerable for residency programs to use for screening purposes.

 

Many ACGME residency programs accept both osteopathic (DO) and allopathic (MD) applicants. Can COMLEX-USA scores be converted into USMLE scores so that those programs can have a common criterion to judge all applicants?

The NBOME understands the desire for a “conversion formula” for osteopathic medical students applying to ACGME-accredited residency programs. However, because of the different natures of the examinations, it is not possible—or even desirable—to make a direct numerical comparison between the scores of the COMLEX-USA examination series and those of the USMLE.  COMLEX-USA and USMLE are two different licensing examination systems. DO applicants and MD applicants are two different test-taking populations by virtue of their training and experience.

The COMLEX-USA series assesses the skills and philosophy unique to the osteopathic medical profession. Osteopathic medical students and residents train for the practice of osteopathic medicine. The curriculum of every osteopathic medical school incorporates distinctive osteopathic principles and prepares students for medical practice as osteopathic physicians. COMLEX-USA incorporates these distinctive osteopathic principles, and it is constructed and validated based on practice patterns that are unique to DOs, including the use of osteopathic manipulative treatment. By contrast, the USMLE does not meet these criteria for assessing osteopathic medical students, and it does not test for osteopathic principles or osteopathic manipulative treatment.

When it comes to examinations, the NBOME encourages residency program directors to consider the COMLEX-USA series as the valid and most appropriate assessment tool for osteopathic medical students.

 

If COMLEX and USMLE scores cannot be converted, how should ACGME programs use COMLEX scores to evaluate osteopathic applicants?

The NBOME encourages ACGME Residency Program Directors to gain a full understanding of the COMLEX-USA examination series, in particular, what it assesses, its format and how standards are set. In addition, all residency program directors may take advantage of the percentile score converter, which can be found on the home page of the NBOME website (www.nbome.org). (Note: A full testing cycle cohort must be complete before a given score can be converted.)

Information on all these areas can be found on the NBOME website:

·         COMLEX-USA cognitive examination overview and blueprint (http://www.nbome.org/comlex-cbt.asp?m=can)

·         COMLEX-USA clinical skills examination overview (http://www.nbome.org/comlex-pe.asp?m=can)

·         NBOME Standard Setting (http://www.nbome.org/standardsetting.asp?m=can)

 

More Questions

If you have any questions, please feel free to contact NBOME Client Services at 1-866-479-6828.