Exploring Test Delivery Options for Computer-Based Exams

UPDATED MAY 29, 2020

The NBOME has been exploring alternate delivery options for computer-based examinations since March 2020 and have provided regular updates to stakeholders. Our goal is to keep our stakeholders up to date without making predictions we cannot accurately substantiate. In this time of uncertainty, we hope to provide you with accurate information.

The NBOME is moving rapidly to set up regional Prometric test centers at COMs. Our goal is to partner with up to 4 regionally distributed COMs to begin testing as soon as July. We distributed a survey about the pilot requirements to all COMs on May 20^th and are currently evaluating responses. Participation in the pilot requires commitment of dedicated testing space, proctor availability and network integrity to be successful. Site selection is based on geographical factors as well as the COM’s ability to meet location, infrastructure and technical parameters, and willingness to host students from other COMs. The NBOME expects to contact COMs for participation as early as next week.

We continue to evaluate all feasible short-term and long-term alternate test delivery solutions of COMLEX-USA computer-based examinations. We have made significant progress with other temporary alternate test delivery options as well, and anticipate additional updates provided publically on or around June 12^th. On our website, we have made available recordings of NBOME’s webinars on the impact of COVID-19 on COMs and candidates as well as a subsequent FAQ page, both of which include information on alternate test delivery.

MAY 20, 2020

The first phase of the evaluation into alternate administration options for COMLEX-USA Level 1 and Level 2-CE included identifying technical and security requirements to protect the validity, reliability and fairness of the examinations. The universal hard-earned trust that state medical and osteopathic medical boards have in the COMLEX-USA licensure examination to assist with medical licensure decisions requires an extremely careful approach to maintain the program’s integrity. This first phase is underway and includes proof-of-concept testing of alternate proctoring solutions. The NBOME continues to explore several potentially viable delivery options, each with different levels of risk, but all in proctored environments, including:

• Satellite Test Centers -- at Colleges of Osteopathic Medicine (COMs) (smaller number of schools)
• Remote Proctored Exams -- at COMs
• Test Events at existing test centers in numerous COMs with simultaneous, web-based delivery
• Remote Proctored Exams -- available anywhere

This week, the NBOME is surveying all COMs to identify those who can demonstrate the technical and other requirements and have the ability to participate in on-site pilot testing. The NBOME will then partner with select COMs in early June for pilot testing. After evaluating the results of those pilot tests, we anticipate implementing one or more alternate test delivery options as early as mid to late July.

We have been communicating directly with all COMs and, through our COM Liaison team, have delivered 16 detailed and customized reports to COMs. We anticipate all COMs will receive an additional report by the end of this week.

As of today, 98% of previously cancelled Level 1 and Level 2-CE appointments have been rescheduled at Prometric Test Centers, with 96% rescheduled in May through July. There is some regional variability due to government-mandated closures and social distancing policies. We sincerely apologize for this as well as several significant shortcomings with Prometric’s lagging scheduling technology and other limitations in customer support. Prometric currently has 284 sites open, and we have been encouraged to hear that 306 sites will be open by June 1. Between May 4, when NBOME examinations restarted, and this Friday, May 22, approximately 600 COMLEX-USA administrations will have been completed. Over 16,000 administrations are scheduled from June to August.

It remains important for us to move forward together in support of our students in these challenging times. The NBOME will continue to assist in any way we can to partner with you all for your assessment needs.

MAY 15, 2020

In response to COVID-19's impact on the safety and feasibility involved in taking examinations at Prometric test centers, the NBOME is actively exploring options for alternate test delivery. Delivering high-stakes examinations outside of test centers involves careful planning to mitigate risks to security, administration, and psychometrics of the examinations. The NBOME is dedicated to rapidly addressing these concerns while continuing to administer exams that are secure, fair, valid, reliable and defensible.

To ensure success, the NBOME is pursuing multiple options simultaneously, including the delivery of examinations through remote proctoring available anywhere, and administering examinations at COMs. We have connected with Prometric, other testing vendors, peer organizations such as NBME and our international colleagues, to assure that every potential solution is investigated on an unprecedented timeline. We have already started testing the feasibility of remote-proctored examination solutions. We aim to partner with COMs within the next month to evaluate options for examination administration at COMs. This multi-pronged approach will allow us to gather information that enables us to determine the best solution for our key stakeholders, including candidates, COMs, state licensing authorities, and the public.

As an organization, we are working around the clock to evaluate and potentially implement alternate test delivery options, as necessary, for both short-term and long-term needs. We plan to publish another update by May 30 with a decision on the feasibility of alternative options potentially as early as June 30, 2020.

MAY 1, 2020

Since the COVID-19 pandemic became a reality, the NBOME has accelerated our research, development and piloting of alternate test delivery options for NBOME assessments. We implemented COMAT-Self-proctored test form options for Colleges of Osteopathic Medicine, with initial use in late March and widespread implementation in April.  In late March, we also launched COMSAE Phase 2 (self-assessments) on our CATALYST longitudinal assessment platform, available to DO students online.

With Prometric Test Centers closed through April, and the limited opening of Prometric Centers in areas of the country where essential services testing was permitted starting May 1, and to be best prepared in the event of any further “second wave” of pandemic activity closing test centers later in 2020 or the winter, we have continued to explore alternate test delivery options for COMLEX-USA exams, including remote proctoring at sites other than Prometric Centers. We are piloting remote proctoring with our COMSAE program in May. We do not envision the current COMLEX-USA Level 2-PE examination, which assesses fundamental clinical skills including physical exam skills and OMT, to be administered remotely or outside of NBOME’s National Centers for Clinical Skills Testing, but have a team exploring the changes that may be needed for this exam as well.

For the COMLEX-USA administration, our objective is to provide candidates the opportunity to complete their examination and demonstrate their competency for licensure as promptly and as safely as possible. As COMLEX-USA is a licensing examination, we are entrusted to ensure that any solution is not only timely, but assures the integrity of the examination program, that it remains valid for this purpose, preserving the reliability, defensibility and fairness of the examination. COMLEX-USA is entrusted in every state for medical licensure for DOs, therefore the integrity of the examination is of paramount concern. We have appreciated the overwhelming support of the colleges of osteopathic medicine and other stakeholders in sharing ideas for solutions and feedback.

In conjunction with identifying supplemental avenues for testing, we continue to listen and collaborate with organizations and individuals in the medical education and the medical regulatory community to address this disruption and its effects, both professionally and personally, for our candidates. Some solutions may involve temporary changes at colleges of osteopathic medicine and residency programs to policy and/or requirements, e.g., relaxing requirements for promotion, graduation, residency applications, or licensing decisions.

We recognize this is yet another stressor in the wake of this pandemic. We empathize with the unique circumstances of osteopathic medical students and residents, and indeed all of us in healthcare, medical education, and our world.  We will continue to provide updated communications regularly, and are committed to an update on alternate test administration options by May 30.