Why osteopathic assessment is important throughout the career of an osteopathic physician and to our patients.

The 4 tenets of osteopathic medicine underlie the osteopathic philosophy of health and medical care. The body, mind, and spirit approach; the connectedness of the body’s structure and function; and consideration of other factors, such as social determinants of health and partnership with patients in overall wellness all contribute to this approach. DOs are trained to integrate these osteopathic principles and osteopathic manipulative treatment (OMT) into patient care.

If you look at an assessment as a milestone for what has been learned and a framework for what is required, then the assessment tools used should align both with the curricular program (what is taught and learned) and what is practiced (what patients expect). If that assessment is further used as part of a profession’s responsibility for self-regulation (i.e. licensure exam), then it is all the more important.

Examinations are developed by a profession, or entity (think driver’s tests or teacher certification) to assess whether a candidate has the knowledge, skills, and attitudes to perform in that profession. The COMLEX-USA licensure examination series is designed to assess the competencies required for the practice of osteopathic medicine. It reflects how DOs approach a patient, and the foundational abilities needed to provide osteopathic medical care, which remain distinctive for DOs.

COMLEX-USA is the only licensing examination that is aligned with the practice of osteopathic medicine, and it is gaining recognition across the globe. It is the only licensing exam accepted in all 50 states and other US licensing jurisdictions for osteopathic physicians. In 2020, the Medical Board of Australia recognized the NBOME/COMLEX-USA as a competent authority pathway for registration (licensure) for comprehensive medical practice in Australia!

Specifically, then, how is COMLEX-USA osteopathically distinctive?

Examination series: The COMLEX-USA series is aligned to be taken at certain developmental times in the osteopathic medical student and residents’ progression toward licensure. Content is designed to be developmentally appropriate and aligned with the osteopathic medical educational programs. COMLEX-USA is endorsed by the AOA’s Commission on Osteopathic College Accreditation (COCA) with graduation standards that include COMLEX-USA examinations. COCA is the recognized accreditation authority by the US Department of Education for osteopathic medical schools.
Blueprint design: The COMLEX-USA examination blueprint includes two integrated dimensions: osteopathic competency domains and clinical presentations. The competency domains define the unique knowledge, skills, experience, attitudes, values, behaviors, and established professional standards of osteopathic medicine. The clinical presentations represent the manner in which a particular patient, group of patients, or a community presents for osteopathic medical care. These are both informed by an evidence-based design for the practice of osteopathic medicine, including the review of national databases of what DOs do and see in practice. ^1-2
Inclusion of the Competency Domain “Osteopathic Principles, Practice and Manipulative Treatment”: COMLEX-USA includes this as a competency domain, and it is important to osteopathic medical education and practice. However, osteopathic principles are also integrated with the other competencies in the assessment. For example, questions classified or “coded” to this competency domain may also be assessing another competency domain or skill. This aligns with practice—osteopathic physicians bring all of their competencies to the care of a patient, at the same time.
Examination construction: Once the blueprint has been designed and the items written, the test is assembled according to specifications for each level (as in, how many of each kind of question goes into an examination, among other considerations). This provides an assessment form that is fair, valid, and aligned with the practice of osteopathic medicine. Because the specifications are so aligned, changing a few items will necessarily change the whole examination. The test specifications are based on DO practice data including that from the National Ambulatory Health Care Surveys, which is actually distinctive from practice data for other physicians. ³
Test question and case construction: Down to the item level, the NBOME uses an osteopathically distinctive approach to developing multiple-choice questions and clinical cases. Interdisciplinary groups of DOs and other professional colleagues collaborate to develop patient scenarios and associated questions based on high-frequency, high-impact clinical presentations. This results in items that address whole patient care—not just assessment of basic recall of scientific facts. COMLEX-USA test items and cases are built to assess the application of knowledge and other competencies—not just knowledge itself. The unique clinical decision-making (CDM) cases in the Level 3 examination are designed to assess a candidate’s ability to think beyond multiple-choice questions. CDM forces candidates to think holistically about the patient’s scenario—whether that means determining the diagnosis based on the available findings or selecting management steps for a patient in an extended multiple-choice question. CDM items are particularly relevant to patient safety and differentiation at the minimal competency threshold.

To help ensure alignment across national standards taught at all of the colleges of osteopathic medicine, the NBOME collaborates with Educational Council on Osteopathic Principles (ECOP) to establish testable concepts and standardized nomenclature related to osteopathic manipulative medicine (OMM). In the Level 2-PE, before that exam was suspended in the pandemic, candidates had the opportunity to perform hands-on diagnostic physical exam maneuvers, and perform OMT as appropriate to the diagnosis and within certain parameters.

On the residency program/GME front, we work at the grassroots-level to help provide information to elective clinical rotation sites and program directors about COMLEX-USA, helping to contribute to the steady increase in DOs being accepted with their own distinctive credentials into ACGME-accredited residency programs.

We have worked with the American Association of Medical Colleges (AAMC) to provide data for the Report on Residents, which for the first time in 2020 included COMLEX-USA scores.
Our partnership with the Electronic Residency Application Service (ERAS^®) assists residency program directors by demonstrating parity in the application dashboard for DO and MD applicants, including a COMLEX-USA percentile score converter and similar transcript presentations and notations to those available for licensing exams for MD applicants.
We serve on the National Resident Matching Program^®’s (NRMP^®) Research Advisory Committee to ensure that the osteopathic voice is represented and that inclusive nomenclature is used for DO stakeholders in NRMP® publications, presentations, and other Match^® initiatives.

Even those less familiar with DOs and COMLEX-USA prior to the Single GME Accreditation System (2015-2020) have learned that DOs take COMLEX-USA. The NBOME has historically advocated for DO students and their credentials, including the DO degree, COMLEX-USA, and AOA board certification both nationally and internationally.

Other examples of osteopathically distinctive assessments deemed to be valid for their purposes include COMAT examinations used by almost every DO medical school campus in the US, AOA board certification assessments in 16 specialties, and numerous residency in-training/in-service assessments designed by osteopathic specialty societies and used by hundreds of residency programs in specialties such as family medicine (ACOFP), internal medicine (ACOI) and surgery (ACOS).

Assessments should be aligned with the educational programs, outcomes, and practice of the profession. A profession must self-regulate and set standards to continue to earn public trust. COMLEX-USA is targeted to the distinct practice of osteopathic medicine to protect that trust. This, along with rigorous standards and continuous quality improvement, has helped COMLEX-USA to be the gold standard licensure examination for osteopathic physicians since the NBOME was founded in 1934. Having a robust, distinctive portfolio of assessments will help DOs to be well positioned to continue to grow and thrive across the United States and the world.

Gimpel JR, Horber D, Sandella JM, Knebl JA, Thornburg JE. Evidence-based redesign of the COMLEX-USA series. Journal of the American Osteopathic Association. 2017;117(4): 253-261.
Horber D, Gimpel JR. Enhancing COMLEX-USA: Evidence-based redesign of the osteopathic medical licensure examination program. Journal of Medical Regulation. 2018;104(3):11-18.
Boulet JR, Gimpel JR, Errichetti AM, Meoli FG. Using National Medical Care Survey data to validate examination content on a performance-based clinical skills assessment for osteopathic physicians. JAOA: 2003; 103(5):225-31.

Original Publication Date: April 19, 2021.

See All

Update: COMLEX-USA Level 1 Transition to Pass/Fail Reporting

Please see a statement from President and CEO John R. Gimpel, DO, MEd, on the intended use of formative performance profiles which accompany pass/fail score reports.

As previously announced, COMLEX-USA Level 1 (Level 1) reporting will transition from a three-digit numeric score to “Pass” or “Fail” outcomes for Level 1 beginning on or after May 10, 2022.

To receive a three-digit numeric score, candidates must take their Level 1 before May 10, 2022.

Dates for Level 1 Transition to Pass/Fail Only:

Level 1	Test before May 10, 2022	Test on or after May 10, 2022
Score Reports	Numeric score AND pass/fail	Pass/fail only
Transcript	Numeric score AND pass/fail	Pass/fail only

Level 1 Candidate Score Report

All candidates will receive their performance information as it relates to the national mean performance of those who have taken COMLEX-USA Level 1 for the first time. This feedback (Low/Average/High) identifies areas of strength and weakness and may be useful to guide a study plan for failing candidates. The minimum blueprint percentage allocated to the competency domains and clinical presentations will also be presented. Discipline scores will no longer be included in either candidate or school reports.

View Sample New Score Report here.

Frequently Asked Questions

1. Q: When will changes to COMLEX-USA Level 1 score reports go into effect?

A: Changes will go into effect for any administrations taken on or after May 10, 2022. Click here to see the sample Score Report.

2. Q: Will COM deans receive the candidate score reports?

A: Yes. COMs will still get up-to-date student performance on COMLEX-USA Level 1 with pass/fail (P/F) and performance profiles by clicking “View Score Report” on their dean’s page.

The NBOME cautions against use of these performance profiles for purposes other than providing candidates with an understanding of their relative strengths and weaknesses in the COMLEX-USA blueprint.

3. Q: Will discipline scores still be available to COM deans?

A: No. Beginning May 10, 2022 administrations, the Level 1 discipline scores will no longer be available for the candidate score reports, Dean’s Page, and aggregate school report. Only performance as it relates to the blueprint competency domains and clinical presentations as seen in the candidate score report will be available to COM deans. This information will be accessible under the “View Score Report” on the Dean’s Page. Click here to see the sample Score Report.

4. Q: What performance data will COMs see?

A: COM deans will be able to view how many of their students have taken the exam for the first time, and performance pass rates for their students who are first-time test-takers as compared to the national pass rate for first-time test-takers. They will see performance as it relates to the blueprint competency domains and clinical presentations as seen in the candidate score report.

5. Q: Will COM deans still get an annual report?

A: Yes, an annual report of school performance will be available to deans in September 2023.

6. Q: What do program directors see in ERAS for candidates who test on or after May 10, 2022?

A: Program directors will only see a pass or fail result in ERAS for candidates who take Level 1 after that date. The minimum passing score of 400 will also be shown in the ERAS transcript, but no numerical scores for candidates will be reported for candidates who test on or after May 10, 2022.

We will continue updating this document with answers to additional questions as we receive them. If your question is not answered here, please email clientservices@nbome.org.

7. Q: How can a student determine if they are prepared to take COMLEX-USA Level 1?

A: Because the COMLEX-USA Level 1 examination is highly integrative, students should review all content areas, especially if they are retaking the examination. The COMLEX-USA Level 1 blueprint is a good place to start.

The NBOME offers self-assessment tools that osteopathic medical students may use prior to sitting for the COMLEX-USA Level 1 examination: COMSAE and WelCOM. These tools are used to gauge students’ knowledge base and abilities, reinforce medical knowledge, and determine which content areas they are most adequately prepared for.

The COMSAE Phase 1 score report provides performance by discipline, competency domain, and clinical presentation, which may be helpful for students as they prepare to take COMLEX-USA Level 1. Please see this article for additional information on the role of COMSAE Phase 1 in COMLEX-USA Level 1 preparation.

WelCOM provides additional learning resources to prepare students for both COMSAE Phase 1 and COMLEX-USA Level 1. After responding to each WelCOM question, candidates are shown the correct answer, the rationale for why that answer is correct, and are provided additional references for further study.

December 17, 2020

PHILADELPHIA, PA. The Board of Directors is pleased to announce that the NBOME will change from reporting three-digit numeric scores to reporting only “Pass” or “Fail” for COMLEX-USA Level 1 examinations administered effective May 1, 2022, aligning with the start of the 2022-2023 testing cycle. This decision was made after several years of analysis and considering input from across the education, training, and licensure continuum.

The NBOME recognizes that this score reporting modification for COMLEX-USA Level 1 (pass/fail only) will create new questions for osteopathic medical students, faculty and deans, state licensing board members, the GME community, and other stakeholders. More information and further specifics will be made available by July 2021 on the particulars and logistics of the new score reporting construct for candidates, and that for schools and other secondary users. Input was considered from stakeholder groups across the continuum, including the licensure community, osteopathic medical students and residents, those involved in undergraduate and graduate medical education, accreditation authorities, and numerous other professional organizations across the profession. Ultimately, the NBOME Board took this action in support of wellness across the continuum and in the interest of its mission to protect the public.

Frequently Asked Questions

These FAQs serve as a complement to the December 2020 announcement of the transition of score reporting for COMLEX-USA Level 1 examinations from reporting numeric scores to only reporting Pass/Fail results starting May 1, 2022. Note that additional information will be provided by July 2021 on the particulars and logistics of the new score reporting construct for candidates, schools and other secondary users.

How long have you been thinking about this decision? What went into making this decision? Did you just do this because USMLE announced the same in 2020?

While there had been considerable discussion and consideration of reporting only pass or fail rather than numeric scores for COMLEX-USA examinations with NBOME’s testing committees and Board of Directors since the first COMLEX-USA redesign in 1995, the concept of the elimination of numeric scores from COMLEX-USA Level 1 garnered considerable attention and analysis by NBOME’s Blue Ribbon Panel for Enhancing COMLEX-USA, which was commissioned in 2009 and published an initial report in March 2012.

At the time, with the secondary use of licensing examination scores by Program Directors and the beginning of discussions regarding a potential transition to a Single Accreditation System for Graduate Medical Education, the decision was to continue to report numeric scores. NBOME continued to engage the medical licensure, regulatory, assessment and education communities on this topic, including inviting input from osteopathic medical students and residents. NBOME has presented on this topic at national meetings including those within the licensure community, but many expressed concerns about change. Students, residents and their advisors in particular cautioned against the unintended consequences of discontinuing reporting of COMLEX-USA Level 1 numeric scores, primarily related to concerns about application for residency programs. While other concerns were cited, the principle apprehension was that DO candidates with a COMLEX-USA score that reported as pass/fail only would not be able to be compared to applicants from MD-granting schools and international medical graduates who applied producing numeric scores. Upon USMLE’s announcement earlier in 2020 that they would discontinue reporting “Step 1” numeric scores beginning in 2022, this concern was mitigated, and stakeholder input changed.

This December 2020 decision was ultimately made after several years of analysis and considering input from across the education, training, and licensure continuum, and in the interest of wellness across the continuum and supporting NBOME’s mission of protecting the public through assessment. The comprehensive analysis included research, surveys of individuals, and solicitation of stakeholder position statements, including those from organizations representing the medical licensure community, undergraduate and graduate medical education, accreditation authorities, and students and residents.

Why May 2022 and not January 2022? Why doesn’t NBOME make this transition sooner?

Our COMLEX-USA test cycles align with the cohorts of students who take the examination and their anticipated graduation year and are established with input from colleges of osteopathic medicine and other stakeholders. This change to the score reporting for COMLEX-USA Level 1 examinations needs to align with the test cycle change, therefore we will make this change in May 2022 (a time when the Class of 2024 begins to take this examination). To make this change earlier would compromise both the psychometric integrity of the examination results of the 2021-2022 test cycle, and would potentially result in half of a graduating cohort having numeric scores, and half having only Pass/Fail results. In addition, this provides stakeholders appropriate notice to prepare for the transition. USMLE has reported that their “Step 1” exam, which is often compared to COMLEX-USA Level 1, will make the change to Pass/Fail reporting as early as January 2022, which is the start of their test cycle. Historically, students at some MD-granting schools and international graduates take Step 1 of USMLE earlier in the calendar year than DO students begin taking COMLEX-USA Level 1, who do not start taking Level 1 until May.

Will candidates or COMs get additional information on candidate performance as part of the COMLEX-USA Level 1 score reports after May 1, 2022?

It is anticipated that comprehensive score reports provided to candidates and available to schools for COMLEX-USA Level 1 examinations taken on or after May 1, 2022 will still include individual candidate performance profiles as they currently do. These demonstrate individual areas of strength and weakness within the examination blueprint for the benefit of continuous professional development, and, when relevant, to use as part of remediation and to enhance potential for success. These reports will likely continue to be generated for candidates and available to COM deans in their secure NBOME Portal accounts. However, it is anticipated that this information will NOT be reported in examination transcripts provided for residency program applications in ERAS or to state medical and osteopathic medical licensing boards. Particulars and logistics of the new score reporting construct for candidates as well and schools and other secondary users will be made available by July 2021.

Will score reports for COMLEX-USA Level 1 examinations administered prior to this transition look any different?

No. If you take/took your COMLEX-USA Level 1 examination on or prior to April 30, 2022, NBOME will continue to report numeric scores AND pass/fail result. Your numeric score will remain on your COMLEX-USA transcript and will be reported in ERAS and to state licensing boards as such. It will not change or update with this transition. Reports for Level 1 examinations starting May 1, 2022 will no longer report numeric scores.

###

About the NBOME:

NBOME is an independent, non-governmental, non-profit assessment organization committed to protecting the public by providing the means to assess competencies for osteopathic medicine and related health care professions. NBOME’s COMLEX-USA examination series is a requirement for graduation from colleges of osteopathic medicine and provides the pathway to licensure for osteopathic physicians in the United States and numerous international jurisdictions.

MEDIA INQUIRIES

Renee Cree
Director for Communications
RCree@nbome.org

See All

Tell us what inspired you to become a DO, specifically.

I was first introduced to osteopathic medicine when I was a patient in the emergency department during my sophomore year of college. At the time, I was already on the path to go to medical school, but I didn’t know the difference between an MD and a DO. While I was at the ER, I had a strong impression of the doctor treating me there, who I later found out was a DO. That discovery was what triggered me to research more about what osteopathic medicine was. After that, I found out that I strongly believed in the core foundations of osteopathic medicine—our body is a unit composed of the body, mind, and spirit—all of which are vital to assess in each of our patients.

Prior to medical school, I watched my grandmother battle cancer. Initially, her physical body was deteriorating, but her mind remained sharp and her spirit uplifted. Witnessing this is what allowed me to truly realize that her body, mind, and spirit each played a role in her overall health. And when I thought about my view of healthcare—how I wanted to treat my patients, I knew that pursuing osteopathic medicine was ultimately the right path for me.

Congratulations on your Match into Emergency Medicine! How did you choose this specialty?

I’d known I wanted to be a physician since high school, but I was first introduced to the exciting field of Emergency Medicine while I was a scribe during my gap year in the emergency department. I chose to scribe for a full year, then did a one-year master's program for my Masters of Arts in Biomedical Science at Midwestern University, going then to their medical school directly after that.

Everyone always told me that I would find my specialty when I found my people, and it was literally less than 15 minutes on the first day of my emergency medicine rotation during my third year of medical school that I knew this was the best specialty for me. I love the diverse medical pathology that presents every day, along with the ability to treat patients within the community from all different walks of life, while using our knowledge and skills to address a variety of emergencies.

We understand you only used COMLEX-USA scores when applying to residency—what prompted that decision, and what advice would you give students who are thinking of doing the same?

I was initially scheduled to take COMLEX-USA Level 1 and USMLE Step 1 five days apart from each other. I did this because it seemed like it was what everyone else was doing—so many people told me that I also needed to take USMLE because I was planning on applying to a competitive specialty. However, during my first dedicated board study period, I realized preparing for both exams was like trying to run two completely different marathons. The exams are very different, and studying for both of them simultaneously posed a challenge for myself. That was why I ultimately decided to focus my efforts on only COMLEX-USA Level 1, and cancelled my USMLE Step 1. And when it came time for the next round of boards, I only scheduled the COMLEX-USA Level 2. But because I’d made that decision, it was so much less stressful—I felt I had more focus and could maintain a positive and optimistic attitude since there was only one exam I had to focus on.

DOs take COMLEX-USA because we are osteopathic students and it is aligned with our curriculum. A large part of what we are tested on is if we understand the osteopathic foundation—if we understand the tenets of what a DO physician is. Also, osteopathic manipulative treatment—none of that would be on USMLE. COMLEX-USA could ask some very specific pathology but might weave in osteopathic foundation into the question as well. Ultimately, USMLE and COMLEX-USA are very different exams.

I was actually, and thankfully, in the position where I was able to decline several residency interview invites because I had sufficient COMLEX-USA scores, despite not having a USMLE Step 1 score. This isn't talked about very much, and I want to bring attention to it, with the hope that our profession can have more osteopathic students matching into competitive specialties with just COMLEX-USA. At first, I felt embarrassed that I didn’t have a USMLE score when applying to a competitive specialty—I felt like I was navigating uncharted territory as an applicant, and it really shouldn’t be the case. I want to be a voice for the students coming after me to know that they can do this with just COMLEX-USA scores.

I also ultimately decided that what I wanted in a residency program was for them to respect my licensing exams as they do the MD examination. DO students shouldn’t have to pay for and take double the amount of board exams to be on an even playing field. Why should residency programs require me as a DO to take both? I am really proud of my osteopathic education and I want residency program directors to acknowledge and respect the decision each of us make to become a DO.

I do think that we are heading in the right direction towards eliminating DO bias, but we still have a long way to go. I hope to see continued acceptance of DOs in graduate medical education across the country, which hopefully allows a greater number of students to match into competitive specialties with only COMLEX-USA scores.

We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you personally?

Our osteopathic training allows me to have a larger understanding of the structure and function within the body, and I think it's so important to be able to have this foundation because it can drastically change someone's life by recognizing that they might not have a disease or a diagnosis that has to only be fixed with medication. I might have a patient that comes into the emergency department with back pain, and I've ruled out all of the other acute pathologies. If I think it’s musculoskeletal, they could really benefit from osteopathic manipulative treatment—such as myofascial release—and walk out of the emergency department feeling a lot better. I want to use osteopathic manipulative treatment when I can to be able to aid the body in healing itself, and I'm so proud to be a DO.

Looking back at how you prepared for your most recent level of COMLEX-USA, would you have taken a different approach to studying? What advice do you have for other COM students who are preparing for COMLEX-USA?

My most recent COMLEX-USA exam was Level 2-CE, and I plan to take the same approach when studying for Level 3. I learned from a lot of mistakes I’d made when prepping for Level 1, and did much better on Level 2 because I’d fixed my approach to studying.

My level one preparation period was focused on passive learning—reading a lot of material versus active learning, such as using practice questions or flashcards, which utilize spaced repetition. You’re not going to be able to read the same textbook five times, but you’ll be able to use a variety of practice questions to ingrain the same principles from them. On top of that, it can be easy to have a resource overload—so be sure to focus on only two to three. That said, I took the active learning approach for Level 2 during a much shorter time period, and it was a huge improvement.

The Road to DO Licensure has many unexpected challenges that I’m sure you’ve experienced, including burnout, stress, and anxiety. Tell me some of the major challenges you’ve faced while taking COMLEX-USA and some advice on how you personally overcame them.

The first time I took COMLEX-USA, it was a challenge being able to focus for the length of the exam while also operating under the time constraints. These exams are timed, and it’s necessary, so we as students need to figure out how we can do our best under these constraints and focus our practice efforts towards preparing for that. I would take four-hour practice exams, and if I could maintain my focus for that amount of time, I knew I could likely extend it the rest of the way. For example, when someone is preparing to run a marathon, they don't practice by running all 26 miles; if they can make it to mile 16, they're likely going to have the stamina to be able to endure until the end.

The dedicated board prep periods are so challenging; they're full of anxiety; they're full of stress. During them, I had the inclination to say ‘no’ to my family and friends if I got a dinner invite. But during my Level 2 prep period, I decided to start saying ‘yes’ instead, and I had better balance because of it. I think that balance allowed me to enter into my exam with a much healthier outlook. My performance also reflected this.

What are you looking forward to the most in the next stage of your journey?

There are so many things I'm looking forward to because I'm finally going to be a resident physician! I would say, ultimately, I am excited to finally apply all of these years of hard work. As an emergency medicine resident physician, I'm looking forward to being able to connect with my patients of all different backgrounds and cultures while treating and advocating for them in their most vulnerable moments.

See All

You were an ICU nurse before you decided to become a doctor—what inspired you to become a DO, specifically?

I was inspired by the osteopathic tenets of mind, body, and spirit being applied to the treatment of patients. This was similar to what I’d already learned during nursing school—you don’t take care of a patient; you take care of a whole person. This is was what I wanted to do at a much higher level, but because I am so big on spending time with my patients and I enjoy being there for them, I didn’t want to lose that in my decision to become a doctor. I felt that by choosing to become a DO, I would be able to maintain that same practice for my patients—that I would always be there for them.

On top of that, DOs also learn OMM. When I discovered this, I thought it was a more hands-on way to take care of somebody without having to rely heavily on medications. Just being able to use your hands to diagnose and treat someone is amazing. Just touching a patient can go a long way. This also aligns with my ideas of prevention because DOs do our duty to avoid prescribing meds if we don’t have to, or if we do, making sure to offer and incorporate non-pharmacologic treatments. OMM is a great way to take care of people that can improve their quality of life with minimal risks and expenses. I wanted to learn that skill so I could become a more well-rounded physician and help patients to the best of my ability.

What mental challenges are you facing as a 2nd-year osteopathic medical student as you prepare for your first level of COMLEX-USA and how are you addressing them?

The sheer amount of information that we need to know can easily be overwhelming—just looking at the big picture. What I realized I needed to do was to break down all of that information into chunks a little bit at a time. I need to understand the bigger concepts before digging down to the smaller details, and then I need to recognize where my weaknesses are so I can go after them. I’ll spend more time on those weaknesses than I will on the things I already know, prioritizing my study time. Also, doing tons of questions—they will show you what’s important and how your brain works—how you’re thinking and if you know why you got something right or why you got it wrong. If you think you’re going to memorize nine text books of medical knowledge, then good luck, but take comfort in realizing that you don’t have to know everything; you just need to know enough. What will make you one of the best doctors is being compassionate with your patients. No matter what anyone does for a living, I think you need to recognize that you are interacting with another human. Empathy should be everywhere—it will make the world a better place.

Also, setting goals is important. I set a daily goal for the number of questions I will do that day. Once I hit that number, it’s over—I’m clocking out! You have to have those breaks in between and not try and do it all in one sitting. Do a set of questions and then take a break to just relax or do something you enjoy. Then, do another batch, and before you know it, you’ll have reached your goal and it wouldn’t have been that tolling.

Although it varies from person to person, what do you feel is most important to a medical student’s mental well-being and why?

You need to do things for yourself. And that may sound selfish, but from what I experienced, taking care of yourself might actually be one of the most unselfish things you can do. Because when you’re in a better place, you’re more able to be there for others. I think if you take care of yourself by making time to exercise on this day at this time or decide that at seven o’clock, you’re going to meditate for 10 minutes no matter what, you’ll keep burnout at bay and find yourself thriving. It’s easy to get caught up with grades and other worries, but I think if people realize that taking these little timeouts for themselves is not going to negatively affect their scores; bur rather improve them, they would be much more likely to do them on a regular basis. There’s a myth that more studying equals better performance, but that may only be true to a certain degree. I believe in quality over quantity, and performing better because you physically and mentally feel better. I think taking some pressure off yourself, especially when you’re always feeling you have to achieve at such a high level, is extremely important.

I hope students can carry this mindset into their practice as physicians. If you adopt—that sure there is school, there is work, but there is also me, my friends, my family, and my life—you will be happier and better for it. I have both witnessed and been someone who just works all the time and is always under this certain level of stress…it’s not exactly the recipe for providing excellent patient care. You’ll find yourself being fatigued, not as sympathetic or compassionate, and even may not be as cognitively present. It becomes easy to miss things that otherwise you would have caught when you’re burned out. It’s a valuable practice to take time for yourself, and I hope it carries over to every facet of your life.

VCOM featured you in an article about practicing gratitude and its positive impact on mental health. What other practices, both mental and physical, do you find helpful for improving wellness while preparing for COMLEX-USA? What advice would you give to other COM students?

For your physical health, you definitely need to be making healthy food choices, drinking lots of water, and you absolutely should be exercising. You don’t have to do them 100% of the time, but if you can pull them off on a regular basis you will still reap the rewards.

As for your mental health, I recommend some form of meditation. And don’t worry about whether or not you’re doing it right— I love the quote: “Meditation, if you're doing it, you're doing it right.” But, especially when you first start out, I think it's easy to get frustrated if your mind wanders off or you internally feel like you're messing up. I would tell anyone that they should expect for this to happen as part of their progression in learning about themselves. You’re only human and it’s natural. Don’t judge yourself when that happens, and instead, use it as a tool to practice self-acceptance and bring yourself back to the present moment.

The goal of meditation isn’t about not having thoughts—the goal isn’t about not having your mind wander; the goal is just to be present in the moment and to be able to let go. Just be there. There is so much time in our day where we are bogged down by work or distracted with other things. I think meditation is an opportunity to practice awareness and “be here now.”

One of the most important things you can do is give yourself permission to relax. A friend of mine, at one point, was constantly quizzing himself so much that he couldn’t sleep. When he came to me with this issue I told him, “Man, you already study for 13 hours a day. It’s okay to go to bed, but you gotta give yourself permission to not think about school.” If you’ve studied 12 hours a day for X number of days for your exam, you’ve already put in the work. Being able to let go and give yourself permission to relax comes from trusting yourself, trusting your work, and trust that you’ve put in enough time.

For me, I anticipate my success on the exam because I know I busted my butt. Now I can relax—now I’m going in to get the reward for all my hard work. And if you’re not succeeding, know you need to go get help. I always believe in asking for help, guidance, advice, or coaching no matter what. From there, you can reformulate how you study.

Work-life-balance is important for every field, how do you recommend osteopathic medical students start now to be sure that they maintain this throughout their career?

This is definitely tough because we do have a lot of pressure. We have school, we get involved in organizations, we do volunteer work, etc. Work-life-balance was definitely something I struggled with, also because I hadn’t gotten involved in many things until going into med school. I wasn’t used to juggling everything.

The best thing you can do is maintain your relationships and stick to your hobbies or find new ones. For me, it’s exercising, hanging out with friends, meditating, learning a new language, NASA/space, and I even had a guitar that I hadn’t touched in 12 years that I just picked back up. I know others who jumped on instruments too. I feel like no matter what you’re passionate about, you need to make time for those things. It’s so important to your mental health and well-being. If you just make time to study for 13 hours a day, then that’s all you’re doing. Having those personal aspects to your life will allow you to succeed more in school and clinically (especially for connecting with patients), but also enjoy yourself and stay who you are.

See All

What inspired you to become a DO, specifically? Tell us your story.

I was a chemist before going into medical school. I’d finished my undergraduate degree at the University of Florida, and I worked for a company in Savannah, Georgia. I was very unhappy, all over the place, and didn’t know what I wanted to do. I dabbled in many things, including being a high school teacher—loved mentoring kids or anything with a group of people—but I hadn’t found my calling. I even earned my Master's degree in molecular medicine. A couple of my friends were in medical school—they were learning pharmacology and I was looking at their material and found myself interested, even enjoying studying the topics with them. That is when they said to me, “Why don’t you go to med school.” I’d never even thought of that. I like people. I like science. I like mentoring. That’s what a physician is and should be—it made sense, so I decided to apply to medical school.

We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you personally?

I was a scribe in the emergency room and met four amazing DO physicians. When I worked with them, I felt like I wanted to be like them. They sat down on the bed of the patient, they held their hand—didn’t even take their computers inside the room. The patient would always laugh, smile, cry—they would tell us things that would help us diagnose them. In just the way those DOs would connect with their patients—I was inspired to go into osteopathic medicine. That’s the main reason why I only applied to DO schools.

Looking back at how you prepared for your most recent level of COMLEX-USA, would you have taken a different approach to studying? What advice do you have for other COM students who are preparing for COMLEX-USA?

COMLEX-USA questions include a lot more detail, are more person-oriented, and have answer choices that are very non-invasive. Every answer choice on every question still tested my scientific knowledge, but I feel as though COMLEX-USA also tests the concept of patient-centered osteopathic medicine in a way that 2nd years can understand.

I was actually surprised by how well I scored. The only regret I really had was inundating myself with too many study resources. I think if I just stuck with two or three and not listened to everyone saying, “This is better than this and that’s better than that,” then I would’ve been more selectively driven. Therefore, I would say to try and find those two or three study resources that work for you and stick with them. Also, do a lot of questions and practice exams. I did a lot because I'm a very anxious test-taker; I don't like taking exams. There’s a lot of pressure.

My best piece of advice is to try to remember that it’s just a test. If you’ve put in all the work prior to taking it and you give it your all, the results will come. You really just need to focus on what you're doing while you're studying—focus on that test, and be as relaxed as possible. I approached each question like I’d treat a patient. Because they are patients; they’re vignettes. I don’t want to misdiagnose my patient so I’m not going to get this question wrong. It’s true!

My final piece of advice I'd give students is, when you're taking practice tests or practice questions, you need to simulate the environment of where you're going to take your exam. Go into a quiet room, have a desk, take it with a similar keyboard, similar mouse, and sit with good posture. Get a chair that's similar to what they have in Prometric testing centers and sit in it as if you're taking the real test every single time, every question. With the right body language, you self-perceive yourself as more confident.

The Road to DO Licensure has many unexpected challenges that I’m sure you’ve experienced, including burnout, stress, and anxiety. Tell me some of the major challenges you’ve faced while taking COMLEX-USA and some advice on how you personally overcame them.

My first two years of school were rough. I was a non-traditional student- I was out of school for three years prior to coming in. I didn't have that study routine down my first year, but I didn’t let that stop me from being who I was. I still joined SOMA. I engaged in extracurricular activities, and I still do—that's what gives me life. It was a struggle to find that rhythm my first year, and there were tears, there were long nights, there were times where I had to tell family, “I can't talk to you today.”

There was a lot of sacrifice—I even had to miss my niece being born. I had to shut people down at the time and ask for forgiveness later. My family knows I love them, but they struggled to see the path I had to take. I’m a 1st generation; my mom came from Mexico and my dad came from Iran. They didn’t go to college—I’m the first. Because of this, I had to be very self-motivated and I had to work my own way through college with four jobs. I worked as a shoe salesman, a waiter at a Mexican restaurant, a barista and a chemistry tutor. My mornings started at four in the morning, working at a coffee shop in the atrium of Shands Hospital, University of Florida. This was all while still being a full-time student.

I also encountered a great deal of financial strain during my first year, almost requiring me to pull out of medical school, but members of my community helped me. I regularly go to the gym to help with stress, and friends there noticed how I was struggling. Without a word to me, they collected donations from community members. They told me I was surrounded by friends: lawyers, dentists, and DO physicians who wanted to help and encourage me. They told me, “We don’t want this paid back. You’re helping our community, and we know what you as a DO physician are doing. We see a light in you and we want to keep that on.” It gives me chills thinking about it. I have been blessed to encounter so many wonderful people on my journey.

I also took my board exams during the roughest period of the pandemic in June, 2020. COVID was messing up all the schedules. The stress level, anxiety, and the trauma of repeatedly having to be rescheduled was extremely high. My exam was pushed back from June to July to the end of July—the day after my birthday. It was stressful, but I looked at it and said to myself that it was more time to study. That extra month gave me the composure to be able to sit down and say this is just a test.

Have you encountered other mentors that have helped get you to where you are today?

One mentor I had was Mr. Timothy Wall, in eighth grade. I was getting C’s and D’s the grade before, but he took one look at me and told me that I could do more. Meanwhile, I couldn’t read because I was a product of immigrants, and learned only broken English. I know four languages. He took me under his wing and guided me. He turned me from a C and D student who started in ESL courses to an honors student who graduated as valedictorian of my eighth grade class.

I want to continue mentoring because of the effect that my mentors have had in my life. I want to continue to give back in the same way.

You were appointed to the Special Commission after your nomination by SOMA—tell us what this means to you.

My involvement with National SOMA began with my desire to contribute to my profession and ensure the student voice will be heard. As I have progressed through my years in medical school, I have witnessed many events that have affected osteopathic schools, students, and the profession as a whole. This appointment grants me the opportunity to ensure that the perspectives of the osteopathic student body are represented during discussions that will impact their future. Moving forward, this appointment will aid in providing me the experience and resources to be an osteopathic physician who confidently advocates for our profession.

What are you looking forward to the most in the next stage of your journey?

My patients. I can’t wait! Every time I think about my future, I think about my patients and the people I’m going to meet—the minds and the hearts that I’m going to touch and change. Hopefully, sharing my story will show others that there is light at the end of a very long tunnel.

See All

MATCH 2023

I believe the hands on exposure I received during my medical school days will help me better assess my patients and sharpen my clinical decision skills.
Akbarali “Akbar” Nabi, Touro University Nevada College of Osteopathic Medicine
Matched to: Physical Medicine & Rehab at the University of Utah

I only took COMLEX-USA. I believe the decision to take only COMLEX aligned best with my educational and career goals. I am happy to talk with any students who may have questions about that decision.

I believe my osteopathic training and skill set have prepared me to treat each patient as the whole, multidimensional person that they are. I am bringing with me to residency a patient-centered mindset and a pair of hands well trained in osteopathic manipulative medicine, which will allow me to identify the root cause of a patient’s concern and deliver high quality, personalized healthcare to them while conveying sincere empathy in both word and action.

Taking only COMLEX-USA was important for me as a proud osteopathic medical student because I wanted to take a test that would evaluate my osteopathic manipulative medicine knowledge so that it remains a core part of the medicine I study and practice. In an ever-changing medical landscape, I find it of the utmost importance that DOs continue to advance and value that which makes us uniquely valuable to our patients and the institutions in which we practice.

Osteopathic principles are not limited to the use of OMM. No, they are, in fact, much more foundational to the practice of medicine in that they bring to the forefront the necessity of treating the patient who has pathology rather than simply treating the pathology itself. Continuing to develop truly Osteopathic curriculums in medical schools and valuing/optimizing the ability of COMLEX-USA to assess a student’s understanding thereof is critical to the advancement of the Osteopathic profession.

Trent Griner, South Georgia Campus-Philadelphia College of Osteopathic Medicine
Matched to: Family Medicine at Atrium Health Floyd Family Medicine Residency Program

My osteopathic training has taught me to see my patients as more than a disease process. To always have a mind of curiosity that strives to understand “why”. And to never stop showing kindness and compassion to every person I meet on this journey.
Abenaa Asare, Edward Via College of Osteopathic Medicine-Virginia
Matched to: Psychiatry at Virginia Commonwealth University (VCU)

Bailey Quinn, Touro College of Osteopathic Medicine–New York
Matched to: Pediatrics at University of Connecticut

I think the osteopathic training I received at MSUCOM and the support I received from [the American College of Osteopathic Surgeons] (ACOS) especially helped me achieve my dream of being a heart surgeon. The personal support and encouragement I received from ACOS was invaluable, especially from Dr. Kristen Conrad-Schnetz and Dr. Tyler Wallen, both proud DOs, on how to thrive as an osteopathic student pursuing surgery.

My advice is to find a mentor and a sponsor, which are very different, that truly supports and understands their needs, and then to work really hard based on the feedback from those people. Don't be afraid to change course if something isn't working, but don't give up hope. It gets easier as you learn more about the process.

Zach Brennan, Michigan State University College of Osteopathic Medicine
Matched to: Integrated Cardiothoracic Surgery at Cedars-Sinai

Knowing that every person is a unit of body, mind, and spirit, I will strive to be as compassionate and comprehensive as possible in my approach to imaging. Also, we learned plenty of useful anatomic landmarks in OMM lab!
Akito Theodore Nicol, New York Institute of Technology College of Osteopathic Medicine-Old Westbury
Matched to: Diagnostic Radiology at Arnot Ogden Medical Center

The most important lessons I will take into my residency training is the focus on the patient as a person and emphasis on a hands-on, patient-centric diagnosis and treatment approach. That is what it means to be an osteopathic physician.
Aaron Byczynski, West Virginia School of Osteopathic Medicine
Matched to: Internal Medicine at the Medical College of Wisconsin in Milwaukee, WI

I was lucky enough to get 20 interviews. I chose FSU as my number 1 because of an amazing culture and camaraderie between the residents. I believe that going through an osteopathic training for me has really helped me better understand the pathophysiology and how an injury to one part of the body can have a downstream affect on a different part. Furthermore I love having an extra set of tools to treat patients through OMT. This will be super helpful for me as an Intern.
Aman Khurana, Lincoln Memorial University Debusk College of Osteopathic Medicine
Matched to: Internal Medicine at Florida State University Sarasota Memorial Hospital

My osteopathic training prepared me immensely in being able to look at the whole body, the human condition, and the holistic lifestyle, which I hope to use in my career as a psychiatrist!
Devan Baijal, Kansas City University College of Osteopathic Medicine
Matched to: Psychiatry at Atrium Health-Carolinas Medical Center

As a former collegiate athlete, I longed to replicate the team environment, adrenaline, and camaraderie that I miss so much about competing. Emergency Medicine is perfect for this desire. I am so appreciative of the education I received at PCOM South Georgia. My training in osteopathic medical education prepared me with a sound foundation in anatomy and “hands-on” experience that is crucial to treating patients in the ER. Furthermore, I applied with COMLEX only! Proud to be a DO and excited for the challenge ahead!
William Brewster, South Georgia Campus-Philadelphia College of Osteopathic Medicine
Matched to: Emergency Medicine at University of Kentucky Medical Center

It is possible to match with only COMLEX. I had plenty of interviews despite a lackluster score. I also knew my strengths and standardized exams are not one of them so I did not attempt USMLE. Emergency Medicine is one of the more DO friendly specialties and COMLEX is acceptable for the vast majority.

That being said, I am glad I only took COMLEX and would not have changed things.

I believe my osteopathic training will be an asset as a resident and attending physician because we are trained to see things differently. We go beyond what is directly in front of us. I have used my OMT with patients while on rotation and several patients have felt immediate relief to some degree. Patients will receive hands on care and my training will be useful in so many cases

Kevin Cabrera, Arizona College of Osteopathic Medicine of Midwestern University
Matched to: Emergency Medicine at Kaiser San Diego

It's amazing to see fellow osteopathic physicians serving alongside our MD colleagues in the military healthcare system. When I was at William Beaumont last summer, I met residents who had graduated from LECOM, TCOM, ATSU-KCOM, PNWU, and others. They worked side by side with MDs to provide care for active-duty and retirees. We were able to share our osteopathic knowledge to help patients on the wards and enrich our coresidents who may not have had prior exposure to osteopathic medicine. These techniques provided additional treatment modalities and helped us provide holistic, patient-centered care.
Ezra Yu, A.T. Still University–School of Osteopathic Medicine in Arizona
Matched to: Internal Medicine at William Beaumont Army Medical Center

My osteopathic training emphasized treating patients holistically which has always been in line with how I envisioned myself as a future physician. Looking at a patient from every angle, for example paying attention to social determinants of health is super important to me because I am a Bronx, NY native who saw these determinants affect so many people I knew personally! I truly love “the mind, body, spirit” tenet and it ties so well into Psychiatry because the mind is so incredibly important to keep healthy! The fact that I will be going into residency with this mindset will reflect strongly in my patient interactions, how I communicate, and assist my overall passion of wanting to make a difference for people in the community as well as disadvantaged populations.

I only took COMLEX-USA. The main thing for me was financial reasons. These exams are so expensive and I just didn’t have the finances to pay for another board exam so I worked on strengthening myself in other areas in case there was any doubt from people looking at my app on not taking both exams. I was also fortunate enough to know an alumni from my school who matched into Psychiatry and she only took COMLEX so I knew it could be done!

Kiyana Harris, MS, Georgia Campus–Philadelphia College of Osteopathic Medicine
Matched to: Psychiatry at the Morehouse School of Medicine

I think my osteopathic training will be extremely beneficial to me in the future as an aspiring sports medicine physician. I believe that my training in various OMT techniques and my extensive knowledge of the musculoskeletal system that I gained in osteopathic medical school will give me a great advantage when practicing sports medicine.
Hailey Chapin, Lake Erie College of Osteopathic Medicine
Matched to: Family Medicine

My education at Pikeville taught me to approach each patient in a holistic manner and treat the person rather than their symptoms. The hands on training I received will help me become a better physician, promote healing and restore mobility.
Morgan Turnow, University of Pikeville Kentucky College of Osteopathic Medicine
Matched to: Orthopedic Surgery at Ohio Health - Doctors Hospital in Columbus, OH

My osteopathic training has taught me to take a more holistic approach to taking care of patients. I feel more confident in my ability to consider alternative or more conservative options before jumping to surgery or medications. Additionally, coming to a MD-heavy program, I hope to be able to share some of my osteopathic skills and knowledge with my co-residents.
Nancy Couturier, West Virginia School of Osteopathic Medicine
Matched to: Pediatrics at University of Michigan

Studying osteopathic medicine has provided me an additional way to connect with and care for patients. I look forward to using my osteopathic skills to assist in the diagnosis and treatment of various ailments.
Rachel Fletcher-Slater, Rowan-Virtua School of Osteopathic Medicine
Matched to: Pediatrics at Brown/Hasbro Children’s

Osteopathic training encompasses a well-rounded approach to medical care that has shown me to always see the patient for who they are beyond their symptoms. The osteopathic lens taught me to recognize that symptoms do not necessarily happen in isolation and encouraged me to adopt a multidisciplinary approach to tackle a condition at its roots. I believe these qualities will allow me to provide a high level of patient care that makes them feel seen and cared for!
Rohan Yarlagadda, Rowan-Virtua School of Osteopathic Medicine
Matched to: Adult Neurology at Georgetown

I only took COMLEX. My training will allow me to treat everyone and provide holistic care to my patients and community. I am confident PCOM-GA prepared me to be successful in residency and my future career
Setu Patel, Georgia Campus-Philadelphia College of Osteopathic Medicine
Matched to: Family Medicine at Wellstar Kennestone Medical Center in Marietta, GA

With an emphasis on whole person healthcare, my osteopathic training will help me see the patient beyond just their signs and symptoms.
Stefanie Worwa, A.T. Still University-Kirksville College of Osteopathic Medicine
Matched to: Internal Medicine

I think my osteopathic training showed me how important it is to treat patients as people. I also feel as if our OMM hands perform physical exams second to none, and I have received complements reflecting that. We have a unique skill set that I am very excited to offer!
Zachary Bloom, West Virginia School of Osteopathic Medicine
Matched to: Family Medicine at Rutgers RWJ Residency at CentraState

I am eternally grateful for my institution’s phenomenal faculty/staff and my amazing mentors. I feel confident that my years of osteopathic medical training have endowed me with the tools needed to not only take on complex medical cases but also render humanistic care.
Odolphe Augustin, Touro College of Osteopathic Medicine-Middletown
Matched to: Internal Medicine (Primary Care) at Johns Hopkins - Bayview

I will be successful in residency because I have and will continue to utilize the osteopathic philosophy/tenets when treating my patients.
Nikoleta Golemi, University of the Incarnate Word School of Osteopathic Medicine
Matched to: General Surgery

I only took COMLEX-USA for Level 1 and 2-CE. I chose an osteopathic education because of my passion for primary care. Before medical school, I shadowed a phenomenal DO (Dr. Tricia Hall - neuromusculoskeletal medicine/FM) and I was amazed at how she could address pain and patient concerns with her hands. Through residency training, I hope to continue building my skills so that I can serve my patients in this way.
Logan Lawson, Philadelphia College of Osteopathic Medicine
Matched to: Family Medicine

MATCH 2022: MATCH 2022

MATCH 2021: MATCH 2021

I matched Emergency Medicine at The Ohio State University and could not be more excited to have years of hard work culminate in this achievement. I hope my fellow DO students always remember that they are amazing, smart, and capable of anything they set their mind to!
Melinda Kizziah, West Virginia School of Osteopathic Medicine
Emergency Medicine at The Ohio State University

I matched at my #1 UAMS South Central! God’s plan never fails! I am beyond grateful! I get to serve my home! The very place I was born, where it all began for me, Pine Bluff, AR. I am overwhelmed and honored to serve my hometown. I promise to do my best to make everyone DO proud!
Asia Colen, New York Institute of Technology College of Osteopathic Medicine
Family Medicine at UAMS South Central

I'm so thrilled to have matched in Emergency Medicine at Trident Medical Center in Charleston, SC! Participating in the 2020-2021 Match cycle presented many unique challenges to an already stressful process. I'm extremely thankful to have my advisors and mentors by my side from audition rotations, to navigating applications, and troubleshooting virtual interviews! Onto the next big adventure!
Andriana Saric, Arizona College of Osteopathic Medicine
Emergency Medicine at Trident Medical Center

As a DO physician and future Physiatrist, I recognize the holistic approach plays a pivotal role in patient care, especially in the world of PM&R. Often times it’s the emotional and or psychosocial components of an individual that contributes and or worsens the physical ailments and vice versa. My osteopathic education has given me this perspective to help improve my patient’s quality of life and be at their best!
Evan Arbit, Michigan State University College of Osteopathic Medicine
Rush Physical Medicine and Rehabilitation

Don't discount yourself from applying to programs even if they have only a few DO residents! You've worked hard and put in the work; you are equally as qualified as our MD counterparts. Show them what you've got!
Alyka Glor Fernandez, Kansas City University
Emergency Medicine at Emory University

When there's a passion, there's a way. I am so incredibly honored and humbled to be the 1st PCOM student to match into cardiothoracic surgery, the only DO student to match into cardiothoracic surgery this year, and the 1st DO to match at the University of Michigan CT Surgery. I can't wait to apply my osteopathic medical training to treat patients holistically within the field of cardiothoracic surgery!
Chi Chi Do-Nguyen, Philadelphia College of Osteopathic Medicine
Cardiothoracic Surgery, University of Michigan

MATCH 2020: MATCH 2020

My Osteopathic education prepared me so well for a residency training in Physical Medicine and Rehabilitation. I could not be more grateful for my school and the physicians who mentored me along the way.
Nicolet Finger, UNTHSC-TCOM
Physical Medicine and Rehabilitation at UT Health San Antonio

The road to get here was surely a rough one, but nothing worth doing in life is meant to be easy. At the end of the day I had faith that my perseverance through the hard times would pay off, and today I found out that they did! To all the future physicians out there, NEVER GIVE UP. The world needs your healing.
Brynne Hunt, WCUCOM
Emergency Medicine at Newark Beth Israel-Newark

I matched to my number one spot! I’ll be a family medicine resident at Saint Joseph Health System in Mishawaka, Indiana. I am so blessed to be able to be a DO and serve the community that raised me! Here is a picture of my cat and me celebrating during quarantine lol. We had a dance party!
Jess Williams, KCOM
Family Medicine at Saint Joseph Health System

I am so grateful to have trained to become an osteopathic physician. I matched into my number one choice for residency in internal medicine! Thankful for all my friends I made along the way and the physicians who mentored me!
Jacob Baer, KCU-COM
Internal Medicine at University of Kansas

I am able to fulfill my dream of being an emergency medicine physician! I am thankful to my school for all the opportunities afforded to me and the people in my life for support! Looking forward to being a DO in the emergency department.
Michael Skaletsky, MU-COM
Emergency Medicine at Doctors Hospital Columbus, OH

With all the craziness that is going on right now, it was great to see how everyone rallied together to celebrate match day virtually. We have all worked so hard to get to this point and I love seeing all of our dreams become a reality.
Amber Hartman, KYCOM
Pediatrics at SIU - Springfield, IL

At Cook County Hospital there’s a plaque that says: 'One doesn’t ask of one who suffers, what is your country and what is your religion? One merely says you suffer. This is enough for me. You belong to me and I shall help you. - Louis Pasteur' This philosophy aligns perfectly with the values we learn at CCOM. Proud to be a part of #DoctorsthatDo and match at my top choice!
Palak Patel, CCOM
Internal Medicine at Cook County

My DO medical education introduced me to mentors, clinical experiences, and a revitalized passion to assist struggling rural locales like my own hometown. I am over the moon about matching into my top choice in family medicine at at NH-Dartmouth Family Medicine Residency in Concord, NH, which offers a rural track that I will carry with me for the entirety of my career in rural family medicine.
Clare O'Grady, NYIT-COM
NH-Dartmouth Family Medicine Residency

My osteopathic medical education provides a unique vantage point into mental health treatment. Residency program leadership also saw this as a valuable asset when assembling their incoming intern class. I am proud to be a DO and will utilize my experiences as a future research psychiatrist!
Grace Sydney Pham, UNTHSC-TCOM
Psychiatry/Research at Baylor College of Medicine

I am beyond thrilled to have matched into Internal Medicine and cannot wait to begin my career as a doctor! The knowledge gained, the experiences lived, and the relationships formed throughout medical school will stick with me as I continue to grow as a physician. To all of my 2020 peers, we made it!
Maxwell Horowitz, TouroCOM NY
Internal Medicine at Mt. Sinai Icahn SOM St. Luke's-West

I could not have picked a better medical school to grow and mature into a young doctor. I am thrilled to have matched into an OB/GYN residency at my top program, St. Luke's University Health Network. I look forward to continuing to pave the way for other DOs!
Kathleen Ackert, PCOM
OB/GYN at St. Luke’s University Hospital in Bethlehem

I think the key to getting a good residency is to go and audition at that location. If you can show programs a good work ethic, coupled with your unique osteopathic education, you might be surprised at how many doors can open up for you.
Phillip Bennett, RVU-COM
Pathology AP/CP at the University of Utah

Matching into my top choice residency is a dream come true! I am incredibly grateful for my medical school peers and physician mentors for helping me build self-confidence and supporting my intellectual, personal, and professional growth throughout this journey.
Priya Shah, CCOM
Emergency Medicine at Duke University

I have loved every moment of this crazy journey through medical school, and I am eager to use the knowledge I have gained as an osteopathic medical student. A quote I once heard was "if you work hard, you'll get lucky" and I am so lucky to have found a place among the amazing physicians who work in Family Medicine. I can definitely say they are my people! I matched at Wright State University in Dayton, Ohio and I can't wait to begin the next 3 years of training.
Megan Miller
Lincoln Memorial University-DeBusk College of Osteopathic Medicine
Family Medicine at Wright State University

Beyond excited to match at Detroit Receiving for Emergency Medicine. I am extremely excited to be a DO in the Emergency Department.
Rajiv Varandani, CCOM
Emergency Medicine at Detroit Receiving

Attending an osteopathic medical school has equipped me with a unique skill set and has provided a one of a kind training experience that will be invaluable to my future role as a physician. The journey was challenging, but with faith, dedication, and support, I made it through! Always remember, your darkest and most difficult moments are never wasted; they simply prepare you for your destiny. I cannot wait to apply all that I have learned to serve and make a positive impact on my patients- the osteopathic way
Anna-Kaye Brown, RowanSOM
Anesthesiology at Temple University Hospital

Incredibly humbled to be the only DO this year to match in Cardiothoracic Surgery I6. Prior to medical school, I worked as an LPN in Cardiac Surgery for 5 years ,while taking night classes dreaming of this day. Thank you very much to the AOA, AACOM, NBOME and WVSOM for all your support and giving this nurse an opportunity to realize his dreams
Jason Gilbert, WVSOM
University of Kentucky in Cardiothoracic Surgery

MATCH 2019: MATCH 2019

Don't think of time spent studying for COMLEX-USA and COMAT exams as time lost. It's an investment in your future career and in your ability to match into your specialty of choice.
Jason Rodriguez, DO
Kettering Health Grandview Medical Center, Anesthesiology

Match can be intimidating, but being an active learner through research, conferences, volunteering, and working hard in preparing for COMLEX-USA, truly pays off. I matched into my #1 choice!
Alexandra Digenakis, DO
University of North Carolina, Emergency Medicine

Medical school and residency applications can cause you to doubt your abilities. Remember that hard work pays off. Have faith in your abilities and push through. You are amazing and can achieve your dreams.
Parth Gandhi, DO
Delaware Christiana Care Health System, Combined Emergency Medicine/Internal Medicine

I matched into my #1 choice! Don't be afraid to step out of your comfort zone and take advantage of new opportunities. Medical school is a time for personal and professional growth, and while grades are important, programs pay attention to other things too.
Jordan Hauck, DO
Medical College of Wisconsin Affiliate Hospitals, Obstetrics and Gynecology

I was able to match into my #1 program! I'm thrilled to be joining the Georgetown team and pursuing diagnostic radiology in Washington DC!
Matthew Bourne, DO
Georgetown University, Diagnostic Radiology

Work hard to shine during your sub-internships and interviews, but most importantly, be kind, positive, and a team player.
Pauline Terbio, DO
University of Nevada Las Vegas, Emergency Medicine

When it comes to residency, the most important thing is to find the right fit for you. Something I considered important was finding a place where I would be happy training. I found some programs more welcoming than others, and I loved the one I ended up in!
Tiffany Sinclair, DO
Franciscan Health Olympia Fields, General Surgery

Put 100% into everything you DO, and it will pay off. Studying hard and surrounding yourself with friends and family that support you is key -- and got me where I am today.
Khadija Farooq, DO
Presence St. Joseph Hospital, Family Medicine

I matched into my #1 choice! I was searching for a program that allowed me to incorporate my love for public health and community health. This one had it all!
Khyrista Valentine, DO
Authority Health Residency Program, Pediatrics

As DOs, we think holistically. As residency candidates, we have to be the same. Be well-rounded on every part of your application: COMLEX-USA, extra-curriculars, volunteer experience, and who you are as a person.
Kayla Prokopakis, DO
St. Elizabeth Boardman Health Center, Emergency Medicine

I wanted to be somewhere that allowed me to help the community, and would also make me the best doctor I could be. All my dreams came true and I matched at my #1 choice!
Hajiraj Ishaq, DO
Doctor's Hospital Ohio Health, Emergency Medicine

My fiance and I were both going for the same specialty in the same year. We couples-matched in pediatrics and will move from being classmates to being co-interns. All of my hard work was worth it!
Este Marks, DO
Oklahoma State University, Pediatrics

I'm very proud of my osteopathic medical training and COMLEX-USA best reflects this. It was important to me to find a program that valued my osteopathic roots as much as I do.
Victoria Starr, DO
Beaumont Hospital Obstetrics & Gynecology

Keep your mind open and trust the process. You might be surprised sometimes what you want initially changes when you stay open to possibilities and explore new things.
Joseph Schreiner, DO
Jefferson Health New Jersey Emergency Medicine

See All

Option	Features
In-person proctored	Highest exam security with proctors on campus or at rotation sites
Remote-proctored	Provides exam security by COM proctors monitoring remote administrations
Self-proctored	Most flexible for location and timing but least secure

How did you know that the programs you ranked were right for you?

I ranked the programs that I knew I would be happy matching into—that’s the best you can do in a process that’s ultimately out of your control. My situation was unique since I rotated at my top ranked program as a third and fourth year student, and I knew it would be a good fit since I had a great experience there. Ultimately there is no one formula for how to rank programs. Personally, I used the same criteria for all my program, location and the people or the overall “fit” of the program. Residents change every year but program structure and faculty are less likely to turnover as frequently. If you don’t think you’ll be happy there based on your non-negotiables, intuition or general impression then don’t rank it. It’s also important to consider whether or not your interests will be supported. For instance, are there faculty or recent graduates who’ve done fellowships, additional training or research in their areas of interest? Is there support in pursuing new or away electives, scholarly activities, or responses to major life events (births, deaths, illness, etc.)? What are their wellness initiatives, and do the residents seem to have good work-life balance? Have residents felt supported during the pandemic (rotation changes and response, PPE)? These are all important factors to consider when finding the programs that are right for you.

What factors from your interview were the most meaningful in affecting how you created your rank list?

The main factor I considered was whether or not I could imagine myself at the program. I considered how the residents interacted with each other, and if it seemed genuine or forced. The availability of the attending who would become my next mentors and the culture of the program were important to me as well. I would also recommend considering the type of resources and support the program provides to ensure their residents succeed.

Did you receive feedback from any programs after the interview? If so, did this make a difference?

Yes, from some, and it’s helpful but I definitely wouldn’t rank a program lower because you don’t hear back. It’s also worth reaching out if you’re interested afterwards; every program is different and just because you didn’t hear back, doesn’t mean they don’t regard you highly. It’s also important to remember that programs are unable to contact applicants between the Rank Order List Deadline and the start of the Match Week, so don’t worry if you didn’t hear anything beforehand!

How did you go about creating your rank list? Did you rank according to your preference or did you try to anticipate which programs would rank you highly?

I was given the advice to rank where you would want to be and not based on the program you think will rank you highest. If I hadn’t taken that advice, I may have missed out. Ultimately, you can only match into one spot, and I wouldn’t want to risk losing a spot somewhere I’d love to be because of trying to predict where I’d actually end up. For example, it’s unlikely that the top six candidates for a program all want to be there equally when most people apply to numerous programs to increase their odds. But if you’re honest with yourself, you have a greater chance of matching at your ideal program. Rank in order of where you truly want to be, and it will all work out the way it’s supposed to in the end. If you are still feeling a little unsure about how to create your list, the NRMP has created a video detailing the Guidelines for Applicants for Creating a Rank Order List, as well as a helpful video that explains exactly how the matching algorithm works.

The rank order list submission is open until March 3, but we recommend trying not to stress and end up waiting until the last minute to submit your list on the National Resident Matching Program (NRMP) website. Remember, when you know, you know! If you’re still looking for additional help on creating your list, there is some helpful advice available from the American Osteopathic Association as well as published resources from the NRMP.

Navigating GME | Match Resources

What inspired you to become a DO, specifically? Tell us your story.

My grandmother was unfortunately sick most of my life with type 1 diabetes and hypertension, and I found out from my mom when I was older that many of her doctors were osteopathic physicians. I’m from Michigan, and Michigan is a mecca for DOs.

I spent so much time in the hospital growing up. I have to give my respect to the nurses and doctors—one even sat down with me after my mother had donated a kidney to my grandmother to draw me a diagram and explain what was happening. At eight years old, it eased my soul when I was so scared in the middle of the hospital. I remember feeling alone, and I’m so grateful he took the time to go over it with me. It was a huge moment to be able to connect with someone beyond the world of medicine just as a person.

I’m a first generation medical school student—a trailblazer trying to figure it all out myself. So that moment when I was a little girl helped me do that. The decision to become a DO is rooted in my background, and is why I wanted to take this path. It’s not just something I found online that I thought was cool; it’s integrated into my history.

The DO approach I feel is what a physician should be. I always talk about this one time I had hurt my neck, and I went to see a doctor who happened to be an MD. He didn’t even touch me! He just looked at me, asked a question, and gave me some painkillers. And they were heavy painkillers—even during this opioid crisis we’re having. That wasn’t what I needed. He could have just touched it to try and see what was going on, but he didn’t even want to engage with me.

One of the best things about starting off at a DO school is how physical you are immediately. I was taught how to be comfortable with touching—how to palpate—how to engage in a permissible way. We’re taught to ask the patient, “Is this comfortable for you? If it’s okay, I’m going to palpate real quick and see some of the tense areas.” There’s even a double board certified doctor at my school in Neuromusculoskeletal medicine and also in OB/GYN, and she uses both to treat her patients! Who wouldn’t want to be a DO?

We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you personally?

I can talk about that all day! I’ve rounded with some MD students and we are equally capable. However, I do think there are some benefits that DOs have that MDs may not. We know more about lymphatics and nervous systems; we know more about how things are connected—how the body’s systems work together as a unit. I feel as though DOs are distinctive in that we have a deeper background in the body: how it moves, how it works, how it doesn’t work, what we can do to help it flow better, how we can assist it in its own healing process, and how we can help it, period.

I feel MDs focus more on problem solving. They are the first ones to say: “Here’s the problem and here’s the solution.” I love that, but that’s not always the right approach to patient care. Sometimes, you need to be able to connect with the patient and understand why they have these issues—look at their stress levels, look at their support system, and look at what they’re eating. What other histories do they have that contribute to their issues? Being a DO has taught me how to look at the person as a whole picture instead of a problem. I feel that’s what osteopathic distinctiveness is. I don’t want to go into a patient room having read and judged their chart and that be my whole understanding of the patient. How do I gain their trust? How do I understand where they’re coming from? You can tell them all day what to do until you’re blue in the face, but if they don’t trust you, they’re not going to do it.

I just finished my OB/GYN rotation, and I was working with high risk pregnancies when a lady who was having twins came in. The MD had done a urinary drug screen on her and it came back positive for cocaine. I asked the doctor before we went in if we were going to talk about it, since she was already high-risk. The doctor said we weren’t. I was very disappointed that we did not address her screening; we needed to understand why she was positive and what was going on. Why is she doing cocaine? Is someone trying to put cocaine in her system? There are so many questions whose answers could potentially help her. This was the opportunity for us to partner with her. That’s another big thing about osteopathic distinctiveness—we partner with our patients. Just as much as I’m trying to help you in here, you have to help me out there.

I’m grateful for the training I’ve had about empathy. A lot of physicians don’t have it at all. It’s just a job sometimes, and it’s become almost robotic. I could never look at my patients like that. I could never see a positive cocaine drug screen and not talk about it. As a person, I can’t do that, and as an osteopathic physician, I definitely can’t do that.

In your speech to COCA, you mentioned how high your stress levels were when preparing for COMLEX-USA during both a pandemic and civil unrest. Tell us about the major challenges you’ve faced and how you overcame them.

In my second year, I really struggled with some aspects of medical school academically. I hit roadblocks with some classes, and I struggled to believe that I could make it. I didn’t open up about it. I was depressed. I didn’t know I was depressed. Thankfully, I got the help I needed to get through that, but then 2020 happened! And it just changed everything. My COMLEX-USA kept getting pushed back, and then my study schedule was off. Then, in the middle of that, civil unrest. We were struggling as a group of URMs (underrepresented minorities). Alongside studying for my exam, trying to figure out when I was going to take it, people were dying. There was protesting everywhere—the election; it was a mess. There were moments I didn’t even feel safe going outside of my house. I didn’t know if I was going to be a target. I know a lot of my friends truly struggled. We were afraid, and that fear was heavy.

A few students, including myself, were working with our school to increase our diversity, equity, and inclusion efforts. We reached out to our school to get support so we could feel safer. We wrote a petition of asks, similar to what the SNMA sent to COCA. We had over 400 signatures on it. I think that changed a lot of things not just at my college, but at our whole campus. How can we make TCOM better? It’s a great school, and we are working to make everybody feel like they have a space here.

All throughout that, my test kept getting pushed back, pushed back, pushed back. I finally was able to take my exam in October after I was originally supposed to take it in August. With everything going on, I still had to dig down and figure out what I needed to know to pass COMLEX-USA. What I ended up doing was hiring a board prep program. My tutor helped me so much, and I needed somebody to be with me through this because I knew I couldn’t do it alone.

Did you also take USMLE, and if not, why didn’t you?

All throughout my time in medical school, I was told over and over again that if I wanted to be an OB/GYN, I needed to take USMLE. But I’m not going to school for Step! I’m going to school for COMLEX-USA—for osteopathic medicine, and I’m very proud of that. I don’t feel the need to prove myself to another branch of physicians in order to be able to serve.

I don’t think it’s fair to ask students who work just as hard, if not harder than MDs (because we have an extra 300 hours of training) to take two separate exams in addition to medical school. I’m not taking another exam. I am a DO through and through, and that’s how I want to be received in a program. For me, there’s a reason I came to this school—there’s a reason I’m going to be a DO. I personally feel that I’m confident in my score, I’m confident in my application, and I’m going to be confident in my letters of recommendation.

What advice would you give to other COM students preparing for COMLEX-USA?

Reach out for help when you need it, and figure out what works for you. Even the best of the best students need a resource to help them. But because a lot of people come from backgrounds like me: single parent home, first person to do this, nobody around me knows what to do—they may not have the resources or the know-how to get those resources because they’re the first person trying to figure it out.

For me, I am a talkative person and I needed to interact with someone. I need someone to look at me through the screen and ask me: “What do you think about this?” and “Let’s do some practice questions together!” And then I’d teach the material back to them so I’d know I knew it.

What are you looking forward to the most in the next stage of your journey?

The thing I’m looking forward to the most is having ‘Dr. Zooper’ on a piece of paper. The last thing my grandmother told me when I was a sophomore in college—she was dying and didn’t have a lot of strength left, but she wanted to make sure her only grandbaby finished. She told me to finish. I knew exactly what that meant. She didn’t have to say anymore—just, “Sabri, I want you to finish.” So, I’m finishing! I’m going to finish and I’m going to go through this.

Being a doctor is an opportunity for me to be able to be the inspiration that so many others were for me. I couldn’t have gotten through undergrad had I not seen other black women graduate in biology and go forward. I wouldn’t have been able to even think about medicine had it not been for my mom. They opened all these doors. I want to be that for others too and pass the torch—to keep that going. I don’t want to stop at being a doctor. I want to keep going and give back in medical education.

As an African American, one of the biggest reasons for our success and our progression has been our ability to educate ourselves—being able to read, go to school, have professions, and build lives for ourselves. It’s very important to continue that legacy—of going on to be an educator at the highest level. I want to be a dean. It’s important for all students to see different colors in leadership—different types of people and different perspectives. I want to be a part of the next generation of physician educators. That’s what I look forward to, and I’m excited!

I think the biggest power that I have is understanding that my life just isn’t about me. Yes, I’m the leading lady in my life, but, at the end of the day, it just isn’t about me. I think when you take the focus off yourself, there is so much more joy and wholesomeness in that. It’s about the impact you leave.

I heard a quote earlier this week: “When you’re born, you look like your parents, but when you die, you look like your choices.” What kind of choices am I going to make? I want to make the right ones! The most influential people are those who made you feel something—that made you believe something—that made you believe in yourself. I’ve been through so many things in my life where I didn’t feel good enough, strong enough, and I didn’t feel like I would ever be enough. I know what that feels like, and I know what it’s like to feel unseen, unheard, and unimportant. I want to make sure the people I come in contact with—no matter if they are doctors or not—that those people know I see them. You’re a human; I’m a human. Let’s be humans together! And let’s help each other because that’s what it’s about.

See All

What made you decide you wanted to do a fellowship?

I must say that I did wrestle with this question a bit during residency. I love Internal Medicine. I love the patient population, the variety in cases, the opportunity to build differential diagnoses, and testing these. However, I realized that any fellowship after an Internal Medicine residency would ensure that I could not only continue to do all of these things, but also be very skilled in one particular area. My genuine interest in hematology and oncology ultimately convinced me that a fellowship was the right way to go.

How did your osteopathic curriculum influence your interest in pursuing a fellowship and help you prepare for success?

My osteopathic curriculum at RowanSOM taught me to be an excellent clinician with great bedside manner and clinical reasoning skills. We were taught to always take the time to listen to our patients and ensure that they were part of the decision making for their treatment plan every step of the way. Being a DO helped me to be a great internist, which is the root of all sub-specialties within Internal Medicine.

As a DO, what advice would you give to those coming after you who are also thinking about applying for a fellowship?

The best advice I have is to keep an open mind when you start residency and to learn your field. Anchoring on a specialty too early will deter you from learning. The other thing to consider is that fellowship means more years of training, which means that there are financial implications and life implications. So it's important to ask yourself if this really is the best thing for you. There is nothing wrong with not wanting to pursue fellowship at all! But if you decide that you want to go ahead and focus on a sub-specialty, it is important to start to find a mentor or mentors in the field who can give you guidance, suggest research opportunities, and connect you with others.

Regarding research, please note that it is not a one-size-fits-all. Some fields value research more than others, and quantity is not always more important than quality. The types of projects and publications you need to be competitive in each field is different and that is where a strong mentor will be most helpful.

Lastly, when it comes time to apply, spend some time and consider what your priorities are. There are many factors that may be important to you, and can really affect the types of programs you will need to apply to help you be successful in your future career.

What made you choose hematology-oncology?

From a basic science perspective, I have been interested in hematology/oncology for quite some time. I majored in biochemistry and cell and molecular biology in college and so we frequently looked to cancers as a real-life illustration of what happens when the cell's intrinsic system of checks and balances goes awry. Through shadowing experiences and rotations throughout medical school and residency, respectively, I saw myself doing this every day. I witnessed the deep appreciation and trust patients have for their hematologists and oncologists. I wanted to the person to have the privilege of gaining that trust and ensuring that I was there for them on their best days and their worst days.

What aspects of being an osteopathic physician do you think will be a benefit to your Hematology-Oncology patients?

Central to Osteopathic Medicine is the tenet that each person is a unit of mind, body, and spirit. Dean Cavalieri at RowanSOM, where I went to osteopathic medical school, reminded us of this quite frequently. I have worked hard throughout my training as a student and a resident to be the best clinician, researcher, teacher, leader, and educator that I can be and I am very proud of that. I believe that these traits, my determination, and my holistic approach to medicine really came through when I interviewed—places that include some of the most elite and prestigious programs in the country. As a future hematologist/oncologist, I will continue to ensure that I find ways to connect with all of my patients on multiple levels using my osteopathic training to ensure that they feel heard, informed, and respected.

How excited are you about your match to Vanderbilt?

I am absolutely elated to have matched for my Hematology/Oncology fellowship at Vanderbilt University Medical Center in Nashville, Tennessee. Despite it being an atypical year for interview season—seeing that all of my interviews were virtual—I felt a sense of community amongst all of the faculty and the fellows just from observing their interactions and the way they spoke of each other. Furthermore, I was searching for a program with a strong commitment to teaching its fellows, a faculty who have varieties of research interests, and opportunities to grow as an educator. I saw all of this and more in the program and cannot wait to begin my fellowship.

That being said, I would be remiss if I did not acknowledge the support of my parents, my siblings, my fiancée, my incredible family, and friends who have supported me each and every day. Along the way, I have been incredibly fortunate to have had the most phenomenal teachers and mentors throughout my time in elementary school, high school, at Drew University, at Rowan University School of Osteopathic Medicine, at the Hospital of the University of Pennsylvania, and at Pennsylvania Hospital, who have always inspired me and motivated me to keep pushing the bar higher and making me the best clinician that I can be. I am also thankful to NBOME for giving me the chance to be a Resident (and future Fellow) Ambassador. I look forward to taking you all on my fellowship journey with me!

Navigating GME | Match Resources

Additionally, we would like to congratulate and welcome the following National Faculty members who have been appointed to 2021 National Faculty Chair positions:

	Clinical Sciences Department Chair – Internal Medicine, Geriatric Medicine and Dermatology Wayne R. Carlsen, DO – Ohio University Heritage College of Osteopathic Medicine – Athens Dr. Carlsen began his appointment in July 2020.
	Clinical Sciences Department Chair – Obstetrics and Gynecology Stephanie L. Zeszutek, DO, RPh – Touro College of Osteopathic Medicine – Middletown
	Clinical Sciences Department Chair – Osteopathic Principles & Practice / Neuromusculoskeletal Medicine Danielle L. Cooley, DO – Rowan University School of Osteopathic Medicine
	Preventive Medicine & Health Promotion Division Chair – Medical Ethics, Jurisprudence & Professionalism William R. Blazey, DO – New York College of Osteopathic Medicine of New York Institute of Technology – Old Westbury
	COMAT Examination Chair – Family Medicine Sarah M. Hall, DO – Oklahoma State University Center for Health Sciences College of Osteopathic Medicine
	COMAT Examination Chair – Osteopathic Principles and Practice / Neuromusculoskeletal Medicine Rebecca E. Giusti, DO – Western University of Health Sciences/College of Osteopathic Medicine of the Pacific

The NBOME is honored to have such talented and committed thought leaders that represent all aspects of clinical and foundational biomedical science disciplines in our National Faculty. Learn about our other National Faculty leaders and how to join our National Faculty here.

###

About the NBOME:

MEDIA INQUIRIES

Jeanne M. Sandella, DO
Associate Vice President for Research and Communications
jsandella@nbome.org

See All

How to Enter:

Access our post here: INSTAGRAM | FACEBOOK
Comment on the post and share some of your best tips and tricks to managing your wellness while preparing for COMLEX-USA.

Official Rules and Fine Print

Submission Period: December 1, 2020 – December 12, 2020 at 12:00am EST.
Submitted photos or text must be original. You hereby warrant and represent that (a) you own all rights to all Entry Materials submitted by you; and (b) all such Entry Materials are original works of authorship on your part and have not been copied, in whole or in part, from any other work and do not violate, misappropriate or infringe any copyright, trademark or other proprietary right of any other person or entity. You hereby verify that you will provide full written permission of any recognizable person, and their guardian if they are a minor, who are depicted in your Entry Materials.
12 winners will be randomly selected on December 14th and PM’d by NBOME for additional information through messenger. Please note you will need to accept the message request if you have not messaged with us before.
Winners must respond within 24 hours to remain eligible, otherwise a replacement winner will be selected.
Quotes from the comments may be utilized for future RTDOL social media posts by the NBOME. By responding to the giveaway, you also give us permission to use your suggestions in our campaign.
The awarded COMSAE is non-transferable.

Forms

Name Change Form

Third Party Transcript Request Form

Rescheduling Fee Waiver

Instructions to Request Test Accommodations

Request Accommodation

Scoring & Reporting

After completing an exam, candidates will receive a Score Report and Performance Profile which provides a numerical standard score for the total test and a graphic presentation for performance on three groups of content areas.

Score Report

The COMSAE report consists of a three-digit numerical score for the total test and a graphic representation of performance by content area. COMSAE does not involve a numerical minimum passing score. Instead, based on standard scores, it provides the following three suggested performance levels:

Standard Score	Performance Level
lower than 400	Lower Performance
400 - 649	Average Performance
higher than 649	Higher Performance

When using the performance profile to assess strengths and weaknesses, examinees should be aware that information provided in content areas consisting of relatively few questions may be less reliable than information provided in content areas with a larger number of questions. Therefore, it is possible that some subtest score patterns are not precisely aligned with a candidate’s numerical score for the total test. COMSAE should not be used to predict performance on the COMLEX-USA cognitive examinations.

You might be familiar with Alin Gragossian, DO, from her blog, A Change of Heart, which provides insight into her experience as an ICU-fellow-on-call turned ICU-survivor.

Originally from Los Angeles, CA, Dr. Gragossian is a 2016 graduate of Lincoln Memorial University-DeBusk College of Osteopathic Medicine. From there, she accepted a residency in Emergency Medicine at Drexel Hahnemann in Philadelphia, which was interrupted when she was hospitalized and received an emergency heart transplant in the middle of her third year of residency.

NBOME sat down with Alin to talk about her extraordinary Road to DO Licensure and how she persevered to reach her goals.

Let’s start at the very beginning. What first inspired you to become a physician and why did you choose to become a DO specifically?

I can’t tell you of a specific event that inspired me, and I don’t have anyone in my family who is a doctor. When I was five, a neighbor got hurt on our street and there were a bunch of ambulances. My parents said I was very interested in what was happening. Maybe, subconsciously, my drive to become a doctor came from that experience.

I started volunteering in the ER in high school and I found myself wanting to go there all the time. I continued volunteering there throughout college, going on to become an EMT and shadowing the same physician. I was driven to do everything in that field.

As to why I chose to become a DO—osteopathic medicine stood out to me because of the way DOs approach the patient. They don’t just look at one part or one system, they look at the patient as a whole. I was intrigued by that as a basic principle.

Do you think the osteopathic approach helps a lot in the ER?

When it comes to the more chronic things like back pain, foot pain, or knee pain that have been going on for a while, or patients with chronic conditions, you definitely look at it from a more osteopathic approach. Having that background knowledge absolutely helps

There is so much about being a DO that is special and unique. We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you, personally?

I’m not an MD—it’s not that MDs are no good—it’s just that I went to school to become a DO and that’s my distinction. I’m part of a minority of physicians that went through a special kind of training to learn more about the osteopathic approach to medicine. I’m very proud of being a DO and if the credentials on my badge are mistakenly printed with MD after my name, I will specifically call to get it changed because I really like showing that I’m a DO.

Thinking back to your COMLEX-USA Level 1 and 2 prep, what was your approach to studying?

I took COMLEX-USA Level 1 after my first two years of medical school. We actually had a one month break to take the exam. I would wake up and literally study from 8am until 8pm—I even made a calendar for myself. I would do one topic in the morning, followed by questions in the afternoon.

For COMLEX-USA Level 2-CE and 2-PE, I took them after my third year during rotations and we didn’t have a break to study. Yet, at the same time, that test was more on clinical application, which we did a lot of during practice. It was a lot easier to remember that than the basic science. I would go to my rotation in the morning and in the afternoon, I would dedicate time to studying certain topics and then do questions at night. I definitely had a set way of studying and always allocated a time of day to do it.

What advice do you have for students preparing to take COMLEX-USA now?

Just answering questions—do as many question banks as you can possibly get your hands on. It’s not so much about getting the question right—it’s more about learning from what you got wrong by reading the answer and retaining what it was.

How would you advise DO students considering residency programs that might not be as familiar with COMLEX-USA?

Talk to somebody in the program, like the program coordinator, about the process. And if you’re unsure whether or not they are familiar with COMLEX-USA—ask! And if they don’t accept COMLEX-USA, find out why. Maybe it’s because they’ve never had a DO student before. Maybe it’s because they aren’t sure how it works. For the most part, most programs do accept DO students. There are probably some that historically haven’t, but in the future, everything should be more balanced.

For most of the emergency medicine programs I applied to, I already knew somebody in the program through an emergency medicine interest group at my COM. The program I ended up going to—which was Drexel—even had a resident who was an alumni from LMU-DCOM. Check out the various specialty interest groups at your school, see if there is anyone in a program you are interested in, and ask them for advice.

In your 3^rd year of residency, you received a heart transplant, how did you pick up where you left off?

I was in the middle of my third year of residency and had just matched into my ICU fellowship. A few weeks later, I got very, very sick. It was so sudden when I went into cardiac arrest and needed a heart transplant. So, instead of starting an ICU fellowship, I became an ICU patient. I went back with 6 months left of residency only to find out that Drexel was shutting down the whole hospital. So not only did I have to work through my health condition, but I also had to find a new program.

During the five months I was recovering, I continued to do board review questions so I wouldn’t forget emergency medicine. Then I found a new program at UPMC Pinnacle in Harrisburg, PA. Initially, I couldn’t see patients face-to-face because I was still immunosuppressed. I did mini shifts and then finally started seeing patients in October of 2019. The first couple of days were awkward, but muscle memory kicked in and I remembered everything pretty quickly.

I obviously didn’t know I was ever going to need a heart transplant—I’m 30 years old and completely healthy. But I was lucky to be in a residency program that was truly supportive. Faculty, attendings and co-residents were always there for me in case I didn’t feel comfortable or if I needed help.

As an ICU-survivor, but also as an ICU-fellow-on-call, you must have experienced some unexpected challenges. Tell me a little about what you’ve overcome to get to where you are on your Road to DO Licensure.

As acute care physicians, there’s always a lot of craziness around us—there could be a code, a trauma, and a stroke all at the same time and you have to know how to stay calm. You can’t just sit there and panic. I don’t know if I was always like this or if emergency medicine shaped me into who I am, but having the ability to stay calm is essential. That’s also what helped me get through my heart transplant. As scary as it was—as annoying, as angry, and as sad as I was—I had to just take it, deal with it, and show people that you can go back to living like you normally would.

We know there is already a tremendous amount of stress and anxiety tied to this journey—how did you manage all of it? What worked for you?

You have to have balance. Recently I found out I had been put on the schedule for multiple days in a row without realizing I had two 24s that week and no day off. Speaking up about that is important. I could have been fine that whole week without the day off, but I said something about it and told them, ‘this has been a crazy week for me.’ It ended up just being a mistake. Speaking up when you feel overwhelmed, making time for yourself, and having an outlet is very important.

I personally like to write. That’s just my way of dealing with things. Different people like to do different things—some of my colleagues write music, some of them like to draw, and one of my co-residents even makes videos and has channeled it into doing things related to COVID-19. Find the thing that makes you happy, know when you feel overwhelmed, and be able to speak up.

You just moved to New York City to start the latest chapter of your life. Tell me a little about the Critical Care fellowship you’re currently doing.

I’m doing a critical care fellowship at Mount Sinai Hospital in New York City. It’s a medicine and surgical-focused two-year multidisciplinary fellowship. We rotate through all of the ICUs throughout the Mount Sinai Hospital system—neuro ICU, transplant ICU, surgical ICU, and medical ICU. At the end, we take the critical care boards. It’s similar to doing a home critical care fellowship, just without the pulmonary part. Most of my peers are all medicine trained—there are a few of us who are emergency medicine trained too so it’s interesting to have a good mix of people around me.

What’s next after this? What are you looking forward to the most?

After my fellowship, it would be awesome to stay on the east coast (because it stole my heart, quite literally). I would love to have a job where I do both ER and ICU because I’ll be boarded in both. I want to stay in academics and work with residents, teaching medical students what I know, learned, and experienced.

Based on your hands-on experiences, what advice do you have for those who are coming after you?

Don’t let anything stop you. There is going to be a lot that challenges your way of thinking and challenges you personally. There are going to be things that happen to you that you wouldn’t ever think would happen to you. There will be people who tell you that you’re not going to make it, and things that happen that will make you question whether or not you should be doing this. But, at the end of the day, if you really want to do it, don’t let anything stop you.

Feeling inspired by Dr. Gragossian’s story? Become an organ donor and advocate for others to become donors too. There are many myths around organ donation, and it’s important to realize how unfortunate circumstances can breathe new life into someone else who is in need.

See All

NBOME Partners with ABCS on Successful Transition to a Virtual Exam

#####

About American Board of Cosmetic Surgery: The American Board of Cosmetic Surgery (ABCS) is the only board dedicated exclusively to excellence in the medical specialty of cosmetic surgery. For nearly 40 years, the ABCS has demonstrated the highest standards of training, knowledge, and expertise in cosmetic surgery, in addition to promoting safe and ethical practice. You can find more information about the ABCS online at www.americanboardcosmeticsurgery.org

About the NBOME: NBOME is an independent, non-governmental, non-profit assessment organization committed to protecting the public by providing the means to assess competencies for osteopathic medicine and related health care professions. NBOME’s COMLEX-USA examination series is a requirement for graduation from colleges of osteopathic medicine and provides the pathway to licensure for osteopathic physicians in the United States and numerous international jurisdictions. You can find more information about NBOME online at: www.nbome.org.

Media Contacts: Dr. Wilbur Hah, ABCS, (425) 689-5665; Susan Peters, Director of Marketing and Communications, NBOME, speters@nbome.org

To read this as it originally appeared on PR Newswire, please click here.

Fantastic news to share with US-educated doctors of osteopathic medicine who are interested in practicing in Australia — The Medical Board of Australia just established a new registration pathway for DOs, recognizing the National Board of Osteopathic Medical Examiners (NBOME) as a competent authority. This means that US DO graduates who have successfully completed the NBOME’s COMLEX-USA examination series since 2005, and have also completed two years of an ACGME- or AOA-accredited residency program will be qualified for provisional registration. “Registration” is the term used in numerous other countries for what we refer to as “medical licensure” in the United States.”

According to the Medical Board of Australia, until recently, doctors who had been awarded the DO degree in the United States were eligible only for limited registration in Australia. The new pathway streamlines the registration process and enables US DOs to be granted general registration after 12 months of supervised practice in Australia. In Australia, “DO USA” refers to physicians with a degree in osteopathic medicine to avoid confusion amongst patients with other types of health practitioners who hold qualifications in osteopathy.

The NBOME is the first international authority to receive competent authority for medical licensure in Australia in 15 years. The evaluation by the Medical Board of Australia included a robust and comprehensive review of the COMLEX-USA examination series, the licensure examination for DOs in the USA. COMLEX-USA Levels 1 and 2 (which includes a clinical skills evaluation) are also required by the American Osteopathic Association–Commission for Osteopathic College Accreditation (AOA-COCA) for graduation from a college of osteopathic medicine. The NBOME provided extensive documentation demonstrating adherence to quality assurance standards for validity, reliability, defensibility, and fairness to assess whether the COMLEX-USA examination program’s processes result in physicians who have the knowledge, clinical skills and professional attributes necessary to practice in the medical profession in Australia.

“Registration in Australia is all about safety to practice,” shared Dr. Anne Tonkin, Chair of the Medical Board of Australia, “We have streamlined our process and continue to welcome DO USA graduates so they can contribute to our profession and our community.”

The news was especially welcome to Nayla Boulad, DO, who submitted her application under the new pathway as soon as it was available and is eager to be able to practice osteopathic medicine in her new home in Australia. After completing her residency training in Pediatrics at the University of Connecticut Health Center in Farmington this summer, Dr. Boulad relocated to Perth, Australia to be nearer to her husband’s family, “I was literally working on my application when I got this fantastic news. I cannot believe the timing, this is so great!” she shared.

Prior to the Board’s recognition of the NBOME and COMLEX-USA, Dr. Boulad was preparing to complete the standard pathway application, a process that entailed extensive requirements, including completing another residency in Australia and taking other medical licensing examinations “In a time where travel and immigration is severely curtailed due to COVID-19, Australia faces a shortage of doctors and I am happy to have the opportunity to start working to fill those gaps. I am looking forward to continuing my learning and training here with hopes to be able to share what I learn with my peers back in the US.”

NBOME President and CEO, John R. Gimpel, DO, MEd, commented on the significant achievement and milestone for US DOs, NBOME’s COMLEX-USA examination program, and (former) AOA-accredited residency programs, now accredited by the ACGME. “Australia joins all United States medical licensing boards in entrusting NBOME’s COMLEX-USA program in the medical licensing process, on the merits of the evidence and the rigor of the exam program,” said Dr. Gimpel, “This is exciting news for DOs, the osteopathic medical profession, and the patients and communities of Australia. A special thanks to the AOA, the Australian Medical Council, the Medical Board of Australia, and all across the profession in the US and Australia who collaborated for more than a decade to make this a reality.”

PHILADELPHIA, PA. The National Board of Osteopathic Medical Examiners (NBOME), an independent, not-for-profit organization who provides testing for osteopathic medical licensure, is pleased to share news that the Medical Board of Australia recently established a new pathway for osteopathic physicians to become registered to practice medicine in Australia.

Referred to as Category G UNITED STATES National Board of Osteopathic Medical Examiners (NBOME), this new pathway enables US educated DO applicants who have passed all levels of COMLEX-USA (including the Level 2-Performance Evaluation/clinical skills exam) and completed at least two years in a residency training program accredited by the Accreditation Council for Graduate Medical Education (ACGME) and/or by the American Osteopathic Association (AOA) to apply for provisional registration for medical practice in Australia. The evaluation by the Medical Board of Australia included a robust and comprehensive review of the COMLEX-USA examination series, including NBOME’s high standards for quality assurance in areas such as validity, reliability, defensibility and fairness. COMLEX-USA is the first international program to receive competency authority for medical licensure in 15 years.

“Australia joins all United States medical licensing boards in entrusting NBOME’s COMLEX-USA program in the medical licensing process, on the merits of the evidence and the rigor of the exam program,” said NBOME President & CEO, John R. Gimpel, DO, MEd. “This is exciting news for DOs, the osteopathic medical profession, and the patients and communities of Australia. Kudos and special thanks to everyone across the profession in the USA as well as in Australia who collaborated to make this a reality.”

###

About the NBOME
NBOME is an independent, non-governmental, non-profit assessment organization committed to protecting the public by providing the means to assess competencies for osteopathic medicine and related health care professions. NBOME’s COMLEX-USA examination series is a requirement for graduation from colleges of osteopathic medicine and provides the pathway to licensure for osteopathic physicians in the United States and numerous international jurisdictions.

Media Inquiries
Susan Peters, Director of Marketing and Communications
speters@nbome.org

Each year since 1991, the NBOME Board of Directors has convened its annual Liaison Committee Meeting, hosting representatives from organizations from across the continuum of osteopathic medical education and the house of medicine. The committee meets to share their experiences, ideas and concerns as they pertain to NBOME assessments as well as other challenges and opportunities facing the profession. This year’s Liaison Committee, while transitioned to a virtual and somewhat shortened format, was attended by 20 leaders from 12 different organizations, including members of the undergraduate and graduate medical education community from AACOM, AOGME, OPDA, ACGME; osteopathic medical students and residents from COSGP, SOMA, and the AOA-Bureau of Emerging Leaders; members of the licensure community from AAOE and FSMB; and professional organizations — AOA, AMA. NBOME also welcomed, for the first time ever, the National Resident Matching Program (NRMP), who shared results from the 2020 Program Director Survey, including a record high 86% of ACGME residency directors requiring COMLEX-USA for DO applicants.

This year’s Liaison Committee Meeting theme, chosen even before the pandemic, was Professional identity formation across the continuum — from medical education through practice. Dr. Geraldine O’Shea chaired the meeting with support from Board Vice-Chair Richard LaBaere, DO, and NBOME President and CEO, John R. Gimpel, DO, MEd. NBOME was further represented by members of the NBOME team, including Liaison Committee Lead, Sandra Waters, MEM, Melissa Turner, MS, and Marie Fleury, DO, MBA.

Following NBOME and COMLEX-USA updates, participants engaged in a facilitated discussion focused on COMLEX-USA scoring (numeric vs. pass/fail), osteopathic distinctiveness, and strategies to advance professional identity formation for DO students, residents, and practicing physicians. Participants showed support for NBOME’s agility and innovation throughout the pandemic, including the creation of a COM Liaison Team, development of COMAT self-proctored administration options, and enhanced collaboration with Prometric to test over 14,000 displaced COMLEX-USA candidates this summer.

Historically, this process has resulted in numerous opportunities to make improvements to NBOME initiatives, products and services. These improvements have shown to help meet the evolving needs of our candidates, colleges of osteopathic medicine, the licensing community, and others who rely on what we DO. The NBOME Liaison Committee meeting enables us to continue to promote a culture of collaboration, while endeavoring to continuously improve as we serve others and remain steadfast in our mission to protect the public via high-quality, valid and reliable assessment.

COMLEX-USA Computer-Based Examination Outline

Section	Questions
1	44	Time: 4 Hours Questions can be answered, reviewed, and changed one section at a time. Individual sections are not timed. Once you move to the next section the previous section is no longer accessible.
2	44
Authorized 10-minute break
3	44
4	44
Authorized 40-minute lunch break
5	44	Time: 4 Hours Questions can be answered, reviewed, and changed one section at a time. Individual sections are not timed. Once you move to the next section the previous section is no longer accessible.
6	44
Authorized 10-minute break
7	44
8	44

Available Examination Features

Examination Review Screen
A review page is presented for each section. This provides candidates with information about the status of the items in the current section, which includes questions completed, questions marked for further review, and questions left incomplete (i.e., no answer is given).

Advancing through Examination Sections
Once a section is completed, the candidate cannot return to review or change any answers within that section.

Time Limitations
Although each session is 4 hours in length, the individual sections within a session are self-paced. A clock is provided to assist with time management. Warnings are given of the time remaining before the end of each 4-hour session.

Examination Visual References
Use of an “Exhibit” button may be required in order to see graphic materials related to a test question. These exhibits consist of images, videos, or audio avatars. In some cases, multiple exhibits may be used in order to answer a question.

Additional Tools
Functional features are available, including a standard calculator and laboratory values with reference ranges (where applicable) embedded directly into exam questions.

It was an historic Match Day for DO students and graduates. This year’s NRMP Main Residency Match was the largest on record, with over 40,000 applicants applying to positions in 5,048 programs across the US. Applicants in this year’s Match increased by 4.5% over last year to 37,256 available positions. On March 20th, 6,215 4th year osteopathic medical students and graduates who matched to PGY-1 positions learned where they’ll be spending the next three or more years in residency training programs in the specialty in which they’ll work.

NRMP President and CEO, Donna Lamb, DHSc, MBA, BSN, reported, “We are especially excited that the 2020 Match marks a milestone for the medical education community: The first Single Match for U.S. MD and DO senior students and graduates and the inclusion of DO senior students as sponsored applicants.”

The results of this year’s Match, the first Single Match under the single accreditation system, continue to trend positively for osteopathic graduates entering residency training. Despite being socially distanced and self-quarantined due to the COVID 19 pandemic, students, schools, programs, specialty societies and others held remote “Virtual Match Day” celebrations in observance of the highest number of DO seniors who matched to PGY-1 positions—5,968—at a rate of 90.7%, compared to a match rate of 80.8% for all applicant types. Applicants who did not match to a residency position participated in the Match week Supplemental Offer and Acceptance Program (SOAP) to obtain one of 1,897 positions available. More DO applicants matched during SOAP, with data to be released in early May.

DO applicants in this year’s Match accounted for the largest increase in applicant groups — up by 19.2%, (1,153 applicants) and resulting in a 20.1% increase in DO seniors (up 1,103) compared to last year. Increased match rates for DO seniors ‒ up 2.6% from 2019 and up 13% from 2016 ‒ are especially impressive considering the increase number of applicants. The match rate for all osteopathic applicants (seniors plus prior graduates) also rose to 86.9%, up from 84.6% in 2019. Positive results by specialty include the 3 most popular specialty matches for DO senior applicants: Internal Medicine, Family Medicine, and Emergency Medicine. Orthopedic Surgery and General Surgery results for DO seniors also showed significant increases.

NBOME President and CEO, John R. Gimpel, DO, MEd shared his congratulations with students, COMs, AACOM, the AOA and many others who contributed to the success of this year’s single Match, “This year’s Match highlights the opportunity for the betterment of quality care and clinical learning involvement in our nation’s GME programs and the patients and communities we have the privilege to serve.” Dr. Gimpel remarked about the equivalent manner in which US DO and MD seniors and graduates are included in the NRMP Match reports, infographics and other communications. For more on the Match 2020 results, follow NBOME social media or read DO student Match stories here. Additional details about this year’s NRMP outcomes, can be accessed in the NRMP 2020 Advance Data Tables.

Jon Bardahl, DO, is currently training in pediatric medicine at OSF HealthCare. In the upcoming academic year starting in July 2020, Dr. Bardahl will be starting his fellowship in pediatric hematology-oncology at Duke University. Prior to medical school, Dr. Bardahl received his bachelor’s degree in biology from Governor’s State University and earned his osteopathic medical degree from Midwestern University in Chicago, Illinois. Throughout the years, Dr. Bardahl has been a consistent advocate for osteopathic medicine, and has been highly involved with the NBOME as a Resident Ambassador helping to spread information regarding COMLEX-USA to various student groups and stakeholder through social media.

You’ve almost finished your residency – we hear a fellowship is your next step, tell us about that — what made you choose hematology-oncology as a subspecialty?

I’m excited to be starting Pediatric Hematology-Oncology fellowship this July at Duke Children’s Hospital. I have been interested in cancer biology for some time, and have been lucky enough to be able to pursue it. Prior to medical school I conducted research on a protein expressed by Epstein-Barr virus and its influence when it comes to tumor development in certain cancers. While in medical school, I got involved in the St. Baldrick’s Foundation which helps raise money for childhood cancer research in exchange for participants shaving their heads. In residency I truly developed an appreciation for the field and loved the continuity with patients and their families, the pathology, the opportunity for research, and advocacy opportunities involved in the field.

Any advice for DO students and residents who might be thinking about completing a fellowship?

I think the biggest advice I would give any upcoming student or resident is to find a mentor and “pick” their brain. I have had so many mentors along the way that have influenced me both personally and professionally and I’m so grateful for the lessons I have learned and the advice they have given me.

You only took COMLEX-USA when you applied to residency – what was your thinking about that?

I’m a proud DO and fully support COMLEX-USA! I was only interested in residency programs that supported COMLEX-USA as well so the USMLE was never something I considered.

How would you advise DO students today in considering a specialty?

Students should try to keep an open mind when it comes to medicine. If they have a specialty they are interested in, they can identify mentors or research opportunities which may be helpful when applying to residency programs.

What about applying to residency?

Here are some things that I would recommend doing:

Perform well on your clinical rotations to obtain the best LOR you can
Get involved in various organizations (e.g., SOMA, COSGP, and AOA)
Join advocacy opportunities to grow as a leader and well- rounded physician
Do research – it’s always a plus!

How would you advise DO students considering programs that might not be as familiar or accept COMLEX-USA scores?

Reach out to programs and ask them! If a program is unfamiliar you can always supply them with resources from the NBOME website or notify NBOME so they can advocate on your behalf.

Applying to residency has gotten so stressful – students are applying to more programs every year. How many programs did you apply to and interview at?

I applied to 20-30 programs for both residency and fellowship and I interviewed at 10-12 programs.

Did you do an audition rotation at your residency or fellowship program?

I did not. Audition rotations are a wonderful opportunity to learn and get a “feel” for a program, but it’s not mandatory. I have loved training at my current residency program and don’t think an audition rotation would have changed that.

What are some tips you would give DO students who are preparing to apply to residency?

These are some of the things that worked for me, and I hope they help others as well:

Stay on top of your deadlines
Reach out to LOR writers early
Review your CV for gaps
Start your personal statement early and have others review it
Reach out to upperclassmen for advice
Continue to perform well on your clinical rotations
Take time to “recharge” and find balance in your life

Were you always interested in pediatrics? When did you know that’s what you wanted to do and how did you decide that’s what you wanted to do?

Pediatrics was at the top of my list! I did consider family medicine at one point as well, but decided to pursue pediatric medicine after my 3rd year core rotation in pediatrics. I have always been interested in pathology and academic medicine, and wanted the opportunity to specialize in Hematology-Oncology to provide continuity of care by working with patients and their parents.

See All

#DOLOVE

The brain-sweat, tears, and hard work that are vital on your journey to DO licensure does not need to evoke feelings of guilt for not studying every hour of the day. So let’s tackle the study burnout and help you find ways to carve out time to keep doing what you love. Here’s how:

Schedule it

Perhaps most of your life is meticulously mapped on a calendar, moment by moment—classes, clinical rotations, crazy amounts of studying, eating, sleeping, laundry, brushing your teeth, more studying. But your hobbies and passions are just as important to your well-being and your mental headspace, and they deserve dedicated time on your calendar too. Yes, yes, we know, there are only so many hours in a day, but who says your passions need to take hours? Take a 10-minute run (instead of your normal 10-mile jog), cook a 15-minute meal (instead of a multi-course masterpiece), read one chapter of a fiction book (instead of the whole thing), doodle for 10 minutes…you get the idea. Your hobbies and passions, just in smaller doses. Clear that brilliant brain of yours, nourish yourself with things that bring you joy, and then get back to the books!

Stop canceling

You’ve made the appointment, but are you going to show up? This is the tough part because we all know how easy it is to cancel, especially when you are too tired, too busy, and too guilty (when you think you should be studying even more than you are). Showing up for yourself amidst COMLEX-USA exam prep might seem like a joke, but doing what you love is a big part of recharging and fueling what comes next. Be confident in the schedule you create and the choices you make, more importantly when they involve taking care of you. If you don’t do it, no one else is going to do it for you.

Mix it up.

It’s easy to shelve the hobbies you’ve had for a long time. While they still bring you joy, they don’t give you the same butterflies-in-your-stomach feeling they did years ago. Maybe it’s time to try something new: rock climbing, origami, ballroom dancing. Step out of your comfort zone and make use of an entirely different part of your brain or body. Getting out of your study-cave and into something new might also help ignite some of those old passions, clear your mind, and help energize you so you can get back to the studying. Plus, adding to your arsenal of interests may eventually help you connect and empathize better with your patients as well.

When studying starts to bleed into the things you love, take a step back and remind yourself that you’re a person before you are a physician—and even more important: you’re a person even after you become one. During this month of passion and #DOLove, encourage yourself, empower yourself, and break through your schedule to keep doing the things that bring you joy and keep you, you.

See All

Osteopathic International Alliance (OIA) Conference

From October 4-6, 2019, NBOME Board Vice-Chair Geraldine O’Shea, DO, and President and CEO, Dr. John R. Gimpel, attended the annual Osteopathic International Alliance Conference in Bad Nauheim, Germany.

The OIA is in official relations status with the World Health Organization, and “envisions a world in which every person has access to high-quality osteopathic medicine.” Next year’s AGM and conference will be held in Rio de Janeiro, Brazil from September 30-October 2, 2020.

National Resident Matching Program® (NRMP®) Conference

From October 3-5, 2019, NBOME Associate Vice President for Strategy and Quality Initiatives, Melissa Turner, MS, attended the National Resident Matching Program® (NRMP®) Conference in Chicago. The NBOME provided attendees with a COMLEX-USA update.

This year’s stakeholder conference, titled, “Transition to Residency: Conversations Across the Medical Education Continuum,” set a record for its 300 registrants. Focusing on a variety of topics related to residency, speakers included Ezekiel Emanuel, MD, PhD, Helen Fisher, PhD and Lawrence G. Smith, MD, MACP.

The COMLEX-USA Composite Examination Committee (CCEC) Meeting

The COMLEX-USA Composite Examination Committee (CCEC) met on October 14 and 15 in the Philadelphia Executive offices. This committee reviews all levels of the COMLEX-USA examination series, including statistics and candidate feedback, and provides a report to the NBOME Board of Directors. At this meeting, CCEC reviewed performance and innovations happening within the examination series — including the potential of reducing test items in Level 1 and 2-CE, as well as the Level 2-PE team researching possible modifications to the Humanistic and Biomedical/Biomechanical Domains. CCEC convenes the Blueprint Subcommittee, which regularly reviews the COMLEX-USA Master Blueprint to assure it keeps up with the evolving practice of osteopathic medicine.

The committee also discussed hot topics related to licensure examinations, such as the possibility of switching to a pass/fail scoring system, or keeping some form of numeric-based scoring. The CCEC is also reviewing the current maximum number of attempt limits per examination level. The Point-of-Care Knowledge, Education and Testing (POCKET) process was reviewed and decisions were recommended regarding next steps with this process.

Osteopathic Medical Education Conference (OMED)

From October 25-28, 2019, the NBOME participated in the Osteopathic Medical Education Conference (OMED) in Baltimore, Maryland. The American Osteopathic Association’s (AOA) annual conference brought together thousands of osteopathic physicians, medical students, and other health professionals from across the country for medical education, inspiration, networking and entertainment.

NBOME exhibited at the conference, featuring our new COMAT-Foundational Biomedical Sciences portfolio, as well as the COMLEX-USA examination series, the CATALYST platform and opportunities for doctors to explore the NBOME National Faculty Program. Attendees visiting the NBOME booth were greeted by staff who had meaningful conversations with many visitors and were on hand to answer student, faculty, practicing physician and others’ questions.

On day one of the conference, the American Osteopathic Foundation hosted its annual Honors Gala, presenting awards to a number of NBOME National Faculty members, including AOF Educator of the Year to Richard Jermyn, DO, from RowanSOM. In addition, and in honor of NBOME’s 85th anniversary year, the NBOME made a contribution to the William Anderson, DO Minority Scholarship Fund.

Early Sunday morning, a team of NBOME runners and walkers came out to join in the Advocates for the American Osteopathic Association (AAOA) Fit for Life Run 2019. The run benefitted osteopathic student scholarships and the NBOME was a featured sponsor as well.

Association of American Medical Colleges (AAMC) Annual Conference

From October 8-12, NBOME sent John R. Gimpel, DO, MEd to attend the AAMC Learn, Serve, Lead 2019 Conference in Phoenix, AZ. The meeting covered many of the successes and challenges of academic medicine nationwide, and was attended by medical educators from across the country. Retired President and CEO of AACOM, Steve Shannon, DO, received an achievement award.

Council of Medical Specialty Societies (CMSS) & Organization of Program Directors Associations (OPDA) Annual Meeting

From November 21-22, 2019, NBOME Vice President for Collaborative Assessment & Initiatives Sandra Waters, MEM and Associate Vice President for Strategic & Quality Initiatives, Melissa Turner, MS, attended the Council of Medical Specialty Societies (CMSS) Annual Meeting and Specialty Forum in Arlington, VA.

Together, they presented NBOME and COMLEX-USA updates at The Organization of Program Directors Associations (OPDA) meeting. “OPDA is dedicated to promoting the role of the residency program director and program director societies in achieving excellence in graduate medical education.”

The NBOME Test Accommodations Committee (TAC) Meeting

Between November 21-22, The NBOME Test Accommodations Committee, which is comprised of osteopathic physicians and other subject matter experts who review applications for special accommodations from COMLEX-USA candidates in cooperation with NBOME staff.

The Committee met to discuss trends and developments in the test accommodations realm, as they apply to high-stakes testing agencies like the NBOME.

Coming Up

In the next quarter, we’ll be making appearances at the following conferences and meetings:

IMSH 2020 January 18-22
AOA 2020 Midyear Business Meeting February 26-27
2020 CREOG & APGO Annual Meeting February 26-29
LEAD Conference February 27-28
2020 ACGME Annual Educational Conference February 27-29
2020 CREOG & APGO Annual Meeting February 26-29
Ottawa Conference 2020 February 29-March 4
CORD Future Academic Assemblies 2020 March 8-11
AAO Convocation 2020 March 11-15
ACOFP 57th Annual Convention and Scientifi c Seminar March 19-22
AACOM Annual Conference March 25-27
ATP Conference March 29-April 1

PHILADELPHIA, PA. The National Board of Osteopathic Medical Examiners (NBOME), an independent, not-for-profit organization that provides competency assessments for osteopathic medical licensure and related health care professions, announced Lori Kemper, D.O., M.S., FACOFP, as its newest Secretary-Treasurer. At their December Board of Directors Meetings, the NBOME elected Dr. Kemper to a two year term.

“I’m thrilled to have even been considered, let alone chosen, as the NBOME’s next Secretary-Treasurer,” said NBOME board member, Lori Kemper, D.O., M.S., FACOFP. “I look forward to working towards our mission in this new, exciting officer role.”

Dr. Lori Kemper’s more than 30-year career encompasses both independent practice and graduate medical education. Since 2007, she has been the dean of Midwestern University, Arizona College of Osteopathic Medicine, where she previously served as associate dean of graduate medical education and associate professor in the department of family medicine. She currently serves as a commissioner to the American Osteopathic Association (AOA) Commission on Osteopathic College Accreditation (COCA) and is the chair of the Board of Deans of the American Association of Colleges of Osteopathic Medicine (AACOM). Dr. Kemper currently serves the NBOME as a member of the Test Accommodations Committee and the Awards Committee.

“We’re ecstatic to announce Dr. Kemper’s election to Secretary-Treasurer of the NBOME,” said NBOME Board Chair Geraldine O’Shea, DO. “Her tenure on our Board of Directors has resulted in great strides for us as an organization, and we look forward to what she’ll help us accomplish moving forward.”

PHILADELPHIA, PA. The National Board of Osteopathic Medical Examiners (NBOME), an independent, not-for-profit organization that provides competency assessments for osteopathic medical licensure and related health care professions, today introduced Juan F. Acosta, DO, MS, as its newest board member. He was recommended to the NBOME Nominating Committee via the Assembly of Graduate Medical Educators (AOGME, formerly known as the Association of Osteopathic Directors of Medical Education- AODME), filling the seat previously held by new NBOME Vice-Chair Richard J. LaBaere II, DO, MPH. He was elected at the annual NBOME Board Meeting in December.

“Joining the Board of Directors at the NBOME is an exciting opportunity for me,” said NBOME new board member, Juan F. Acosta, DO, MS. “I’d like to express my gratitude to Dr. Gimpel and his colleagues at the NBOME Board for electing me to this position.”

Dr. Acosta recently moved to New York where he serves as the Associate Medical Director for the Emergency Department at Saint Catherine of Siena Medical Center in Smithtown. He is also actively involved with the Disaster Medical Assistance Team (DMAT) and serves as a reviewer for the Journal of Emergency Medicine and a section editor for the West-JEM Journal. Dr. Acosta is an Oral Board examiner for American Osteopathic Board of Osteopathic Emergency Medicine (AOBEM). He is presently the secretary for the American College of Osteopathic Emergency Physicians (ACOEP) and Secretary for the Association of Osteopathic Directors and Medical Educators (AODME). Dr. Acosta also serves on the American Association’s Commission on Osteopathic College Accreditation (COCA) and the Committee on Continuing Medical Education (CCME).

“We are very fortunate to welcome Dr. Acosta to the Board,” said NBOME Board Chair Geri O’Shea, DO. “His enthusiasm, decorated professional career and experience in graduate medical education, add to our Board at an exciting time for the NBOME and the osteopathic medical profession.”

COMAT Product Updates

The COMAT exam series will expand in January 2020 to include the new Foundational Biomedical Sciences (FBS) Targeted exams. Each of the 14 exams focus on a specific organ body system or basic science discipline introduced to osteopathic medical students in years one and two. Click here to see the full list of available FBS Targeted subject exams.

Since the introduction of the FBS Comprehensive exam in December 2018, a total of 20 Colleges of Osteopathic Medicine have used or plan to use the FBS exams for pre-clerkship assessments to complement their use of the COMAT Clinical discipline exams for their end-of-rotation needs in years three and four.

The October issue of the Journal of Graduate Medical Education included research on the concurrent and predictive validity of the COMAT discipline exams and COMLEX-USA Level 2-CE. The findings indicated statistically significant, positive associations between COMAT and COMLEX-USA Level 2-CE scores, which can support the use of COMAT for osteopathic medical schools.

COMSAE Product Updates

After the first several score releases for COMLEX-USA Level 1 beginning mid-July 2019 and Level 2-CE beginning mid-August 2019, the NBOME conducted a comprehensive evaluation of scores on COMSAE Phase 1 and Phase 2 and subsequent scores on COMLEX-USA Level 1 and Level 2-CE. Following this evaluation, both the COMSAE Phase 1 and Phase 2 new score reports and scoring will be implemented on February 3, 2020. Please note that the COMSAE Phase 2 cut-score remains the same as 2019.

In addition, the NBOME has conducted a concordance correlation study, which demonstrated a positive and significant correlation, around 0.70, with COMSAE Phase 1 and COMLEX-USA Level 1 and COMSAE Phase 2 and COMLEX-USA Level 2-CE. This concordance study finding is consistent with those of recent years for all forms purchased by COMs with timed administrations.

As always, caution should be exercised when using COMSAE scores to estimate subsequent COMLEX-USA scores or for uses other than those for which they were developed.

We will continue to communicate regularly with COMs regarding new information related to COMSAE scores and their relationship to COMLEX-USA scores, as well as other COMSAE program updates as they become available.

“When you show deep empathy toward others, their defensive energy goes down, and positive energy replaces it. That’s when you can get more creative in solving problems.” – Stephen Covey

Empathy has always been the root of human connection, and in that, stems the foundation of our capacity to help others—whether family, friend, or patient—it all comes down to the same core values. And yet, it is the humanistic domain that many question including as part of the DO licensure exam. How important is it?

Having worked in osteopathic medical education and licensing for over 25 years, I am frequently posed the question, what makes DOs different? My answer is always the same, it’s about patient empathy. This isn’t to say that MDs don’t possess this trait; they do. However, there’s a heightened sense of empathy and patient understanding that seems to steer certain candidates toward osteopathic medicine.

The DO approach is based on the unique connection between mind, body, and spirit as it relates to patient care. It’s this holistic, 360-degree assessment and desire for enhanced understanding that fuels empathy and a different shade of patient care. It also involves empathic inquiry, developing understanding beyond just the problem at hand, but also what other life factors are impacting the patient. As a doctor, understanding how these many dimensions interact and intersect on a deeper level is the basis of the DO approach.

To clarify, empathic doctors are not internalizing or ‘taking on’ a patient’s pain or discomfort in a therapeutic way. Rather, they’re attempting to understand the patient’s illness experience. A patient once told me, “I don’t need my doctor to love me, but I do need them to understand me.” That deeper level of understanding is what brings humanistic values back into the medical encounter, allowing for the establishment of a knowing relationship. Research has shown that empathy helps to build trust, is linked to better diagnoses, improves patient outcomes, and decreases malpractice lawsuits.

Now that we have better understanding of the importance of empathy, how do we measure it in a clinical setting? The COMLEX-USA Level 2-Performance Evaluation has been assessing interpersonal and communication skills and professionalism of candidates for the past 15 years. And empathy is one of six dimensions assessed. Based on evidence that patients place great value on their human connection to their doctor, there are several guiding principles that support the role empathy plays in patient care:

Empathy is connection.

Attend to the patient both verbally and non-verbally. Listen to them. Make eye contact. Actively respond to their condition or pain. Avoid giving patients the ‘clinical cold shoulder’ by focusing only on their symptoms.

Empathy is curiosity.

This is especially important for young doctors who don’t have a lot of patient experience. Learn from the patient. Explore their illness experience. Discuss their lifestyle, their belief system, their stress levels, what motivates them. Give patients the feeling of being understood.

Empathy is compassion.

This is the ability to imagine what a patient is experiencing without being overwhelmed by their pain or distress. Research has shown that people are selective when expressing empathy towards others. It’s hard to feel compassion, for example, when a patient is difficult, unlikeable, or struggling with unhealthy behaviors that put them at risk. But it’s these patients who are most deserving and in need of our compassion and understanding.

Empathy is not stress.

Stress is in opposition to empathy. It’s difficult to connect to a patient, or anyone for that matter, when we feel anxious and overwhelmed. Likewise, physicians who have difficulty managing their feelings towards patients are themselves at risk. Although the stress of working with patients is an unavoidable part of a physician’s work life, one goal of medical education should be to equip students with the skills to manage stress in healthy ways.

A generation ago, few were talking about the role of empathy in healthcare. But today, cognitive neuroscience has enabled us to look critically at what ignites and motivates our behaviors, including the empathic ones. This new knowledge and learning, coupled with a heightened focus on developing higher quality patient care, shines a bright light on the need and desire for greater empathic engagement. That said, empathy is not what makes good doctors; it’s what makes good doctors even better.

Demonstrating empathy is important to becoming a DO, and since 2004, passing this assessment has been required to obtain the DO degree, move into residency training, and obtain a license to practice osteopathic medicine.

Contributed by Tony Errichetti, PhD | Director of Doctor-Patient Assessment | NBOME

See All

COMLEX-USA | New Level 1 and Level 2-CE Exams Have Launched

We are pleased to announce the completed launch of all elements of the enhanced COMLEX-USA exam series under the new COMLEX-USA Enhanced Master Blueprint. Level 1 successfully released this spring, followed by Level 2-CE’s in late summer. These exams have joined Level 3 and Level 2PE, which launched in 2018 and earlier this year, respectively. New passing standards for COMLEX-USA Level 1 and 2 have also been implemented for the 2019-2020 test cycles.

This multi-stage release is the culmination of nearly 10 years of work in evidence-based design by experts and leaders from across the organization and the country who contributed in all areas to the creation and deployment of this state-of-the-art assessment.

The exams launched to heavy candidate volume with over 1,500 candidates completing each exam during the first weeks. To date, over 5,000 Level 1 examinations have been administered, with similar numbers for Level 2-CE.

These examinations also mark a move to Prometric’s new test driver, SURPASS, where NBOME already administers its Core Osteopathic Recognition Readiness Examination (CORRE), as well as the latest version of COMLEX-USA Level 3. Since the move to SURPASS, some students have encountered performance problems during their administrations, including latency and examination restarts. Prometric continues to investigate the cause and has made system upgrades in early June to address issues. NBOME is currently offering online tutorials for Levels 1, 2-CE and 3 for candidates who would like to learn more about the new test interface being offered at Prometric Testing Centers.

Please visit the COMLEX-USA pages of our website to learn more.

COMAT | Foundational Biomedical Sciences (FBS) Exams Available this Academic Year

Since the inception of the COMAT Clinical Exams in 2011, osteopathic medical students have taken over 250,000 COMAT Clinical exams. As a result, we have seen dramatic improvement in COMLEX-USA Level 2-CE scores by osteopathic medical students. This is particularly important in the era of Single Accreditation for GME.

Celebrating 85 years of protecting the public through valid and reliable licensing exams, NBOME has spent the last 5 years developing an expanded COMAT portfolio to assist in DO student success.

COMAT Clinical exams initially focused on assessment of clinical education and knowledge typically found in year 3 and 4 COM curriculum. The success of this initial series of exams led the NBOME, in collaboration with its National Faculty, to expand its offerings and develop assessments for the Foundational Biomedical Science (FBS) curriculum which take place during year 1 and 2. After careful development and testing, the COMAT FBS Comprehensive (FBS-C) exam became available in December 2018. Since its inception, the 5-hour, 250-question COMAT FBS-C has successfully been utilized by many COMs across the country. This assessment has enabled both COM students and faculty to better understand the effectiveness of the school’s classroom curriculum and identify areas for student development.

Scheduled for release in January 2020, the suite of 14 FBS Targeted (FBS-T) exams further supports osteopathic medical student professional success. These exams are divided between 6 core science disciplines, including Anatomy and Pharmacology, as well as 8 body systems, including musculoskeletal and cardiovascular. Each 90 minute, 62 question COMAT FBS-T exam is designed to evaluate a student’s knowledge in a focused subject area. Timely score reports detail areas of strength and challenge, and will provide COM faculty and students insight to guide COMLEX-USA Level 1 preparation.

Should you have any questions about COMAT or the new FBS examinations, please visit the COMAT pages of our website.

Contributed by: Michael Finley, DO | Senior VP for Assessment | NBOME

CATALYST | Continuous Learning Platform

Wouldn’t it be nice if you could make learning new material easier? And what if you could avoid taking another traditional multiple-choice exam to demonstrate what you’ve learned?

Inspired by research from leading cognitive psychologists, as well as by the success of the American Board of Anesthesiology’s MOCA-Minute, the NBOME began its research journey into developing its own continuous learning platform – CATALYST.

CATALYST is a formative assessment platform designed as an alternative to traditional physician competence and practice-relevant assessment. Based on the outcomes of several successful pilots conducted in 2017 and 2018, the NBOME has expanded its partnership with ITS to develop a more sophisticated platform that can be customized to meet various client assessment needs.

With the primary goal of eliciting user feedback on the newly designed platform, the CATALYST 2.0 Platinum Pilot was released on June 5. Participants included osteopathic medical students, residents, NBOME National Faculty and NBOME staff. Learners were asked to answer 70 multiple-choice questions during a five-week period and were offered a choice of receiving 2 items a day, 14 items a week, or all 70 items at once. Following the completion of each item, participants were asked to gauge their confidence in answering the question and the question’s relevance to their specialty / field of study. Whether or not the question was answered correctly, the participant was provided with immediate feedback including the correct answer, a rationale for the answer, as well as references and links to additional learning resources.

Feedback has been very positive — 94% said the platform met their expectations and 91% found the system easy to navigate. What did participants like most about CATALYST? One responded that it was “very easy to navigate, good questions.” Another “liked that the platform showed the learning objectives of each question, helped identify why the question was being asked, and identified what the learning goals were for each question.” And a National Faculty member liked that “it could be done on my own time across multiple platforms and devices.”

The cross-functional CATALYST team, led by Sandra Waters, MEM, VP for Collaborative Assessment & Initiatives, is preparing for the next CATALYST release in September which will include an enhanced dashboard with normative statistics, highlighting of item components, and streamlined registration. Preparation has also begun for the delivery of COMSAE Phase 2 on CATALYST, providing COMLEX-USA Level 2-CE candidates the opportunity for alternative learning through formative assessment.

Contributed by: Dot Horber, PhD | Director for Continuous Professional Development | NBOME

In this section

Browne, M, Wojnakowski M, Horber DT. Choosing Wisely: So Many Options for Assessment Administration. Which will Enhance Your Exam’s Validity and Fairness? Paper presented at the 2019 Innovations in Testing Conference, Orlando FL, March 2019.

Short Description

With advances in assessment, credentialing organizations are presented with myriad options to “enhance” test format and administration. Two organizations have been conducting research and pilot testing to explore some options alone and in combination – use of resources while testing, and, high stakes testing in remote proctored conditions.

Reference availability may increase an assessment’s fidelity to real life clinical situations, but it raises many implementation questions: Which references will be useful and what is the best way to make them available? What is the effect on test time needs, outcomes, and validity?

Remote proctoring is attractive to candidates as a convenience and can offer some cost savings. In reality though, just how easy is it to test from home? What are the security implications? Copyright treats remote proctored tests differently; how can this be addressed?

The presenters will discuss obstacles encountered, comparison of outcomes, and best practices found.

Full Description

With advances in assessment, credentialing organizations are presented with myriad options to “enhance” test format and administration. Two organizations have been conducting research and beta testing to explore some options alone and in combination – use of resources while testing, and, high stakes testing in remote proctored conditions.

As certification organizations move toward nontraditional assessments, provision of reference resources during assessment is one of many areas of uncertainty. Although reference material availability likely increases an assessment’s fidelity to real life applicable clinical situations, it raises many implementation questions as well as concerns about test outcomes and validity.

Remote proctoring, long a hot topic, has rarely been contrasted with in-person proctoring in a high stakes examination. The differences that materialized in candidate acceptance, test administration and outcomes can inform much constructive discussion.

One organization is researching options for continued professional certification for its 50,000-plus certificants. A 1,500-participant research study incorporating open-book features and different proctoring conditions was completed in October 2018. The research divided the participants into six different experimental conditions – in-person proctored vs remote proctored, no resources, e-resources, and hard copy resources. Presenters will discuss development of the research design as well as the research outcomes.

Presenters from the second organization will discuss aspects of the development of an innovative item format that focuses on competency domains other than clinician knowledge recall. This incorporates the use of online resources to locate clinical diagnostic and treatment information to answer questions. The item format contains a clinical case scenario with associated multiple-choice items that would require most examinees to access online resources in order to answer the questions. In current day-to-day practice. Presenters will discuss the item development process and relate quantitative and qualitative data obtained from the Proof-of-Concept study. Lessons learned from this study and planned next steps will provide insights to organizations seeking more authentic modes of assessment of clinical behavior and decision-making.

NCME Paper 2019

The Effects of Test Familiarity on Person-Fit and Aberrant Behavior

Hotaka Maeda, Ph.D. & Xiaolin Wang, Ph.D.

Abstract (50 words)

The person-fit to the Rasch model was evaluated for examinees taking multiple subject tests with a similar structure. The evaluation considered which test in the sequence (i.e., first, second) was taken. Compared to an examinee’s first test, person-fit improved for later tests. Test score reliability may improve with test familiarity.

Introduction

Aberrant behaviors are unusual test-taking behaviors that introduce noise to test data. They introduce nuisance constructs that are not intended to be measured and thus threaten measurement validity. One source of aberrant behavior is unfamiliarity with tests (Meijer & Sijtsma, 2001; Rupp, 2013). Examinees who take a new and unfamiliar test are likely to struggle to understand the test structure, gauge how much time they have for each item, navigate through a computer-based test, and handle their nerves. In contrast, examinees who are familiar with the test structure are likely to be less stressed, know how to prepare, and be able to complete the test efficiently. Compared to first-time takers’ results, scores for examinees who are familiar with the test structure may be less affected by the nuisance construct of test unfamiliarity and be more representative of their underlying ability. To the authors’ knowledge, this speculation has not been investigated and reported in the literature. Therefore, the purpose of this study is to examine the effects of test familiarity on person-fit and aberrant behavior using observed data.

Method

The instrument used in this study is a comprehensive medical achievement examination composed of eight clinical subject tests. Medical students typically take the test at the end of their clinical rotation in a given clinical subject. All clinical subject tests are structured identically:

They are administered through the same platform.
Item stems are worded similarly as they all target commonly encountered patient scenarios.
All items in all tests are multiple-choice items with only one best answer.

Many examinees take all eight clinical subjects, but they do not take them in the same order. They can also choose to retake any clinical subject test. Therefore, the context of the instrument used in this study can be considered a quasi-experimental setting for assessing the effects of test familiarity on person-fit and aberrant behavior, where test familiarity can be defined by the number of clinical subject tests (including retakes) a candidate has taken.

Response data in all clinical subjects from July 2017 to June 2018 were used. Exploratory factor analysis with no rotation was conducted for each subject separately in order to identify high-quality items. Items were removed from the data if the factor loadings on the first dimension were less than 0.1. Then, the data were modeled using the Rasch model. For each subject, test forms were equated through concurrent calibration. Ability was estimated with maximum likelihood, which was standardized as N(0,1) and bound between [-5, 5] so that the values could be compared across subjects.

Aberrant behavior was assessed using the lz* person-fit statistic (Snijders, 2001). The lz* is asymptotically distributed as N(0,1), where positive values represent good person-fit, and negative values represent poor fit. If examinees respond to the items in a reasonable manner (e.g., not aberrant because of the familiarity of tests), lz* should be a high value, which shows that their responses fit well to the model. The lz* is uncorrelated with ability when aberrant behavior is not present. One of the typical cutoffs for determining poor person-fit is -1.645, which is equivalent to the one-tailed .05 alpha level.

The degree of person-fit (i.e., lz*) was regressed on the sequence of tests using two separate two-level random intercept models. As examinees took multiple tests, the tests were modeled as nested within examinees. Model 1 included three exam-level predictors: 1) examinee age in years at the time of the exam, 2) standardized test score, and 3) whether the subject being taken is a retake. The only predictor at the examinee-level was the number of times the person had ever retaken any clinical subject test (0, 1, 2, and >2). The model could be written as:

Model 1: lz* ~ age + test.score + subject.retake + total.retake

Model 2 included all the predictors in Model 1 in addition to the test sequence as a categorical variable from 1 to 11 (i.e., the order in which the examinees took the test, such as first test, second test, etc.).

Model 2: lz* ~ age + test.score + subject.retake + total.retake + test.sequence

The test sequence for some students did not start with “first” if they had taken the tests prior to July 2017. The test sequence can extend longer for students who retake some clinical subject tests.

Residual plots were used to confirm that the residuals were approximately normally distributed with the same mean and standard deviation at every fitted value. Because Model 1 was nested within Model 2, they were compared using a likelihood-ratio test.

Result

For the purpose of this specific study, 1,422 out of 5,594 items were removed from analysis, many of which were pretest items. All subjects achieved unidimensionality after the removal of such items. In addition, response data from 55 tests were removed because of an abnormally high test sequence due to retakes (12 or more). The final sample size across all test subjects was 4,172 items on 42,903 test administrations given to 10,135 examinees (see Table 1). Each test contained an average of 96.7 items (SD = 9.3). A majority of examinees had no history of retaking any clinical subject test (68.4%). Only 6.7% of the tests were retakes.

Table 1. Number of Exams by Sequence and Clinical Subject

Test Sequence	Clinical Subject
Test Sequence	A	B	C	D	E	F	G	H	Total
1	71	672	568	394	2656	585	468	507	5,921
2	138	1,261	881	809	476	730	646	678	5,619
3	172	705	744	824	577	769	767	884	5,442
4	192	700	760	825	473	744	753	738	5,185
5	198	697	660	721	642	803	681	774	5,176
6	231	598	737	705	924	698	676	667	5,236
7	527	590	683	617	574	615	629	689	4,924
8	1181	352	334	288	575	262	287	353	3,632
9	207	124	175	90	109	99	90	125	1,019
10	228	51	64	26	75	37	41	35	557
11	75	4	7	8	80	8	6	4	192
Total	3,220	5,754	5,613	5,307	7,161	5,350	5,044	5,454	42,903

Note. Many examinees take all eight clinical subjects, but they do not take them in the same order. Although there are only eight clinical subjects, the test sequence can extend beyond eight because of retakes.

Mean lz* was 0.04 (SD = 1.09), while mean standardized test scores was 0.02 (SD = 1.14). Mean SE of the standardized test scores was 0.51 (SD = 0.07). Mean standardized test scores for those who had a history of retaking any clinical subjects test was lower (M = -0.36, SD = 1.13) than those who did not (M = 0.25, SD = 1.08). The percent of the test records exhibiting poor person-fit (i.e., lz* < 1.645) was 6.7%. Standardized test scores were positively correlated with lz* (r = .23).

A likelihood-ratio test showed that the addition of the test sequence predictor significantly improved the model fit, χ²(10)=75.05, p<.001. Controlling for examinee age, total historical test retake count, whether the subject being taken is a retake, and standardized test score, the student person-fit was the poorest for the first test compared to all later tests (p < .05). The coefficients from Model 2 are shown in Table 2. Compared to the first test, person-fit improved for the second exam by 0.07, and on the 11^th test by 0.27.

Table 2. Model 2 Coefficients

	Coef	SE	df	t	p
(Intercept)	0.41	0.05	32,754	8.65	<.001
Examinee-level predictors
Retake total = 0 (Reference)	–	–	–	–	–
Retake total = 1	-0.04	0.02	10,131	-2.62	.009
Retake total = 2	-0.11	0.02	10,131	-4.61	<.001
Retake total > 2	-0.19	0.03	10,131	-6.73	<.001
Test-level predictors
Standardized score	0.19	0.01	32,754	38.17	<.001
Examinee age in years	-0.02	0.00	32,754	-9.30	<.001
Retaking the clinical subject	0.10	0.02	32,754	4.44	<.001
Test sequence = 1 (Reference)	–	–	–	–	–
Test sequence = 2	0.07	0.02	32,754	3.63	<.001
Test sequence = 3	0.08	0.02	32,754	4.27	<.001
Test sequence = 4	0.10	0.02	32,754	5.03	<.001
Test sequence = 5	0.13	0.02	32,754	6.62	<.001
Test sequence = 6	0.13	0.02	32,754	6.57	<.001
Test sequence = 7	0.10	0.02	32,754	4.66	<.001
Test sequence = 8	0.13	0.02	32,754	5.86	<.001
Test sequence = 9	0.10	0.04	32,754	2.79	.005
Test sequence = 10	0.20	0.05	32,754	4.12	<.001
Test sequence = 11	0.27	0.08	32,754	3.27	.001

Note. Person-fit was modeled using a two-level random-intercept model.

Model 2 also showed that those who had a history of retaking any clinical subject test tended to have lower person-fit than those who did not (p <.05). However, retaking the same clinical subject test was associated with an increase in person-fit by 0.10 (p <.001).

Discussion

This study shows that person-fit to the Rasch model improves as examinees gain experience in taking a series of tests with a similar structure. Improvements in person-fit were observed beyond the first and second tests. Test familiarity increased lz* by 0.1 or more. For reference, an increase in lz* from 0 to 0.1 is equivalent to an increase in person-fit by 3.98 percentiles. The findings indicate that the reliability of the test scores may improve with test-taking experience, and they show the importance of examinee familiarity with the test structure. The improvement in person-fit by increased test familiarity supports the provision of practice materials in order to minimize the negative impacts from test unfamiliarity and to promote measurement validity.

When interpreting the data, retakes of the same clinical subject exams needed to be considered. The option to retake any test allowed the test sequence to go beyond the number of available clinical subjects (i.e., eight). Clearly, a person who has taken the same test multiple times (despite taking a different form every time) should be more familiar with the test than the first-time takers. The examinees who have retaken any of the clinical subject exams tend to be lower achievers and have lower person-fit compared with non-retakers. However, their person-fit improved upon retaking the same clinical subject test. Also, results suggest poor person-fit occurred due to spuriously low aberrant behavior (i.e., poor performance) such as running out of time, more often than spuriously high-scoring behavior such as item pre-knowledge. This led many of the poor performers to retake the test. However, regardless of the test-retaking behavior, familiarity of the test structure led to increases in person-fit.

The study is limited in that we did not directly investigate whether improvement in person-fit is in fact associated with an increase in the accuracy of the standardized test scores. This is rather difficult to show empirically, but it should be pursued in the future. Further, a quasi-experimental design was used, where some factors were uncontrolled, including allowing examinees to retake any test at their own will. These test-retaking patterns were not random as they were correlated with important variables such as the standardized test scores. The study should also be replicated using other psychometric models and test data.

References

Meijer, R., & Sijtsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.

Rupp, A. A. (2013). A systematic review of the methodology for person fit research in Item Response Theory: Lessons about generalizability of inferences from the design of simulation studies. Psychological Test and Assessment Modeling, 55, 3-38.

Snijders, T. (2001). Asymptotic null distribution of person-fit statistics with estimated person parameter. Psycho

New York Colleges of Osteopathic Medicine Educational Consortium (NYCOMEC)

This presentation to osteopathic medicine residency directors focused on preparing for their Clinical Learning Environment Review (CLER), an ACGME program instituted as part of the next accreditation system. The goal of CLER is to ensure residency programs train residents to ensure patient safety. The presentation focused on what is required to ensure patient safety, i.e. “learner safety.” It presented how to debrief residents using “good judgment” (a focus on performance gaps) and “empathic inquiry” (debriefing that develops self-reflection and self-correction). The talk provided examples of effective and ineffective feedback and debriefing approaches.

Parshall C, Julian E, Parikh S, Horber DT. Using Nudges for More Effective Exam Programs. Paper presented at the 2019 Innovations in Testing Conference, Orlando FL, March 2019.

Short description:

Nudges are small, deliberate tactics we can use to help our test-takers (and our SMEs) do the things they want to do. While our testing programs have many points that can derail candidates, through small and subtle changes we can help them persist through the life cycle of application, testing (and retesting), and ongoing certification. For example, framing tactics in messaging can effectively decrease the number of test-takers who fail to show. Nudges can also be used with SMEs to increase JTA survey response rates and committee volunteer numbers. Join us for a panel discussion with researchers and practitioners using nudges in testing.

Full description:

Behavioral nudges have been used forever to help people remember to do things, or follow through on things they started. New research has identified the strategies that are most effective, as well as the research tools for increasing their success in a specific environment. As a result, the use of nudges is moving from ad hoc to intentional and systematic. Educators, corporate offices, and governmental institutions are formally incorporating nudges into their interactions with the public and their staff, and testing programs can use them to support examinees, subject-matter experts, staff, and employers in doing what they already want to do.

The underlying goal is to influence, or “nudge,” people in positive ways that are in their own best interest, as defined by themselves. This presentation will discuss ways that a variety of testing programs are already using nudges and will share the evidence of their effectiveness.

This session will have a panel that includes researchers and practitioners effectively using nudge tactics in the field of testing. They will share real-world successful (and unsuccessful) examples of nudges in testing.

Presentations will include:

an overview of nudges: what they are, the evidence for their effectiveness, and a simple research plan for implementing nudges effectively.
a discussion of common areas in testing programs where people have agreed to do things, but often need help carrying them out: e.g., examinees would benefit from nudges to meet registration deadlines, study, stay honest, show up for the test on time with appropriate accouterments; SME’s would benefit from nudges to volunteer, write items, review items.
a presentation on before-and-after data on how timely phone calls decreased candidates “no-show” for a medical licensure performance exam; additional nudge interventions from the program’s in-development continuous assessment will be included.
a case study of nudging applied in a high school equivalency program, with specific behavioral techniques and overall results.

References:

Ariely, D. (2008). Predictably Irrational: The Hidden Forces That Shape Our Decisions. New York: HarperCollins.

Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus, and Giroux.

Thaler, R.H., & Sunstein, C.R. (2008). Nudge: Improving Decisions About Health, Wealth, and Happiness. New Haven, CT: Yale University Press.

Authorship

Kimberly M. Hudson, PhD, National Board of Osteopathic Medical Examiners

Yue Yin, PhD, University of Illinois at Chicago

Tsung-Hsun Tsai, PhD, National Board of Osteopathic Medical Examiners

Grant Number/ Funding Information

Not applicable

Corresponding Author

Kimberly Hudson, 8765 West Higgins Road, Suite 200, Chicago, Illinois 60631; 773-714-0622; Kimberly.shay86@gmail.com

Key Words

Equating, Automated Test Assembly, Optimal Test Assembly, IRT, Rasch Model, CINEG

Abstract

As early as the 1960s, testing organizations began implementing Automated Test Assembly (ATA) to simplify the laborious process of manually assembling test forms and to enhance the psychometric properties of the examinations (Wightman, 1998; van der Linden, 2005). But it is unclear what impact transitioning to ATA has on equating outcomes. The purpose of this research study was to evaluate outcomes from different IRT scale linking and equating methods when a testing organization transitioned from manual test assembly to ATA.

After crossing each scale linking procedure with each equating method, I calculated error and bias indices (e.g., RMSD, MAD, MSD) and evaluated the decision consistency of the equating outcomes.

The results showed that the mean/mean scale linking procedure paired with the IRT preequating method produced the lowest bias and error, and highest level of decision consistency.

The results of this study support the importance of aligning psychometric and test development procedures. The findings of this study suggest that the equating outcomes were related to the similarity in statistical test specifications. ATA resulted in more parallel test forms with better psychometric properties than forms assembled manually. Therefore the modifications to assembly practices warrant the reconsideration of a new base form for scaling and standard setting.

Introduction

In high-stakes medical licensure testing programs, test developers and psychometricians work together to develop multiple test forms that can be administered simultaneously to examinees to enhance examination security. Although the volume of forms may differ between testing programs, it is crucial that all test forms are built according to the same test specifications (von Davier, 2010). Furthermore, scores on the test forms must be interchangeable and candidates should perceive no difference between the test forms administered (Kolen & Brennan, 2014). The test development processes and psychometric procedures are inherently connected and both must be considered when developing multiple test forms.

Traditionally, test developers have manually assembled multiple test forms according to a set of content requirements. Test developers typically evaluate statistical criteria such as mean proportion of correct responses (p-value) or mean point-biserial correlation upon completion and make adjustments to confirm that statistical specifications are met. Manual test assembly (MTA) is a time-intensive process, typically requiring the attention and work of multiple test developers. However with the widespread use of computers, testing organizations can improve the laborious manual process by developing and employing computer programs to automatically assemble tests. If staff members possess technical computer programming skills, they might create computer programs that can assemble multiple test forms simultaneously by balancing the content and statistical constraints.

When assembling tests manually, test developers use a variety of informational inputs, or constraints, to create multiple forms of an assessment that are balanced in terms of content, difficulty of items, item formats, contextual information of items (e.g., the patient’s life stage), item duration, word count, and exposure rate. Test developers first compile an item pool, which contains a selection of items that meet some basic requirements for inclusion on a test. Scorable items function as operational or anchor items and often have known item parameters based on a prior administration. Test developers iteratively select a group of items that meet the minimum proportions of each domain as specified by the test blueprint and evaluate the range of item statistics or average item statistics, such as p-values and point-biserial correlations. The number of parallel test forms and the number of constraints undoubtedly impacts the complexity of manually assembling forms. Moreover, many testing organizations implement this resource-intensive process across numerous testing programs on an annual or semi-annual basis.

Automated Test Assembly (ATA) is an efficient alternative to this laborious process with unique challenges (Wightman, 1998). Unlike MTA, ATA programs utilize the test information, the summation of item information across the ability continuum, in the creation of multiple parallel test forms. Thus, ATA improves the manual procedure by not only saving time and resources, but also enhancing the psychometric quality of balanced forms according to a predetermined set of constraints and maximization of the specified objective function. ATA may improve reliability across examination forms due to the standardization of the test development process. Therefore the impact of ATA is not just a question of “Can the computer do it,” but rather “Can the computer do it better?”

In medical licensure examinations there is a critical need for score comparability across test forms to not only ensure that scores are an accurate, reliable representation of examinee ability, but also to make pass/fail distinctions based on the scores. Earning a passing score on a medical licensure examination allows examinees to enter into supervised medical practice. Therefore psychometricians work to maintain decision consistency, regardless of the test assembly method and the form administered to examinees. Decision consistency refers to the agreement of an examinee’s pass/fail decisions on two (or more) independent administrations of unique forms and decision accuracy refers to the agreement between an examinee’s pass/fail decision and whether the same decisions made based on an examinees’ true ability (Livingston & Lewis, 1995). These two indices are necessary to evaluate in high-stakes medical licensure testing. In this research, I compare the decision consistency of equated results after implementing ATA.

The results of this research provide a psychometric framework to evaluate results from different equating methods upon the implementation of ATA. When testing organizations implement new test development processes, it is critical to examine the impact on examinee scores (AERA, APA, & NCME, 2014). Testing organizations monitor and evaluate scores and decision consistency of scores on examinations that ultimately license examinees to practice medicine in supervised or unsupervised settings. Neglecting to examine this may inadvertently lead to passing unqualified physicians, or failing qualified physicians.

In ATA, psychometricians and test developers often define linear and/or non-linear constraints in order to maximize a specific objective function, typically the test information function (TIF), at a given score point on the true-ability continuum (van der Linden, 2005). In a high-stakes licensure examination, the minimum passing standard (or cut-score) is commonly used for optimization because it maximizes test information near the cut-score and minimizes the standard error of measurement (SEM) at the cut-score. This leads to increased reliability of scores closest to the cut-score and better accuracy of pass/fail distinctions. Therefore, ATA is designed to enhance the psychometric qualities based on prior item information (i.e., higher reliability coefficients, and lower standard error of measurement near the cut-score), and the efficiency of assembling test forms. However, research has not yet addressed the impact of transitioning from MTA to ATA on results from equating methods. In this study, I investigate the differences in equated results between MTA and ATA forms.

Most ATA processes use a framework of Item Response Theory (IRT) to construct forms with computer programs integrating item-level information according to a set of predetermined constraints. The use of IRT typically goes hand-in-hand with the psychometric framework utilized by the testing program. In IRT, items have a set of unique characteristics; some items are more informative than others at different ability levels. Psychometricians investigate the individual contribution of an item to a test by reviewing the item information function (IIF). The TIF is the summation of IIFs across the ability continuum. The TIF in the ATA represents the characteristics and composition of all items on each test form. Moreover, in the context of medical licensure examinations, using the minimum passing standard as the value for optimization ensures that scores are precise ability estimates for minimally qualified examinees. Thus, when the TIF is optimized at the cut-score, it ultimately reduces the probability of Type I error (unqualified examinees passing the examination). Furthermore, Hambleton, Swaminathan, and Rogers (1991) suggest that the test characteristic curve (TCC) creates the foundation for establishing the equality of multiple test forms, which is certainly the case when optimizing the TIF. The TIF provides aggregate information from each item on the examination, whereas the TCC shows the probability of an expected raw score with a given ability level, . If we wish to create parallel test forms, then the TCC provides evidence that a given ability level relates to similar expected scores for two parallel forms of the same test. Furthermore, the use of content and statistical constraints in ATA computer programs provides evidence that all test forms are balanced in terms of statistical specifications.

Once parallel forms are assembled, reviewed, published and administered, the results must be analyzed and equated. Equating refers to the use of statistical methods to ensure that scores attained from different test forms can be used interchangeably. Equating can be conducted through a variety of designs, approaches and methods (Kolen & Brennan, 2014). Although there are key differences between IRT and the Rasch model, this research will focus on the applicability of IRT equating methods to a testing program that utilizes the Rasch model as a psychometric framework. Within IRT equating methods, both preequating and postequating methods are widely implemented in K-12 educational settings to ensure scores can be used interchangeably (Tong, Wu, & Xu, 2008). Psychometricians may use IRT to preequate results prior to the start of examination administration, which assuages the tight turnaround time between examination administration and score release. Alternatively, postequating methods use response data from complete current examination administrations (Kolen & Brennan, 2014).

In IRT preequating methods, item parameters are linked from prior calibration(s) to the base form of an examination. For the purpose of this research, item difficulties will be the only item parameter used, which is in alignment with the testing program’s psychometric framework (the Rasch model). The base form (denoted as Form Y) is the form in which the cut-score was established. In order to implement preequating methods, item difficulties for scorable items must be estimated prior to examination administration. Prior to ATA, scorable item difficulties must be known to calculate and maximize the TIF. The alignment of previously calibrated item statistics that are used both for assembling forms using ATA and for preequating may support the applicability of this equating method.

Measurement Models and ATA

IRT allows test developers to “design tests to different sets of specifications and delegate their actual assembly to computer algorithms,” (van der Linden, 2005, p.11). By setting constraints for computerized test assembly, including blueprint domain representation or reasonable ranges for item statistics, test developers can create multiple forms of examination that are parallel in difficulty. ATA can incorporate item details regardless of the psychometric paradigm used to calibrate or score examinees and can be applied to polytomously or dichotomously scored examination. As discussed previously, this study uses data previously calibrated using the Rasch model.

While CTT, IRT, and Rasch approaches to ATA can utilize population dependent item statistics (i.e., p-values and discrimination indices) as constraints, in CTT there is no equivalent metric to the TIF. In order to construct parallel test forms in ATA, Armstrong, Jones and Wang (1994) maximized score reliability through a network-flow model. The authors stated that it was advantageous to use the CTT approach because it was computationally less expensive and produced comparable results in relation to the IRT approach to ATA. When this research was published, computational power was indeed a challenge; however, advances in computer memory and technology are much greater now, so the cited advantage does not hold the test of time. As such, IRT or Rasch approaches to ATA are more supported in the literature and are the focus of this study.

Prior to beginning ATA, test assemblers must calibrate response data to estimate item parameters from the sample population. Psychometricians often examine the goodness-of-fit of the data to determine the best IRT model (i.e., 1-PL, 2-PL) or confirm that the data fit the Rasch model. Once the examination is administered, psychometricians anchor item parameters based on prior calibrations to estimate examinee ability (van der Linden, 2005). In this study, I calibrate data using the Rasch model and will provide some evidence supporting the appropriateness of the model.

In ATA, test assemblers often optimize the TIF and evaluate the similarity of the forms by comparing the TIFs. However, even well-matched TIFs do not necessarily yield equitable score distributions (van der Linden, 2005). Thus, psychometricians must also continuously evaluate and monitor the score distributions once the examination forms are administered. The main question of this study is which IRT equating method (IRT observed score, IRT true score, or IRT preequating) yields the most comparable scores and decision consistencies when transitioning from MTA to ATA. In the following section, I provide a foundation of linking, equating, and scale linking as it pertains to this study.

Equating

Equating is the special case of linking in which psychometricians transform sets of scores from different assessment forms onto the same scale. By definition, equating methods are only applied to assessment forms that have the same psychometric and statistical properties and test specifications. The primary goal of equating is to allow scores to be used interchangeably, regardless of the form that an examinee was administered (Holland & Dorans, 2006; Kolen & Brennan, 2014).

Assessment programs can employ a variety of equating designs and methods, each design with unique characteristics and assumptions. Assessment programs often administer examinations within and across years. For the purpose of this section, I notate an original form of an examination as Y and a new form of an examination as X, with the understanding that assessment programs may administer multiple new forms ( ) or multiple original forms ( ). CINEG design are commonly used and require previously administered items from original forms of an examination to be included on new forms by a set of common or anchor items. The CINEG design is considered a more secure design than the random groups design because only a set of common items are exposed from an original form, rather than exposing an entire original form.

The CINEG design not only accounts for the difference in form difficulty, but also accounts for the difference in the population of test-takers. The statistical role of the common items is to control for differences in the populations, therefore removing bias from the equating function. In order to implement the common items design, the common items must meet several requirements (Dorans et al., 2010). First, the common items must follow the same content and statistical specifications as the entire full-length test. Second, there should be a strong positive correlation between scores on the full-length test form and scores on the common items because the common items follow the same specifications as the full-length test. Thirdly, measurement and administration conditions for the common items must be similar across new and original forms. Lastly, prior research recommends the use of common item sets include at least 20% of the full-length test, or consist of at least 30 items (Angoff, 1971; Kolen & Brennan, 2014). Satisfying these requirements ultimately ensures that the reported scores and the decisions based on the reported scores are accurate and reliable. The testing program used for this study meets the conditions described above.

IRT equating methods can be applied to data calibrated using the Rasch model and are the focus of this study. In this section, IRT equating methods are discussed in detail; however, first psychometricians must use scale linking procedures to examine the relationship between newly estimated item parameters and original estimations of item parameters from two independent calibrations. Due to the assumption of item invariance, if item parameters are known, no equating or scale linking is necessary and IRT preequating methods can be implemented prior to test administration (Hambleton et al., 1991). However, in practice it is important to implement scale linking procedures because there are often differences in item parameter estimates (Stocking, 1991).

Scale linking is the process by which independently calibrated item difficulties are linked onto a common scale. Several methods can be used to calculate scaling constants in order to place the item difficulties from form X on the same scale as Y (Hambleton et al., 1991). The mean/mean, mean/sigma, and TCC methods are discussed in their application to this study. Prior research supports the performance of TCC methods over other methods (i.e., mean/mean or mean/sigma) for scale linking due to the stability of the results and the precision, even when item parameters had modest standard errors (Kolen & Brennan, 2014; Li et al., 2012). Other research investigated the adequacy of different scale linking procedures within the Rasch model.

The mean/sigma method calculates scaling constants A and B based on the mean and standard deviation of the difficulty parameters of the common items on form X. There are two main TCC scale linking procedures, which are iterative processes that utilize item parameter estimates; the focus of the current student is on the Stocking and Lord (1983) procedure. The scale indeterminacy property of IRT is used in this method, such that an examinee with a given ability will have the same probability of answering an item correctly regardless of the scale used to report scores. The Stocking and Lord TCC procedure calculates the probability of correctly answering an item on the original scale ( ) and the new scale ( ) for each common item ( ) by taking the difference in examinee ability into consideration. Equation 10 represents the difference in TCCs ( ) between common items administered on form Y and form X, respectively. Then an iterative process solves for A and B by minimizing across all examinees.

(10)

(11)

Once item parameters are on the same scale, IRT equating methods are employed. IRT true score equating is the most commonly used IRT equating method. In IRT true score equating, true scores ( ) are represented as the number-correct score for examinee ( ) with given ability ( ; Kolen & Brennan, 2014). Additionally, true score equating assumes that there are no omitted responses (von Davier & Wilson, 2007). In a simplistic example, psychometricians first identify a true score on form X, then estimate the corresponding ability level is determined (see equation 12). Then, the true score on form Y ( ) is determined by using the corresponding ability level (see equation 13). Therefore, the equivalent score is the inverse of the ability distribution. This process is iterative, which typically involves the Newton-Raphson Method (Kolen & Brennan, 2014; Han et al., 1997).

and (12)

(13)

Unlike IRT true score equating methods, the IRT observed score equating method depends on the distribution of examinee abilities. The IRT observed score equating method is similar to equipercentile equating methods without the application of additional smoothing techniques, as previously discussed. It requires specifying the distributional characteristics of examinees prior to equating, using prior distributions (Kolen & Brennan, 2014).

All of the IRT equating methods previously discussed require data from the current test administration cycle. However, the IRT preequating method can be used when items are pretested prior to operational use. Once items are on the same scale, psychometricians generate raw-to-scale conversion tables prior to form administration, which ultimately decreases the workload for score release (Kolen & Brennan, 2014). Many testing organizations utilize IRT preequating in order to shorten the window for score release after examination administration. Testing organizations may also prefer IRT preequating methods due to their flexibility when equating scores for computer-based examinations that are administered intermittingly over a long testing cycle.

Researchers have compared the results among equating designs, methodologies and procedures; yet no researchers have compared scale linking or equating outcomes among ATA and MTA forms. In the current study, RMSD, MSD and MAD were calculated to examine the error and bias associated with scores. Researchers have commonly used these indices to evaluate the comparability of equating methods (Antal, Proctor, & Melican, 2014; Gao, He, & Ruan, 2012; Kolen & Harris, 1990).

Decision Consistency

In the context of testing programs that aim to categorize examinees into one or more groups based on their scores, such as medical licensure examinations, classification accuracy is a measurement of whether examinees were accurately classified based on their true ability (Lee, 2010).

Research Questions

The goal of this study is to compare the equating results when a testing organization moved from MTA to ATA. The research questions address the comparability of outcomes from three different methodological approaches to equating, after combining three IRT equating methods with three scale linking procedures.

Which method of IRT equating (e.g., IRT observed score, IRT true score, or IRT preequating methods) minimizes error and bias associated between MTA and ATA developed forms?
Which method of IRT equating (e.g., IRT observed score, IRT true score, or IRT preequating methods) yields the highest expected decision consistency of pass/fail distinctions between MTA and ATA developed forms?

METHOD

Data

In this study I used two years of response data from a large-scale medical licensure examination. From the 36 Y forms, I selected four forms (denoted ). There was item overlap among the 36 Y forms, which made it possible to concurrently calibrate data from the 36 Y forms simultaneously using the Rasch model.

First, I aggregated key information for each form, where pretest items were not embedded within each Y form. The pretest design used a total of 12 unique pretest blocks, each consisting of 50 items with overlap. The test administration vendor randomly assigned pretest blocks to examinees. Therefore, pretest items needed to be reviewed and selected for the form by form calibration for CINEG design. Figure 5 shows the design of an intact Y form (denoted form A) of operational items and plausible assignment of six pretest blocks. In Figure 5, Form A consists only of operational items, the test vendor randomly assigned a pretest blocks from group A (PTA) and a pretest blocks from group B (PTB). The diagram is a simplified depiction of the true design, which can ultimately yield more than 5,100 different combinations. Therefore, I employed a threshold of 30 responses to determine which pretest items had sufficient exposure for inclusion in the form by form calibration. Despite anchoring the item difficulties, at least 30 exposures ensured there was sufficient data to investigate data-model fit.

The concurrent calibration of the operational and pretest items on Y forms resulted in item difficulties on the same scale of measurement. Additionally, I selected four X forms (denoted ). I used three criteria to select the eight forms for this study: (a) X forms with the highest volume of administrations after the first several weeks following the examination launch, (b) X forms and Y forms with at least 20 percent overlap or at least 30 common items for scale linking purposes, and (c) the common item set was representative of the test blueprint (Angoff, 1971; Kolen & Brennan, 2014). The data design is shown in Figure 6. The common set of items on and is denoted as , the common set of items on and is denoted as , the common set of items on and is denoted as , and the common set of items on and is denoted as .

Approximately 7,600 examinees took one of the Y forms in year 1 and 4,300 first-time examinees took one of the X forms in the first testing window of year 2. After selecting the four forms, as previously described, I used data from the approximately 1,300 examinees who were administered or and the approximately 1,200 examinees who were administered or . Table IV displays a summary of the data selected for this research study.

Response data from year 1 on all 36 Y forms were concurrently calibrated using WINSTEPS® (Linacre, 2017). The estimated item difficulties were then used as anchors for each separate form calibration of . Y is considered the base form of the examination and therefore no equating on original forms was conducted.

Data Analyses

All data management and analyses were conducted in RStudio (2016), unless otherwise specified. The criterion of was used to examine the statistical significance of tests, unless otherwise specified.

1. Research Question 2: Equating Methods and Error

Which method of IRT equating (e.g., IRT observed score, IRT true score, or IRT preequating methods) minimizes error and bias associated between MTA and ATA developed forms?

I employed three scale linking approaches (mean/mean, mean/sigma, and Stocking-Lord TCC) and three equating methods (IRT observed score, the IRT true score, and the IRT preequating). I utilized the PIE computer programming to implement IRT observed score and IRT true score equating methods (Hanson et al., 2004b). To assess the equating results, I compared the root mean squared difference (RMSD), mean absolute difference (MAD), and mean signed difference (MSD) on X’ to Y. I then evaluated which method minimizes bias by identifying RMSD values close to 0 and evaluated which method minimizes error by identifying MSD and MAD close to 0. Higher indices indicate an accumulation of error and are not preferred. Findings from prior research show that IRT preequating methods often have higher levels of error associated with the examinee scores. However, due to the alignment of using precalibrated item difficulties for both ATA and preequating methods, I expect that the design of ATA may have an impact on the equated results.

2. Research Question 3: Equating Methods and Passing Rates

Which method of IRT equating (e.g., IRT observed score, IRT true score, or IRT preequating methods) yields the highest expected decision consistency of pass/fail distinctions between MTA and ATA developed forms?

Using the outcomes from research question 1, I estimated decision consistency indices using Huynh’s methodology (1990), which uses the probability density function, item curve functions (ICFs) and relative frequencies of a single population to estimate to common decision consistency indices: a raw agreement index, and kappa, (see equations 18 and 19). The raw agreement index, is calculated using the cumulative distribution function of test scores, and relative frequencies of test scores. Kappa is calculated as the difference between the raw agreement index, and , the expected proportion of consistent decisions if there is no relationship between test scores. Kappa indicates the decision consistency beyond what is expected by chance (Subkoviak, 1985).

(18)

Where

, (19)

, (20)

And

(21)

Where represents the ability level at a given raw score, ;

represents the difference in cumulative distribution functions of the raw cut-score, at ability level, ;

represents the relative frequency distribution at and

represents the number of classifications.

RESULTS

Research Question 2

To evaluate the adequacy of the results, I calculated the RMSD, MSD, and MAD. RMSD is a measure of bias, and MSD and MAD are measures of random error. Values closer to 0 indicate no raw score point differences between MTA and ATA forms. Overall, there were large differences in the amount of bias and error associated across forms and equating methods, therefore RMSD, MSD and MAD are presented separately for each form (see Table XV). Across all forms the equating and scale linking method with the least amount of error and bias was the mean/mean preequating method.

Table XV

BIAS AND ERROR INDICES BY EACH SCALE LINKING AND EQUATING METHOD

Observed Score

True Score

Preequating

Form

Index

RMSD

19.35

20.46

21.85

19.51

20.55

22.10

8.02

8.41

8.278

MSD

-18.83

-20.27

-21.38

-18.97

-20.35

-21.62

-7.71

-8.05

-7.96

MAD

18.83

20.27

21.38

18.97

20.35

21.62

7.71

8.05

7.96

RMSD

10.37

5.47

9.40

10.35

5.70

9.40

4.74

5.24

4.91

MSD

-10.31

-4.18

-9.35

-10.29

-4.29

-9.34

-4.31

-4.75

-4.46

MAD

10.31

4.34

9.35

10.29

4.49

9.34

4.32

4.75

4.47

RMSD

2.53

4.57

1.53

2.51

4.53

1.53

2.51

2.96

2.68

MSD

-2.32

-4.45

-0.62

-2.27

-4.39

-0.58

-1.69

-2.08

-1.89

MAD

2.38

4.48

1.29

2.35

4.41

1.28

1.99

2.34

2.15

RMSD

12.42

12.46

15.91

12.92

13.03

16.84

2.55

3.01

2.82

MSD

-11.92

-11.83

-15.29

-12.30

-12.26

-16.02

-1.90

-2.27

-2.20

MAD

11.92

11.83

15.29

12.30

12.26

16.02

2.07

2.43

2.31

Note. MM represents mean/mean scale linking, MS represents the mean/sigma scale linking, and SL represents the Stocking and Lord TCC scale linking procedure. Due to the disparate index values across forms, results are shown for each form separately.

Boldface signifies values more favorable results with indices close to 0 per index per form (by row).

Preequating Method

Across the three equating methods paired with the three scale linking procedures, the results indicated that the mean/mean scale linking procedure with the preequating method performed the most favorably for three of the four forms ( ). For , the mean/mean preequating method resulted in lower bias and error in comparison to all other methods (RMSD = 8.02, MSD = -7.71, and MAD = 7.71), whereas the highest amount of bias was related to the Stocking and Lord TCC procedure paired with the IRT true score equating method (RMSD = 22.10, MSD = -21.62, MAD = 21.62). For , the mean/mean preequating method produced the most favorable results in comparison to all other methods (RMSD = 4.74, MSD = -4.31, and MAD = 4.32). However, the Stocking and Lord TCC scale linking procedure paired with the preequating method produced only slightly higher results than the mean/mean preequating method (within 0.5 raw score points). The small difference of 0.5 raw score points in RMSD, MSD and MAD between the scale linking procedures within the preequating method was present across all forms. For form 3, the mean/mean preequating method produced slightly higher RMSD in comparison to the Stocking and Lord true score equating method (RMSD = 2.51, RMSD = 1.53, respectively). These differences relate to a difference of about 1 raw score point. Therefore, the results from the mean/mean preequating method showed a slight improvement over the other scale linking procedures within the preequating method, although there was very little practical difference in the results across each scale linking procedure.

True Score and Observed Score Equating Methods

The results from the true score and observed score equating methods with each scale linking procedure were comparable across all forms. For , the true and observed score methods yielded very consistent results. Specifically, the mean/mean observed score method and the mean/mean true score method resulted in similar levels of error (MSD = -18.83, MSD = -8.97, respectively) and the maximum deviation between raw scores in terms of MSD of the mean/sigma true score and observed score methods was approximately 0.35 raw score points. Furthermore for , the Stocking and Lord TCC scale linking procedure paired with the observed score and true score methods produced similar high amounts of error (RMSD = 9.400, 9.399, respectively). Unique to form 3, the Stocking and Lord true score and observed score methods produced the lowest bias (RMSD =1.53) and error (MAD =1.279 and MAD = 1.292, respectively) across all other conditions. The results from combining each scale linking procedure with the true score and observed score methods were varied across forms; in some cases, the Stocking and Lord TCC procedure performed least favorably ( ), while in other cases, the mean/sigma scale linking procedure performed the least favorably ( ). Therefore the findings are inconclusive in terms of the preferred scale linking procedure for the IRT observed score and true score methods, although the evidence suggests that the Stocking and Lord TCC procedure produced higher levels of errors for two forms.

Research Question 3

Overall, the mean/mean preequating method and the Stocking and Lord TCC preequating method performed the most favorably ( ). The true score and observed score methods produced similar levels of decision consistency, indicating not much practical difference.

Figure 10. Mean decision consistency indices, (blue) and (red) across all forms. MM represents mean/mean scale linking, MS represents the mean/sigma scale linking, and SL represents the Stocking and Lord TCC scale linking procedure.

The average decision consistency indices across all ATA forms improved in comparison to baseline estimates using MTA forms. For example, the raw agreement index was 1% to 3% greater for ATA forms than MTA forms. For 3 of the 4 forms, the raw agreement index was the highest for preequating methods. Similar to the findings from research question 2, the results from the decision consistency evaluation indicated that the Stocking and Lord TCC scale linking procedure paired with the IRT true and observed score methods performed the most favorably for form 3 (see Figure 11). Results for each form and equating method are displayed in Appendix B.

Discussion

In order to examine the differences in equating outcomes that transitioning from MTA to ATA introduces, I employed three scale linking procedures and three equating methodologies. I calculated the RMSD, MSD, and MAD in order to determine which combination of scale linking procedure and equating method resulted in the least amount of bias and error. The preequating method with the mean/mean scale linking procedure produced the most favorable results for three of the four forms, even when the number of common items did not meet recommended criteria. Lastly, results from the decision consistency analyses indicated that the preequating method outperformed the true score and observed score equating methods in terms of , however the true score and observed score methods produced the most favorable decision consistency in terms of . The variation in error terms of equated scores across forms suggests that MTA and ATA forms cannot be directly compared. If testing organizations begin to implement ATA for form assembly, they should give thoughtful consideration to the use of MTA forms as the base forms for equating purposes.

Research Question 2: Equating Methods and Error

Due to the nature of implementing IRT equating methods following scale linking, the results of the equating methods are based on the quality of the results on the scale linking procedures. The common item sets were sufficiently sized for only form pair 1 and form pair 3; therefore the generalizability of the equating results of forms 2 and 4 are limited. Yet all previous known item statistics were used for preequating, not just those included in the common item set.

It is important to note that for the purpose of this research raw scores were used to evaluate the error and bias associated with scores that were equated for each method. Overall, the optimal method for three of the four forms was the mean/mean scale linking procedure paired with the preequating method. The mean/mean preequating method produced the lowest amount of bias as measured by RMSD. While other methods produced slightly lower MSD or MAD, there was not an appreciable or practical difference in these values from others (typically less than 0.2 raw score points). For the two forms with sufficient common item sets, the mean/mean preequating method and the Stocking and Lord TCC true score methods produced the most favorable results. Similar favorable findings from the mean/mean preequating method were found across the remaining forms; meaning, despite having an insufficient amount of items for scale linking purposes, the results still supported preequating methods. This may be due in part to the similarity between the Rasch equating model and the mean/mean preequating method.

There were large differences in the magnitude of results of RMSD between true score, observed score and preequating methods across the forms. Specifically, form 1 had the highest values of RMSD across all equating methods, whereas form 3 had the lowest values of RMSD across all equating methods. The differences in RMSD across the forms provide additional evidence that the new and original forms were not built to the same statistical specifications.

Results from prior research indicated that preequating methods have performed poorly in comparison to postequating methods. Kolen and Harris (1990) reported that the IRT preequating method resulted in the highest values of RMSD and MSD in comparison to IRT postequating methods. Tong and Kolen (2005) compared the adequacy of equated scores from the traditional equipercentile, IRT true score, and IRT observed score equating methods using three criteria and found that the IRT true score method performed least favorably in comparison to the IRT observed score. In this respect, the results from the current study disagreed with previous literature. Yet, the goal of the current study was to evaluate differences in equating outcomes between MTA and ATA forms. At this point in time, no research studies have compared outcomes from different equating methods when testing organizations transitioned from MTA to ATA, therefore the lack of consistent findings with prior literature may be in relation to the change in test development procedures, the differences in psychometric framework (i.e., IRT 2-PL versus Rasch model) or differences in the nature and purpose of the testing program (i.e., K-12 versus medical licensure). For example, much of the body of literature on equating utilizes K-12 assessment programs to investigate differences in equating methodologies. K-12 assessment programs are built to different test specifications as the purpose of these examinations may be to evaluate and monitor student growth rather than passing or failing examinees. Often these types of assessment programs have different characteristics than medical licensure examinations, including shorter administration windows and mode of delivery. The difference in results can also be explained by the utility of a purposeful equating design. Although the testing program used for the current study did not operationally implement a CINEG design, I employed this design by selecting data that conformed to the design (i.e., requirements were met). Two of the four forms had sufficiently sized common item sets. Yet the favorable findings for the preequating method were in agreement for three of the four forms used. This may be due to the fact that the preequating methodologies relied on a quality bank of linked items rather on a small common item set.

The RMSD, MAD, and MSD are commonly used measures to gain an overall understanding of the differences between equated scores and those on the base form. Yet the standard error of equating is another commonly used approach to evaluate the adequacy of equating results and can be used to gain a better understanding of the error associated across the distribution of equated scores. The standard error of equating replicates hypothetical samples to approximate the standard deviation of each equated score (Kolen & Brennan, 2014). Future research can expand on this study by calculating the standard error of equating for the IRT true and observed score equating methods.

The results of this research question suggest that prior to implementing ATA, the equating design should be thoroughly considered in light of the purpose of the assessment. In agreement with the best principles of test assembly, any time new test development procedures are implemented, results processing should be carefully considered (AERA, APA, & NCME, 2014). The implementation and variation in test assembly procedures necessitates the need for reviewing and evaluating current psychometric procedures (e.g., standard setting, equating designs, etc.). A key recommendation for practitioners is to discontinue the use of MTA forms as the base form when an organization is newly implementing ATA procedures as the findings from this research suggest that there is more variation in the statistical specifications of MTA forms to support continual use as the base form.

Research Question 3: Equating Methods and Decision Consistency

Although there are many ways to evaluate decision consistency, decision consistency was measured using two estimates, the proportion of raw agreement ( ), and the kappa index ( ) which corrects the raw agreement index by what is expected by chance. Both the mean/mean preequating methods and Stocking and Lord TCC preequating methods produced the highest raw agreement indices across forms ( ). This finding supports the findings from research question 2, in which the lowest error and bias were found in the same methods. The raw agreement indices for other equating methods differed slightly within each form, typically within 1%. In comparison, the estimates were inconsistent across forms (see Appendix B). Moreover, the decision consistency index of ATA forms was 1% to 3% greater than that of the MTA forms. This finding provides additional evidence that ATA enhances the psychometric properties of examinations that make pass/fail distinctions.

Although decision consistency is an important aspect for psychometricians to explore, it does not fully explain outcomes of examinations with pass/fail decisions. Specifically without simulation studies, where true ability is known, one cannot know for certain that decisions are accurate. Although there are ways to explore and provide evidence of decision accuracy, it was beyond the scope of the current study. Future research is warranted on different approaches like Lee (2010) to evaluate both decision consistency and decision accuracy when testing programs newly implement ATA.

Limitations

The testing program did not implement a CINEG design operationally; however, the data easily lent itself to the implementation of a CINEG design based on the use of anchor blocks in ATA. Although I confirmed key equating requirements and controlled for others by carefully selecting the forms used in study, the common item sets were not a perfect representation of the content or statistical specifications to that of the entire test. Moreover, the pretest item design had also changed between MTA and ATA forms, which may have influenced the findings. Specifically, pretest item blocks were assigned randomly to examinees in Y forms, whereas pretest items blocks were embedded within each X form. Although understandable when using operational data, there are limitations in the findings. Lastly, the 0.3 logit criteria used to establish the common item set is operationally employed although there are alternative methods one can use to identify outlying items in the common item set. Therefore, future research is warranted to address the replicability of this study when considering the purpose of the examination and equating designs in conjunction with test design decisions (i.e. ATA).

In addition, the assessment used in this study has unique and complex characteristics (e.g., test specifications, blueprint, and constraints) that may limit the generalizability of the results. For example, the forms assembled using ATA programs involved approximately 70 content and statistical constraints (i.e., domain representation, life stage of patient, clinical setting, mean item p-value, etc.), and maximized the TIF to create parallel forms. Although MTA and ATA forms were built according to the same domain representation, in MTA other variables were not as controlled as they were in ATA. Furthermore, ATA employs over 70 standardized constraints, whereas test developers loosened the constraints one by one, form by form during MTA. It is expected that the similarity of constraints between ATA and MTA procedures influenced the findings of this study. The results of this study shed light on the enhancement in quality of ATA forms; however this improvement necessitates the reevaluation or reconsideration of continuing to use MTA forms as the base examination. Future research may address the similarities or differences in MTA and ATA procedures by simulating ATA conditions and assessing the outcomes from different equating methods. Future research would provide insight as to how the similarity (or differences) between assembly procedures may influence different outcomes from equating methodologies.

Prior researchers have developed a variety of models that can be used in order to implement optimal test design. A brief overview of the different models is discussed in van der Linden (1998). It is well-documented that forms developed via ATA produce more favorable psychometric properties than MTA due to the overall test design and the defining attributes of ATA (Luecht, 1998; van der Linden, 2005). This research study provides some evidence that ATA creates more parallel test forms, not only in terms of content and statistical specifications, but also with respect to test information, data-model fit, and decision consistency. Yet, very few studies have provided empirical evidence of the quality improvement of ATA over MTA. Due to the growing popularity of ATA, more research is warranted on the replicability of this study (e.g., simulation studies), on other psychometric advantages that result from implementing ATA, and on the application to assessment programs that have different purposes and test designs.

Summary

The widespread implementation of ATA procedures has alleviated the workload of test developers by allowing computer programs to create multiple parallel test forms with relative ease. ATA procedures provide an efficient and cost-effective alternative to assembling parallel test forms simultaneously. The integral psychometric goal of ATA is the minimization of the SEM and maximization of the test score reliability (van der Linden, 2005). However, ATA is not only a question of computer programs easing the workload, but rather if computer programs improve the psychometric quality of assembled test forms. The results presented in this research study provide empirical evidence of the improvement in psychometric qualities of ATA forms. Whenever testing organizations newly implement test development practices, it is important to evaluate the outcomes (AERA, APA, & NCME, 2014). Assessing the adequacy of score outcomes of various equating methods is one way to investigate the relationship between psychometric quality and new implementation of ATA programs. In this research study, I evaluated the adequacy of different equating methods by estimating the bias, error, and decision consistency associated with score outcomes of newly developed ATA forms.

The context of evaluating equating methodologies with respect to test assembly procedures is important in today’s operational psychometric work as many testing organizations move towards ATA. Although testing organizations may utilize item parameters estimated using the Rasch model or IRT models for ATA, no previous research has connected differences in test assembly procedures to outcomes of equating methods. The results of this study further support the importance of planning and aligning the psychometric procedures to the test development procedures. The findings of this study suggest that the error and consistency of scores were related to the similarity in statistical test specifications. ATA led to the development of parallel forms that had better psychometric properties and less variation in content and statistical specifications than test forms assembled manually.

The results indicated that despite the differences in statistical specifications, the mean/mean preequating method performed the most favorably. This finding may be explained by the alignment among the mean/mean preequating method, psychometric framework of the Rasch model, and that ATA utilizes the same item difficulties to build each form. Conceptually, the mean/mean preequating method is similar to the Rasch equating method, which anchors known item difficulties. Therefore, the mean/mean preequating method is aligned with the Rasch anchored equating method. Furthermore, because ATA utilizes the same known item difficulties to build forms that have similar TIFs that peak at the cut-score of the examination, all of these methods are complimentary and work in tandem. Future research should expand on these findings by investigating the outcomes from similar equating methodologies when ATA forms are used as the base forms.

Cited Literature

Ali, U. S., & van Rijn, P. W. (2016). An evaluation of different statistical targets for assembling

parallel forms in item response theory. Applied Psychological Measurement, 40(3),

163-179.

Antal, J., Proctor, T.P., & Melican, G.J. (2014). The effect of anchor test construction on scale

drift. Applied Measurement in Education, 27(3), 159-172, doi: 10.1080/08957347.2014.905785

American Educational Research Association, American Psychological Association, National

Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (AERA, APA, & NCME). (2014). Standards for educational and psychological testing (pp. 75-94). Washington, DC: American Educational Research Association.

Armstrong, R. D., Jones, D. H., & Wang, Z. (1994). Automated parallel test construction using

classical test theory. Journal of Educational Statistics, 19(1), 73-90. doi:10.2307/1165178

Andrich, D. (2004). Controversy and the Rasch model: A characteristic of incompatible

paradigms? In Smith, E. V., & Smith, R. M. (Ed). Introduction to Rasch Measurement: Theory, models and applications. Maple Grove, MN: JAM Press.

Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.),

Educational measurement (pp. 508–600). Washington, DC: American Council on Education.

Babcock, B., & Albano, A. D. (2012). Rasch scale stability in the presence of item parameter and

trait drift. Applied Psychological Measurement, 36(7), 565-580. doi:10.1177/0146621612455090

Bock, D. R. (1997). A brief history of item response theory. Educational Measurement: Issues

and Practice, 16(4), 21-33.

Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York:

Holt, Rinehart, and Winston.

Debeer, D., Ali, U. S., & van Rijn. P. W. (2017). Evaluating the statistical targets for assembling

parallel mixed-format test forms. Journal of Educational Measurement, 54(2), 218-242.

Dorans, N., Moses, T. & Eignor D. (2010) Principles and Practices of Test Score Equating. ETS

RR-10-29. ETS Research Report Series.

Eignor, D. R. & Stocking, M. L. (1986). An investigation of possible causes for the inadequacy

of IRT preequating (Report No 86-14). Princeton, NJ: ETS Research Report Series. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/j.2330-8516.1986.tb00169.x/epdf

Embretson, S. E., & Reise, S. P. (2009). Item response theory for psychologists. New York, NY:

Psychology Press.

Gao, R., He, W., & Ruan, C. (2012). Does preequating work? An investigation into a preequated

testlet-based college placement examination using post administration data (Report No 12-12). Princeton, NJ: ETS Research Report Series.

Hambleton, R., Swaminathan, H., & Rogers, H. (1991). Fundamentals of Item Response Theory.

Newbury Park, CA: Sage.

Hambleton, R., & Slater, S. (1997). Item response theory models and testing practices: Current international status and future directions. European Journal of Psychological Assessment, 13(1), 21-28. doi: 10.1027/1015-5759.13.1.21

Hanson, B.A., Zheng, L., & Cui, Z. (2004a). PIE: A computer program for IRT equating

[computer program]. Iowa City, IA: education.uiowa.edu/centers/casma

Hanson, B.A., Zheng, L., & Cui, Z. (2004b). ST: A computer program for IRT scale linking

[computer program]. Iowa City, IA: education.uiowa.edu/centers/casma

Han, T., Kolen, M., & Pohlmann, J. (1997). A comparison among IRT true- and observed-score

equatings and traditional equipercentile equating. Applied Measurement in Education, 10(2), 105-121. doi:10.1207/s15324818ame1002_1

Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied

Psychological Measurement, 9(2), 139-164.

Holland, P. W., & Dorans, N. J. (2006). Linking and equating. In Brennan, R. L. (Ed.),

Educational measurement (pp. 187-220). Westport, CT: Praeger.

Huynh, H. (1990). Computation and statistical inference for decision consistency indexes based

on the Rasch Model. Journal of Educational Statistics, 15(4), 353-368. doi:10.2307/1165093

Huynh, H., & Rawls, A. (2011). A comparison between robust z and 0.3-logit difference

procedures in assessing stability of linking items for the rasch model. Journal of Applied Measurement, 12(2), 96.

Karabatsos, G. (2017). Elements of psychometric theory lecture notes. Personal Collection of G.

Karabatsos, University of Illinois at Chicago, Chicago, Illinois.

Kolen, M. J. (1981). Comparison of traditional and item response theory methods for equating

tests. Journal of Educational Measurement, 18(1), 1-11.

Kolen, M. J., & Brennan, R. L. (2014). Test equating scaling and linking (3^rd ed). New York,

NY: Springer.

Kolen, M. J., & Harris, D. J. (1990). Comparison of item preequating and random groups

equating using IRT and equipercentile methods. Journal of Educational Measurement, 27(1), pp. 27-30.

Lee, W. (2010). Classification consistency and accuracy for complex assessments using item

response theory. Journal of Educational Measurement, 47(1), pp. 1-17.

Li, D., Jiang, Y., & von Davier, A. A. (2012). The accuracy and consistency of a series of IRT

true score equatings. Journal of Educational Measurement, 49(2), 167-189. doi: 10.1111/j.1745-3984.2012.00167.x

Lin, C.-J. (2008). Comparisons between Classical Test Theory and Item Response Theory in

Automated Assembly of Parallel Test Forms. Journal of Technology, Learning, and Assessment, 6(8).

Linacre, J. M. (2017). Winsteps® Rasch measurement [computer program]. Beaverton, OR:

Winsteps.com.

Linacre, J. M. (2017). ). Fit diagnosis: infit outfit mean-square standardized. Retrieved from

http://www.winsteps.com/winman/misfitdiagnosis.htm

Livingston, S. L. (2004). Equating test scores (without IRT). Princeton, NJ: ETS. Retrieved

from: https://www.ets.org/Media/Research/pdf/LIVINGSTON.pdf

Lord, F. M. (1977). Practical applications of item characteristic curve theory. Journal of

Educational Measurement, 14(2), 117-138. doi:10.1111/j.1745-3984.1977.tb00032.x

Lord, F. M. & Novick, M. R. (1968). Statistical theories of mental test scores. Redding, MA:

Addison-Wesley Publishing Company.

Lord, F. M., & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile

observed-score “equatings.”. Applied Psychological Measurement, 8(4), 453-461. doi: 10.1177/014662168400800409

Luecht, R.M. (1998). Computer-assisted test assembly using optimization heuristics Applied

Psychological Measurement, 22(3), 224-236. doi: 10.1177/01466216980223003.

Mead, R. (2008). A Rasch primer: The measurement theory of Georg Rasch. Psychometrics

services research memorandum 2008–001. Maple Grove, MN: Data Recognition Corporation.

Penfield, R. D. (2005). Unique properties of Rasch model item information functions. Journal of

Applied Measurement, 6(4), 355-365.

O’Neill, T., Peabody, M., Tan, R. J. B. & Du, Y. (2013). How much item drift is too much?

Rasch Measurement Transactions, 27(3), 1423-1424. Retrieved from: https://www.rasch.org/rmt/rmt273a.htm

RStudio Team (2016). RStudio: Integrated Development for R. RStudio, Inc. [computer

program]. Boston, MA Retrieved from http://www.rstudio.com/.

Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen,

Danish Institute for Educational Research. Chicago: The University of Chicago Press.

Smith, E. V., Stearus, M., Sorenson, B., Huynh, H. & McNaughton, T. (2018). Rasch DC

[computer program]. Unpublished, Department of Educational Psychology, University of

Illinois at Chicago, Chicago, IL.

Smith, E. V. (2004). Evidence of the reliability of measures and validity of measure

interpretation: A Rasch measurement perspective. In Smith, E. V., & Smith, R. M. (Ed.),

Introduction to Rasch Measurement: Theory, models and applications. Maple Grove, MN: JAM Press.

Smith, E. V. (2005). Effect of item redundancy on Rasch item and person estimates. Journal of

Applied Measurement, 6(2), 147-163.

Smith, R. M. (2003). Rasch measurement models: Interpreting Winsteps and FACETS output.

Maple Grove, MN: JAM Press.

Smith, R. M. (1991). The distributional properties of Rasch item fit statistics. Educational

and Psychological Measurement, 51, 541-565.

Smith, R. M. & Kramer, G. (1992). A comparison of two methods of test equating in the Rasch

model. Educational and Psychological Measurement, 52, 835-846.

Stephens, M. A. (1974). EDF Statistics for Goodness of Fit and Some Comparisons, Journal of

the American Statistical Association, 69, pp. 730-737.

Subkoviak, M. J. (1985). Tables of reliability coefficients for mastery tests. Paper presented at

the Annual Meeting of the American Educational Research Association, Chicago, IL.

Stocking, M. (1991, April). An experiment in the application of an automated item selection

method to real data. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Boston, MA.

Taylor, C. S., & Lee, Y. (2010). Stability of Rasch scales over time. Applied Measurement in

Education, 23(1), 87-113. doi: 10.1080/08957340903423701

Tong, Y., & Kolen, M. J. (2005). Assessing equating results on different equating criteria.

Applied Psychological Measurement, 29(6), 418-432. doi: 10.1177/0146621606280071

Tong, Y., Wu, S-S., Xu, M. (2008, March). A comparison of preequating and post-equating

using large-scale assessment data. Paper presented at the Annual Conference of the American Educational and Research Association. New York City, NY.

van der Linden, W. J. & Hambleton, R. K. (1997). Handbook of modern item response theory.

New York, NY: Springer.

van der Linden, W. J. (2005). Linear models for optimal test design. New York, NY: Springer.

doi:10.1007/0-387-29054-0

van der Linden, W. J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22(3), 195-211. doi: 10.1177/01466216980223001

von Davier, A. A. (2010). Equating and Scaling. In Peterson, P., Baker, E. & McGaw, B. (Ed.), International Encyclopedia of Education. Amsterdam: Academic Press, pp. 50-55.

von Davier, A. A. & Wilson, C. (2007). IRT true score test equating: A guide through

assumptions and applications. Journal of Educational and Psychological Measurement, 67(6); 940-957. doi: 10.1177/0013164407301543

Wightman, L. F. (1998). Practical issues in computerized test assembly. Applied Psychological

Measurement, 22(3), 292-302. doi: 10.1177/01466216980223009

Wilson, M. (1988). Detecting and interpreting local item dependence using a family of Rasch

models. Applied Psychological Measurement, 12, 353-364.

Wright, B. D. (1977). Solving measurement problems with the Rasch Model. Journal of

Educational Measurement, 14(2), 97-115.

Wright, B. D. & Mok, M. C. (2004). An overview of the family of Rasch Measurement

Models. In Smith, E. V., & Smith, R. M. (Ed.), Introduction to Rasch Measurement: Theory, models and applications. Maple Grove, MN: JAM Press.

Wright, B. D., & Linacre, M. (1994). Reasonable Mean-Square Fit Statistics. Rasch

Measurement Transactions, 8(3), 370. Retrieved from: https://www.rasch.org/rmt/rmt83b.htm

Yen, W. M. & Fitzpatrick, A. R. (2006). Item Response Theory. In Brennan, R. L. (Ed.)

Educational measurement, fourth edition. Portsmouth, NH: Praeger.

Yi, H. S., Kim, S., & Brennan, R. L. (2007). A method for estimating classification consistency

indices for two equated forms. Applied Psychological Measurement, 32(4), 275-291.

APPENDIX B

TABLE XVII

COMPLETE RESULTS FROM DECISION CONSISTENCY ANALYSES

Form	Equating	Scale Linking		Error		Error
	Baseline Comparison		0.949	0.006	0.682	0.033
	OS	MM	0.958	0.007	0.764	0.035
	OS	MS	0.955	0.007	0.736	0.035
	OS	SL	0.948	0.008	0.743	0.035
	TS	MM	0.958	0.007	0.763	0.035
	TS	MS	0.955	0.007	0.734	0.035
	TS	SL	0.948	0.008	0.745	0.035
	PE	MM	0.971	0.007	0.633	0.063
	PE	MS	0.971	0.007	0.641	0.061
	PE	SL	0.972	0.007	0.646	0.061
	Baseline Comparison		0.958	0.006	0.657	0.040
	OS	MM	0.958	0.007	0.649	0.049
	OS	MS	0.967	0.006	0.714	0.048
	OS	SL	0.963	0.007	0.651	0.052
	TS	MM	0.958	0.007	0.652	0.049
	TS	MS	0.966	0.006	0.712	0.047
	TS	SL	0.963	0.007	0.651	0.052
	PE	MM	0.971	0.006	0.697	0.053
	PE	MS	0.969	0.006	0.697	0.052
	PE	SL	0.970	0.006	0.699	0.052
	Baseline Comparison		0.947	0.006	0.647	0.034
	OS	MM	0.960	0.006	0.617	0.044
	OS	MS	0.954	0.007	0.631	0.040
	OS	SL	0.965	0.006	0.587	0.050
	TS	MM	0.960	0.006	0.612	0.044
	TS	MS	0.954	0.007	0.623	0.040
	TS	SL	0.965	0.006	0.580	0.051
	PE	MM	0.959	0.006	0.699	0.037
	PE	MS	0.958	0.006	0.709	0.036
	PE	SL	0.958	0.006	0.701	0.037
	Baseline Comparison		0.935	0.009	0.719	0.034
	OS	MM	0.955	0.007	0.791	0.029
	OS	MS	0.955	0.007	0.793	0.029
	OS	SL	0.950	0.007	0.802	0.026
	TS	MM	0.956	0.007	0.803	0.028
	TS	MS	0.958	0.007	0.814	0.027
	TS	SL	0.952	0.007	0.818	0.025
	PE	MM	0.963	0.007	0.678	0.044
	PE	MS	0.963	0.007	0.697	0.042
	PE	SL	0.963	0.007	0.686	0.044

Note. Boldface signifies maximum value. MM represents mean/mean scale linking, MS represents the mean/sigma scale linking, and SL represents the Stocking and Lord TCC scale linking procedure. OS represents IRT observed score equating, PE represents IRT preequating, and TS represents IRT true score equating.

Association of Standardized Patient Annual Conference, June 9-11, 2019

RESEARCH PRESENTATION:
Standardizing Judgment: A Qualitative Study of How SPs Co-Construct Meaning

This presentation reported on the results of a discourse analysis of 22 Standardized Patient (SP) interviews. The research received IRB approval through the University of California, San Diego. The research questions were: 1) How do SPs maintain “standardization” in role performance and assessment, 2) to what degree to SPs adhere to standardization? The results concluded that 1) the term “standardization” is co-constructed by test developers, psychometricians, SP trainers, and SPs, 2) SP trainers employ non-standardized approaches in their training, and 3) SPs are highly invested in maintaining a standard of role portrayal and assessment but the personal resources they bring to it are highly subjective.

PHILADELPHIA, PA. The National Board of Osteopathic Medical Examiners (NBOME), an independent, not-for-profit organization that provides testing for osteopathic medical licensure and related healthcare professions, is excited to mark 2019 as the 85^th anniversary of its founding.

Since its founding in 1934, and particularly in recent years, the NBOME has experienced substantial growth and opportunities given the many changes in healthcare, medical education and medical regulation. The NBOME’s assessment portfolio has grown to encompass the continuum of education, training, and continuous professional development for osteopathic medicine and a number of other healthcare professions, both in the United States and in other countries.

“We continue to be steadfast in our commitment to the NBOME’s mission and producing valid, reliable, and defensible national standardized assessment tools, and acknowledging the critical role that osteopathically distinctive assessment plays in protecting the public,” said John R. Gimpel, DO, MEd, NBOME President and CEO, “We are thankful for the strategic direction and leadership our Board of Directors has provided throughout our history, and the unparalleled support NBOME receives from the osteopathic medical community and our numerous other partners in health care.”

Coinciding with the 85 year celebration is the launch of NBOME’s enhanced blueprint for the COMLEX-USA examination series. This multi-stage update to the organization’s flagship exam series is the culmination of nearly 10 years of work in evidence-based design led by NBOME’s National Faculty, including subject matter experts from the education, licensure and clinical practice communities. In addition, NBOME has expanded the COMAT product portfolio with the Foundational Biomedical Sciences exam series, and also introduced the CATALYST longitudinal assessment platform for formative testing and continuous professional development.

A provisional logo and accompanying information resources focused on NBOME’s 85 year legacy, including as a dedicated webpage and social media campaign, are planned for the remainder of the year.

About the NBOME

The NBOME congratulates former board member Ronald Burns, DO, on his new appointment as president of the American Osteopathic Association on Saturday, July 27th 2019.

Dr. Burns assumed the presidency before an estimated 500 osteopathic physicians (DOs) at the American Osteopathic Association’s annual business meeting in Chicago. The organization represents the professional interests of the nation’s more than 145,000 DOs and osteopathic medical students.

From 2009-2018, Dr. Burns served on the NBOME Board and participated in the COMLEX-USA Level 3 Advisory Committee, Executive Committee, and chaired the Nominating Committee. He has also previously served on the Cognitive Testing Advisory Committee.

Dr. Burns is a board-certified family medicine physician, with a private practice in Orlando, Florida. He has represented the state of Florida in the AOA House of Delegates for the past 19 years. He has also been a member of the Florida Osteopathic Medical Association for over 29 years and served as its president from 2004–2005.

“My goal is to ensure the AOA is ready to meet the current and future needs of its ever-growing body of constituents,” said Dr. Burns. “By necessity, that means having strong collaborative relationships with AOA’s affiliate organizations.”

We wish Dr. Burns best of luck, and look forward to working with our longtime colleague in his new position.

Mathematical programming has been widely used by professionals in testing agencies as a tool to automatically construct equivalent test forms. This study introduces the linear programming capabilities (modeling language plus solvers) of SAS Operations Research as a platform to rigorously engineer tests on specifications in an automated manner. To that end, real items from a medical licensing test are used to demonstrate the simultaneous assembly of multiple parallel test forms under two separate linear programming scenarios: (a) constraint satisfaction (one problem) and (b) combinatorial optimization (three problems). In the four problems from the two scenarios, the forms are assembled subjected to various content and psychometric constraints. Assembled forms are next assessed using psychometric methods to ensure equivalence about all test specifications. Results from this study support SAS as a reliable and easy-to-implement platform for form assembly. Annotated codes are provided to promote further research and operational work in this area.

To read this article as it was initially published in Applied Psychological Measurement click here.

Contributed by:

Can Shao | Research Scientist | Curriculum Associates

Hongwei “Patrick” Yang | Assistant Professor | The University of West Florida

Silu Liu | Epidemiological Research Manager | DCHealth Technology

Tsung-Hsun “Edward” Tsai, PhD | Associate Vice President of Assessment Services and Research | NBOME

National Board of Osteopathic Medical Examiners, Inc.
Corporate Offices: 8765 W. Higgins Rd Suite 200 · Chicago, IL 60631
Executive Offices: 101 West Elm Street Suite 230 · Conshohocken, PA 19428
contact us · site map

The names NBOME and the National Board of Osteopathic Medical Examiners are registered service marks of the National Board of Osteopathic Medical Examiners, Inc. (NBOME). The NBOME seal, the assessment names COMAT, COMLEX, COMLEX-USA, COMSAE, COMVEX, WelCOM^TM and CATALYST, and all other designated marks and styles are trademarks or service marks of the NBOME. Unauthorized use of any NBOME mark to promote the goods or services of others is prohibited and subject to all rights and remedies under applicable federal and state law.

Website developed by Zero Defect Design LLC.

	2020 COMLEX-USA Level 1 Item Writer of the Year: Hannah Coulson, DO Dr. Coulson is an assistant professor of pathology at Edward Via College of Osteopathic Medicine – Carolinas Campus. She has been a significant contributor to all levels of COMLEX-USA since she joined the National Faculty in 2017.
	2020 COMLEX-USA Level 2-CE Item Writer of the Year: Robert Gioia, DO, DDS Dr. Gioia is an attending physician of family medicine at Spectrum Health in Michigan and has been a member of our national faculty since 2018.
	2020 COMLEX-USA Level 2-PE Case Author of the Year: Kym Carpentieri, DO Dr. Carpentieri is an assistant professor of family medicine at New York College of Osteopathic Medicine of New York Institute of Technology-Old Westbury. She has been a member of the Case Development Committee since 2013.
	2020 COMLEX-USA Level 3 Item Writer of the Year: Suzanne Rogers, DO Dr. Rogers is an assistant professor of pediatrics at Idaho College of Osteopathic Medicine. She has been a member of our National Faculty since 2018, and provided significant contributions to the COMLEX-USA Level 2-CE and Level 3 examinations as well as the COMAT Clinical Pediatrics examination.
	2020 Clinical Decision-Making (CDM) Case Writer of the Year: Alesia Wagner, DO Dr. Wagner is the Vice Chair for Primary Care and an Assistant Professor at Touro University College of Osteopathic Medicine – California, and has been a long-standing member of our National Faculty, as well as the CDM committee. She has provided significant contributions across all of the COMLEX-USA examinations.
	2020 COMLEX-USA Osteopathic Principles and Practice (OPP) Item Writer of the Year: Edward Shadiack, III, DO Dr. Shadiack is a Clinical Fellow at the Children’s Hospital of Philadelphia. He has been a member of our National Faculty since 2016, and has been heavily involved with all levels of the COMLEX-USA examinations and COMAT.
	2020 COMLEX-USA Preventative Medicine/Health Promotion (PMHP) Item Writer of the Year: Theresa McCann, PhD, MPH, CHSE Dr. McCann is a Professor and Discipline Chair for Epidemiology, Community & Public Health, and Preventive Medicine at Edward Via College of Osteopathic Medicine – Virginia Campus. She is a new member to our National Faculty in 2020, and has been heavily involved in all levels of COMLEX-USA examinations.
	2020 COMAT Clinical Item Writer of the Year: Bernadette Riley, DO Dr. Riley is an associate professor of Family Medicine at New York College of Osteopathic Medicine of New York Institute of Technology-Old Westbury and a long standing member of our National Faculty. She has been a significant contributor to both COMAT and COMLEX-USA examinations.
	2020 COMAT Foundational Biomedical Sciences (FBS) Item Writer of the Year: Rebecca Pratt, PhD Dr. Pratt is a professor of anatomy at Oakland University William Beaumont School of Medicine. She has been a member of our National Faculty since 2014, and contributes to both COMAT Foundational Biomedical Sciences and COMLEX-USA Level 1 examinations. Dr. Riley was also awarded Item Writer of the Year for COMAT Foundational Biomedical Sciences in 2018.

	Clinical Sciences Department Chair, Osteopathic Principles & Practice / Neuromusculoskeletal Medicine Mark Sandhouse, DO – Nova Southeastern University Kiran C. Patel College of Osteopathic Medicine Dr. Sandhouse has served the NBOME in collective National Faculty leadership roles for over 15 years, as COMLEX-USA Composite Examination Committee Member: 2016-2020, COMLEX-USA Level 1 Advisory Committee Member: 2016-2020, CTAC Member: 2013, 2015 and National Faculty Department Chair of Osteopathic Principles and Practice / Neuromusculoskeletal Medicine: 2015-2020.
	Clinical Sciences Department Chair, Obstetrics and Gynecology Eric Carlson, DO, MPH – Philadelphia College of Osteopathic Medicine
	COMAT Chair, Osteopathic Principles & Practice / Neuromusculoskeletal Medicine Danielle L. Cooley, DO – Rowan University School of Osteopathic Medicine
	Clinical Sciences Division Chair, Medical Ethics, Jurisprudence and Professionalism Sarah Hall, DO – Oklahoma State University Center for Health Sciences College of Osteopathic Medicine
	COMAT Chair, Family Medicine, served as Charter COMAT FM Chair Tracy Middleton, DO – Arizona College of Osteopathic Medicine of Midwestern University Dr. Middleton has been elected to NBOME Board of Directors starting January 2021

Login

Information for...

You may also like

Update: COMLEX-USA Level 1 Transition to Pass/Fail Reporting

Level 1 Candidate Score Report

Frequently Asked Questions

You may also like

Tell us what inspired you to become a DO, specifically.

Congratulations on your Match into Emergency Medicine! How did you choose this specialty?

We understand you only used COMLEX-USA scores when applying to residency—what prompted that decision, and what advice would you give students who are thinking of doing the same?

We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you personally?

Looking back at how you prepared for your most recent level of COMLEX-USA, would you have taken a different approach to studying? What advice do you have for other COM students who are preparing for COMLEX-USA?

The Road to DO Licensure has many unexpected challenges that I’m sure you’ve experienced, including burnout, stress, and anxiety. Tell me some of the major challenges you’ve faced while taking COMLEX-USA and some advice on how you personally overcame them.

What are you looking forward to the most in the next stage of your journey?

You may also like

You were an ICU nurse before you decided to become a doctor—what inspired you to become a DO, specifically?

What mental challenges are you facing as a 2nd-year osteopathic medical student as you prepare for your first level of COMLEX-USA and how are you addressing them?

Although it varies from person to person, what do you feel is most important to a medical student’s mental well-being and why?

VCOM featured you in an article about practicing gratitude and its positive impact on mental health. What other practices, both mental and physical, do you find helpful for improving wellness while preparing for COMLEX-USA? What advice would you give to other COM students?

Work-life-balance is important for every field, how do you recommend osteopathic medical students start now to be sure that they maintain this throughout their career?

You may also like

What inspired you to become a DO, specifically? Tell us your story.

We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you personally?

Looking back at how you prepared for your most recent level of COMLEX-USA, would you have taken a different approach to studying? What advice do you have for other COM students who are preparing for COMLEX-USA?

The Road to DO Licensure has many unexpected challenges that I’m sure you’ve experienced, including burnout, stress, and anxiety. Tell me some of the major challenges you’ve faced while taking COMLEX-USA and some advice on how you personally overcame them.

Have you encountered other mentors that have helped get you to where you are today?

You were appointed to the Special Commission after your nomination by SOMA—tell us what this means to you.

What are you looking forward to the most in the next stage of your journey?

You may also like

MATCH 2023

MATCH 2022

MATCH 2021

MATCH 2020

MATCH 2019

You may also like

You may also like

In this section

How did you know that the programs you ranked were right for you?

What factors from your interview were the most meaningful in affecting how you created your rank list?

Did you receive feedback from any programs after the interview? If so, did this make a difference?

How did you go about creating your rank list? Did you rank according to your preference or did you try to anticipate which programs would rank you highly?

You may also like

You may also like

What inspired you to become a DO, specifically? Tell us your story.

We often hear the phrase ‘osteopathic distinctiveness’ used across the profession. What does that phrase mean to you personally?

In your speech to COCA, you mentioned how high your stress levels were when preparing for COMLEX-USA during both a pandemic and civil unrest. Tell us about the major challenges you’ve faced and how you overcame them.

Did you also take USMLE, and if not, why didn’t you?

What advice would you give to other COM students preparing for COMLEX-USA?

What are you looking forward to the most in the next stage of your journey?

You may also like

What made you decide you wanted to do a fellowship?

How did your osteopathic curriculum influence your interest in pursuing a fellowship and help you prepare for success?

As a DO, what advice would you give to those coming after you who are also thinking about applying for a fellowship?

What made you choose hematology-oncology?

What aspects of being an osteopathic physician do you think will be a benefit to your Hematology-Oncology patients?

How excited are you about your match to Vanderbilt?

You may also like

In this section

Convenient

Flexible

Responsive

Enhanced Learning

Modern Approach

You may also like

You may also like

You may also like

Additionally, we would like to congratulate and welcome the following National Faculty members who have been appointed to 2021 National Faculty Chair positions:

You may also like

Examination Calendar

You may also like

You may also like

How to Enter:

Official Rules and Fine Print

Forms

Scoring & Reporting

Score Report

You may also like

You may also like

Breanne Jaqua, DO, MPH

Eleanora Yeiser, DO