Updates and News

In addition, they reviewed student survey feedback from participants in the Phase 1 pilots. Some examples:

798 students completed the post-activity survey, for a 98% response rate. They rated elements of the C3DO assessment and offered a total of 1,865 comments. 92% of students indicated that the C3DO provided at least a partial assessment of their skills (with additional questions asking about individual skill performance), and 89% agreed that the cases presented appropriate clinical challenges. Sample comments include:

• “I think because the cases were so diverse and many of them required you to think about their health holistically, and OMM was integrated in the case, I really did feel like I was able to integrate my osteopathic training.”
• “Overall, this was a great experience and very comfortable being in my home institution.”
• “Feel that this format was a formal and very thorough assessment of my clinical knowledge and skills up until this point in clinical training. Highly recommend”

NBOME staff is creating a brief video highlighting the student experience—watch for it coming soon!

The Task Force reviewed a draft request for proposals (RFP)—the guide for COMs thinking about submitting a proposal for their inclusion in the Phase 2 pilots. The RFP is being crafted to reflect lessons learned from Phase 1 and to make the process more user-friendly, including streamlining applications by using an online tool to apply. The 2023 RFP will be released on September 29, and proposal responses will be accepted through November 3, 2023.

Our goals were to identify opportunities to assess the feasibility of clinical skills assessment at the COMs while ensuring a national, standardized experience for fair evaluation of all osteopathic medical students. Our deep gratitude to all four schools for their hard work on the C3DO pilots and their gracious hosting of NBOME representatives during those visits.

The NBOME is now turning to analysis of the vast stores of information gathered during the pilot, from site visit reports, examination data, analytics from online training courses for pilot COM staff and standardized patients (SPs), and feedback received through surveys of COM staff, SPs, and students. Information from all of these sources will be joined with lessons learned over the past several months of pilot preparation to inform the C3DO Task Force and Advisory Panel in the coming months, as they consider next steps in C3DO development.

Additional announcements about the future of the pilot program as well as the current COM Dean clinical skills attestations from the NBOME Board of Directors are expected in late July.

• “I appreciated the preparation I made for this as it really helped push my training and understanding.”
• “After going through 2 years of clinical rotations, this was an opportunity to demonstrate what I have learned. I felt like my training as a medical student trained me well and this was my chance to show it.
• I think the assessment itself was reasonable but the preparation material could be more specific in explaining what is expected of us–will we be using aspects of different screening exams together etc.”
• “There is enough variety to touch on most fundamental osteopathic clinical skills.”
• “I would recommend giving a better understanding of the schedule and the expectations.”
• “Need OMT to be more apparent.”

Pilots continue through the late spring: MUCOM is administering their pilot sessions now and CUSOM and MWU-AZCOM are beginning their sessions in early June. The task force will meet again in early summer to review additional data.

A centralized group of OMT physician examiners chosen from across the country have been oriented and trained to provide evaluation of the performance of OMT on 3 cases per session. We believe that this model will provide an independent assessment of these skills during the patient encounters.

The C3DO Advisory panel met on April 12. Please see the last update for the names of these individuals and their represented organizations. This diverse group of individuals represents organizations from across the profession of osteopathic medicine and the learning continuum (including students and residents) as well as patient safety and advocacy. Each person had the opportunity to identify benefits, unintended consequences, and suggestions for the program. They will continue to provide advice regarding the project, the exam prototype, and feedback on reports generated from the C3DO Task Force.

NBOME leaders are meeting again with the state medical and osteopathic medical licensing community on May 3-May 6 for further input on this initiative at the national Federation of State Medical Boards (FSMB) Annual Conference and annual meeting of the American Association of Osteopathic Examiners (AAOE).

Pilot format

The pilot format will include 8 standardized patient encounters (14 minutes each); scored elements of history and physical examination (checklist); OMT (centrally scored; distributed physician examiner pool); and communication and respectfulness evaluation (global rubric). There will not be a SOAP note component, but students will instead answer two questions following their encounters: the first an open-ended differential diagnosis (or problem list) format pertinent to the SP presenting problem, and the second an extended multiple choice question aimed at key features of the diagnosis or treatment for the patient they encountered.

Pilot goals

Feasibility: Can the COMs administer a centrally developed examination? What is the experience like for them? The NBOME will use measures such as surveys of staff, faculty, Standardized Patients (SPs), and students, among other measures. As the participating COMs have selected uses for this investigation that align with their own curriculum, it will be investigated how this can integrate into their current processes.

Standardization: Can the resulting examination be standardized across COMs? The NBOME will analyze student performance evaluations, SP performance, and administration.

Advisory Panel

After an extensive application cycle with nominee solicitation from multiple stakeholder groups (AACOM [BOD, AOGME, ECOP, COSGP], AOA, AAOE, Licensing Board members, residents, students, and others), the chosen members of the C3DO Advisory Panel were notified on March 1, 2023. Their first meeting will be in April 2023, and they will review the progress and reports from the C3DO Task Force and Pilots. Based on the pilot research and the current performance assessment environment, the panel will also advise the NBOME Board of Directors. David Kuo, DO, has agreed to chair this panel.

AACOM BOD, Sharon J. Obadia, DO
AACOM BOD, Italo Subbarao, DO, MBA
AACOM BOD, Andrea P. Mann, DO
AACOM BOD, Heath A. Parker, DO
AAOE, Christine M. Khandelwal, DO
AAOE, William B. Swallow, DO
AAOE, David E. Tannehill, DO
ACOFP, Peter F. Bidey, DO, MSEd
ACOFP, Gautam J. Desai, DO
ACOI, Laura M. Rosch, DO, CS, MS
ACOI, David L. Tolentino, DO
ACOS, Mary K. Carneval, DO
ACOS, Alyssa R. Pastorino, DO
AOA BOT, Robert Dolansky, DO
AOA BOS, Carolyn Quist, DO
AOA BEL, Selena Raines, DO
AOA BEL, David O. Shumway, DO, HBS
AOA COCA, Janice A. Knebl, DO, MBA
AOSED, Brian Bowles
COSGP, Edoardo G. Frezza, OMS
NF At-large, Rebecca Giusti, DO
NF At-large, Natalie A. Hyppolite, DO, MBS
Patient Safety, Maureen P. Barnes, ARM, CPHRM
Patient Safety, Bruce Anderson
SOMA, Lawsen Parker, OMS
Educator, Sherman Gorbis, DO
Educator, Amanda Cain, DO
Educator, Suzanne L. Rogers, DO
Educator, Darren Cohen, DO
Resident, Brysen Keith, DO

Next steps

Pilots begin in mid-April at the COMs’ selected times. We have been able to accommodate all COM requests for their preferred schedule to align with their existing activities.

NBOME staff and consultants have been meeting with representatives of the COMs, keeping the Physician Leads engaged with C3DO development and beginning work with SP training teams from the COMs that are administering the first pilot sessions. Training is competency-based, designed to ensure standardized performance across the COMs while letting the COMs draw on their expertise as they work with their SPs.

On Thursday, February 16, Jeanne Sandella, DO, VP for Professional Development Initiatives and Communications, and Amy Lorion, Senior Director for Professional Development Initiatives, presented at AACOM’s Community Forum Meeting. She shared updates and answered questions from the AACOM Board of Deans regarding the C3DO pilots.

The goals of this first set of pilots are to examine the feasibility of training, administration, and scoring at each site and to evaluate the standardization of the examination across the sites. The four selected COMs are excellent partners; all have clinical skills testing programs with experienced and dedicated staff and reflect different regions of the country and different sized classes.

A call for members of the C3DO Advisory panel, a larger oversight group formed of broader representation including from specialty colleges, student and patient advocacy groups, and other representation of the profession (e.g., AACOM/AACOM Groups, AOA and COCA, AAOE/Licensure Boards, other GME groups) was sent out earlier this month. Input from across the profession remains integral to the C3DO initiative.