- January 29, 2024 – C3DO Phase 2 Physician Lead Kickoff Meeting
- On January 19, 2024, the NBOME held a kickoff meeting for lead physicians and staff from the eight colleges of osteopathic medicine (COMs) participating in phase 2 of the C3DO pilots.
The workshop was designed to introduce the teams to the phase 2 project plan, starting with a summary of phase one and lessons learned. Important updates to the program that were a direct result of feedback from schools and student participants in 2023 included enhanced case development and item editing, simplification of training materials for standardized patients, improvements in data exchange, expanded student informational materials, and the creation of a comprehensive COM “toolkit,” as a few examples.
Sessions included how to create an exam form (the set of cases a candidate would see on a given testing day); review of Phase 2 administrative and IT elements; an introduction to the NBOME’s online Learning Center and the courses it hosts for C3DO standardized patients (SPs) and staff; discussion of OMT in the C3DO; an overview of SP training; review of resources and timelines; and a discussion about scoring considerations. Physician leads from the three COMs that had also participated in Phase 1 joined with NBOME staff to share insights from Phase 1 as well as recommendations regarding communicating with and preparing students to participate.
On January 22, each lead received their C3DO “toolkit,” allowing them and their teams to dive more deeply into the topics covered at their own pace and begin their preparations. The next step for each COM, depending on their testing dates, is a kickoff meeting for the COM’s core team. Pilots begin in March 2024 for returning colleges, and April 2024 for colleges participating for the first time, and run throughout the year.
- December 15, 2023 – NBOME Names Eight COMs for C3DO Pilot Project Phase 2
- The NBOME Board of Directors has accepted recommendations from the Core Competency Capstone for DOs (C3DO) Task Force for a new cohort of colleges of osteopathic medicine (COMs) to participate in Phase 2 pilots for C3DO. Learn more here.
- November 30, 2023 – Gearing up for Phase 2
- The C3DO Case Development Committee met in our Philadelphia offices on October 20-21. Physicians from across the country convened onsite to review case performance from Phase 1 and develop additional case materials and post encounter assessments for Phase 2. Participants remarked,
“I enjoyed meeting others in the osteopathic profession and learning about what goes into case and testing development.”
“This was a fantastic experience!”
“Great couple of days working on and editing cases. This is extremely valuable work by all involved and I can’t wait to see the final outcome.”
Additional opportunities will be available to participate through the year. If you are interested in participating, the best way to begin is by joining the NBOME National Faculty and indicating your interest in C3DO.
Proposals from COMs interested in participating in the C3DO Phase 2 pilots were due November 3. De-identified proposals were shared with members of the C3DO Task Force in advance of their November meeting.
The Task Force met on November 28. Members received an update on Phase 1 pilot scoring trends and correlations and the October Case Development Committee meeting. They also continued review of student feedback from Phase 1 to address concerns regarding value, timing, student preparation, and student and COM feedback in anticipation of Phase 2 pilots in 2024.
The Task Force reviewed the evaluation criteria for the Phase 2 proposals, discussed and debated points of difference among the proposals, and highlighted their relative strengths.
After the meeting, Task Force members individually evaluated each proposal, noted whether they would recommend the submitting COM for inclusion in the Phase 2 pilots, and ranked the submissions for inclusion. Information from the survey will be compiled and recommendations presented to the NBOME Board of Directors at their December meeting.
- September 27, 2023 – Preparing for Phase 2: Release of the RFP and Preparations for Case Development
- The NBOME will release the Request for Proposals (RFP) for Phase 2 of the C3DO pilots on Friday, September 29. The RFP is a guide for COMs considering submitting a proposal to be included in Phase 2, with details on the assessment and instructions for applying. The RFP reflects lessons learned throughout Phase 1, including use of an online tool for applications. Applications will be accepted through November 3, 2023.
The RFP was sent to COM deans along with a registration link for an optional informational webinar. For information about the RFP or webinar, please reach out to your dean.
The C3DO Advisory Panel will meet on October 10. This is a panel made up of representatives of multiple stakeholder groups (AACOM [BOD, AOGME, ECOP, COSGP], AOA, AAOE, Licensing Board members, residents, students, and others). See March update for list of members. They will review the progress and reports from the C3DO Task Force and Pilots and advise the NBOME Board of Directors.
Also in October, a group of osteopathic physicians, including representatives from COMs that participated in Phase 1, will be convening in Conshohocken, PA for a C3DO case development meeting, creating and revising cases for use in the upcoming Phase 2 pilots.
- August 29, 2023 – Task Force meets to review preliminary data and student feedback, discuss plans for Phase 2
- The C3DO Task Force met on Tuesday, August 29. The Task Force reviewed preliminary data regarding both feasibility and reproducibility across COMs.
In addition, they reviewed student survey feedback from participants in the Phase 1 pilots. Some examples:
798 students completed the post-activity survey, for a 98% response rate. They rated elements of the C3DO assessment and offered a total of 1,865 comments. 92% of students indicated that the C3DO provided at least a partial assessment of their skills (with additional questions asking about individual skill performance), and 89% agreed that the cases presented appropriate clinical challenges. Sample comments include:
- “I think because the cases were so diverse and many of them required you to think about their health holistically, and OMM was integrated in the case, I really did feel like I was able to integrate my osteopathic training.”
- “Overall, this was a great experience and very comfortable being in my home institution.”
- “Feel that this format was a formal and very thorough assessment of my clinical knowledge and skills up until this point in clinical training. Highly recommend”
NBOME staff is creating a brief video highlighting the student experience—watch for it coming soon!
The Task Force reviewed a draft request for proposals (RFP)—the guide for COMs thinking about submitting a proposal for their inclusion in the Phase 2 pilots. The RFP is being crafted to reflect lessons learned from Phase 1 and to make the process more user-friendly, including streamlining applications by using an online tool to apply. The 2023 RFP will be released on September 29, and proposal responses will be accepted through November 3, 2023.
- July 18, 2023 – Initial Core Competency Capstone for Osteopathic Physicians (C3DO) Pilots Completed; Attestations accepted through Class of 2027
- Initial analysis of pilot studies in 2023 at six colleges of osteopathic medicine (COM) campuses show promising results regarding the feasibility of on-campus standardized assessment of fundamental osteopathic clinical skills of osteopathic medical students. Learn more here.
- June 29, 2023 – Pilot Assessments Completed at All C3DO Sites
- All four C3DO pilot schools have now administered the pilot assessment of a core competency capstone (C3DO) prototype at 6 campuses nationwide. NBOME staff visited the participating colleges of osteopathic medicine (COMs) during their final preparations and the first days of their sessions to assist the COMs as needed and to observe their processes. The NBOME gathered a wealth of information during these visits as well as from exam data.
Our goals were to identify opportunities to assess the feasibility of clinical skills assessment at the COMs while ensuring a national, standardized experience for fair evaluation of all osteopathic medical students. Our deep gratitude to all four schools for their hard work on the C3DO pilots and their gracious hosting of NBOME representatives during those visits.
The NBOME is now turning to analysis of the vast stores of information gathered during the pilot, from site visit reports, examination data, analytics from online training courses for pilot COM staff and standardized patients (SPs), and feedback received through surveys of COM staff, SPs, and students. Information from all of these sources will be joined with lessons learned over the past several months of pilot preparation to inform the C3DO Task Force and Advisory Panel in the coming months, as they consider next steps in C3DO development.
Additional announcements about the future of the pilot program as well as the current COM Dean clinical skills attestations from the NBOME Board of Directors are expected in late July.
- May 25, 2023 – May C3DO task force meeting includes review of pilot data
- The Core Competency Capstone for DOs (C3DO) task force met virtually on May 11. The task force reviewed and discussed the feedback from the C3DO Advisory Panelists’ April meeting. Additionally, task force members reviewed data from the first pilot at RVUCOM, which completed initial runs in April with great success in learning more about opportunities and challenges of administering this type of activity at colleges of osteopathic medicine. Data included survey results from staff on the pilot implementation and form creation processes – how they put together the case combinations for the students to experience and trained their local staff. Task force members discussed using feedback from these initial pilot locations for the remaining schools in this first phase, and allowed for adoption of additional tools to iteratively improve the activity.The task force reviewed students’ feedback on participation in the activity, including their impression on the types and distribution of the cases, and whether they felt the pilot adequately assessed their osteopathic clinical skills. Students favorably reacted to an updated post encounter exercise which included clinical decision making items pertaining to the case they saw (as opposed to a SOAP note). Some comments from student feedback included:
- “I appreciated the preparation I made for this as it really helped push my training and understanding.”
- “After going through 2 years of clinical rotations, this was an opportunity to demonstrate what I have learned. I felt like my training as a medical student trained me well and this was my chance to show it.
- I think the assessment itself was reasonable but the preparation material could be more specific in explaining what is expected of us–will we be using aspects of different screening exams together etc.”
- “There is enough variety to touch on most fundamental osteopathic clinical skills.”
- “I would recommend giving a better understanding of the schedule and the expectations.”
- “Need OMT to be more apparent.”
Pilots continue through the late spring: MUCOM is administering their pilot sessions now and CUSOM and MWU-AZCOM are beginning their sessions in early June. The task force will meet again in early summer to review additional data.
- “I appreciated the preparation I made for this as it really helped push my training and understanding.”
- May 2, 2023 – First pilot testing begins; Feedback from across the profession continues with Advisory Panel and licensing community
- April was a busy month for the C3DO pilot project. The RVU Utah and Colorado campuses completed their main pilot testing dates and we have received great constructive feedback from staff, administration, standardized patients (SPs), and students. Thank you to all who participated. Marian University College of Osteopathic Medicine, Campbell University School of Osteopathic Medicine, and Midwestern University – Arizona College of Osteopathic Medicine continue preparations for their pilots this spring.We have already made some improvements for the colleges still to come in the pilot, with added support on training and administration through the use of time-based checklists. We have placed training surveys directly in the learning modules to make completion easier for COM staff and SPs. We have developed robust orientation materials for colleges to inform students about the project.
A centralized group of OMT physician examiners chosen from across the country have been oriented and trained to provide evaluation of the performance of OMT on 3 cases per session. We believe that this model will provide an independent assessment of these skills during the patient encounters.
The C3DO Advisory panel met on April 12. Please see the last update for the names of these individuals and their represented organizations. This diverse group of individuals represents organizations from across the profession of osteopathic medicine and the learning continuum (including students and residents) as well as patient safety and advocacy. Each person had the opportunity to identify benefits, unintended consequences, and suggestions for the program. They will continue to provide advice regarding the project, the exam prototype, and feedback on reports generated from the C3DO Task Force.
NBOME leaders are meeting again with the state medical and osteopathic medical licensing community on May 3-May 6 for further input on this initiative at the national Federation of State Medical Boards (FSMB) Annual Conference and annual meeting of the American Association of Osteopathic Examiners (AAOE).
- March 27, 2023 – Structure of C3DO; Advisory panel named
- The C3DO pilot is in development at our various sites. We appreciate the collegial spirit and value the feedback and expertise of our COM partners.
The pilot format will include 8 standardized patient encounters (14 minutes each); scored elements of history and physical examination (checklist); OMT (centrally scored; distributed physician examiner pool); and communication and respectfulness evaluation (global rubric). There will not be a SOAP note component, but students will instead answer two questions following their encounters: the first an open-ended differential diagnosis (or problem list) format pertinent to the SP presenting problem, and the second an extended multiple choice question aimed at key features of the diagnosis or treatment for the patient they encountered.
Feasibility: Can the COMs administer a centrally developed examination? What is the experience like for them? The NBOME will use measures such as surveys of staff, faculty, Standardized Patients (SPs), and students, among other measures. As the participating COMs have selected uses for this investigation that align with their own curriculum, it will be investigated how this can integrate into their current processes.
Standardization: Can the resulting examination be standardized across COMs? The NBOME will analyze student performance evaluations, SP performance, and administration.
After an extensive application cycle with nominee solicitation from multiple stakeholder groups (AACOM [BOD, AOGME, ECOP, COSGP], AOA, AAOE, Licensing Board members, residents, students, and others), the chosen members of the C3DO Advisory Panel were notified on March 1, 2023. Their first meeting will be in April 2023, and they will review the progress and reports from the C3DO Task Force and Pilots. Based on the pilot research and the current performance assessment environment, the panel will also advise the NBOME Board of Directors. David Kuo, DO, has agreed to chair this panel.
AACOM BOD, Sharon J. Obadia, DO
AACOM BOD, Italo Subbarao, DO, MBA
AACOM BOD, Andrea P. Mann, DO
AACOM BOD, Heath A. Parker, DO
AAOE, Christine M. Khandelwal, DO
AAOE, William B. Swallow, DO
AAOE, David E. Tannehill, DO
ACOFP, Peter F. Bidey, DO, MSEd
ACOFP, Gautam J. Desai, DO
ACOI, Laura M. Rosch, DO, CS, MS
ACOI, David L. Tolentino, DO
ACOS, Mary K. Carneval, DO
ACOS, Alyssa R. Pastorino, DO
AOA BOT, Robert Dolansky, DO
AOA BOS, Carolyn Quist, DO
AOA BEL, Selena Raines, DO
AOA BEL, David O. Shumway, DO, HBS
AOA COCA, Janice A. Knebl, DO, MBA
AOSED, Brian Bowles
COSGP, Edoardo G. Frezza, OMS – through June 2023
COSGP, Nichelle Chastain, OMS – added October 2023
NF At-large, Rebecca Giusti, DO
NF At-large, Natalie A. Hyppolite, DO, MBS
Patient Safety, Maureen P. Barnes, ARM, CPHRM
Patient Safety, Bruce Anderson
SOMA, Lawsen Parker, OMS
Educator, Sherman Gorbis, DO
Educator, Amanda Cain, DO
Educator, Suzanne L. Rogers, DO
Educator, Darren Cohen, DO
Resident, Brysen Keith, DO
Pilots begin in mid-April at the COMs’ selected times. We have been able to accommodate all COM requests for their preferred schedule to align with their existing activities.
- February 17, 2023 – Planning continues for first pilots; Presentation to AACOM Board of Deans
- NBOME staff and consultants spent the first half of February preparing examination materials, creating training protocols and materials, and working closely with the pilot COMs.NBOME is providing cases for the pilots. Pilot COMs will be able to select among these cases to create exam forms (the combination of cases students see in an exam session) that fit their needs while matching the C3DO blueprint, designed to align with the COMLEX-USA Master Blueprint Clinical presentations.
NBOME staff and consultants have been meeting with representatives of the COMs, keeping the Physician Leads engaged with C3DO development and beginning work with SP training teams from the COMs that are administering the first pilot sessions. Training is competency-based, designed to ensure standardized performance across the COMs while letting the COMs draw on their expertise as they work with their SPs.
On Thursday, February 16, Jeanne Sandella, DO, VP for Professional Development Initiatives and Communications, and Amy Lorion, Senior Director for Professional Development Initiatives, presented at AACOM’s Community Forum Meeting. She shared updates and answered questions from the AACOM Board of Deans regarding the C3DO pilots.
- January 26, 2023 – C3DO task force begins pilot planning; call for advisory panel members
- The C3DO Task force met again on January 19, 2023. They discussed scoring elements: assessment of OMT; communication, interpersonal skills, and professionalism; history and physical examination. Also under discussion were form requirements-the types of clinical skills cases that should be seen by each candidate.Core NBOME staff and consultants have worked tirelessly to put the Task Force recommendations into place. Some of these activities include: building the exam-day structure; creating exam forms and selecting cases; refining the assessment tools for OMT and for communication, interpersonal skills, and professionalism; and refining standardized patient portrayal protocols. The first pilot exams will be held in mid-April, and pilot sessions will be held through June according to the schedule preferred by the COM.
The goals of this first set of pilots are to examine the feasibility of training, administration, and scoring at each site and to evaluate the standardization of the examination across the sites. The four selected COMs are excellent partners; all have clinical skills testing programs with experienced and dedicated staff and reflect different regions of the country and different sized classes.
A call for members of the C3DO Advisory panel, a larger oversight group formed of broader representation including from specialty colleges, student and patient advocacy groups, and other representation of the profession (e.g., AACOM/AACOM Groups, AOA and COCA, AAOE/Licensure Boards, other GME groups) was sent out earlier this month. Input from across the profession remains integral to the C3DO initiative.
- January 11, 2023 – COMs begin pilot planning with NBOME
- Over the past two weeks, NBOME staff and consultants held meetings with representatives from each of the four COMs participating in the 1st round of C3DO pilots. They discussed pilot planning within the context of each COM’s particular needs and circumstances. This strategy allows for collaboration with the schools as the details of the first pilot are finalized. These discussions will be shared with the C3DO task force at their next meeting on January 19. Future meetings with the COM staff will focus on supporting their efforts on local implementation of the pilot plan.
- December 9, 2022 – NBOME Names Four COMs for C3DO Pilot Projects
- The NBOME Board of Directors has selected four colleges of osteopathic medicine (COMs) to participate in the first set of pilot projects for the Core Competency Capstone for DOs. Learn more here.
- November 28, 2022 – Task Force reviews proposals
- The C3DO task force held their final meeting of 2022 on November 28. They reviewed data from the COM pilot proposals and discussed important considerations in the selection process. Recommendations garnered from the task force members will be presented to the NBOME Board of Directors at the December meeting. Selected COMs will be notified by December 16.
- November 15, 2022 – Task force meets to address proposal evaluation; discussion of OMT assessment begins
- November 15, 2022 marked the due date for colleges of osteopathic medicine to submit proposals to participate in the first phase of C3DO pilots in 2023. On November 14, the C3DO task force held their monthly meeting. During the meeting, Task Force members discussed the selection criteria for the pilot locations and began discussions on strategies for the assessment of OMT in the pilot.
- October 24, 2022 – Letters of Intent received; Webinar hosted for COM faculty
- This afternoon, the NBOME hosted an informational webinar to assist COMs in writing their proposals to become host sites for the 2023 C3DO pilots. Attendees discussed the proposal and plans for the pilots with NBOME staff, led by Consulting Coordinator Dennis J. Dowling, DO, MA.An updated FAQ document, reflecting questions asked during the webinar, will be shared with the COMs that submitted a letter of intent.
- September 30, 2022 – Request for Proposals Sent to Deans
- This morning, COM Deans received from the NBOME a Request for Proposals (RFP) for the 2023 Core Competency Capstone for DOs (C3DO) pilot. The RFP includes information about the C3DO project as well as instructions for COMs interested in submitting a proposal to be included in the pilot.NBOME will host an informational webinar on October 24, 2022 for those COMs that submit a non-binding Letter of Intent (LOI) by the LOI deadline of October 21.
- September 19, 2022 – NBOME Announces Task Force Members for New Capstone Project
- The NBOME has announced the members of the task force that will provide oversight for its Core Competency Capstone for DOs project. Learn more here.