WGEA/WGSA/WOSR 2017

11:15 AM - 11:30 AM

Creation of 4th year elective for Medical School Admissions   
B.R. Chan, D. Himes, G. Kacinski, University of Utah School of Medicine
Abstract Body: Context Medical schools have their 4th year medical students participate in a variety of ways in the admission process, but it is unknown if students understand the history of the admissions process. In 2015, at the University of Utah School of Medicine we implemented a fourth year elective that stipulates that the history of medical school admissions is also the history of medical education. Medical students receive credit by attending lectures (which are also accessible online due to residency interview travel schedules) that range from subjects such as the Flexner Report, Holistic Review, and race and ethnicity in the admissions process by reviewing historical and recent Supreme Court Cases. Students are also trained on item writing and contribute scenarios for our Multiple Mini Interview (MMI) and Situational Judgment Test (SJT) scenarios that are used for the current Class of 2021 application cycle. Objectives To measure the short-term impact of the Admissions elective we monitored how many students participated from the 4th year medical school class. We also tracked the number and quality of MMI and SJT scenarios that were created by the students each year. We were also able to monitor quiz completion (based on the Flexner Report for example). Key Message This has proven to be a popular elective as 50% of the 4th year class participates over the past 2 years, and we have received feedback that this helps them transition to internship. The medical students have also helped create over 50 MMI and SJT items that are currently used in our application process. Conclusions Additional research will be needed to ascertain lasting influence of this elective as our medical students transition into residency and attending leadership positions.  

11:00 AM - 11:15 AM

Situational Judgment Test (SJT) compared to Multiple Mini Interview (MMI), Medical College Admission Test (MCAT), and Grade Point Average (GPA)  
B.R. Chan, J. Colbert-Getz, K. Pippitt, C. Knupp, M. Onofrietti, University of Utah School of Medicine
Abstract Body: Context Medical school admissions rely on limited data to make decisions. The limitations of the Medical College Admission Test (MCAT) and Grade Point Average (GPA) are apparent. Recent innovations, such as the Multiple Min Interview (MMI), have attempted to add additional information for admissions committees. However, MMI can be labor and time intensive, and there still can be variation in between evaluators on how they assess different scenarios. The next logical step is to better test non-cognitive skills, such as a Situational Judgement Test (SJT), where a standardized test is created. SJTs can inquire about different competencies, such as teamwork, professionalism, and communication skills. However, if an applicant does well on MCAT, GPA, or MMI, is there a positive correlation between SJT performance? We administered a MMI and SJT to 499 applicants to the University of Utah School of Medicine. Objectives To determine if SJT performance is correlated with MCAT, GPA, or MMI performance. We hypothesize that we are measuring different attributes and traits, such as teamwork skills, communication, professionalism that would not necessarily be detected with MCAT or GPA. Key Messages Comparing performance across MCAT, GPA, MMI, and SJT, we found that there was a positive, but low correlation (<0.30) between SJT across the domains. Thus, superior test taking (MCAT) and studying skills (GPA) and impromptu interview performance (MMI) was considered separate from SJT. This was found to be reassuring insofar that SJT appears to be assessing different domains. Conclusions Additional research will be required to longitudinally follow matriculated medical students in regards to these domains. 

California Longitudinal Evaluation of Admission Practices (CA-LEAP): Variability in Predictors of Acceptance to Medical Schools by Institution and Disadvantage Status  
E.J. Griffin, M. Henderson, C. Kelly, P. Franks, A. Jerant, UC-Davis, UC-San Diego
 Abstract Body: Introduction In 2014, deans of admissions from 5 University of California (UC) Medical Schools formed the California Longitudinal Evaluation of Admission Practices (CA-LEAP) consortium, supported by a grant from the Edward J. Stemmler Fund. Our study includes nearly 8,000 interviews from nearly 5000 individuals from three consecutive matriculation cycles (2011-20110) and includes information about applicant characteristics and demographics, interview method (MMI versus traditional), and admissions outcomes. The consortium has accumulated data from multiple institutions to longitudinally evaluate the relationships between applicant characteristics, interview and admissions practices, and performance outcomes in medical school and beyond. Research question We sought to assess the extent to which UC Medical Schools are using holistic review in the applicant selection process. To explore this question, we obtained qualitative information about interview practices from each school and analyzed the relationship between applicant demographics, disadvantage status, undergraduate metrics, and interview performance, and whether and acceptance offer was extended (yes/no). Methods Admissions and medical school performance data were collected and analyzed at UC Davis. Application and interview records were linked by a unique ID. Descriptive analyses were conducted to explore applicant characteristics and acceptance offer outcomes across the five schools. A series of multivariate logistic regression models were used to estimate the odds of receiving an acceptance offer versus not receiving an offer, as a function of applicant characteristics including gender, age, self-reported disadvantage status, undergraduate GPA and MCAT scores, and standardized interview performance score. Analyses were conducted within and across schools. Results Average undergraduate GPA and MCAT scores were similar among interviewees across the schools (range= range = 3.67-3.80). Schools interviewed equal proportions of men and women. The percentage of self-identified disadvantaged interviewees varied widely by school (15%-34%). Interview score was a significant predictor of receiving an acceptance offer at all schools (OR range=3.5-13.2). MCAT score and GPA were modestly positively predictive of an acceptance offer at 4 of 5 schools (OR range=3.5-13.2). Disadvantage status predicted offers in 3 of 5 schools. Discussion The consistent strong effect of interview score and the more modest effect GPA in predicting offers suggest that in the context of the overall admissions process, schools are being holistic both within and across institutions in that interview performance, metrics and disadvantage status all contribute to admissions decisions. The role of GPA and MCAT scores are used to screen applicants thus are understated in this study; those with low values generally do not receive an interview. Disadvantage status is associated with a greater likelihood of receiving an offer at some schools but not others, likely reflecting mission-based recruitment practices. Finally, within DA and non-DA interviewees, interview performance remains the strongest and most consistent predictor of receiving an offer, with variable influence of GPA and MCAT score. Subsequent studies will examine interview performance across schools and interview method (traditional versus MMI), relationships with pre-clinical and clinical performance in medical school. 

11:00 AM - 11:15 AM

Clinicians’ Perspectives on the Challenges of Standard Setting for Objective Structured Clinical Examination (OSCE)  
E.A. Hernandez, D. Kahn, E. Ha, R. Brook, C. Harris, M. Plesa, M. Lee, David Geffen School of Medicine at UCLA
Abstract Body: Introduction: All eight California schools administer an Objective Structured Clinical Examination (OSCE) at the end of the third year of medical school. Although the same cases are used, each school uses a different cut off for pass/fail. The David Geffen School of Medicine at UCLA decided to use the modified Angoff method for setting criterion-based standards to determine the passing score of the exam for the graduating class of 2017. Research questions: What were some of the challenges encountered by clinician educators who participated in standard setting process for the OSCE? How were these challenges overcome? What could have been done differently? Would the clinician educators use this method for other tests in the future in other settings? Methods Six clinician educators were briefed by a Professor of Education on the modified Angoff method. The method was subsequently used to determine the passing score for an eight-station OSCE administered at the end of the third year of medical school. A survey with the above four questions were sent to the six clinician educators who participated in the standard setting. Results The most common challenges clinician educators encountered were 1) determining what a minimally competent third year medical student would know (4/6), 2) scheduling a time when all clinicians could meet (3/6), and 3) not allowing clinicians’ own experience to influence their decision (2/6). Challenges to the first problem were overcome by drawing on the experience of faculty who had more experience teaching medical students (3/4). 2 out of 6 respondents thought having videos of sample target students available prior to discussion could have been helpful. All six clinicians thought they could use this in other settings they teach though all acknowledged logistical issues that would make it difficult. Discussion The OSCE administered at the end of the third year is a perfect opportunity for the medical school to assess the clinical competency of their medical students. The modified Angoff method helps set a criterion-based standard that determines borderline performance. However, the main challenge encountered by the clinician educators in this study were defining what a minimally competent student would know or perform. A better definition of minimal competency may be achieved by more extensive review of student videos. Clinicians thought they might use this method for other tests in the future. 

11:15 AM - 11:30 AM

Clinicians’ Perspectives on the Challenges of Standard Setting for Objective Structured Clinical Examination (OSCE)   
E.A. Hernandez, D. Kahn, E. Ha, R. Brook, C. Harris, M. Plesa, M. Lee, David Geffen School of Medicine at UCLA 
Abstract Body: Introduction: All eight California schools administer an Objective Structured Clinical Examination (OSCE) at the end of the third year of medical school. Although the same cases are used, each school uses a different cut off for pass/fail. The David Geffen School of Medicine at UCLA decided to use the modified Angoff method for setting criterion-based standards to determine the passing score of the exam for the graduating class of 2017. Research questions: What were some of the challenges encountered by clinician educators who participated in standard setting process for the OSCE? How were these challenges overcome? What could have been done differently? Would the clinician educators use this method for other tests in the future in other settings? Methods Six clinician educators were briefed by a Professor of Education on the modified Angoff method. The method was subsequently used to determine the passing score for an eight-station OSCE administered at the end of the third year of medical school. A survey with the above four questions were sent to the six clinician educators who participated in the standard setting. Results The most common challenges clinician educators encountered were 1) determining what a minimally competent third year medical student would know (4/6), 2) scheduling a time when all clinicians could meet (3/6), and 3) not allowing clinicians’ own experience to influence their decision (2/6). Challenges to the first problem were overcome by drawing on the experience of faculty who had more experience teaching medical students (3/4). 2 out of 6 respondents thought having videos of sample target students available prior to discussion could have been helpful. All six clinicians thought they could use this in other settings they teach though all acknowledged logistical issues that would make it difficult. Discussion The OSCE administered at the end of the third year is a perfect opportunity for the medical school to assess the clinical competency of their medical students. The modified Angoff method helps set a criterion-based standard that determines borderline performance. However, the main challenge encountered by the clinician educators in this study were defining what a minimally competent student would know or perform. A better definition of minimal competency may be achieved by more extensive review of student videos. Clinicians thought they might use this method for other tests in the future. 

11:30 AM - 11:45 AM

Bridging the Gap: Integrating Hypothesis-Driven Physical Exam and Clinical Reasoning for the 21st Century Physician   
S. Narayana, H. Nye, A. Chang, J. Stein, A. Richards, A. Ishizaki, S. Cornes, Neurology, UCSF 
Abstract Body: Context: The physician physical exam (PE) is a core skill that yields data for clinical decisions. Inaccuracies, missed maneuvers and misinterpretation of physical findings contribute to medical error. PE learning taught in a “head-to-toe” manner may lack clinical context. Many experts advocate learning diagnosis-focused (“hypothesis-driven”) PE to facilitate retention and integrate clinical reasoning. The optimal instructional method for early learners in this vein has yet been determined. Objectives: Design and implement case-based standardized patient (SP) sessions in which first and second-year medical students apply history-taking and hypothesis-driven PE skills, while integrating the clinical reasoning principles of data acquisition, problem representation, and illness scripts. Key Message: With expert input, we identified 12 essential chief complaints: fatigue, weight loss, shortness of breath, ear pain, vision loss, loss of consciousness, shoulder pain, abdominal pain, falls, cognitive impairment, rash and fever. We subsequently developed seven four-hour SP sessions around these chief complaints. Each session begins with a faculty didactic on essential concepts related to the upcoming SP encounter (e.g. how to identify various cardiac sounds, how to take vital signs, how to hold the otoscope, etc). Students then obtain a focused history and complete a PE. Abnormal PE findings are simulated when appropriate. While one student is performing these clinical skills, another functions as a time-keeper and prompts her peer with possible PE maneuvers from a comprehensive checklist developed by local content area experts. Meanwhile, the third student in the room is developing an evolving problem representation and differential diagnosis as data is acquired from the history and PE. At the end of the session, students regroup with their faculty preceptor to discuss their problem representation and differential. Prior to each session, students are expected to watch skills-based videos or read content-relevant material and complete an online self-assessment as preparation. During each session, students receive focused feedback from their preceptor on their history-taking and PE technique. They also receive feedback from their SPs on communication, basic PE technique, and flow of the clinical encounter. We recruited SPs to be representative of a diverse patient population and uphold an institutional mission to promote inclusion. The evaluation plan includes: 1. Focus groups with students and faculty. 2. Observed Standardized Clinical Encounters (OSCE) performance. Conclusion: A clinical skills curriculum incorporating focused history-taking, hypothesis-driven physical exam, and clinical reasoning principles, is feasible and may address the gap of integration of previously isolated clinical skill techniques. 

12:45 PM - 1:00 PM

Physician, Know Thyself: Reflecting on Identity and Medical Practice
C. Chow, G.A. Case, University of Utah
Abstract Body: Context: As our nation’s population continues to diversify, it is important to prepare physicians to work with patients from different backgrounds. While medical education trains students to become physicians, there is less emphasis on creating self-awareness around how personal experiences inform professional practice. We developed a workshop session that provides clinicians with the opportunity to explore their social and professional identities and reflect on how their identities might influence the delivery of culturally responsive and inclusive patient care. We have presented this session to four different samples: (1) to first-year medical students during their orientation training, (2) to fourth-year medical students in an elective course, (3) to residents and fellows during a didactic session, and (4) to faculty attending a health science educators symposium. Objectives: Describe aspects of identities that matter in personal lives and professional careers Analyze how these identities are socially constructed, particularly with respect to the identities of patients Develop strategies for bridging identity differences with patients Key Message: We conducted pre- and post-surveys at the beginning and end of our workshops in order to assess our first objective: how participants perceive the intersection of personal and professional identities. We adapted an existing scale to ask participants questions about identity and belonging, and how personal identities inform professional identities. We used a two-sample t-test to assess whether there were any significant changes on these measures between the surveys. In addition, we asked an open-ended question: Thinking back over today’s session, was there any particular concept that resonated with you? Did you have an “ah-ha” moment? If so, what was it about?” Our quantitative analysis reveals that participants were more likely to agree that their social groups reflect who they are after participating in the workshop. Additionally, they were more likely to agree that their social identities are connected to their decisions to pursue medicine after completing the workshop. The open-ended question was coded for themes. 18% of respondents did not respond to this question, or said “no”. The remaining 82% of participants’ responses fall into four themes: (1) awareness of social identity; (2) awareness of professional identity; (3) recognition of professional relationships; and (4) acknowledgement of privilege and difference. Conclusion: This workshop is useful in promoting thought and reflection around social identities, professional identities, the intersection of social and professional identities, and concepts of privilege and difference. Extending this effort is a worthwhile curricular endeavor. 

1:00 PM - 1:15 PM

Diversity and inclusion competencies for faculty educators
H. Nishimura, P. O'Sullivan, A. Teherani, A. Jackson, A. Rincon, D. Davis, A. Martinez, E. Bautista, University of California San Francisco
Abstract Body: Introduction Educators have a significant impact on learner success by creating a learning environment that is inclusive of diverse students and perspectives. As we actively recruit students from diverse backgrounds, we must ensure that medical educators possess skills to effectively teach diverse learners. Based on recommendations by LCME to increase the culture of inclusiveness and results from the University of California Climate Survey, it became clear that the University of California San Francisco (UCSF) needed to improve the climate of inclusion. UCSF is well known for its success in increasing student body diversity, yet instructors still face challenges in effectively teaching and mentoring students from diverse backgrounds. We identified that the Multicontextual Model for Diverse Learning Environments (MMDLE model) was a useful framework for explaining the organizational, curricular and individual level factors that impact the campus climate and thus learners. This study sought to identify faculty educator competencies for creating an inclusive learning environment for educating diverse learners aligned with the MMDLE. Research question(s) What are competencies faculty must master to successfully teach diverse learners in a health care environment? Methods This study took a multi-step approach: 1) we conducted a scoping review of literature on teaching diverse learners for medical and health professions faculty to identify existing competencies; 2) we interviewed 12 key stakeholders; 3) we reviewed learning objectives for 7 existing diversity and inclusion curricula at UCSF targeting faculty, trainees, or staff. Lastly we clustered the objectives and mapped them to the domains of the MMDLE to ensure coverage. Results From our analysis, we developed 7 essential faculty educator competencies which were then mapped to the MMDLE. Domain 1: Instructor Identity Build foundational knowledge of key diversity concepts such as privilege, power, unconscious bias, cultural humility, microaggressions, intersectionality etc. Establish awareness of one’s own identity(ies). Develop ability to recognize, remedy and monitor conscious and unconscious bias and discrimination. Domain 2: Pedagogy and Teaching Methods Communicate effectively across racial, ethnic, gender, social other differences for learners and for patients. Develop educational skills for creating an inclusive learning environment for learners and for patients. Domain 3: Course Content Apply curriculum development skills to include diversity in course and curricular design and content. Represent and invite diverse perspectives in the structure of learning opportunities. Discussion Enabling a truly diverse and inclusive climate for learners requires faculty to be equipped with skills to teach diverse learners. This methodology for developing competencies may have implications for other academic institutions looking to enhance faculty development offerings related to diversity and inclusion in a manner that is contextually specific to the needs of the faculty, learners, and institution.

1:15 PM - 1:30 PM

Outcomes and Participant Perspectives Following a UCSF-CORO Faculty Leadership Training Program
J.Y. Tsoh, J. Cheng, A. Kuo, J. Barr, I. Merry, M. Fisch, B. Alldredge, A. Azzam, University of California San Francisco
S. Shain, L. Whitcanack, Coro Northern California
Abstract Body: Context Fostering professional development and academic advancement are critical elements for improving faculty satisfaction, success and retention. Since 2005, the University of California San Francisco (UCSF) has offered the UCSF-CORO Faculty Leadership Collaborative, a 10-session (75 program hours) leadership training program for cohorts of up to 16 faculty members. The Coro Northern California conducts the training with goals to catalyze individual and collective change to benefit both the participant and the broader UCSF community. The Collaborative has high completion rates with promising objective leadership outcomes; however, little is known about participants’ perceived values of the program, particularly longer term (>12 months) impacts after program completion. Objectives We sought to assess long-term impacts of the Collaborative from the 2005-2012 graduates' perspectives with an online survey conducted during August - October 2013. Using a mixed methods design incorporating quantitative and qualitative items, we integrated quantitative findings with emergent themes from content analyses of participant comments on open-ended questions. Key Message Between 2005–2012, 175 faculty members applied to the Collaborative: 139 (79%) were accepted of whom 136 (98%) completed the program. Graduates (64% women and 65% Whites) included faculty from schools of medicine (79%), dentistry (10%), nursing (6%), and pharmacy (5%) representing > 30 departments. The academic ranks at time of program participation included full (29%), associate (45%) and assistant (26%) professors. Since program completion, 11 (8%) graduates have obtained a UCSF leadership position as chairs or deans, of whom 9 (82%) were women. Survey respondents (N=72, response rate: 55%) included 66% women, 63% Whites, 19% Asians, 10% Blacks, and 6% Latinos. Most (92%) agreed the sponsorship of the Collaborative demonstrated the University’s commitment to foster faculty development, and indicated noticeable changes in leadership attitudes or behaviors (92%) and skills (99%). Quantitative and qualitative data revealed that graduates perceived impacts at multiple levels. At an individual level, most believed that it led to personal growth with increased self-awareness, confidence, and aspiration. The program fostered development of leadership skills in conflict resolution, team management, and giving and receiving feedback with 93% indicated an increased ability to lead in challenging times. At an interpersonal level, graduates described the program led to new collaborations. Most (90%) reported increased interpersonal leadership skills. More women than men perceived their program participation encouraged them to expand their leadership roles in professional or volunteer organizations outside of UCSF (73% vs. 48%, p=0.03). At an organizational level, some graduates described experiencing increased cohesion within departments or units. A majority (92%) said the program had increased their understanding of UCSF as an organization. More full professors agreed to the statement that the program improved the climate for UCSF than associate or assistant professors (79% vs. 45%, p=0.005). While 31% did not perceive a positive impact on recruiting or retaining faculty, the Collaborative was described as a means to provide support, connectedness, empowerment, and formal mentorship, particularly for women and minority faculty. Conclusion The UCSF-CORO Faculty Leadership Collaborative, as perceived by the program participants 12 months to 8 years after program completion, has generated positive and sustaining impacts at multiple levels, from individual to organizational. The Collaborative successfully reached a diverse faculty audience, created a faculty leadership network and led to new leadership opportunities. Although the impact on recruitment and retention are unclear, faculty are appreciative of the University’s investment in faculty development through leadership training. 

12:45 PM - 1:00 PM

Progress towards mastery learning: converting foundational science assessments from partially summative to purely formative
L. Sells, M.M. Metzstein, T. Stocks, M. Vo, J.M. Colbert-Getz, J.E. Lindsley, University of Utah School of Medicine
Abstract Body: Introduction: As part of our goal to support student self-efficacy and move our culture from performance-based to mastery-based, the University of Utah School of Medicine is exploring the effects of replacing summative quizzes with formative assessments in the pre-clerkship curriculum. This change began in the fall of 2016 with one 9-week year 2 course in which the three MCQ quizzes were changed from each being worth 12% (2015) to 0% (2016) of the course grade. The final knowledge MCQ exam was changed from contributing 37% (2015) to 60% (2016), and became a must-pass element of the course grading. Using Kirkpatrick’s evaluation framework, we investigated students’ satisfaction and learning outcomes to determine the impact of this change. Research questions: Does a switch to purely formative assessments change student performance within one 9-week MS2 course? Do students perceive a change in their ability to master course content? Do students experience increased pathologic stress with more heavily weighted final exams? Methods: We compared performance on the three quizzes and final MCQ examination between MS2s in 2015 (N = 101-102) and MS2s in 2016 (N = 114-118) with Mann Whitney U tests limited to items that were similar for both years. Students’ satisfaction with the change was measured by the percentage of MS2 in 2016 agreeing with an end of course evaluation question: The formative quiz structure enhanced my mastery of course content. To determine if the change caused increased pathologic stress in students, psychological services usage data were queried. Results: There were no significant performance differences between 2015 and 2016 MS2s on quizzes 1 (76% for both classes, p = 0.546), 3 (78% for both classes, p = 0.934) and the final (83% vs. 81%, p = 0.194). On quiz 2, MS2s in 2015 did perform significantly better than MS2s in 2016, 82% vs. 78%, p = 0.011. The majority of students (75%, N = 112) reported that the formative quizzes enhanced their mastery of course content. The number of psychological sessions provided to MS2s was similar during the first four weeks (34 in 2015 and 33 in 2016) and dropped during the final 5 weeks (48 in 2015 and 18 in 2016) of the course. Discussion Switching to formative-only mid-course assessments within one integrated MS2 course did not decrease performance on two quizzes and a final and the majority of students were satisfied with the change. No evidence of increased student stress was seen. 

1:00 PM - 1:15 PM

Correlation between advanced thinking skills and USMLE Step One licensure exam scores.
J. Rebman, Student, UNRMED
T.K. Baker, J. Weinert, G.S. Shonkwiler, B.J. Janes, OME, UNRMED
K.C. Facemyer, Pharmacology, UNRMED
Abstract Body: Bloom’s taxonomy (1956) is a widely used tool for describing and classifying educational objectives into cognitive domains of increasing complexity of thinking. As such, advanced thinking skills are associated with higher Bloom’s level, such as “Evaluation”, while more basic thinking skills are associated with lower Bloom’s level, such as “Knowledge”. We tested the hypothesis that higher performance of advanced thinking skills (variable one) [as described by Blooms Level 6 verbs] involved in Team Based Learning (TBL) experiences, correlates positively with higher USMLE Step 1 scores (variable two). Using Bloom’s taxonomy (1956), we rated and sorted individual Readiness Assurance Test (iRAT) questions according to the verb used in the assessment. Performance on batches of Knowledge level (Bloom’s Level 1 [B1]) questions was compared with performance on batches of Evaluation level (Bloom’s Level 6 [B6]) questions. We sorted student’s performance on B6 questions into Above Average, Average and Below Average groups. We compared (and matched and controled) their underlying aptitude (equivalent based on their entry MCAT performances) with their USMLE Step 1 performances. Using a sample of mid curriculum iRAT TBL scores, we observed that students with above average performance on B6 questions, averaged 6.00 points better than their B6 average peers, while students with below average B6 question performance, averaged 13.45 points below their B6 above average peers. The MCAT performance of these 3 groups is identical and if this is a proxy for equating individual medical school aptitude, then the difference in the performances on B6 level questions suggests that a) some educational treatment (perhaps the practices associated with TBL) facilitated this differentiation, and that b) demonstrating Above Average B6 performance is a predictor of advanced USMLE Step 1 performance. The quantitative connection between these two variables supports the hypothesis that advanced thinking skills inherent in TBL are associated with advanced USMLE Step 1 performance. 

1:15 PM - 1:30 PM

Development and Validation of an Inquiry Assessment Tool for the UCSF Bridges Curriculum: A Modified Delphi Study
S. Brondfield, C. Boscardin, G. Strewler, K. Hauer, M. Hermiston, K. Hyland, S.A. Oakes, University of California, San Francisco
Abstract Body: Introduction The new UCSF Bridges curriculum teaches an ‘inquiry habit of mind’: the process of approaching the unknown with curiosity and skepticism, challenging current concepts, and creating new knowledge. The four-year curriculum includes weekly inquiry small group cases, didactic immersions, and individualized capstone projects. To assess inquiry habit development, a tool is needed to enable faculty to monitor students’ behaviors and provide feedback. Research question We aimed to develop and validate a tool for small group facilitators to assess early medical students’ inquiry behaviors. Methods We followed established guidelines for designing an assessment tool and gathered evidence for content validity. We conducted a literature review to identify essential inquiry elements, verified findings with a UCSF expert faculty focus group, and synthesized the literature and feedback into 40 inquiry behaviors for faculty facilitators to assess. UCSF faculty educators (n=33) and final-year Health Professions Education Pathway medical students (n=14) participated in a modified Delphi survey using a Likert scale (1=absolutely do not include, 5=very important) to verify and refine the behaviors. In the second round, participants rerated each item after viewing their individual first round responses alongside the group’s mean and standard deviation data (n=31 faculty and 10 students). Inclusion threshold was a second-round median rating of 5 with 70% consensus. In three structured cognitive interviews with expert faculty educators who had not participated in the Delphi study, identified items were further refined for clarity. The authors then wrote anchors and descriptors. The tool was piloted with first-year medical student inquiry small group facilitators. Results Two-round response rate was 77% (79% faculty, 71% students). Five items met the threshold: Does the student 1) select relevant questions to pursue? 2) justify explanations with evidence? 3) critically evaluate his/her explanation in light of alternative possibilities? 4) allow for the possibility that his/her own knowledge may not be completely correct? 5) collaborate well with peers? After a pilot, the authors reviewed the data, and three small group instructors provided additional feedback; the anchors were edited with simplified descriptors. Tool implementation is ongoing. Discussion We designed and began to validate an inquiry assessment tool for early medical students. Challenges included capturing the inquiry concept while keeping the tool short, and difficulty observing complex small group behaviors for every student. Focus groups are planned to solicit feedback from facilitators and students. We hope to promote inquiry skillset acquisition by clarifying these behaviors and guiding feedback to students with this assessment tool. 

12:45 PM - 1:00 PM

The role of third year clerkship students and the potential for change
C. Burke, D.E. Masters, P.S. O'Sullivan, L. Sheu, University of California, San Francisco
Abstract Body: Introduction Preclerkship medical education has undergone extensive reform and the clinical years are growing targets for curricular innovation. Perceptions of the third year medical student (MS3) role vary and are not standardized across clerkships or clinical sites. UCSF is implementing a new undergraduate medical curriculum emphasizing inquiry skills and systems knowledge in the preclerkship years. Understanding the perspective of clinical rotation leadership regarding the current role of MS3s may facilitate evolution and redefining of traditional MS3 roles within a novel curriculum. Research Question(s) What is the current role of the MS3? What factors influence this role? What is the potential for MS3 roles to change, particularly in the context of preclinical curricular change? Methods In this qualitative exploratory study, we interviewed current clerkship directors and site directors for eight core clerkships using semi-structured questions regarding the current MS3 role, factors contributing to the role, and the potential for changing the role in the context of curricular reform. Through an iterative consensus building process, themes relevant to the three research questions were identified. Results Twenty-three clerkship directors and site directors participated. Preliminary results reveal that the MS3 role is determined by intrinsic student-specific factors, supervisor-specific factors, and system constraints. The MS3 role is considered unique and adds value to teams through team-patient communication and inquiry tasks. The role is considered authentic and workplace learning is enhanced when the student is able to function as a “mini-resident,” assisting with patient care tasks. Directors’ positive perceptions of a novel curriculum are associated with identification of new and expanded MS3 roles, such as engagement in interdisciplinary collaboration and care transitions, or formalizing sharing of learning topics with the team; neutral or negative perceptions are associated with concerns about further systems constraints or deviation from traditional clinical skills and knowledge acquisition. Discussion The MS3 role allows students unique opportunities to engage in communication and inquiry tasks as they move towards more central and traditional team roles as “mini-residents.” While system constraints understandably limit the MS3 role, the role is inconsistent and largely determined by individual students and supervisors. Directors’ historical perspectives and reliance on tradition may hinder them from expanding the MS3 role to capitalize on new skills developed within a reformed preclinical curriculum. Thoughtful and deliberate engagement of directors is required to assist in envisioning changes to MS3 roles that utilize novel skills in training 21st century physicians. Target Audience Medical educators, medical students 

1:00-1:15 PM

Utilizing Technological Advances to Improve Surgery Curriculum: Experience with a Mobile Application
C.A. Green, N. Zhao, E. Kim, P. O'Sullivan, H. Chern, University of California, San Francisco
Abstract Body: Introduction: Technology provides opportunity to improve instructional approach. Previously, we published a successful home-video, basic surgical skills curriculum. Unfortunately, implementation required substantial faculty time and resources, and the approach was limited by delayed feedback and technical difficulties with cumbersome recording equipment. To address these limitations we integrated the home-video curricula with a mobile application platform. Our purpose is to describe the format of this application and learner satisfaction. Methods: This mobile application incorporates a patented pedagogical design based on Erikson’s deliberate practice and Bandura’s social learning theory. Within the platform instructors build modules focused on skill acquisition. Each module includes activities at different stages, representing a step-wise approach to learning: Challenge, Peer Review and Recap. In the Challenge phase, learners watch a video of surgical tasks completed by experts. In response, learners upload a video of themselves performing the same task. After submitting their video, learners enter the Peer Review phase where they are randomly assigned peer videos (of the same task) to review. Learners complete three peer video assessments using a grading rubric highlighting essential components for the task. After completion, learners “unlock” the final Recap stage where they receive individual feedback and can review their own videos. Using our basic surgical skills home-video curricula, we created 16 different modules with associated grading rubrics. We then invited 2 different learner groups to participate, graduating medical students and matriculating surgical residents. In addition to use of the mobile application, learners participated in 2-4 lab sessions run by surgical faculty focused on technical skills and completed a final survey about their experience with the platform. Results: In total 50 different learners submitted videos of assigned tasks and completed peer reviews. Learners testified to positive experiences specifically for the Peer Review Stage, structured home practice, ease of mobile access to submit and review videos and ongoing immediate feedback. Over half of the learners reported spending at least 10-30 minutes practicing skills before recording their videos and over 80% re-recorded at least 2 times before submission. Discussion: Based on these findings, learners appreciated the practice and peer feedback. The ability to do these steps was greatly facilitated by the electronic platform. Learners reported motivation to re-record prior to submission, indicating use of the application resulted in skill repetition. Peer feedback significantly decreased faculty resources compared to our prior implementation. Future investigation could determine the sufficiency of this platform as a stand-alone curriculum to teach surgical skills. 

1:15-1:30 PM

Do Surgical Preparatory Courses Give Incoming Residents a Technical Advantage?
C.A. Green, E. Huang, N. Zhao, P. O'Sullivan, E. Kim, H. Chern, University of California, San Francisco
Abstract Body: INTRODUCTION: Graduating medical students (GMS) often participate in courses to facilitate transition from medical school to residency. For those entering surgery, curriculum frequently emphasizes technical skills. However, the sustainability and benefits of this skill acquisition once in residency remains uncertain. This study assessed technical skill performance of GMS before (T1) and after a preparatory course (T2) and then again 2 (T3) and 4 (T4) months later as surgical residents, with comparison to surgical interns without such a course. METHODS: In April 2016, 16 GMS took the surgical preparatory course. In July-August, 2016, the GMS as interns completed the basic skills curriculum for all surgical interns. Both courses included a home video curriculum with completion of the same four technical exercises at the start and conclusion of the course. Three expert surgeons scored the video exercises and we calculated average reviewers’ scores across the four tasks. Overall scores were examined for GMS across the 4 time points. Course naive (control) interns were compared to these GMS at T3 and T4. RESULTS: Seven of the 16 GMS enrolled in the preparatory course matched to our institution, and 41 residents completed the intern basic skills curriculum. Of these interns, 32 completed all pre/post course assessments (T3 and T4), and the 7 GMS-interns completed assessments at all 4 time points. Results reveal score increases for GMS from 74.5%(T1) to 94.1%(T2) (p<0.001), and maintained elevated performance in residency (89.08% (T3) and 93.02% (T4)). Control interns also improved with a course (68.2%(T3) to 82.9%(T4), p<0.001). The GMS-interns scored higher at the start of residency compared to the control interns (T3, 89.08% vs 65.03%, p<0.001), with both groups achieving near the maximum score at the end of the curriculum. DISCUSSION: This study corroborates existing evidence that preparatory courses improve performance but adds evidence that the skills are maintained upon matriculation. The study supports that our structured curriculum consistently benefits learners, but those without a preparatory course start further behind their peers, requiring a steeper learning curve. Furthermore we illustrate a potential solution for the often-feared and highly publicized “July Effect” (gap in resident skill during the first month of residency). Our GMS show technical gains that accompany them into residency, erasing the gap seen in their intern control peers. 

1:45 PM - 2:00 PM

Entrustable professional activities (EPAs) in undergraduate medical education (UME): How supervisors make entrustment decisions about medical students 
C. Boscardin, K.E. Hauer, A. Teherani, Medicine, University of California, San Francisco
S. Oza, Albert Einstein College of Medicine, New York
P. Walstock, University of Groningen and University Medical Center Groningen
Abstract Body: Introduction Trust is at the nexus of EPAs as an assessment tool. Work to date on EPAs in undergraduate medical education (UME) has defined and provided content validity evidence for EPAs. Little is known about how entrustment decisions are made for UME EPAs. Research Questions We investigated what levels of entrustment are typical for clinical year students at different levels, how supervisors make entrustment decisions, and how EPA ratings relate to other performance measures. Methods We implemented two institutionally-defined EPAs in the longitudinal integrated clerkships at the mid- and end-of third year (MS3) and end-of sub-internship in the fourth year (MS4). The two EPAs were: (1) Evaluate and care for a patient with an acute complaint and (2) Evaluate and care for a patient with a chronic medical problem. Supervising clinicians rated students on a 5-point scale developed by ten Cate and colleagues in which level 4 indicates the ability of the student to act independently. Results MS3s and MS4s were both predominantly rated at level 3 (may act under reactive supervision). As a group, MS4s were rated slightly higher than MS3s. The primary means by which clinical supervisors gathered information to judge students’ readiness for independent work was through direct observation of a range of activities. Although clinical supervisors saw the EPAs as representative of the activities students’ partake in, some felt that ratings of entrustment were less relevant, particularly in the third-year, when students would not be able to achieve independence. We found low to moderate correlations between EPA scores and overall ratings by clerkship directors and comprehensive standardized patient examination scores for both MS3s and MS4s. All students scored between 3 (good) and 4 (outstanding) on clerkship director ratings, and no scores below 3 were recorded. In contrast, supervisors used the full 5-point EPA scale, resulting in greater performance differentiation across learners. Discussion MS3s and MS4s are trusted to complete an activity with supervision readily available upon request. Supervisors’ reliance on direct observation indicates that EPAs may help facilitate high quality assessment. EPAs might be used to diversify the information on which to base consequential summative decisions. Our recommendations for how EPAs can be structured to improve judgments of trust include focus on scale use, EPA details, and faculty development. Advancing the conversation on EPAs entails inquiry into how trust forms across the continuum of medical school education starting in early clinical education.  

2:00 PM - 2:15 PM

The importance of faculty development in implementing assessments aligned with EPAs
S. Stern, K. Miller, J. Wold, K. Anderson, T. Glasgow, G. Zinkhan, D. Roussel, S. Lamb, J. Colbert-Getz, University of Utah School of Medicine
Abstract Body: Context A critical aspect to ensure that medical students achieve milestones in a competency-based framework is developing faculty members’ understanding of the Entrustable Professional Activities (EPAs). However, it is unknown how much faculty development is needed before faculty can be entrusted to accurately assess medical students. In 2013 the University of Utah School of Medicine implemented a two-year clinical method curriculum (CMC) using the EPA framework to develop goals and assessments. Learning communities of 10 students/faculty member are used to deliver the curriculum to each cohort of 100-120 students. Objectives 1. To describe faculty development for learning community faculty who rate medical students on assessments aligned with EPAs. 2. To identify barriers, as reported by learning community faculty, in faculty development relating to EPAs Key Message Learning community faculty received the following EPA faculty development: Two brief presentations of EPAs by CMC course directors in lecture with students and faculty present; presentation and discussion of EPAs at the annual faculty retreat; annual “just-in-time” instruction prior to completing EPA progress forms for students; and post-hoc faculty development discussions after completing EPA progress forms. Barriers to EPA faculty development reported by core faculty members were: variability of faculty knowledge about how EPAs are used and assessed in other aspects of medical training (i.e. GME specific EPAs); the length of the EPA-based assessments and applicability to student level of training; and the level of detail associated with each EPA that faculty are expected to assess. Conclusion Little research exists on the amount and type of faculty development necessary for rating students using assessments developed with the EPA framework. Next steps are to survey learning community faculty on their post training knowledge of EPAs and their confidence in assessing students, and to evaluate the quality of feedback provided to students through the EPA-based assessments. 

2:15 PM - 2:30 PM

How much explicit curriculum do students need about EPAs?
J. Wold, K. Miller, S. Stern, K. Anderson, T. Glasgow, G. Zinkhan, D. Roussel, S. Lamb, J. Colbert-Getz, University of Utah School of Medicine
Abstract Body: Context As medical schools adopt the Entrustable Professional Activities (EPAs) as a framework for curriculum development and assessment it is important to consider students’ knowledge about this framework. Recent research has focused on faculty use of EPA-based assessment of students, but little or no attention is given to another important stakeholder in the process - the students. If students are expected to meet milestones and eventually be deemed “entrusted” then it is important to know how much explicit curriculum is needed about the EPAs. In 2013 the University of Utah School of Medicine implemented a two-year clinical method curriculum (CMC) using the EPA framework to develop goals and assessments. Learning communities of 10 students/faculty member are used to deliver the curriculum to each cohort of 100-120 students. Objectives To describe the explicit curriculum to students on EPAs during a two-year clinical method curriculum Key Message The explicit curricular elements to students about EPAs were: Introduction to the EPAs during CMC orientation; CMC course syllabi goals and objectives notated with specific EPAs; presentation and discussion of EPA’s prior to students completing self-assessments; “Where we have been and where were are going” discussions related to the EPAs; detailed description of EPA components in the self-assessment tool students complete biannually; one-on-one student meetings with learning community faculty to discuss portions of the EPA-based self-assessment; and citation of the AAMC EPA blueprint made available to students. Conclusion The more knowledgeable students are about EPAs, the more focused and invested they may be in achieving entrustment. Further study as to which components are most effective and the depth to which students should understand the EPAs is necessary. Our next step is to determine if students’ construct effective individual learning goals focused on EPA milestone development once they have received the explicit EPA curriculum. 

Full-time basic science educators: a “gold-standard” for medical student academic support 
S.D. Schneid, K. Brandl, Skaggs School of Pharmacy and Pharmaceutical Sciences & School of Medicine, University of California, San Diego (UCSD)
A. Apperson, N. Laiken, Office of Educational Support Services, University of California, San Diego (UCSD) School of Medicine
Abstract Body: Context Academic support programs are meant to enhance student learning and promote successful progression through a challenging medical curriculum. While many medical school academic support programs offer tutoring services for their students, the majority of tutors are medical students or graduate students1. Without the presence of full-time experienced professional educators dedicated to being the most current on the content and providing significant accessibility to students, tutoring services will be less than optimal. And as students with multiple acceptances make their final decision about where to attend medical school, the quality of academic support available may have significant implications for optimal recruitment of applicants. Objectives The University of California, San Diego (UCSD) School of Medicine (SOM) created the Office of Educational Support Services (OESS) to provide comprehensive academic assistance in the basic sciences. Uniquely, the OESS is staffed by four full-time educators. As part of their job training, OESS educators were expected to attend all medical school lectures and become content experts in a broad range of basic science disciplines so they could maximally help medical students learn this content. On a weekly basis, they provide approximately ten hours of “drop-in” office hours and three hours of review sessions available to the entire class. The remainder of their available time is filled meeting with students individually during a course and during the summer for those requiring remediation. Because the OESS educators are full-time employees and are involved with multiple courses throughout curriculum, they are able to establish long-term and meaningful interactions with the medical students. All academic assistance provided by the OESS is done collaboratively with the course directors and faculty advisors, and has the full support of the SOM administration. Key Message The AAMC collects student satisfaction data from medical school graduates annually using the Medical School Graduation Questionnaire (Likert scale rating: 1 = not at all satisfied and 5 = very satisfied). From 2014-2016, UCSD medical students (n=220) reported a high level of satisfaction with tutoring provided by UCSD SOM. The average at UCSD SOM was 4.5 compared to 3.9 for all other medical schools (p<0.0001). In fact, 67% (SD = 3.0%) of UCSD medical students reported being very satisfied compared to 34% (SD = 0.6%) of students from the other 136 US medical schools (n=16,756). Also, in 2015, 126 first-year UCSD medical students were solicited to fill out a survey regarding OESS. Eighty-two students responded and of the 48 students who reported receiving multiple medical school acceptances, 52% reported that the presence of OESS influenced their decision to attend UCSD. Conclusion Making a long-term investment in the training of full-time basic science educators has built a unique and robust academic support service that provides a high level of medical student satisfaction that is nearly double that of other US medical schools (67% versus 34% being very satisfied). Furthermore, the presence of a strong academic support program can play an important role when a student is deciding which medical school to attend. References 1. Saks, NS, Karl S. Academic Services in U.S. and Canadian Medical Schools. Med Educ Online 9:6. Available from http://med-ed-online.net/index.php/meo/article/view/4348  

2:45 PM - 3:00 PM

Intended and Emergent Learning in Interprofessional Scenario-Based Simulation
B.C. OBrien, M. Wamsley, J. Rivera, UCSF
Abstract Body: Introduction: Interprofessional (IP) scenario-based simulations are designed to support participants’ learning with, about, and from one another. Educators’ make decisions about scenario content, participants, and materials based on intended learning objectives. However, even highly-structured scenarios leave room for interpretation by participants, which can yield unanticipated, or emergent, learning. Examining emergent learning in scenario-based simulations may provide insights that can enhance their design. Research question(s): What learning opportunities emerge during an IP scenario-based simulation? How do they relate to the intended learning? Methods: We qualitatively analyzed 9 videos from an IP standardized patient simulation of a 70 year-old woman who recently fell, has multiple chronic conditions and a complex medication regimen. We used interaction analysis techniques to document how medical, NP, pharmacy, physical therapy, and dental students worked in teams of four to distribute and coordinate tasks and perspectives during the 15 minute pre-visit huddle portion of the simulation. Using intended and emergent learning as an analytic lens, we focused on the relationship between specific design features (e.g., instructions given to students, sequence of the scenario, and learning objectives) and students’ interactions during the scenario. Results: The pre-visit huddle gives students time to read the case, discuss roles and tasks, and decide the order in which they will each interview the patient for 12 minutes. By design, the huddle provides an opportunity for students to learn about one another’s expertise and coordinate efforts to optimize patient care. Some groups took up this learning opportunity by having each member describe their concerns and suggested approach, then trying to decide a logical order. Other groups focused more on the details of the case, trying to develop a shared understanding of content such as medications or disease processes. In these groups, participants had few opportunities to practice coordinating and negotiating tasks and roles. Several groups struggled to decide a logical order. Behaviors potentially contributing to this struggle included: no acknowledgement of overlapping roles, reluctance to take responsibility for tasks several team members could perform, and framing the circumstances as unrealistic. These behaviors created emergent learning opportunities that appeared to diverge from intended learning. Discussion: Literature on formal, informal, and hidden curricula has raised awareness of differences between stated objectives and actual practices across learning environments. Our findings add to this work by highlighting ways in which students actively construct, and thereby contribute to, learning opportunities in a simulated environment that is designed for IP learning. 

3:00 PM - 3:15 PM

Teaching Intensive Care Medicine from the Learners’ Perspective: A Multicenter Evaluation
L. Santhosh, UCSF Medical Center
W.G. Carlos, Indiana University
A. Brady, University of Washington
M. Sharp, Johns Hopkins University
 Abstract Body: INTRODUCTION/RESEARCH QUESTION: It is important for teaching physicians to know what qualities are most valued by learners. To date, only two studies have been published addressing internal medicine residents’ perceptions of teaching faculty [1,2]. We sought to expand on the evidence about this topic through a multicenter study at four geographically diverse academic medical centers. Our study focused on teaching characteristics of intensive care unit (ICU) physicians that learners perceive are most impactful. METHODS: The study was conducted at Indiana University, Johns Hopkins University, UCSF, and University of Washington. Internal medicine residents completed an anonymous online survey rating the importance of characteristics of ICU attending role models. Questions on our 37-item-questionnaire were derived from prior studies and from the Stanford Faculty Development Center for Medical Teachers Clinician Teaching program [1,2,3]. Learners also named impactful role models at their institutions. T-tests were used to compare scores. RESULTS: 260 residents responded to the survey. The attributes most commonly rated as “very important” to trainees were that the attending enjoyed teaching house staff, demonstrated empathy and compassion to patients and families, explained clinical reasoning & differential diagnoses, treated non-MD staff members respectfully, and showed enthusiasm on rounds. Factors that trainees rated as less important were having numerous research publications, having served as a chief resident, sharing personal life with house staff, and organizing end-of-rotation social events. DISCUSSION Our study provides new information to teaching faculty striving to impact their learners’ education. While prior data demonstrated that learners valued attendings having served as a chief resident and sharing personal information with house staff, our study did not replicate this. We confirmed that learners appreciated teachers who are perceived to enjoy teaching. We also discovered that expression of empathy, explanation of clinical reasoning, and qualities of professionalism were influential. This may reflect a new generation of learners, differences between ICU versus ward teaching, or institutional variations. Next steps include analyzing course evaluations of named attending role models and conducting thematic analysis to identify predictors of teaching excellence. REFERENCES: Wright, S., et al. "Attributes of excellent attending-physician role models." New England Journal of Medicine (1998). Wright, S. "Examining what residents look for in their role models." Academic Medicine (1996). Skeff, K., et al. "The Stanford faculty development program: a dissemination approach to faculty development for medical teachers." Teaching and Learning in Medicine. (1992). 

2:45 PM - 3:00 PM

The GME program conundrum: A grounded theory of valued characteristics Multicenter Evaluation
G.F. Martinez, K.S. Knox, K. Spear-Ellinwood, K. Moynahan, C. Clemens, University of Arizona
Abstract Body: Introduction Hoekzema et al. state that Graduate Medical Education (GME) program quality is an ill-defined construct with no widely-agreed upon metrics. Yet, directors are required to complete reports regarding the quality of their programs. Previous studies explore residency director and resident perceptions and propose metrics to assess quality. Traditional metrics include: board pass rate, in-training exams, and accreditation status while others look at graduate trajectories, clinical performance measures or a combination of all. Little is known about what values inform the definition of program quality to other educational leaders. Research Question The purpose of our study is to learn how program quality is defined by a broader scope of educational leaders not included in previous studies. We ask: what metrics are valued that inform leaders about the quality of our GME programs that may be under recognized nationally? Methods In our IRB exempt study, we applied the inductive methodology of Grounded Theory to categorize concepts and formulate a hypothesis. In-depth hour long individual and focus group interviews were conducted and transcribed verbatim between August 2015 and May 2016. Participants included department chairs, vice chairs for education, residency directors and associate directors interviews (N =17) from five large clinical departments at the University of Arizona College of Medicine-Tucson. Constant comparison analysis was conducted. Results Overall, there was a belief that programs are only as good as the caliber of their residents, teaching faculty, curricular structure relative to departmental training missions. Valued metrics included: ratio of primary care or subspecialty career aspirations to actual matriculation achieved, percentage of residents presenting scholarly projects at national conferences, rate of clinical or educational research productivity of teaching faculty, degree of research opportunities, trainee diversity, percentage going into academic positions versus community practice, and faculty turnover impact. Differences in the perceived value in retaining students and residents into fellowships and faculty positions as a quality indicator emerged. Some assigned high value to the “pipeline” theme or hiring those they trained while others valued graduates leaving to prestigious academic institutions more. U.S. News & World Report and Doximity residency rankings were perceived as not valuable as criteria and methodologies were seen as irrelevant or flawed. We theorize that the above outcomes measured against specific department missions best indicates quality for those in our study. Discussion Knowing desired mission outcomes of departments is important to contextualizing quality and should be considered in annual reviews and self-studies. 

3:00 PM - 3:15 PM

Efficacy of asynchronous teaching (flipped classroom) model in point-of-care ultrasound for medical students and first year emergency medicine residents: A pilot study
P. Aguilera, B. Lara, F. Vargas, D. Sanchez, Pontificia Universidad Catolica de Chile, Santiago, CHILE
M. Lee, V. Sigalov, A. Chiem, University of California, Los Angeles
Abstract Body: Introduction The use of Point of care ultrasound (POCUS) in medical education has grown during the last ten years. Ultrasound in undergraduate medical education has gained increasing popularity. One of the difficulties faced for its implementation is the time needed to teach practical skills and image acquisition by faculty members and also to translate it to clinical relevant information. We utilized a flipped classroom model to train novices in POCUS. Research Questions How effective is the flipped classroom model in teaching POCUS to medical trainees? Methods: We recruited eight volunteer emergency medicine PGY1 and seven last year medical students that completed an emergency medicine clerkship. The participants’ baseline knowledge and image acquisition skills for E-FAST ( extended focused assesment sonography in trauma), lung, IVC ( inferior vena cava) and basic cardiac ultrasound (ECHO) were evaluated by a test (written and online) consisting of 15 questions and cases and a standardized hands-on exam according to American College of Emergency Physician imaging guidelines (9 items, scored from 1 to 5, with 5 being the best score). After baseline evaluation, participants received a flipped classroom intervention consisting of 80 minutes online videos covering imaging acquisition techniques and normal/abnormal ultrasound findings recognition. Then they received a 90-minute hands-on training with ultrasound certified experts. Their performance and subjective evaluation of the intervention were assessed a week after the intervention. Data analysis was conducted using Wilcoxon matched-pair test. Results: Fifteen participants completed all pre- and post-tests. Their image acquisition skills improved from 20 to 33 points out of 45 maximum score (p<0.05). 13 of the 15 subjects improved in the knowledge assessment portion by at least one point, which did not reach statistical significance (p=0.5). Discussion Basic ECHO and E FAST POCUS teaching of medical students and EM interns is feasible. Asynchronous learning is a good mechanism to teach and learn image interpretation. Image acquisition, however, requires more hands-on training to acquire good images. The trainees seem to have good ultrasound baseline knowledge. The content coverage and difficulty level of the pre-test on knowledge may be increased to better detect trainees deficiency. This cohort will be followed to assess retention capacity 60 days after. This pilot study will serve as a baseline for future research in POCUS teaching. 

3:15 PM - 3:30 PM

Inclusion of a formal pediatric curriculum in an adult rheumatology fellowship training program for application in areas underserved by pediatric rheumatologists
S. Stern, A. Woodward, A. Hersh, A. Sawitzke, D. Lebiedz-Odrobina, University of Utah
 Abstract Body: Context There is a critical shortage of pediatric rheumatologists in the Intermountain West and in other geographic regions across the United States as there are states without a full-time pediatric rheumatologist (ID, WY, MT) and others with severely limited access to pediatric rheumatology care. The current number of fellows being trained in pediatric rheumatology is unlikely to meet this need. Therefore, adult rheumatologists are frequently asked to evaluate pediatric patients’ care with limited training in pediatric rheumatology, pediatrics, or transition medicine. Given the substantial differences between pediatric and adult rheumatology practices, this has the potential to lead to significant delays in care. In July 2016, the University of Utah School of Medicine Rheumatology Division implemented a curriculum involving a 10% compulsory pediatric rheumatology experience to address this need. Objective To describe a curriculum for adult rheumatology fellowship that focuses on enhancing pediatric rheumatology knowledge. Key Message The adult and pediatric rheumatology divisions at the University of Utah collaborated to develop a 10% pediatric rheumatology curriculum which is incorporated into the general adult rheumatology fellowship training program. During the pediatric rheumatology rotation, fellows participate in inpatient and outpatient rheumatology care, manage phone calls from pediatric patients and their families, triage pediatric rheumatology phone consults, attend lectures in pediatric rheumatology, and present at pediatric rheumatology journal clubs focusing on gaps in pediatric rheumatology knowledge and transition care. This collaboration has strengthened the bond between the rheumatology and pediatric rheumatology divisions and enabled increased knowledge sharing. Barriers to the implementation of this program have been lack of knowledge of pediatrics among internal medicine trained rheumatology fellows, the addition of another hospital system with its own electronic health record, and uncertainty of the impact this program will have on increasing access to pediatric rheumatology expertise in geographic areas lacking a pediatric rheumatologist. Conclusion This is an innovative curriculum that incorporates a 10% pediatric rheumatology experience in an adult rheumatology fellowship training program. There is a lack of research to inform the education of internal medicine trained rheumatology fellows in pediatric rheumatology. The next step is to assess the comfort and knowledge base gained in pediatric rheumatology by the rheumatology fellows and to evaluate how this program impacts the fellows’ future clinical practice. 

10:00 AM - 10:15 AM

Is there an “Honors” level of competency or is it time to retire the “H” grade in clerkships? 
J. Colbert-Getz, M. Northrup, D. Roussel, A. Smith, University of Utah School of Medicine
Abstract Body: Purpose: With more and more medical school using the Entrustable Professional Activities (EPAs) as a framework for competency it is unclear if grading systems beyond pass/fail are necessary. However, a pass/fail clerkship grading system is in conflict with residency program selection, which place great emphasis on “honors” in a program specific clerkship (1). The main purpose of this study was to determine if faculty and residents who evaluate clerkship students conceptualize competency for “honors” as a norm-referenced standard or a criterion-referenced standard. It is important to understand how raters conceptualize honors because norm-referenced standards are not typical in a competency-based framework. The secondary purpose was to characterize the domains in which honors students stand out from non-honors students as perceived by faculty and resident raters. Approach/Methods In 2015-2016 the University of Utah School of Medicine required Critical Care Clerkship included a question on an EPA-aligned global rating form: Does this student perform at an honors level; if so, what distinguishes the student from a non-honors student? Two raters independently coded all qualitative responses (1) by any reference to criterion- or norm-referenced judgment for determining honors and (2) by EPA or non-EPA areas based on grounded theory for the later categorization. Any disagreement was discussed till consensus was reached between the raters. Results/Outcomes: There were 99 global rating forms completed on 81 students. Fifty-six of the forms indicated honors level performance and of those 20% described students’ performance in terms of norm-referenced judgments while the other 80% were based on criterion-referenced judgments. The top five topics mentioned for students with honors were work ethic (mentioned on 25% of 56 forms), patient-centered care (21%), teamwork or EPA 9 (21%), active learning (18%) and knowledge (16%). Discussion: When raters form a judgment about the honors student they are most likely to do so in a criterion-referenced manner. However, with the exception of teamwork, what distinguishes an honors student from a non-honors student is not necessarily captured in the EPAs. Significance Assessments aligned with EPAs may be useful for determining entrustment decisions, but may not accurately capture the honors clerkship student as conceptualized by faculty and resident raters. More large-scale research is needed before it can be determined if the honors grade is warranted in a competency based framework. References 1. National Resident Match Program. Results of the 2016 NRMP Program Director Survey. June 2016. Available online: http://www.nrmp.org/wp-content/uploads/2016/09/NRMP-2016-Program-Director-Survey.pdf 

10:15 AM - 10:30 AM

Clinical Skills Temporal Degradation Assessment in Undergraduate Medical Education
J. Fisher, R. Viscussi, A. Ratesic, C. Johnstone, R. Kelley, J. Bates, E. Situ-Lacasse, W. Adams Rappaport, R. Amini, University of Arizona College of Medicine
Abstract Body: Abstract: Objectives: Medical students’ ability to learn clinical procedures and competently apply their skills on patients is an essential component of medical education. Complex skills with limited opportunity for practice have been shown to degrade without continued refresher training. To our knowledge there is no evidence that objectively evaluates temporal degradation of clinical skills in undergraduate medical education. The objective of this study was to evaluate temporal retention of clinical skills among third year medical students. Methods: A cross-sectional study at an urban academic medical center evaluated 45 novice third year medical students for temporal retention of pigtail thoracostomy, femoral line placement, and endotracheal intubation skills. Prior to the start of third-year medical clerkships, medical students participated in a two-hour didactic session geared to teach clinically relevant procedures. Prior to their respective surgery clerkships students were asked to perform the same three procedures and were evaluated by trained emergency medicine and surgery faculty for retention rates using three validated checklists. Results: Retention rates were shown to rapidly decline between six and 12 weeks with statistical significance (p < 0.05) for all three procedural skills. Conclusion: Further data needs to be collected in a variety of clinical skills to determine set points for implementing refresher training. 

10:30 AM - 10:45 AM

Relationship between Propensity to Trust and Entrustment-based Clinical Evaluations 
M.B. Farmer, University of Colorado
J. Shea, C.J. Dine, J. Lapin, J.R. Kogan, University of Pennsylvania
Abstract Body: Introduction: Trust is an increasingly recognized construct in medical education both for its role in clinical education and its utility in evaluation. The decision to trust a trainee with less supervision can be used as an indicator of competency. For example, entrustable professional activities are increasingly being used to assess ACGME competencies and milestones. Given the central role of trust, it is important to understand the factors that affect entrustment decisions. The field of personality psychology has long recognized trust as an element of personality, noting some people are generally more trusting than others. To best achieve the goal of measuring the trainee’s competency, it is important to elucidate and alleviate supervisor-dependent factors (sources of error). Research Question: To determine if there is a relationship between clinical supervisors’ natural propensity to trust (personality trait) and their entrustment-based evaluations of trainees. Methods: 196 internal medicine attendings at the University of Pennsylvania were invited to participate. Propensity to trust was measured using the Trust facet of the NEO Personality Inventory-3 Big Five personality survey; the Ideas facet was used as control trait. Each facet contained 8 statements rated on a 5-point scale (Strongly Disagree to Strongly Agree). Participants were emailed the questionnaire; three reminders were sent to non-responders. Results were correlated with attendings’ previously completed entrustment-based clinical competency evaluations of internal medicine interns (PGY1) and residents (PGY2/3). This study was IRB approved. Results: 137 attending physicians participated (70%). Of these, 118 (60%) had sufficient evaluation data (> 6 evaluations each of interns and residents) for analysis. No correlation existed between either personality trait (Trust and Ideas) and participants’ ratings of trainees using an entrustment-based evaluation scale [for trust: r = -0.010, p = 0.913; r = 0.009, p = 0.927; for ideas: r = 0.026, p = 0.780; r = 0.060, p = 0.517, for evaluations of PGY1s and PGY2-3s, respectively]. Discussion: Trust as a personality trait does not seem to measurably effect how attendings complete entrustment based evaluations suggesting trainee trustworthiness (competence) may be a key determinant of trust in longitudinal clinical relationships. Propensity to trust may be most important in novel situations, diminishing as information is gained about the trustworthiness of the individual being trusted. Propensity to trust may be less important for grounded, summative entrustment decisions, such as those formal clinical evaluations intend to measure. Generalizability of findings is limited given the single institution study of internal medicine attendings. 

10:00 AM - 10:15 AM

Novel Approaches to Predictive Modeling for Understanding Influences of Practice Behavior: An Example Using Alzheimer’s Disease
J. Reiter, J. Perez, S. Tordoff, W. Faler, CME Outfitters
Abstract Body: Introduction. An essential component of improving patient outcomes through medical education is ensuring healthcare providers (HCPs) perform according to best practices. Traditional statistical comparisons of pre- versus post-activity performance are important for demonstrating performance improvement. However, they do not provide information regarding the factors that influence practice behaviors, which will help guide needs assessments for future activities and ensure the appropriate topics, formats, questions, and audiences are targeted. PredictCME utilizes a form of predictive modeling known as CHAID (chi-square automatic interaction detection). Although frequently used in data mining, CHAID has not been utilized in medical education. PredictCME can be used to determine which variables most impact knowledge, competence, behavior, or other endpoints. It has two main advantages over linear or logistic regression: 1) decision tree-based output which allows for a more informative and user-friendly interpretation, and 2) ability to utilize both continuous and categorical data. This presentation provides results from a CHAID analysis of real-world data from an educational activity on Alzheimer’s disease (AD). Research Question. What factors influence practice behaviors in HCPs seeing patients with Alzheimer’s disease? Methods. Data from 262 HCPs participating in an educational activity on AD were analyzed using PredictCME. A question related to practice behavior was entered into the model as the response variable, with variables such as knowledge, number of patients seen with AD, years in practice, and confidence entered as predictor variables. Results. Results showed that the strongest predictor of practice behavior was confidence. A secondary predictor was the number of patients with AD seen by the learners. Discussion. For future activities, it will be important to consider ways to improve HCP confidence as well as address the needs of HCPs who don’t see a large number of patients with AD. Findings from the PredictCME analysis demonstrate the utility in using predictive modeling to better understand the influences of practice behavior, which in turn will help maximize the impact of future activities, and ultimately patient outcomes. 

10:15 AM - 10:30 AM

“We’re on the same team”: What non-physician team members want new physicians to know about their role.
M. Garth-Pelly, E. Shearer, A.J. Millet, S. Bereknyei, A. Aaronson, D. Svec, S. Stafford, Stanford Hospital
Abstract Body: Introduction The ability to use knowledge of our own and other’s role to address the health needs of patients and populations is a core competency for interprofessional collaborative practice1. This research seeks to illuminate topics that non-physician healthcare professionals would like new physicians to know about their role, with the aim of promoting improved interprofessional collaboration. Research Question What do non-physician members of the interdisciplinary team want new physicians to know about their profession and role? Methods We conducted focus groups at one academic center with pharmacists, social workers, case managers, dietitians, nurses, and rehab therapists in adult care. Participant recruitment was via word of mouth, email, fliers, and tabling. A semi-structured interview guide was employed to gather information including 1) self-described role on the healthcare team, 2) experiences collaborating with medical teams, 3) most misunderstood elements of their profession. One medical student researcher conducted all focus groups. Two coders used an exploratory (inductive) approach to develop and apply the codebook. An inter-rater reliability test was performed to assess for coding drift. Team-based theme analysis was performed to identify within-group and across health professional group themes. Results We conducted 7 focus groups with 42 total participants. Preliminary results identified 3 themes. 1) Each profession identified misconceptions they want addressed (eg. rehab therapists described being over-consulted when new physicians believe they need rehab to ensure a patient is mobilizing—they describe mobilization as a team effort and the rehab therapist’s role is to address more skilled needs). 2) Professionals felt their expertise is not sufficiently incorporated into care plans (eg. dietitians feel their expertise in medical nutrition therapy can have a large impact on outcomes but they have insufficient opportunities to impact care). 3) Participants wanted new physicians to show more curiosity to learn about non-physician expertise and roles, stating that if new physicians are unsure, part of the interprofessional experience is to ask. Discussion The experiences of non-physician members on the interprofessional healthcare team show that more remains to be done in medical education to ensure that new physicians understand the roles and how to incorporate the expertise of non-physician colleagues. Medical students should have focused curriculum on healthcare professional roles they will interact with in the clinical setting. 1Interprofessional Education Collaborative (2016). Core competencies for interprofessional collaborative practice. 

10:30 AM - 10:45 AM

A method for calculating the costs of medical education and opportunities for value analysis
S. Lamb, J. Lindsley, D. Roussel, T. Tsai, L. Boi, S. Petersen, M. Lauder, W. Samuelson, A. Stevenson, K. Shaffer, J. Colbert Getz, University of Utah
 Abstract Body: Context The skyrocketing cost of healthcare is a global problem. Likewise, debt for medical school graduates has been rising faster than inflation over the last 20 years (Youngclaus 2012). Despite increasing student tuition, the total cost of medical student education is rising even faster. Changing economics, fiscal pressures and new focus on higher quality and lower cost require a new operating model for academic medicine; every aspect of academic medical centers is undergoing transformation including how care is delivered, how students and residents are educated and how research is funded (Enders 2014). There is increasing recognition that the whole issue of cost and value in health professions education is important (Walsh 2014). Yet, to date the field has not figured out how best to determine the cost of medical education. Objectives 1. Describe the method used to calculate actual costs of the University of Utah School of Medicine (UUSOM) undergraduate medical education program 2. Review annual cost data for the UUSOM medical education program 3. Propose opportunities to use cost information for data driven analysis of resource utilization in medical education programming and planning Key Message The UUSOM has adapted a tool utilized at the University of Utah Health Sciences Center that successfully lowers costs of healthcare while improving patient outcomes (Lee 2016) for use by the educational program. The tool, known as Value Driven Outcomes (VDO), aggregates Professional and Facility Costs and assigns these costs to the corresponding clinical encounters (Kawamoto 2015). Data are aggregated in the University’s Data Warehouse where they are then available for reporting and analytics. We adapted the VDO framework to calculate the cost of the medical school program. This is providing an opportunity for data-driven analysis of resource utilization in medical education by the UUSOM. We believe this method can be replicated by other medical schools to allow them to calculate the actual costs of education for their program. Future utilization of this method can inform decisions about new programming, program change and quality improvement in education. Conclusion The costs of medical education likely varies among medical school programs; few can say they know the real cost of medical education. As a first step we have identified the categories of cost and actual dollar amounts for undergraduate medical education. This effort combined with future collaborative work with other institutions will help leaders make informed decisions about fiscal planning relating to education. 

10:00 AM - 10:15 AM

Transition to Medical School: A Novel Approach to New Student Orientation
K. Shaffer, S. Baumann, J. Colbert-Getz, T. Hurtado, School of Medicine, University of Utah
Abstract Body: Context This presentation will describe the implementation of a Transition to Medical School course offered by the University of Utah School of Medicine. Recognizing the need for explicit self-regulated learning training for new medical students, and acknowledging the limits of traditional first year orientation to provide students with the resources needed to succeed academically and personally, the Offices of Curriculum and Student Affairs collaborated on creating Transition to Medical School (TTMS) in place of the existing first year orientation. To design the course we used Kern’s six-step approach to curriculum development, beginning with a needs assessment (recently published in Teaching and Learning in Medicine). The result was a hybrid, conference-style course where logistical information and compliance forms were stored in an LMS for students to complete on their own time, and to refer back to as needed. This helped to reduce extraneous load by allowing students to focus on information that was more germane to their learning. The in-person event was a week long, and included a combination of plenaries and optional breakout sessions divided into four tracks: Academic Success, Professionalism and Culture, Interpersonal and Wellness, and Social Connections. Objectives 1. Describe the course development process: problem identification, needs assessment, goals and objectives, educational strategies, implementation, and evaluation. 2. Discuss feedback and next steps. Key Message Rather than spending valuable face-to-face time providing didactic and logistical information to new students, that time is better spent preparing them to be self-regulated learners by setting the foundation with just-in-time, learning to learn sessions. Conclusion While more time and study are needed to evaluate whether the course had a sustained effect on student learning, the feedback from MS1s (N = 128, 100% response rate), and presenters (N = 36, 50% response rate) was overwhelmingly positive. Students enjoyed having the opportunity to decide which breakout sessions to attend and the option to skip sessions they felt were irrelevant to them. This resulted in an overall satisfaction with the experience. Further, a welcome side effect of the new format was that incoming students recognized the time, care, and effort that went into the course, which set a tone of mutual trust and appreciation between students, faculty, and administrators. 

10:15 AM - 10:30 AM

Students perceptions of their experiences and contributions during a longitudinal clinical immersion course for first year medical students
M. Sow, K. Osborn, S. Bereknyei Merrell, E. Schillinger, Stanford University School of Medicine
Abstract Body: INTRODUCTION: There is growing consensus that early patient experiences are an important part of medical education. At our institution, we offer an elective course that provides year-long, longitudinal clinical experiences for first-year medical students, matching students with a clinical site and mentor for one half day a week for the academic year. In 2015-16, students received training in health coaching, medication reconciliation, motivational interviewing, and change leadership, gained early insight into patient-team interactions, the healthcare system, and began to integrate their evolving skills into the clinical environment. RESEARCH QUESTION(S): What are student perceptions of the value of their experiences working with and contributing to patient health, communities and healthcare systems during an early, longitudinal clinical immersion experience? METHODS: Students were asked to complete quarterly post-course surveys. Scope was narrowed to two qualitative free-response items: 1) student experiences working with patients and communities and 2) contribution to the improvement of patient health or healthcare systems or communities. Systematic qualitative analysis was applied to summarize data and identify trends. Responses were analyzed by 2 independent analysts, compared and adjudicated for code application discrepancies. RESULTS: Of 13 participants, 12 responded to the survey at end of quarter 1, 10 at quarter 2, and 10 at quarter 3. The 64 responses gathered were parsed by quarter. Quarter 1 themes revealed the experience had positive impact on some students, while some found it a challenge to integrate themselves into clinic in a meaningful way. Variability was based on perceived level of site and team integration. Quarter 2 themes highlighted an increased student perception of patient encounters within interactions: telephone health coaching, team based practice, and discharge procedure planning skills. Most participants reported increased impact due to continued longitudinal clinical interactions, though integration remained a challenge for a subset. Quarter 3 themes elucidated an increased awareness of socio-economic impact, enhanced communication skills, sustained positive impact; healthcare system contributions more broadly remained mixed, but improving. DISCUSSION: The course aided acclimatization to clinical practice, and development of clinical and communication skills, for most students. Student perception of value varied, with more positive experiences reported by students who integrated at sites into roles that utilized patient communication skills. Additional research needs to be conducted to assess the impact of curricular and clinic onboarding improvements made in the current academic year, to better understand which variables correlate to perceived positive contribution to patient health and/or healthcare systems. 

10:30 AM - 10:45 AM

Development of a Content Outline for Undergraduate Critical Care Education using a Modified Delphi Method
A. Smith, K. Campell, University of Utah
J. Brainard, University of Colorado
Abstract Body: Introduction Critical care education is an important part of undergraduate medical education (UME). The Association of American Medical Colleges states that physicians entering residency should be competent in recognizing and initiating the management of a deteriorating patient who requires emergent care. One opportunity to learn such skills and knowledge is by caring for critically-ill patients. However, there is currently no available data or consensus on what students should learn regarding critical care. This lack of consensus is in contrast to other specialties, which have national content outlines that assist educators in designing curriculums. Other specialties have utilized Delphi consensus methods to develop content outlines. The aim of this research is to develop a national critical care content outline through a multidisciplinary expert consensus process to improve medical student education related to the management of deteriorating and critically-ill patients. Research Question 1) What are the core critical care skills and knowledge that should be learned during a UME critical care experience? Methods We use a modified Delphi process to reach consensus on the core skills and knowledge that should be learned during a UME critical care experience. The Delphi panel included 3 expert groups: 1) UME critical care educators identified through the Society of Critical Care Medicine; 2) residency program directors nominated by their respective national organizations; and 3) residency-level trainees nominated to represent residents of their respective specialties. Over three, iterative rounds, the Delphi panel will reach consensus on the critical care skills and knowledge expected of graduating medical students. Results The Delphi is currently beginning its third round. We expect completion of the Delphi by December 2016. The panel consists of 28 experts. The following specialties are represented: Internal Medicine (n=4), Emergency Medicine (n=6), Surgery (n=5), Anesthesia (n=6), Pediatrics (n=5), Ob/Gyn (n=2). Seventeen participants have sub-specialty training in critical care. Eight participants are currently in residency training. Consensus on the critical care skills and knowledge expected of graduating medical students is pending. Discussion The results of our Delphi will provide a national, consensus content outline for critical care in UME. By including experts from various disciplines, our content outline will be meaningful for all graduating students, independent of their intended specialty. The content outline will represent the first step for educators in the development of local and national UME critical care curriculums. 

11:00 AM - 11:15 AM

Show Me Your Objectives: Nutrition Education in the University of New Mexico School of Medicine Curriculum, 2015-2016
A. Robinson, University of New Mexico School of Medicine, Albuquerque
Abstract Body: Introduction: More than 2/3 of adults and 1/3 of children in the United States are overweight or obese which contributes to a vast array of adverse health outcomes. The population of New Mexico has been significantly impacted by this trend. Medical educators have been called upon to respond to these trends by improving nutrition education in undergraduate medical education. Course objectives provide the gold standard for documenting course content. The first step in creating a foundation for conversation regarding the optimal content and timing of nutrition curriculum is to identify the content and location within the existing curriculum. Key stakeholders at our institution expressed interest in this process prior to creation of this study, because no one had ever systematically assessed the content of our curriculum through the lens of nutrition education. Dialogue regarding the study findings is ongoing. Other United States medical schools may wish to utilize a similar process to evaluate nutrition education within their undergraduate curriculum to provide a foundation for conversation on improving their curriculum. Research questions: The goal of this project was to survey the content of the Phase I and Phase II course objectives within our undergraduate medical education program in an attempt to determine to what extent and within which courses our program provides foundational material on nutrition. A secondary question emerged during the research process: was there material on nutrition education presented within courses that may not be captured through survey of course objectives? Methods: A data collection instrument was designed based upon the list in "Nutrition Competencies for Graduating Medical Students" from the Nutrition in Medicine Program at the University of North Carolina. 61 subtopics were examined within 6 major categories. Course objectives of all first and second year required courses from 7/1/15 through 6/30/16 were surveyed. Thus, course objectives for 21 major blocks and clerkships and 5 additional courses were surveyed in our curriculum management program, One45. Following this survey, there was communication with the block chair or clerkship director from each course to discuss and verify the data.This was a mixed-methods study. Results: 17/26 (65%) of the courses surveyed included course content in at least one nutrition subtopic. "Water and electrolytes" and "Sources, bioavailability, action, deficiency, excess of micronutrients" were the subtopics most frequently covered within Foundational Topics, each addressed in five courses.Additionally, contact with course directors provided significantly more detail on course objecives within 4/17 (24%) of these courses. Discussion: There was more variability than anticipated in terms of identifying course objectives. We have opportunities for improvement in the quality of documentation of objectives and depth of curriculum content in nutrition education.This may facilitate communication between course directors to reduce redundant curriculum and elucidate gaps. The data collection was complicated by the level of detail in the survey tool. Variability of results was likely minimized by the fact that a single researcher was applying the same tool to each course with an identical approach. Other broad questions emerged from this study: should nutrition education be implemented as a longitudinal curriculum throughout our four year medical school program? Precisely what do we categorize as nutrition education from the standpoint of quantifying the number of hours of nutrition education that is provided to our learners?  

11:15 AM - 11:30 AM

Where to begin: Developing a comprehensive sexuality curriculum
B. Austin, P. Basaviah, B. Herman, D. Matsuda, M. Bryant, Stanford School of Medicine
Abstract Body: Context As an integral member of the healthcare team, patients expect that medical students will have the training and expertise related to their healthcare inclusive of sexual health and activities. To address this theme, the curriculum oversight committee at our institution established the Sexuality, Sexual Function and Gender curriculum working group. The group was tasked with with updating our Stanford curriculum to be on the forefront of national and regional standards. The workgroup conducted a thorough review of the current Stanford curriculum, as well as those across the nation, leading to the development of new learning objectives and specific recommendations for an integrated thread of topics on Sexuality, Gender and Sexual Function to be addressed across the medical school continuum. The group identified the following areas of focus for the new curriculum: Respectful, sensitive communication with patients and colleagues Identification of sexuality issues across all medical disciplines (sexual orientation, identity, age, function and quality of life) Sensitivity in case development, i.e., including LGBTQI populations as part of cases centered on common medical conditions, such as hypertension; and, conversely, avoiding stereotypes (e.g., avoiding a gay male stereotype in HIV-related cases) Transgender health issues Sexuality issues in geriatric populations Consideration of a name change for the HHD Women’s Health block, where many sexuality-related topics related to all genders are currently focused Objectives Describe a framework for development of a comprehensive curriculum on sexuality, sexual function and gender. Describe one approach to integrating AAMC objectives into institution-specific objectives and curricular modalities Share established institutional benchmarks Explain the importance of involving key stakeholders that represent the diversity of expertise on this topic (including faculty, students, medical education staff, deans, etc.) Discuss successes, challenges and barriers in establishing curriculum Key Message and Conclusion This is one example of one institution's experience in implementing curricular change with regards to sexuality, sexual function and gender. Establishing a comprehensive sexuality curriculum is a multi-faceted, team effort and there are opportunities across the region to collaborate on best practices in diversifying curriculum and provide strategies for success.

11:30 AM - 11:45 AM

Culinary Medicine: Teaching Medical Students about Nutrition Through Hands-on Application
C. Diamant, Z. Memel, E. Clarke, G. Harlan, Keck School of Medicine of USC
S. Chou, Nutrition, LA Kitchen, Los Angeles, California
 Abstract Body: The Institute of Medicine recommends that every medical school integrate a minimum of 25 hours of nutrition into their curriculum. In 2014, however, the University of North Carolina surveyed every US medical school and found that only 29% of schools provide the recommended 25 hours of nutrition education. In 2016 we conducted a needs based assessment of the nutrition attitudes and knowledge of Keck medical students and found that 83% of students were not satisfied with the quantity and quality of nutrition education at Keck. This translates into deficiencies in lifestyle promotion training with only 55% of surveyed students having ever counseled a patient on lifestyle recommendations and 64% of surveyed students not feeling confident to discuss dietary interventions with patients. In addition, only half of students reported eating at least three fruits or vegetables each day. In response to these findings, the Culinary Medicine selective was created as part of our Introduction to Clinical Medicine-Professionalism (ICM-P) course to address the need for more nutrition education, lifestyle promotion training, and community-based partnerships at the Keck School of Medicine. By utilizing hands-on culinary classes, we aim to educate students on their own healthy nutrition choices and improve their knowledge to pass along to patients. Nutrition education is essential to help future doctors learn how to integrate preventative practices and the appropriate sources of referral into their practice to maximize patient care. Course Objectives: 1. Respond to lack of nutrition curriculum after conducting a needs based assessment 2. Create a team-oriented, interdisciplinary course incorporating dietetics, culinary skills, and patient education 3. Fulfill LCME requirement to integrate lifestyle promotion skills into school curriculum 4. Create a sustainable course that honors USC’s long standing tradition of community partnership with a non-profit organization Key Message: The Culinary Medicine ICM-P Selective aims to prepare future physicians to serve, heal, and empower patients and communities through a curriculum that incorporates culinary skill and preventative medicine. In order to engage with the local community, Keck School of Medicine partnered with LA Kitchen, a local non-profit organization that distributes healthy meals to the elderly and provides free culinary training and job placement to people recently released from incarceration or foster care. In collaboration with LA Kitchen, medical students practice culinary techniques as they learn about the specific dietary options to treat common diseases. With this knowledge, students can counsel patients on how to create low-cost, healthy meals on a budget. Honoring the courses’ interdisciplinary theme, each class focuses on a specific disease (hypertension, diabetes, heart disease) taught from three different approaches: medical perspective (clinical case discussions), nutrition perspective (discussing appropriate diets with each morbidity), and culinary perspective (preparing a meal applying the nutrition lessons just learned). We also include a chef-guided trip to a grocery store, a session with several community partner organizations, and a final “team cook-off” session, in which each team prepares a nutritious meal targeted for a specific illness on a limited budget. In order to measure the impact of this course on students’ nutrition and culinary knowledge, a pre-course survey was completed and a post survey will also be administered. We anticipate that the survey results will demonstrate how a hands-on nutrition course can effectively improve students’ confidence in lifestyle counseling, nutrition knowledge, and personal culinary skills. Conclusion: Implementing a hands-on, community-based nutrition course provides students with a collaborative approach for promoting their personal health and dietary habits while simultaneously enhancing their confidence and knowledge needed to apply nutrition interventions in the clinical setting. 

11:00 AM - 11:15 AM

Applying lessons from the MD curriculum to the 1st-year PhD Bioscience curriculum at the University of Utah
M. Kay, University of Utah
Abstract Body: Context: Over the past seven years, the foundational science curriculum for medical students at the University of Utah (UU) has evolved to become more integrated, with a focus on developing the competencies of knowledge application, teamwork and self-directed learning. By contrast, the Bioscience PhD Programs in Molecular Biology and Biological Chemistry, 1st-year umbrella programs that serve multiple departments in the School of Medicine, College of Pharmacy, and College of Science, continued to have a discipline-based, lecture-heavy curriculum. However, it became apparent that many of our graduate students were not being adequately prepared to succeed on their oral preliminary exams, to be competitive for NIH and NSF training grants, and to graduate within about 6 years. Objectives: 1. Develop a new PhD curriculum that is more focused on critical evaluation of primary research literature, problem solving, hypothesis generation, and small-group learning. 2. Create new courses in critical thinking and literature review, as well as guided research proposal preparation. 3. Utilize active small group learning with faculty and senior graduate student facilitators modeled on successful case-based learning techniques pioneered in the MD curriculum. 4. Provide more time for the new courses by condensing the foundational curriculum and encouraging course leaders to introduce more active learning into their lecture-based courses. 5. Implement an end-of-year capstone exam in which students present and defend the original research proposal produced in class to an independent faculty panel not involved in the courses. For most departments, this capstone exam will replace the previous preliminary exam on a topic unrelated to their thesis taken in the second or third year. Key message: Actively involving foundational science researchers in the medical student curriculum can have unexpected benefits. Science faculty who have participated as case-based learning facilitators with medical students and witnessed the benefits of peer-teaching are more likely to support the adoption of student-centered, active learning strategies. The capstone exam was beta-tested last year as a non-binding exam, and the results helped to refine our strategy for this year's official launch. Conclusion: The UU Bioscience PhD program has just undergone a significant curriculum revision designed to help students achieve the competencies expected of independent scientists. Current efforts are focused on evaluating the effectiveness of these changes, providing support to instructors, and mentoring a new generation of faculty to make the new curriculum self sustaining. The medical school curriculum revisions helped inspire these changes. How might the strengths of PhD programs be used to improve medical education? 

11:15 AM - 11:30 AM

Conceptual Frameworks: an Old Idea Helps Students Learn Cutting Edge Biomedical Science 
J. Lindsley, T. Formosa, M.K. Tophman, University of Utah School of Medicine
Abstract Body: Context: First year Medical students have diverse backgrounds, making it challenging to effectively deliver introductory content in a broad range of topic areas. The traditional lecture format allows a large amount of material to be covered quickly, but this mode of delivery can encourage memorization without comprehension, resulting in poor retention. In order to address this problem for a first year medical student (MS1) course on Hematology and Cancer Biology at the University of Utah School of Medicine, we adopted David Ausubel's approach of starting each section of new material by providing an intellectual framework (which he called an Advance Organizer1) to help students appropriately structure ideas and facts. Objective: Our objective was to regularly provide a framework for students to help explain, integrate and interrelate upcoming material with content and ideas that they had previously learned. As course directors we sought to retain the value of having content experts deliver their scientific and clinical perspectives on complex topics while neither overwhelming the students nor requiring excessive dilution of the material. Key message: Providing a framework lecture at the beginning of each week of instruction has been an effective and very popular approach for structuring our integrated MS1 course. The course was organized into themes for each week ("leukemia" or "breast cancer" for example) and each framework session gave basic background on that topic and anticipated the content to be covered. Potential areas of difficulty were identified, and overarching principles of organization were explicitly provided. The general goal of these frameworks was to allow students to see the overall organization of the weekly theme so that they could securely distinguish the core concepts from the supporting details while the material was being presented. Further, a summary lecture was provided at the end of the week to help students assimilate problematic topics and resolve conflicting ideas or the different levels of emphasis provided by some experts with the level of mastery expected by the course directors. Conclusion: The framework format coupled with a summary session allowed students to extract more value from standard lectures as the intellectual structure for the topic was already in place, providing enhanced comprehension, less reliance on memorization, and greater long-term retention of core ideas. 1. Ausubel, D.P., The Use of Advance Organizers in the Learning and Retention of Meaningful Verbal Material. Journal of Educational Psychology, 1960. 51(5): p. 267-272.