Abstract [from journal]
Context: Residency programs invest considerable time and resources in candidate interviews due to their perceived ability to reveal important social traits. However, studies examining the ability of interviews to predict resident performance have shown mixed findings, and the role that the interview plays in candidate evaluation remain unclear. This mixed methods study, conducted in an anesthesiology residency program at a large academic medical center, examines how interviews contributed to candidate assessment as well as whether the addition of behavioral questions to interviews altered their role in the evaluation process.
Methods: During 2018-2019 residency selection season in the Department of Anesthesiology & Critical Care at the University of Pennsylvania, independent ratings for each interviewee were collected from faculty interviewers. Consensus ratings subsequently established by committee were also collected. Committee meetings were audio-recorded and transcribed for qualitative analysis. Behavioral questions were integrated into half of interview days. Ratings of candidates interviewed on behavioral question days were compared statistically with candidates interviewed on non-behavioral days.
Results: Qualitative analysis showed that interviewers heavily emphasized candidates' application files in evaluating the interviews. Interviewers focused on candidates' academic records and favored candidates whose interview behaviors were consistent with their applications and whose applications demonstrated similarities to interviewers' traits. The addition of behavioral questions demonstrated little ability to alter these dynamics. Quantitatively, there were no significant differences in candidate rating outcomes between behavioral and nonbehavioral interviewing days, while higher medical school rating and higher USMLE Step 1 score were associated with more favorable consensus rating.
Conclusions: Residency candidates' application files predisposed interviewers' experience and evaluation of interviews, preventing the interviews from providing discrete assessments of interpersonal qualities-even when behavioral questions were included. In the continued effort to perform well-rounded assessments of resident candidates, further research and reflection on the role of interviewing in evaluation is necessary.