Feasibility of an automated interview grounded in Multiple Mini Interview (MMI) methodology for selection into the health professions: an international multi-methods evaluation

Objectives: Global, Covid-driven restrictions around face-to-face interviews for healthcare student selection have forced admissions staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in Multiple Mini-Interview (MMI) methodology. This study aimed to explore test re-test reliability, acceptability, and usability of the system.Design, setting and participants: Multi-method feasibility study in Physician Associate (PA) programmes from two UK and one US university during 2019 - 2020.Primary, secondary outcomes: Feasibility measures (test-retest reliability acceptability and usability) were assessed using intra-class correlation (ICC), descriptive statistics, thematic and content analysis.Methods: Volunteers took (T1), then repeated (T2), the automated MMI, with a seven-day interval (+/- 2) then completed an evaluation questionnaire. Admissions staff participated in focus group discussions.Results: Sixty-two students and seven admission staff participated; 34 students and four staff from UK and 28 students and three staff from US universities.Good-excellent test-retest reliability was observed with T1 and T2 ICC between 0.62-0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005, individual site (all ICC≥ 0.76 p<0.001) and mean test retest across sites 0.82 p