Medical Education Research Seminar - Dr Joan Li

Comparing Medical Students’ learning outcomes and experience in Multidisciplinary Practical Exam Prior to and during COVID-19

Joan Li, Suja Pillai, Mark Midwinter, Tiffany Hansen, Jeremy Hunt, Keunwoo Jeong and Kali Juracek

Recognition and interpretation of gross specimen is essential for students in the assessment of Pathology and Anatomy during their pre-clinical medical training. Prior to COVID-19, multidisciplinary practical exams (MDPE) were conducted as in person exams (3D) using  pathological pots and cadaver for Year1 and Year 2 MD students in University of Queensland. Due to  COVID-19 lockdown, MDPE  had to be conducted as an online image-based exam (2D) using an electronic platform (Examsoft). This study aims to evaluate and compare students’ performance and experience in the online, or in-person format of MDPE.

RELEVANCE

This study is very relevant as the higher education sector is still dealing with the ongoing to the pandemic and possibly facing significant changes in teaching and learning.

METHODS

Student performance in (3D) practical exam (2019) and (2D) practical exam (2020) were evaluated and compared, both at  cohort level  and at course grade level. Psychometric analyses were used to assess the validity and reliability of the assessment. In addition, students were also invited to complete an online questionnaire about their experience of online vs in-person MDPE.

RESULTS

The reliability score Kuder–Richardson Formula 20 (KR-20) was >0.74 for both assessments with the average cohort performance score being 72% (3D) and 67% (2D). Difficulty index and Point-Biserial correlation showed discipline specific difference but overall, there was no significant difference between the two mode of exams. However, students’ feedback indicated that image-based exam during COVID-19 was more challenging compared to in person exams due to lack of 3D visualization and orientation. Analysis also showed that this change had very little impact on the performance (≤4%) of students in higher course grade (≥5) but it greatly affected the performance (>11%) of lower grade students (≤3).

CONCLUSION
The study suggests that  the overall cohort performance wasn’t impacted by the change from in-person to online practical exam, however it is important to  assess the impact of this change on the performance of lower grade students.