Development of a benchmarking tool for pharmacy students using threshold learning outcomes

Benchmarking is crucial for ensuring the quality of the learning experience offered by a curriculum. This project aimed to develop, refine and validate a tool based on the pharmacy threshold learning outcomes (TLOs) to facilitate benchmarking of students' performances in verbal 'capstone' assessments in Australian pharmacy programs. A pilot tool was developed and trialled by local and external examiners during the University of Tasmania (UTAS) 2013 Bachelor of Pharmacy Fourth Year verbal examinations. Usability and acceptability were assessed using a survey and the tool was evaluated for validity and inter- and intra-rater reliability using Pearson correlation coefficients (r). The tool was refined using these results, and a validation exercise is planned for late 2014. In the initial evaluation, agreement on TLO scoring was excellent between UTAS examiners (r values .73-.88; n=29 comparisons); and very good between the Tasmanian and external examiners for most TLOs (r=.59-.68; n=12). Agreement between allocated marks and TLO scores were also very good (r=.62-.85; n=29). Survey results recommended improving alignment of the TLOs with the assessment activity, simplifying the rating scales and providing training regarding the tool. Progress is underway towards the development of a relatively simple, flexible and validated benchmarking tool to facilitate high quality student outcomes across Australian pharmacy programs. Training in the use of the tool has been identified as a major enabler of its successful and consistent application, and a scenario-based training video is currently being produced. The tool and training video will be showcased in this poster presentation.