Usability of augmented reality technology in situational tele-mentorship for managing clinical scenarios : a feasibility study

Introduction
Many rural and remote areas of Australia experience health workforce shortages. Workers in these areas can feel isolated and have fewer opportunities for professional development. Telementorship provides a way to maintain the professional skills of isolated rural healthcare workers. Tele-mentoring uses a technological communication device to provide instruction from an expert to a less-experienced practitioner at a different geographic location in real time.
Augmented reality (AR) may enhance the efficacy of tele-mentoring by overlaying computer-generated three-dimensional content created by the expert and displayed for the practitioner using a headset. The incorporation of AR technology into tele-mentoring systems has been reported internationally. However, little has been reported on the use of AR technology in mentoring clinical healthcare professionals remotely. The aim of this study is therefore to evaluate the usability of augmented reality technology in a situational tele-mentorship for managing clinical scenarios in a high-fidelity clinical skills training laboratory.
Methods
A quasi-experimental design was used. Four experienced health professionals and fifteen novice health practitioners were recruited for the roles of mentors and mentees, respectively, and then trained in the use of the AR setup. In the experiment, each mentee wearing the AR headset (Microsoft HoloLens) was asked to respond to four different clinical scenarios: Acute Coronary Syndrome (ACS), Acute Myocardial Infarction (AMI), Pneumonia Severe Reaction to Antibiotics (PSRA), and Hypoglycaemic Emergency (HE). Their mentor used a laptop to provide remote guidance following treatment scripts and protocols developed for each scenario.
Outcome variables were mentors’ and mentees’ perception of the AR usability, mentorship effectiveness, mentees’ self-confidence, and skill performance. Scale ratings were used to measure the first three outcomes and checklists were used to assess the skill performance of the mentees that were videotaped. Descriptive statistics were applied to demographics, AR usability, mentorship effectiveness, self-confidence, and skill performance. Scores were compared within and between groups across age, gender, and years of experience using an independent t-test for parametric data and a Mann-Whitney U-test for non-parametric data. Manifest content analysis was employed to analyse narrative comments in each survey.
Results
The four scenarios were delivered in a random sequence to each mentee. This generated 60 performance cases captured in over 31 hours of HoloLens use and 35 hours of video recordings. All seven core AR features of the HoloLens were applied in the clinical scenarios.
Both mentors and mentees were positive about using the AR setup, despite some technical issues and the time required to become familiar with the device. Most mentees were satisfied with the display, the natural field of view and clear audio signals of the HoloLens. Participants were satisfied with the fidelity of the simulation, their immersion with each scenario and the effectiveness of tele-mentoring provided.
The positive experience of tele-mentorship was highlighted. All the mentees and mentors were satisfied with the relationship with mean scores of 4.80 and 4.25 out of 5, respectively. Mentees recognised the elevated level of responsiveness and clinical expertise, support, and encouragement of the mentors. As anticipated, mentees’ confidence in managing each of the four scenarios improved after tele-mentoring (p<0.001).
The mentees found the performance in the PSRA scenario the most difficult, followed by the HE, AMI, and ACS scenarios. Voice prompts were used the most frequently by mentors (833 times), followed by visual cues (104 times) and annotation (47 times). All mentors and mentees reported the usefulness of visual promptings, such as sharing images of treatment protocols, allowing the mentees to extend their capability and provide a level of care that would not have been possible without the HoloLens.
Discussion and Conclusion
It is the first study to use AR technology in a situational tele-mentorship for managing full-clinical scenarios. There are four key findings: (1) mentor’s and mentee’s positive perception and usability of the AR setup; (2) mentor’s and mentee’s positive perception and effective tele-mentorship; (3) significant improvement in self-confidence amongst mentees; and (4) high individual skill performance ratings of mentees by mentors. Limitations of this study include the small number of participants, limited number of simulated experimental conditions (scenarios), the short amount of training provided to participants and the pragmatic quasi-experimental design of this study. The study provides evidence on the usability and acceptability of AR technology in remote tele-mentorship for managing these four clinical scenarios. The findings suggest the potential for this technology to be used to support health workers in real-world clinical environments and point to new directions for research. For example, long-term studies with larger participant groups are needed to evaluate the AR device under different conditions and with different clinical scenarios. This study also raises the need to develop standards and guidelines for the mentors providing situational tele-mentorship.