HandsOnVideo: Towards a Gesture based Mobile AR System for Remote Collaboration

There is currently a strong need for collaborative augmented reality (AR) sys-tems with which two or more participants interact over a distance on a mobile task involv-ing tangible artefacts (e.g., a machine, a patient, a tool) and coordinated operations. Of in-terest to us is to design and develop an AR system for supporting a mobile worker involved in the maintenance/ repair of complex mining equipment. This paper presents HandsOn-Video, our on-going work towards a gesture based mobile AR system for remote collabora-tion. HandsOnVideo is designed and developed using a participatory design approach. Fol-lowing this approach, we learnt that helpers found it natural to use their hands for pointing to objects and explaining procedures and that workers found it valuable to see the hands of the person guiding them remotely. On the other hand, we observed that this form of hand gesture interaction supported by HandsOnVideo resulted in network latency. This paper de-scribes some of the design tradeoffs we came across during the design process and tested with representative end users. These tradeoffs will be further investigated as part of our re-search agenda.