University of Tasmania
Browse

File(s) under permanent embargo

Real-time automated detection of older adults’ hand gestures in home and clinical settings

journal contribution
posted on 2023-05-21, 15:21 authored by Guan HuangGuan Huang, Son TranSon Tran, Quan BaiQuan Bai, Jane AltyJane Alty
There is an urgent need, accelerated by the COVID-19 pandemic, for methods that allow clinicians and neuroscientists to remotely evaluate hand movements. This would help detect and monitor degenerative brain disorders that are particularly prevalent in older adults. With the wide accessibility of computer cameras, a vision-based real-time hand gesture detection method would facilitate online assessments in home and clinical settings. However, motion blur is one of the most challenging problems in the fast-moving hands data collection. The objective of this study was to develop a computer vision-based method that accurately detects older adults’ hand gestures using video data collected in real-life settings. We invited adults over 50 years old to complete validated hand movement tests (fast finger tapping and hand opening–closing) at home or in clinic. Data were collected without researcher supervision via a website programme using standard laptop and desktop cameras. We processed and labelled images, split the data into training, validation and testing, respectively, and then analysed how well different network structures detected hand gestures. We recruited 1,900 adults (age range 50–90 years) as part of the TAS Test project and developed UTAS7k—a new dataset of 7071 hand gesture images, split 4:1 into clear: motion-blurred images. Our new network, RGRNet, achieved 0.782 mean average precision (mAP) on clear images, outperforming the state-of-the-art network structure (YOLOV5-P6, mAP 0.776), and mAP 0.771 on blurred images. A new robust real-time automated network that detects static gestures from a single camera, RGRNet, and a new database comprising the largest range of individual hands, UTAS7k, both show strong potential for medical and research applications.

Funding

National Health & Medical Research Council

History

Publication title

Neural Computing and Applications

Volume

35

Issue

11

Pagination

8143-8156

ISSN

0941-0643

Department/School

School of Information and Communication Technology

Publisher

Springer-Verlag

Place of publication

175 Fifth Ave, New York, USA, Ny, 10010

Rights statement

Copyright The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2022.

Repository Status

  • Restricted

Socio-economic Objectives

Artificial intelligence; Human-computer interaction

Usage metrics

    University Of Tasmania

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC