University of Tasmania
143603 - Human behavior analysis by means of multimodal context mining.pdf (1.73 MB)

Human behavior analysis by means of multimodal context mining

Download (1.73 MB)
journal contribution
posted on 2023-05-20, 22:14 authored by Banos, O, Villalonga, C, Bang, J, Hur, T, Kang, D, Park, S, Huynh-The, T, Le-Ba, V, Muhammad Bilal AminMuhammad Bilal Amin, Razzaq, MA, Khan, WA, Hong, CS, Lee, S
There is sufficient evidence proving the impact that negative lifestyle choices have on people’s health and wellness. Changing unhealthy behaviours requires raising people’s self-awareness and also providing healthcare experts with a thorough and continuous description of the user’s conduct. Several monitoring techniques have been proposed in the past to track users’ behaviour; however, these approaches are either subjective and prone to misreporting, such as questionnaires, or only focus on a specific component of context, such as activity counters. This work presents an innovative multimodal context mining framework to inspect and infer human behaviour in a more holistic fashion. The proposed approach extends beyond the state-of-the-art, since it not only explores a sole type of context, but also combines diverse levels of context in an integral manner. Namely, low-level contexts, including activities, emotions and locations, are identified from heterogeneous sensory data through machine learning techniques. Low-level contexts are combined using ontological mechanisms to derive a more abstract representation of the user’s context, here referred to as high-level context. An initial implementation of the proposed framework supporting real-time context identification is also presented. The developed system is evaluated for various realistic scenarios making use of a novel multimodal context open dataset and data on-the-go, demonstrating prominent context-aware capabilities at both low and high levels.


Publication title






Article number









School of Information and Communication Technology


Molecular Diversity Preservation International

Place of publication

Matthaeusstrasse 11, Basel, Switzerland, Ch-4057

Rights statement

c 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (

Repository Status

  • Open

Socio-economic Objectives

Applied computing; Artificial intelligence; Human-computer interaction