Sensor fusion for analysis of gait under cognitive load: deep learning approach

Research output: Contribution to conferencePaperpeer-review

178 Downloads (Pure)


Human mobility requires substantial cognitive resources, thus elevated complexity in the navigated environment instigates gait deterioration due to naturally limited cognitive load capacity. This work uses deep learning methods for 116 sensors fusion to study the effects of cognitive load on human gait of healthy subjects. We demonstrate classifications, achieving 86% precision with Convolutional Neural Networks (CNN), of normal gait as well as 15 subjects’ gait under two types of cognitive demanding tasks. Floor sensors capturing multiples of up to 4 uninterrupted steps were utilized to harvest the raw gait spatiotemporal signals, based on the ground reaction force (GRF). A Layer-Wise Relevance Propagation (LRP) technique is proposed to interpret the CNN prediction in terms of relevance to standard events in the gait cycle. LRP projects the model predictions back to the input gait spatiotemporal signal, to generate a “heat map” over the original training set, or an unknown sample classified by the model. This allows valuable insight into which parts of the gait spatiotemporal signal have the heaviest influence on the gait classification and consequently, which gate events are mostly affected by cognitive load.
Original languageEnglish
Number of pages6
Publication statusPublished - 11 Mar 2020
EventIEEE Sensors Applications Symposium 2020 - Malaysia, Kuala Lumpur, Malaysia
Duration: 9 Mar 202011 Mar 2020


ConferenceIEEE Sensors Applications Symposium 2020
Abbreviated titleSAS
CityKuala Lumpur
Internet address


  • Deep Convolutional Neural Networks (CNN)
  • Cognitive Load
  • Ground Reaction Force (GRF)
  • Layer-Wise Relevance Propagation (LRP)


Dive into the research topics of 'Sensor fusion for analysis of gait under cognitive load: deep learning approach'. Together they form a unique fingerprint.

Cite this