Gait Activity Classification using Multi-Modality Sensor Fusion: A Deep Learning Approach

Syed Usama YUNAS, Krikor Ozanyan

Research output: Contribution to journalArticlepeer-review


Floor Sensors (FS) are used to capture information from the force induced on the contact surface by feet during gait. On the other hand, the Ambulatory Inertial Sensors (AIS) are used to capture the velocity, acceleration and orientation of the body during different activities. In this paper, fusion of the stated modalities is performed to overcome the challenge of gait classification from wearable sensors on the lower portion of human body not in contact with ground as in FS. Deep learning models are utilized for the automatic feature extraction of the ground reaction force obtained from a set of 116 FS and body movements from AIS attached at 3 different locations of lower body, which is novel. Spatio-temporal information of disproportionate inputs obtained from the two modalities is balanced and fused within deep learning network layers whilst reserving the categorical content for each gait activity. Our approach of fusion compensates the degradation in spatio-temporal accuracies in individual modalities and makes the overall classification outcomes more accurate. Further assessment of multi-modality based results show significant improvements in f-scores using different deep learning models i.e., LSTM (99.90%), 2D-CNN (88.73%), 1D-CNN (94.97%) and ANN (89.33%) respectively.
Original languageEnglish
Article number09424038
JournalIEEE Sensors Journal
Publication statusPublished - 5 May 2021


  • Acceleration
  • Artificial intelligence
  • Feature extraction
  • Foot
  • Sensor fusion
  • Sensor phenomena and characterization
  • Sensors
  • ambulatory inertial sensors
  • artificial neural networks
  • convolutional neural networks
  • deep learning
  • floor sensors
  • inertial measurement unit
  • long short-term memory


Dive into the research topics of 'Gait Activity Classification using Multi-Modality Sensor Fusion: A Deep Learning Approach'. Together they form a unique fingerprint.

Cite this