HARMES Icon

HARMES: A Multi-Modal Dataset for Wearable Human Activity Recognition with Motion, Environmental Sensing and Sound

Robin Burchard1, Pascal-André Brückner1, Marius Bock2,3, Jürgen Gall3,2, Kristof Van Laerhoven1
1 University of Siegen
2 Lamarr Institute for Machine Learning and Artificial Intelligence
3 University of Bonn

arXiv preprint, 2026
Overview of the HARMES dataset, combining wrist-worn motion, environmental, and audio sensing for human activity recognition.

HARMES combines wrist-worn inertial, atmospheric environmental, and audio sensing to support multimodal recognition of activities of daily living in real-world home environments.

Abstract

With each sensing modality exhibiting inherent strengths and limitations, multi-modal approaches for wearable Human Activity Recognition (HAR) are becoming increasingly relevant -- particularly for recognizing Activities of Daily Living (ADLs), where individual modalities often produce ambiguous signals for similar or complex activities. This work introduces HARMES, a multi-modal wearable dataset combining three wrist-recorded modalities: motion sensing via an Inertial Measurement Unit (IMU), atmospheric environmental sensors (humidity, temperature, and pressure), and audio. Collected from 20 participants performing household activities in their own homes, HARMES totals over 80 hours of recorded data, with approximately three hours of labeled activity data per participant across 15 ADL classes. To the best of our knowledge, HARMES is the first dataset to combine this particular sensor trio, and it is nearly six times larger than the previously largest wrist-inertial-acoustic HAR dataset. In an extensive benchmark, we evaluate cross-subject, cross-dataset, cross-modality, and cross-environment generalization and conduct an ablation study revealing that modality contributions are activity-dependent and can provide complementary value, particularly for activities that are ambiguous from motion data alone. HARMES is freely available at Zenodo, alongside example code for loading the dataset and training models on GitHub.

Dataset Overview

HARMES supports research on wearable Human Activity Recognition by providing synchronized motion, environmental, and acoustic sensor data recorded during everyday household activities in participants' own homes. The dataset enables researchers to investigate the complementary information provided by different sensing modalities and evaluate generalization across participants and recording conditions.

20

Participants

15

ADL Classes

80+ h

Recorded Data

Sensing modalities: wrist-worn inertial measurement units (IMUs), atmospheric environmental sensing (humidity, temperature, and pressure), and audio.

BibTeX

@misc{burchard2026harmes,
  title         = {HARMES: A Multi-Modal Dataset for Wearable Human Activity Recognition with Motion, Environmental Sensing and Sound},
  author        = {Burchard, Robin and Brückner, Pascal-André and Bock, Marius and Gall, Jürgen and Van Laerhoven, Kristof},
  year          = {2026},
  eprint        = {2605.02596},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2605.02596}
}

Publications using this dataset

Research papers and preprints citing the HARMES dataset or publication.

5 publications Automatically updated
Zeyu Yang, Zhuchenyang Liu, Yao Zhang, Clayton Frederick Souza Leite, Yu Xiao
2026 ISWC Cites Paper
Mustafa Elhadi Ahmed, Hongnian Yu, Michael Vassallo, Pelagia Koufaki
2026 Research Square (Preprint) Cites Paper
Mohamed Aktham Ahmed, Moceheb Lazam Shuwandy
2026 Zenodo Cites Dataset
Ahmed Mohamady, Robin Burchard, Kristof Van Laerhoven
2026 arXiv:2606.27886 Cites Paper & Dataset
Zipeng Wu, Shiqiao Zhou, Fabian Spill, James Andrews
2026 University of Birmingham Research Portal (Preprint) Cites Dataset