CBA Lab @Georgia Tech

address: CODA, 756 W Peachtree St NW, Atlanta, GA 30308

We are the Computational Behavior Analysis (CBA) research lab at Georgia Institute of Technology.

Our research agenda focuses on applied machine learning, that is developing systems and innovative sensor data analysis methods for real world applications. Primary application domain for our work is computational behavior analysis where we develop methods for automated and objective behavior assessments in naturalistic environments. Main driving functions for our work are “in the wild” deployments and as such the development of systems and methods that have a real impact on peoples’ lives.

news

Jan 18, 2024	Welcome to our new lab members!
Oct 20, 2023	Best paper honorable mention award at ISWC 2023!
Oct 2, 2023	CBA lab at Ubicomp / ISWC 2023
May 5, 2023	Congrats to our graduates!
Apr 16, 2023	End of semester get together

selected publications

ISWC
Generating Virtual On-Body Accelerometer Data from Virtual Textual Descriptions for Human Activity Recognition

Leng, Zikang, Kwon, Hyeokhyen, and Ploetz, Thomas

In Proceedings of the 2023 ACM International Symposium on Wearable Computers 2023

Abs Bib

The development of robust, generalized models for human activity recognition (HAR) has been hindered by the scarcity of large-scale, labeled data sets. Recent work has shown that virtual IMU data extracted from videos using computer vision techniques can lead to substantial performance improvements when training HAR models combined with small portions of real IMU data. Inspired by recent advances in motion synthesis from textual descriptions and connecting Large Language Models (LLMs) to various AI models, we introduce an automated pipeline that first uses ChatGPT to generate diverse textual descriptions of activities. These textual descriptions are then used to generate 3D human motion sequences via a motion synthesis model, T2M-GPT, and later converted to streams of virtual IMU data. We benchmarked our approach on three HAR datasets (RealWorld, PAMAP2, and USC-HAD) and demonstrate that the use of virtual IMU training data generated using our new approach leads to significantly improved HAR model performance compared to only using real IMU data. Our approach contributes to the growing field of cross-modality transfer methods and illustrate how HAR models can be improved through the generation of virtual training data that do not require any manual effort.
@inproceedings{10.1145/3594738.3611361, author = {Leng, Zikang and Kwon, Hyeokhyen and Ploetz, Thomas}, title = {Generating Virtual On-Body Accelerometer Data from Virtual Textual Descriptions for Human Activity Recognition}, year = {2023}, isbn = {9798400701993}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3594738.3611361}, doi = {10.1145/3594738.3611361}, booktitle = {Proceedings of the 2023 ACM International Symposium on Wearable Computers}, pages = {39–43}, numpages = {5}, keywords = {Activity recognition, Virtual IMU Data, Large Language Models, Motion Synthesis, Wearable Sensors}, location = {Cancun, Quintana Roo, Mexico}, series = {ISWC '23}, abbr = {ISWC}, bibtex_show = {true}, selected = {true} }

Percom

If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios

Plötz, Thomas

In 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) 2023

Bib

@inproceedings{plotz2023if,
  title = {If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios},
  author = {Pl{\"o}tz, Thomas},
  booktitle = {2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)},
  pages = {565--570},
  year = {2023},
  organization = {IEEE},
  abbr = {Percom},
  bibtex_show = {true},
  selected = {true}
}

IMWUT
Bootstrapping Human Activity Recognition Systems for Smart Homes from Scratch

Hiremath, Shruthi K., Nishimura, Yasutaka, Chernova, Sonia, and Plötz, Thomas

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2022

Abs Bib HTML

Smart Homes have come a long way: From research laboratories in the early days, through (almost) neglect, to their recent revival in real-world environments enabled through the existence of commodity devices and robust, standardized software frameworks. With such availability, human activity recognition (HAR) in smart homes has become attractive for many real-world applications, especially in the domain of Ambient Assisted Living. Yet, getting started with an activity recognition system in specific smart homes, which are highly specialized spaces inhabited by individuals with idiosyncratic behaviors and habits, is a non-trivial endeavor. We present an approach for bootstrapping HAR systems for individual smart homes from scratch. At the beginning of the life cycle of a smart home, our system passively observes activities and derives rich representations for sensor data-action units-which are then aggregated into activity models through motif learning with minimal supervision. The resulting HAR system is then capable of recognizing relevant, most frequent activities in a smart home. We demonstrate the effectiveness of our bootstrapping procedure through experimental evaluations on CASAS datasets that show the practical value of our approach.
@article{10.1145/3550294, abbr = {IMWUT}, bibtex_show = {true}, author = {Hiremath, Shruthi K. and Nishimura, Yasutaka and Chernova, Sonia and Pl\"{o}tz, Thomas}, title = {Bootstrapping Human Activity Recognition Systems for Smart Homes from Scratch}, year = {2022}, issue_date = {September 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {6}, number = {3}, url = {https://doi.org/10.1145/3550294}, doi = {10.1145/3550294}, journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.}, month = sep, articleno = {119}, numpages = {27}, keywords = {pattern recognition, smart-home, human activity recognition}, html = {https://dl.acm.org/doi/10.1145/3550294}, selected = {true} }
IMWUT
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables

Haresamudram, Harish, Essa, Irfan, and Plötz, Thomas

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2022

Abs Bib HTML

The emergence of self-supervised learning in the field of wearables-based human activity recognition (HAR) has opened up opportunities to tackle the most pressing challenges in the field, namely to exploit unlabeled data to derive reliable recognition systems for scenarios where only small amounts of labeled training samples can be collected. As such, self-supervision, i.e., the paradigm of ’pretrain-then-finetune’ has the potential to become a strong alternative to the predominant end-to-end training approaches, let alone hand-crafted features for the classic activity recognition chain. Recently a number of contributions have been made that introduced self-supervised learning into the field of HAR, including, Multi-task self-supervision, Masked Reconstruction, CPC, and SimCLR, to name but a few. With the initial success of these methods, the time has come for a systematic inventory and analysis of the potential self-supervised learning has for the field. This paper provides exactly that. We assess the progress of self-supervised HAR research by introducing a framework that performs a multi-faceted exploration of model performance. We organize the framework into three dimensions, each containing three constituent criteria, such that each dimension captures specific aspects of performance, including the robustness to differing source and target conditions, the influence of dataset characteristics, and the feature space characteristics. We utilize this framework to assess seven state-of-the-art self-supervised methods for HAR, leading to the formulation of insights into the properties of these techniques and to establish their value towards learning representations for diverse scenarios.
@article{10.1145/3550299, abbr = {IMWUT}, bibtex_show = {true}, author = {Haresamudram, Harish and Essa, Irfan and Pl\"{o}tz, Thomas}, title = {Assessing the State of Self-Supervised Human Activity Recognition Using Wearables}, year = {2022}, issue_date = {September 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {6}, number = {3}, url = {https://doi.org/10.1145/3550299}, doi = {10.1145/3550299}, journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.}, month = sep, articleno = {116}, numpages = {47}, keywords = {self-supervised learning, representation learning, human activity recognition}, html = {https://dl.acm.org/doi/10.1145/3550299}, selected = {true} }

ACM

Applying machine learning for sensor data analysis in interactive systems: Common pitfalls of pragmatic use and ways to avoid them

PlÖtz, Thomas

ACM Computing Surveys (CSUR) Sep 2021

Bib

@article{plotz2021applying,
  abbr = {ACM},
  bibtex_show = {true},
  title = {Applying machine learning for sensor data analysis in interactive systems: Common pitfalls of pragmatic use and ways to avoid them},
  author = {Pl{\"O}tz, Thomas},
  journal = {ACM Computing Surveys (CSUR)},
  volume = {54},
  number = {6},
  pages = {1--25},
  year = {2021},
  publisher = {ACM New York, NY, USA},
  selected = {true}
}

ACM

IMUTube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Sep 2020

Bib HTML

@article{imutube,
  abbr = {ACM},
  bibtex_show = {true},
  title = {IMUTube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition},
  author = {},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  volume = {4},
  number = {3},
  article = {65},
  year = {2020},
  publisher = {Association for Computing Machinary},
  html = {https://dl.acm.org/doi/10.1145/3411841},
  selected = {true}
}