CBA Lab @Georgia Tech

address: CODA, 756 W Peachtree St NW, Atlanta, GA 30308

We are the Computational Behavior Analysis (CBA) research lab at Georgia Institute of Technology.

Our research agenda focuses on applied machine learning, that is developing systems and innovative sensor data analysis methods for real world applications. Primary application domain for our work is computational behavior analysis where we develop methods for automated and objective behavior assessments in naturalistic environments. Main driving functions for our work are “in the wild” deployments and as such the development of systems and methods that have a real impact on peoples’ lives.

news

May 5, 2023	Congrats to our graduates!
Apr 16, 2023	End of semester get together
Mar 17, 2023	Thomas gives keynotes at Percom 2023 and ICLR 2023 workshops
Jan 20, 2023	Thomas gives talk at Aalto University
Dec 7, 2022	Thomas is now Distinguished ACM member

selected publications

Percom

If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios

Plötz, Thomas

In 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) 2023

Bib

@inproceedings{plotz2023if,
  title = {If only we had more data!: Sensor-Based Human Activity Recognition in Challenging Scenarios},
  author = {Pl{\"o}tz, Thomas},
  booktitle = {2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)},
  pages = {565--570},
  year = {2023},
  organization = {IEEE},
  abbr = {Percom},
  bibtex_show = {true},
  selected = {true}
}

IMWUT
Reading the Room: Automated, Momentary Assessment of Student Engagement in the Classroom: Are We There Yet?

DiSalvo, Betsy, Bandaru, Dheeraj, Wang, Qiaosi, Li, Hong, and Plötz, Thomas

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2022

Abs Bib HTML

When in front of a classroom, a skilled teacher can read the room, identifying when students are engaged, frustrated, distracted, etc. In recent years we have seen significant changes in the traditional classroom, with virtual classes becoming a normal learning environment. Reasons for this change are the increased popularity of Massive Open Online Courses (MOOCs) and the disruptions imposed by the ongoing COVID-19 pandemic. However, it is difficult for teachers to read the room in these virtual classrooms, and researchers have begun to look at using sensors to provide feedback to help inform teaching practices. The study presented here sought to ground classroom sensor data in the form of electrodermal activities (EDA) captured using a wrist-worn sensing platform (Empatica E4), with observations about students’ emotional engagement in the class. We collected a dataset from eleven students over eight lectures in college-level computer science classes. We trained human annotators who provided ground truth information about student engagement based on in-class observations. Inspired by related work in the field, we implemented an automated data analysis framework, which we used to explore momentary assessments of student engagement in classrooms. Our findings surprised us because we found no significant correlation between the sensor data and our trained observers’ data. In this paper, we present our study and framework for automated engagement assessment, and report on our findings that indicate some of the challenges in deploying current technology for real-world, automated momentary assessment of student engagement in the classroom. We offer reflections on our findings and discuss ways forward toward an automated reading the room approach.
@article{10.1145/3550328, abbr = {IMWUT}, bibtex_show = {true}, author = {DiSalvo, Betsy and Bandaru, Dheeraj and Wang, Qiaosi and Li, Hong and Pl\"{o}tz, Thomas}, title = {Reading the Room: Automated, Momentary Assessment of Student Engagement in the Classroom: Are We There Yet?}, year = {2022}, issue_date = {September 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {6}, number = {3}, url = {https://doi.org/10.1145/3550328}, doi = {10.1145/3550328}, journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.}, month = sep, articleno = {112}, numpages = {26}, keywords = {Wearable, Students, Engagements, Electrodermal Activity, Momentary Assessments}, html = {https://dl.acm.org/doi/10.1145/3550328}, selected = {true} }
IMWUT
Bootstrapping Human Activity Recognition Systems for Smart Homes from Scratch

Hiremath, Shruthi K., Nishimura, Yasutaka, Chernova, Sonia, and Plötz, Thomas

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2022

Abs Bib HTML

Smart Homes have come a long way: From research laboratories in the early days, through (almost) neglect, to their recent revival in real-world environments enabled through the existence of commodity devices and robust, standardized software frameworks. With such availability, human activity recognition (HAR) in smart homes has become attractive for many real-world applications, especially in the domain of Ambient Assisted Living. Yet, getting started with an activity recognition system in specific smart homes, which are highly specialized spaces inhabited by individuals with idiosyncratic behaviors and habits, is a non-trivial endeavor. We present an approach for bootstrapping HAR systems for individual smart homes from scratch. At the beginning of the life cycle of a smart home, our system passively observes activities and derives rich representations for sensor data-action units-which are then aggregated into activity models through motif learning with minimal supervision. The resulting HAR system is then capable of recognizing relevant, most frequent activities in a smart home. We demonstrate the effectiveness of our bootstrapping procedure through experimental evaluations on CASAS datasets that show the practical value of our approach.
@article{10.1145/3550294, abbr = {IMWUT}, bibtex_show = {true}, author = {Hiremath, Shruthi K. and Nishimura, Yasutaka and Chernova, Sonia and Pl\"{o}tz, Thomas}, title = {Bootstrapping Human Activity Recognition Systems for Smart Homes from Scratch}, year = {2022}, issue_date = {September 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {6}, number = {3}, url = {https://doi.org/10.1145/3550294}, doi = {10.1145/3550294}, journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.}, month = sep, articleno = {119}, numpages = {27}, keywords = {pattern recognition, smart-home, human activity recognition}, html = {https://dl.acm.org/doi/10.1145/3550294}, selected = {true} }
IMWUT
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables

Haresamudram, Harish, Essa, Irfan, and Plötz, Thomas

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2022

Abs Bib HTML

The emergence of self-supervised learning in the field of wearables-based human activity recognition (HAR) has opened up opportunities to tackle the most pressing challenges in the field, namely to exploit unlabeled data to derive reliable recognition systems for scenarios where only small amounts of labeled training samples can be collected. As such, self-supervision, i.e., the paradigm of ’pretrain-then-finetune’ has the potential to become a strong alternative to the predominant end-to-end training approaches, let alone hand-crafted features for the classic activity recognition chain. Recently a number of contributions have been made that introduced self-supervised learning into the field of HAR, including, Multi-task self-supervision, Masked Reconstruction, CPC, and SimCLR, to name but a few. With the initial success of these methods, the time has come for a systematic inventory and analysis of the potential self-supervised learning has for the field. This paper provides exactly that. We assess the progress of self-supervised HAR research by introducing a framework that performs a multi-faceted exploration of model performance. We organize the framework into three dimensions, each containing three constituent criteria, such that each dimension captures specific aspects of performance, including the robustness to differing source and target conditions, the influence of dataset characteristics, and the feature space characteristics. We utilize this framework to assess seven state-of-the-art self-supervised methods for HAR, leading to the formulation of insights into the properties of these techniques and to establish their value towards learning representations for diverse scenarios.
@article{10.1145/3550299, abbr = {IMWUT}, bibtex_show = {true}, author = {Haresamudram, Harish and Essa, Irfan and Pl\"{o}tz, Thomas}, title = {Assessing the State of Self-Supervised Human Activity Recognition Using Wearables}, year = {2022}, issue_date = {September 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {6}, number = {3}, url = {https://doi.org/10.1145/3550299}, doi = {10.1145/3550299}, journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.}, month = sep, articleno = {116}, numpages = {47}, keywords = {self-supervised learning, representation learning, human activity recognition}, html = {https://dl.acm.org/doi/10.1145/3550299}, selected = {true} }

ACM

Applying machine learning for sensor data analysis in interactive systems: Common pitfalls of pragmatic use and ways to avoid them

PlÖtz, Thomas

ACM Computing Surveys (CSUR) Sep 2021

Bib

@article{plotz2021applying,
  abbr = {ACM},
  bibtex_show = {true},
  title = {Applying machine learning for sensor data analysis in interactive systems: Common pitfalls of pragmatic use and ways to avoid them},
  author = {Pl{\"O}tz, Thomas},
  journal = {ACM Computing Surveys (CSUR)},
  volume = {54},
  number = {6},
  pages = {1--25},
  year = {2021},
  publisher = {ACM New York, NY, USA},
  selected = {true}
}

IMWUT

Deriving Effective Human Activity Recognition Systems through Objective Task Complexity Assessment

Shruthi K Hiremath, Thomas Plötz

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. Sep 2020

Bib HTML

@article{deriving,
  abbr = {IMWUT},
  bibtex_show = {true},
  title = {Deriving Effective Human Activity Recognition Systems through Objective Task Complexity Assessment},
  author = {Shruthi K Hiremath, Thomas Plötz},
  journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},
  volume = {4},
  number = {4},
  article = {131},
  year = {2020},
  publisher = {Association for Computing Machinary},
  html = {https://dl.acm.org/doi/10.1145/3432227},
  selected = {true}
}

ACM

IMUTube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Sep 2020

Bib HTML

@article{imutube,
  abbr = {ACM},
  bibtex_show = {true},
  title = {IMUTube: Automatic extraction of virtual on-body accelerometry from video for human activity recognition},
  author = {},
  journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
  volume = {4},
  number = {3},
  article = {65},
  year = {2020},
  publisher = {Association for Computing Machinary},
  html = {https://dl.acm.org/doi/10.1145/3411841},
  selected = {true}
}