TDOST · Megha Thukral

The problem. Smart-home HAR models break the moment a sensor is moved or a new home has a different layout. The signal is the arrangement of sensors, and that arrangement never repeats.

TDOST method overview — TDOST converts each raw sensor event into a textual description of the trigger and then encodes that description with a language model, so that downstream models learn over the *meanings* of the triggers rather than over opaque sensor IDs.

Idea

Instead of learning over opaque sensor IDs, we re-describe each sensor trigger as a short natural-language phrase, for example "the kitchen motion sensor turned on" or "the bedroom door opened". We then embed those phrases with a pretrained language model. The activity classifier subsequently operates over text-grounded representations that are invariant to the specific layout of any one home, since the same activity in a different home produces semantically similar descriptions even when the underlying sensor identifiers are completely different.

Why it matters

Layout invariance is one of the missing rungs in deploying smart-home human activity recognition at scale, because every new home effectively introduces a new domain in which sensor placements, sensor types, and sensor identifiers all change. TDOST allows models trained in one house to transfer to others without retraining, and we demonstrated that the approach works on a real deployment in the Georgia Tech Aware Home as part of the NSF AI CARING demo.

TDOST results — Cross-home generalization gains over sensor-ID baselines.

Cite

@article{thukral2025tdost,
  title  = {Layout-Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions of Sensor Triggers (TDOST)},
  author = {Thukral, Megha and Dhekane, Sourish Gunesh and Hiremath, Shruthi K. and Haresamudram, Harish and Pl{\"o}tz, Thomas},
  journal= {Proc. ACM IMWUT},
  year   = {2025}
}