Neuro-Symbolic Human Activity Recognition

Generally, sensor-based HAR methods are based on one of two prevailing paradigms: Symbolic, knowledge-based methods and data-driven methods. Data-driven HAR methods, in recent years almost exclusively based on deep neural networks, have shown impressive results. However, they typically require extensive amounts of training data and cannot use prior knowledge as a surrogate for training data. Symbolic, knowledge-based methods, on the other hand, can integrate such prior knowledge directly, but are less suited to learn from raw, high-dimensional sensor data and fail to achieve competitive performance for such complex data. This project addresses sensor-based HAR from a hybrid, neuro-symbolic perspective, combining the strengths of data-driven, neural models and knowledge-based, symbolic models.

Relevant application domains for such models are industrial work processes, like order picking in warehousing or assembly tasks, where the goal is to recognize activity sequences of workers from wearable sensor data (e.g. smartwatches), which helps to optimize the economics and ergonomics of these work processes.