Top issue tpami 2022! Behavior recognition based on different data modes: a recent review

human beings Behavior recognition To understand human behavior , And assign a label to the behavior , for example , handshake 、 Eat something 、 Running, etc . It has a wide application prospect , Therefore, it has attracted more and more attention in the field of computer vision . Human behavior can be represented by various data modes , Such as RGB、 skeleton 、 depth 、 Infrared sequence 、 Point cloud 、 Flow of events 、 Audio 、 Acceleration signal 、 Radar and WiFi, These data modes have different advantages in different scenarios . therefore , The existing papers try to study the behavior recognition methods of different data modes based on the mainstream deep learning .

From the point of view of modal , The current behavior recognition methods based on deep learning are comprehensively summarized . say concretely ,

（1） This paper reviews Behavior recognition method based on single mode , These modes include RGB、 skeleton 、 depth 、 Infrared sequence 、 Point cloud 、 Flow of events 、 Audio 、 Acceleration signal 、 Radar and WiFi.

（2） This paper reviews Behavior recognition method based on multimodality , And divide it into Multimode fusion and Cross modal collaborative learning Two types of .

（3） This paper reviews The latest and most advanced in-depth learning methods , Include CNN、RNN、GCN and Transformer, The existing methods and their performances are compared on several benchmark data sets .

Thesis link ：

https://arxiv.org/abs/2012.11866