Video human behavior detection

theory

Video detection usually deals with ideas

Ideas	Application scenarios	principle	shortcoming
Video behavior detection	Behavior recognition	adopt 3D Convolution kernel implementation , Combining spatial semantic information and motion information to recognize human behavior	？
Body posture recognition	Used to detect whether a person falls or gets sick , Or for fitness 、 Automatic teaching of sports and dance	Based on key point detection , Combine time sequence information to judge human behavior	Distortion occurs when the human body is covered , The robustness is not high in complex scenes
Abnormal behavior detection	Abnormal behavior detection technology in video scene is the core technology of intelligent monitoring system	Predict the next frame based on the previous normal behavior , The predicted next frame is compared with the original video , Judge whether it is abnormal behavior

Overview of video behavior detection

• Behavior recognition ： Recognize actions in short videos ——what
• Sequential action detection ： Locate the time period when a specific action appears in a long video ——what+when
• Spatiotemporal motion detection ： Identify and locate people and actions in the video ——what+when+where

Pain points of video detection ：

• How to represent the actions in the video
• How to efficiently process a large amount of video data
• How to reduce the labeling cost of video data

slowfast–2019ICCV

The author's creative ideas come from

SlowFast yes Facebook stay 2019 year ICCV A video recognition paper , Inspired by the types of retinal nerve cells in primates （ about 80% The cells （P-cells） Operate at low frequency , Can identify details ; And about 20% The cells （M-cells） It operates at high frequency , Sensitive to changes in time ）. author A fast and slow network is proposed SlowFast framework , To realize the processing and analysis of time and space dimensions by two branches .

structure

SlowFast The whole algorithm consists of two convolution branches ：

• Slow Branch ： Fewer frames and larger channels learn spatial semantic information .
• Fast Branch ： A larger number of frames and a smaller number of channels learn motion information

Data set features

.avi file You need to collect an image every few frames .jpg file

The length of the video is inconsistent

Model effect

mmaction2 Instructions

ubantu18.04
python == 3.8
torch == 1.8.0
cuda == 11.11

1、 download Data sets and Open source framework mmaction2

from github download mmaction2-init-master github：https://github.com/open-mmlab/mmaction2

download UCF101（ Behavior recognition ） In dataset Data Set、Action Recognition https://www.crcv.ucf.edu/research/data-sets/ucf101/

UCF101 It's from YouTube Collected with 101 Action recognition of realistic action video of operation categories

2、 Install the open source framework

cd mkdir mmaction2
cd mmaction2
unzip mmaction2-master.zip
cd mmaction2-master
pip install -r requirements/build.txt
python setup.py develop
pip install decord -i https://pypi.douban.com/simple

https://github.com/jfzhang95/pytorch-video-recognition

Body posture recognition

Based on graph convolution

Body posture recognition OpenPose Source code compilation - Lost little bookboy's Note Lost little bookboy's Note (xugaoxiang.com)

Video abnormal behavior detection

• Unsupervised learning ： Predict the next frame based on the previous normal behavior , The predicted next frame is compared with the original video , Judge whether it is abnormal behavior
• Supervised learning ： Expand exception samples

Application scenarios

Monitor security violence detection