Automation and disconnection monitoring of video addition

The live broadcast configuration of multiple cameras is automated

demand ： There are many camera hardware that need to be connected to their own platform , Achieve the effect of display and configuration .
scene ：
At present, all the cameras used are Haikang security cameras , Because the scene decides that the number of watching a single live broadcast is not much , There are many cameras . And power is often cut off ！ Broken net ！ Server shutdown ！

There are several ways to achieve

1. Haikang camera links to the video recorder and gets it directly RTSP Address , after ffmpeg Convert to m3u8 Output directly after streaming .
   ffmpeg Stream processing software , To baidu
   advantage ： The advantage is that the process is very simple ,ffmpeg The command pulls the flow to the local . Then the stream address is played directly . The most important thing is not to spend money .

shortcoming ： The disadvantages are obvious , The stream exists for local direct reading , Bandwidth is the bottleneck . When the number of videos played exceeds 20 You'll find up 5M The bandwidth of is obviously insufficient . And when there are many streams , We should carry out flow pulling one by one , If a stream goes wrong , Operation and maintenance is very difficult . For example, we often lose power and network . Think about it .

2. Fluorite cloud .
   The easiest way . Brought by Haikang , The result is right Bring their own CDN Speed up , except Cost money There's nothing wrong with it .

3. ffmpeg+ Tencent live
   This is a little more complicated , It is based on the first idea , Push the stream to Tencent live , Let Tencent live do CDN Speed up It solves the problem of occupying bandwidth when playing streams , And the traffic of live broadcast is very cheap . And Tencent cloud can control whether to pull or not , It can be disconnected when there is no need to pull the flow No need to occupy bandwidth . Our scene doesn't need to see videos all the time , But sometimes I see more than ten or twenty . So I finally chose this .

technological process

The train of thought is certain , Basically, the process is settled .
First, the camera is connected to the video recorder , Get RTSP Address , And then through ffmpeg Circumfluence RTMP , Continue to push to Tencent cloud live . Tencent will help you transcode Format The resolution of the Code Anti-counterfeiting Time limit CDN Operation, etc.
Be careful
ffmpeg Try to put the server on the intranet of the camera , because ffmpeg Is always pulling flow , If you put it on the Internet , When pulling the flow The upstream bandwidth of the network where the camera is located will be the bottleneck .

• Get the video stream address .

rtsp://admin:[email protected]:1554/h264/ch36/main/av_stream 

• Pull the flow to the local

ffmpeg -rtsp_transport tcp -i rtsp://admin:[email protected]:1554/h264/ch36/main/av_stream -vcodec copy -acodec aac -f flv -y  -hls_time 5 -hls_list_size 10 -hls_wrap 10 rtmp://127.0.0.1:1935/hls/

1935 yes nginx Listening port

• To configure nginx Streaming to Tencent cloud live

server {

    listen 1935;  # Listening port 

    chunk_size 4000;


 # rtmp Push stream request path 
    application hls1 {
        live on;
        hls on;
        #  Keep locally hlv Format file 
        hls_path /Users/**/haikang/hls1;
        hls_fragment 10s;
        #  Turn to Tencent cloud 
        push rtmp://***t?bizid=2490&txSecret=d39774d3ce88ad218fe54eb84783a5af&txTime=5C1281FF;
    }

If there are multiple streams here Need to configure multiple application modular .
push It is the forwarding address generated by Tencent cloud .
hls_path Is the video path after streaming transcoding ,
nginx It needs to be installed RTMP Modular .
The configuration of Tencent cloud can be viewed in official documents , In addition, it provides interface operation . You can configure the flow through the interface .
Come here Basically through manual configuration Can generate a CDN Accelerated live address , You just need to save , And display it to the desired position .

monitor

In case of power failure Broken net , We also need to do a monitor , It is detected that a stream is disconnected , Just restart .
stay linux There was a crontab Scheduled tasks for , Set to startup and operation , In this way, the server can continue to automatically pull the stream after it is powered off and restarted . We can use it to implement a monitoring system . Monitor whether each flow is normal in the monitoring script . If it's not normal Then restart the streaming command .

• crontab Execute the script

Execute the monitoring script every minute .

*/1 * * * * /Users/user/myshell/listenshell.sh

• Monitor scripts
  listenshell.sh

#! /bin/bash
PATH=/usr/local/bin:/usr/local/opt/[email protected]/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/Cellar/rabbitmq/3.8.3/sbin:/Users/benxiong/soft/gradle-5.6.4/bin:/Users/benxiong/soft/apache-maven-3.6.3/bin:/usr/local/Cellar/rabbitmq/3.8.3/sbin
# This script is used to monitor FFPEMG  Whether it is normal to pull the local flow 
#  Application -- The camera is powered off when pulling the stream   Broken net  ffpemg The server is powered off and restarted 
# adopt Crontab  Execute the script every ten minutes ,  When it is detected that the number of folder files storing video streams decreases, it is judged 
 The disconnection has been disconnected   Or cut off the network   Or power off . Then carry out the flow pulling operation again . For ease of operation   The pull flow command exists in > The name in the pull stream folder is getStream.sh

#HLS Stored folder and path 
folder=`ls /Users/username/haikang/`
var1="/Users/username/haikang/"

# Log name 
log="/Users/username/myshell/logs/shell.log"  # Operation log storage path 
fsize=2000
exec 2>>$log  # If there are error messages during execution, they are output to the log file 
# Disconnected flow   Log files 
logErr="/Users/username/myshell/logs/shellErr.log"  # Operation log storage path 
# Log function 
# Parameters 
 # Parameter one , Level ,INFO ,WARN,ERROR
    # Parameter two , Content 
# Return value 
function zc_log()
{
  # Judgment format 
  if [ 2 -gt $# ]
  then
    echo "parameter not right in zc_log function" ;
    return ;
  fi
  if [ -e "$log" ]
  then
    touch $log
  fi

  # current time 
  local curtime;
  curtime=`date +"%Y%m%d%H%M%S"`

  # Determine file size 
local cursize ;
  cursize=`cat $log | wc -c` ;

  if [ $fsize -lt $cursize ]
  then
    mv $log $curtime".out"
    touch $log ;
  fi
  # write file 
  echo "$curtime $*" >> $log;
}
time2=$(date "+%Y%m%d%H%M%S");
echo " present time " $time2 "---  Start the test " >> $log;
echo " present time " $time2 "---  Start the test "
# Circular video storage directory   Judge whether the current number of files is greater than 4  If it is less than 4  Then it is judged as disconnected   Try to restart >.
for sfile in $folder; do
 { count=`ls  $var1$sfile | wc -w`;
  if [ $count -gt 4 ]
   then echo $time2$var1$sfile " The number of videos under the path is -“ $count “- Recording is normal   Continue to record > system ···" >> $log;
 else
        echo $time2$var1$sfile " The number of videos under the path is -“ $count “- Recording may have stopped   take > To restart recording ··" >> $logErr;
          echo $time2$var1$sfile"/getStream.sh  Ready to restart "
        . $var1$sfile"/getStream.sh";
        echo $time2$var1$sfile"/getStream.sh  Restart successfully "


  fi
} &

done
echo " sign out "

This script needs attention ,folder=ls /Users/username/haikang/ The left and right points of the string to the right of the equal sign It's the keyboard 1 Left button .PATH Need to add , Because this script is through crontab perform , The process he executes cannot read the system path .

The basic principle is the pull flow partition under the pull flow path , If the flow is disconnected This file will be deleted in a short time , So we judge that the number of files in it is less than a number We can judge whether he is still pulling . If the flow is disconnected , Then execute the pull flow command again . And print the error log , If not disconnected Then continue to pull the flow .

getStream.sh

ffmpeg -rtsp_transport tcp -i rtsp://admin:[email protected]:1554/h264/ch36/main/av_stream -vcodec copy -acodec aac -f flv -y  -hls_time 5 -hls_list_size 10 -hls_wrap 10 rtmp://127.0.0.1:1935/hls/

Come here Basically monitoring There are no more problems .

Add cameras automatically

When there are too many cameras , It will lead to cumbersome configuration Operating the configuration file is error prone .
Now complete the whole video configuration process There are several manual configurations that need to be repeated .

• Tencent video gets the streaming address
• Pull stream file getStream.sh Each time you add a new camera, you need to add a streaming file and modify the corresponding streaming address
• nginx The configuration file nginx.conf Add one for each increase application Fragments of
• Tencent cloud live broadcast configuration

Mainly these four , Tencent cloud live provides interface configuration , So we can operate through the interface 1 and 4.
As for the command of generating flow And modify Nginx Configuration file for restart nginx Wait for the operation , We can go through JAVA Code to remote connection operation .
The second point generates getStream.sh Directly splice according to the pull flow path , Then the generated file is uploaded to the server , Create a new streaming folder , Just put it inside . Note that the folder name must match yours nginx The names configured inside are consistent
Third nginx It is troublesome to modify the configuration file , You need to download the file locally first , Then read the file , Find the mark made in advance , That is, put a recognizable sign on the position you want to add . Then replace this logo . Then back up the files on the server , Upload the replacement file , restart nginx.

server {

    listen 1935;  # Listening port 

    chunk_size 4000;

    # rtmp Push stream request path 
    application hls {
        live on;
        hls on;
        #  Keep locally hlv Format file 
        hls_path /Users/username/haikang/hls;
        hls_fragment 10s;
        #  Turn to Tencent cloud 
        push rtmp:// domain name /live/rtsptest?bizid=2490&txSecret=d39774d3ce88ad218fe54eb84783a5af&txTime=5C1281FF;
    }
    #ABCD
}

among #ABCD It's a sign , When adding #ABCD Replace with application+#ABCD The position of such replacement is guaranteed to remain unchanged forever . Not bad application Put signs on both sides , For the convenience of deleting configuration . such as
#hls+application+#hls+#ABCD
In this way, it is convenient to find the name hls Delete and modify the configuration of .

thus , We will implement through an interface , Pass in the streaming address of a video recorder , Automatic monitoring can be realized Configured video platform