Shell script implements application service log warehousing MySQL

Today I'd like to share a shell Scripting tools , adopt shell Script vs mysql The combination of , Store the error output log of a specific service into the specified mysql In the table , So as to facilitate the positioning and analysis of error problems .

On a daily basis , Often need and linux The system deals with , for example ： Service deployment 、 Log and service status viewing , and shell Scripts are and linux A common method of interaction . The following code example is in shell Embedded in the script mysql The configuration information drops the error log information into the database .

shell The script is as follows ：

#!/usr/bin/env bash

hostName="l-beta.io"
dbPort=3306
dbUser="beta"
dbPassword="123456"
dbName="test_log"
tableName="beta_error_log_record"
env="test1"

base="/home/q/www"

# Into the dead cycle 
while [[ 1 ]]; do
    # Loop de ergodic base Folder under directory ,-vE  The last two are the items to be excluded 
    for name in `ls $base | grep -vE "default|genesis"`;
    do
        #echo $base/$name
        # Determine if it's a directory 
        if [[ -d $base/$name ]]; then
           # Define the previous minute of the current time , It is used to filter and match the log content 
           before=`date -d "-1 minute" "+%Y-%m-%d %H:%M:"`
           # echo "Watching log $name in $before"
           # Filter and obtain catalina.out Last minute in the file Exception and Error journal 
           result=`grep -C5 "$before" $base/$name/logs/catalina.out | grep -vE "DEBUG|skywalking|10.8.0.37:8800" | grep -C5 -E "Exception|ERROR" | sed $'s/\'/\"/g' `
           # If result If it is zero, skip this cycle 
           if [[ -z "$result" ]]; then
                continue;
           fi
           echo "-------------------------->"
           # Splicing insert sql  sentence 
           insert_sql="insert into $tableName (environment,project_name,content) values('$env','$name','\n\nException:\n\n$result')"
           echo $insert_sql > tmp.sql
           echo "***********start*************"
           echo $insert_sql
           # Start execution sql sentence 
           mysql -h$hostName  -P$dbPort  -u$dbUser -p$dbPassword $dbName -e "source tmp.sql"
           echo "************end************"
        fi
    done
    # Input sleep 60s
    echo -e "sleep 60 seconds\n"
    sleep 60s
done

Copy

above shell The key information in the script is marked with corresponding comments , If you need to practice , Can be mysql The corresponding configuration information and the monitored log service path can be changed and replaced with the target information .

end