ZABBIX monitoring host and resource alarm

Zabbix What is it?
Zabbix It is an enterprise level distributed open source monitoring solution . Be able to monitor the server , Network devices , Objects such as applications . All reports 、 Statistics and configuration parameters can be accessed through Web Front page access .

Zabbix function
https://www.zabbix.com/documentation/3.4/zh/manual/introduction/features
• Data collection
• Flexible threshold definition
• Highly configurable alarm
• Real time chart display
• Web monitor
• Rich visual configuration
• Historical data storage
• Internet discovery
• Web Interface
• Zabbix API
• Authority management system
• extensible Agent
• Web

Zabbix The scope of monitoring
• Hardware ：Zabbix IPMI Interface
• System ：Zabbix Agent Interface
• Java ：Zabbix JMX Interface
• Network devices ：Zabbix SNMP Interface
• Application service ：Zabbix Agent UserParameter
• URL：Zabbix Web monitor

What to monitor

Zabbix Common terms
https://www.zabbix.com/documentation/3.4/zh/manual/definitions
• host (host)
• Host group (host group)
• Monitoring item (item)
• trigger (trigger)
• event (event)
• abnormal (problem)
• action (action)
• upgrade (escalation)
• medium (media)
• notice (notification)
• Remote command (remote command)
• Template (template)
• application (application)
• web scene (web scenario)
• front end (frontend)
• Zabbix API

Zabbix Components
https://www.zabbix.com/documentation/3.4/zh/manual/concepts

Zabbix Common terms
https://www.zabbix.com/documentation/3.4/zh/manual/definitions
• host (host)
• Host group (host group)
• Monitoring item (item)
• trigger (trigger)
• event (event)
• abnormal (problem)
• action (action)
• upgrade (escalation)
• medium (media)
• notice (notification)
• Remote command (remote command)
• Template (template)
• application (application)
• web scene (web scenario)
• front end (frontend)
• Zabbix API

Zabbix Deploy

1. install MySQL
   Deploy MySQL5.7 edition .

# yum -y install yum-utils 
# rpm -ivh https://dev.mysql.com/get/mysql80-community-release-el7-1.noarch.rpm # # # # yum-config-manager --disable mysql80-community
yum-config-manager --enable mysql57-community
# yum install mysql-community-server mysql-community-devel
systemctl start mysqld
systemctl status mysqld
systemctl enable mysqld

 To reset root Account and password ：
# grep 'temporary password' /var/log/mysqld.log
# mysql -uroot -p
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY '[email protected]';

 Modify the configuration file ：
# vi /etc/my.cnf
[mysql]
socket = /tmp/mysql.sock
[mysqld]
user = mysql
port = 3306
datadir = /var/lib/mysql
socket = /tmp/mysql.sock
bind-address = 0.0.0.0
pid-file = /var/run/mysqld/mysqld.pid
character-set-server = utf8
collation-server = utf8_general_ci
log-error = /var/log/mysqld.log

max_connections = 10240
open_files_limit = 65535
innodb_buffer_pool_size = 3G
innodb_flush_log_at_trx_commit = 2
innodb_log_file_size = 256M

# systemctl restart mysqld

2. Deploy Nginx

 Source code compilation and installation ：
# wget http://nginx.org/download/nginx-1.15.3.tar.gz
# yum install gcc pcre-devel openssl-devel –y
# useradd -M -s /sbin/nologin nginx
# tar zxvf nginx-1.15.3.tar.gz
# cd nginx-1.15.3
# ./configure --prefix=/usr/local/nginx --user=nginx --group=nginx --with-http_ssl_module --with-http_stub_status_module
# make && make install

 Modify the configuration file , modify pid Location ：
# vi /usr/local/nginx/conf/nginx.conf
pid        /var/run/nginx.pid;

 To configure systemd management service ：
# vi /usr/lib/systemd/system/nginx.service
[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/var/run/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/usr/local/nginx/sbin/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

 Boot and join boot on ：
systemctl daemon-reload
systemctl start nginx
systemctl enable nginx

3. Deploy PHP

# yum install epel-release –y

 Install dependency packages :
# yum install -y gcc gcc-c++ make gd-devel libxml2-devel \
libcurl-devel libjpeg-devel libpng-devel openssl-devel \
libxslt-devel

 Source code compilation and installation PHP:
# wget http://docs.php.net/distributions/php-5.6.36.tar.gz
# tar zxf php-5.6.36.tar.gz
# cd php-5.6.36
# ./configure --prefix=/usr/local/php \
--with-config-file-path=/usr/local/php/etc \
--enable-fpm --enable-opcache \
--with-mysql --with-mysqli  \
--enable-session --with-zlib --with-curl --with-gd \
--with-jpeg-dir --with-png-dir --with-freetype-dir \
--enable-mbstring --enable-xmlwriter --enable-xmlreader \
--enable-xml --enable-sockets --enable-bcmath --with-gettext
# make -j 8 && make install

 Copy configuration file ：
cp php.ini-production /usr/local/php/etc/php.ini
cp sapi/fpm/php-fpm.conf /usr/local/php/etc/php-fpm.conf
cp sapi/fpm/php-fpm.service /usr/lib/systemd/system/

 To configure systemd management service ：
# vi /usr/lib/systemd/system/php-fpm.service 
[Unit]
Description=The PHP FastCGI Process Manager
After=syslog.target network.target

[Service]
Type=simple
PIDFile=/usr/local/php/var/run/php-fpm.pid
ExecStart=/usr/local/php/sbin/php-fpm --nodaemonize --fpm-config /usr/local/php/etc/php-fpm.conf
ExecReload=/bin/kill -USR2 $MAINPID

[Install]
WantedBy=multi-user.target

 Boot and join boot on ：
systemctl daemon-reload
systemctl start php-fpm
systemctl enable php-fpm

 Import table structure ：
mysql> create database zabbix;
mysql> grant all on zabbix.* to [email protected]'%' identified by '[email protected]';
tar zxvf zabbix-4.0.0.tar.gz
cd zabbix-4.0.0/database/mysql
mysql -uzabbix -p'[email protected]' zabbix < schema.sql
mysql -uzabbix -p'[email protected]' zabbix < images.sql
mysql -uzabbix -p'[email protected]' zabbix < data.sql

4. Deploy Zabbix Server
   Install dependency packages ：

# yum install libxml2-devel libcurl-devel libevent-devel net-snmp-devel mysql-community-devel -y

 Source code compilation and installation zabbix：
# tar -zxf zabbix-4.0.0.tar.gz
# groupadd zabbix
# useradd -g zabbix zabbix -s /sbin/nologin
# cd zabbix-4.0.0
# ./configure --prefix=/usr/local/zabbix --enable-server --enable-agent --enable-java --with-mysql --enable-ipv6 --with-net-snmp --with-libcurl --with-libxml2
# make install

 Modify the configuration file ：
# vi /usr/local/zabbix/etc/zabbix_server.conf
DBHost=localhost
DBName=zabbix
DBUser=zabbix
DBPassword=[email protected]
DBSocket=/tmp/mysql.sock
 To configure systemd management service ：
# vi /usr/lib/systemd/system/zabbix_server.service
[Unit]
Description=Zabbix Server
After=syslog.target
After=network.target

[Service]
Environment="CONFFILE=/usr/local/zabbix/etc/zabbix_server.conf"
EnvironmentFile=-/etc/sysconfig/zabbix-server
Type=forking
Restart=on-failure
PIDFile=/tmp/zabbix_server.pid
KillMode=control-group
ExecStart=/usr/local/zabbix/sbin/zabbix_server -c $CONFFILE
ExecStop=/bin/kill -SIGTERM $MAINPID
RestartSec=10s
TimeoutSec=0

[Install]
WantedBy=multi-user.target

 Import table structure ：
# cd zabbix-4.0.0/database/mysql
# mysql -uroot –[email protected]
mysql> create database zabbix;
mysql> use mysql;
mysql> source schema.sql;
mysql> source images.sql;
mysql> source data.sql;

 Boot and join boot on ：
# systemctl daemon-reload
# systemctl start zabbix-server
# systemctl enable zabbix-server

 start-up Agent, Also monitor this machine ：
# /usr/local/zabbix/sbin/zabbix_agentd

Two 、 monitor Linux The server

1. Deploy Zabbix Agent
   CentOS7.x Operating system installation ：

# rpm -ivh http://repo.zabbix.com/zabbix/4.0/rhel/7/x86_64/zabbix-agent-4.0.0-2.el7.x86_64.rpm

Add configuration file ：

# vi /etc/zabbix/zabbix_agentd.conf
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
DebugLevel=3
Server=<Zabbix The server IP>
ListenPort=10050
ListenIP=< Current server IP>
ServerActive=<Zabbix The server IP>
Hostname=< Current server IP>
HostMetadata=Linux

Boot and join boot on ：

# systemctl start zabbix-agent
# systemctl enable zabbix-agent

2. Connectivity testing
   stay Zabbix The server executes the following command , Determine whether you can connect to Zabbix agent The server .

/usr/local/zabbix/bin/zabbix_get -s 192.168.80.2 -p 10050 -k "system.uptime"
774564

The returned number indicates that the connection is normal . Otherwise, it may be blocked by the firewall of the monitoring machine , Unable to connect , You can try to add a release rule to the monitored machine ：

iptables -I INPUT -s 192.168.80.1 -p tcp --dport 10050 -j ACCEPT

3. Web Page add host
   host , Monitoring item , Triggers and graphics
   

3.1 Add host

Create application set

Create monitor item

Add graphics

Create trigger

view drawings

Templates

Import template

Monitoring item
https://www.zabbix.com/documentation/3.4/zh/manual/config/items/itemtypes
• Monitor item format

• Monitor item type
• Zabbix agent
• SNMP agent
• Simple check
• User parameters （user parameter） ）

UserParameter=<key>,<command>
UserParameter=key[*],command

trigger

https://www.zabbix.com/documentation/3.4/zh/manual/config/triggers/expression
 Expression format ：
{
    <server>:<key>.<function>(<parameter>)}<operator><constant>
 Example 1： come from www.zabbix.com The last load value of the host is greater than 5 When entering PROBLEM state 
{
    www.zabbix.com:system.cpu.load[all,avg1].last()}>5
 Example 2： stay 5 Within minutes CPU iowait The average load is greater than 20 When entering PROBLEM state 
{
    www.zabbix.com:system.cpu.util[,iowait].avg(5m)}>20
 Example 3： When the load is greater than 5 Or the last 10 The load in minutes is greater than 2 When entering PROBLEM state 
{
    www.zabbix.com:system.cpu.load[all,avg1].last()}>5 or {
    www.zabbix.com:system.cpu.load[all,avg1].min(10m)}>2
 Example 4： When file /etc/passwd Check it out checksum When the value is different from the latest value, enter PROBLEM state 
{
    www.zabbix.com:vfs.file.cksum[/etc/passwd].diff()}=1
 Example 5： When the network card “eth0” stay 5 Bytes received in minutes are greater than 100KB When entering PROBLEM state 
{
    www.zabbix.com:net.if.in[eth0,bytes].min(5m)}>100K
 Example 6： stay 30 Within minutes 5 Enter when it is unreachable PROBLEM state 
{
    zabbix.zabbix.com:icmpping.count(30m,0)}>5
 Example 7： stay 5 No data is received within minutes to enter PROBLEM state 
{
    zabbix.zabbix.com:agent.ping.nodata(5m)}=1

The alarm
action （Action）

Define actions based on supported event sources ：
 Triggering event - When trigger The state of the from OK go to PROBLEM Or turn back
 Discover the event - When network discovery occurs
 Automatically register Events - When a new activity agent automatically registers
 Internal events - When the item is not supported or the trigger enters an unknown state

mail
Change the alarm media type

Create action

Create operation

 Alarm host :{
    HOSTNAME1}
 Alarm time :{
    EVENT.DATE} {
    EVENT.TIME}
 Alarm level :{
    TRIGGER.SEVERITY}
 The alarm information :{
    TRIGGER.NAME}
 Alarm items :{
    TRIGGER.KEY1}
 Details of the problem :{
    ITEM.NAME}:{
    ITEM.VALUE}
 current state :{
    TRIGGER.STATUS}:{
    ITEM.VALUE1}
 event ID:{
    EVENT.ID

Set the alarm media

Change trigger , The trigger condition

The test email has been sent

Custom script alert

# yum install mailx
# vi /etc/mail.rc     # Add email information 
set from=[e-mail] smtp=smtp.163.com
set smtp-auth-user=[e-mail] smtp-auth-password=xxxxx # Authorization code 
set smtp-auth=login
# echo "this is test mail." |mail -s "test mail" [e-mail]
 Alarm script ：
# cat /usr/local/zabbix/share/zabbix/alertscripts/sendmail.sh
#!/bin/bash
to=$1
subject=$2
body=$3
FILE=/tmp/mail.tmp
echo "$body" > $FILE
dos2unix -k $FILE     #  Solve the problem that the text becomes an attachment .bin
mail -s "$subject" "$to" < $FILE

Create media types

User specified media type

Change owner , Group test

chown zabbix.zabbix  /usr/local/zabbix/share/zabbix/alertscripts/sendmail.sh

Nail alarm

Nail add robot , Execute the following script

 cat dingding.py
#!/usr/bin/python
# -*- coding: utf-8 -*-
import requests
import json
import sys
import os

headers = {
    'Content-Type': 'application/json;charset=utf-8'}
api_url = "https://oapi.dingtalk.com/roboXXXXXXXXXXXXXXXXX"  # Nail robot Webhook Address 

def msg(text):
    json_text= {
    
     "msgtype": "text",
     "text": {
    
         "content": text
     },
     "at": {
    
         "atMobiles": [
             "186..."
         ],
         "isAtAll": False
     }
    }
    print requests.post(api_url,json.dumps(json_text),headers=headers).content

if __name__ == '__main__':
    text = sys.argv[1]
    msg(text)

Create media types

User specified media type

test