ZABBIX monitoring of traditional operation and maintenance (you get up early, I get up early, and we will get together sooner or later)

Preface

As an operation and maintenance , You need to be able to use the monitoring system to check the server status and website traffic indicators , Use the data of the monitoring system to understand the results released online , And the health status of the website .

Use an excellent monitoring software , We can ：

• Browse all server states of the whole website through a friendly interface
• Can be in Web The front end is convenient to view the monitoring data
• It can trace back to find the system problems and alarm conditions when the accident occurred

There are many monitoring systems , This paper mainly introduces zabbix

One 、zabbix brief introduction

1.1 zabbix What is it? ？

• zabbix It's based on Web The interface provides enterprise level open source solutions for distributed system monitoring and network monitoring .
• zabbix Can monitor all kinds of network parameters , Ensure the safe operation of the server system ; And provide a flexible notification mechanism to allow system administrators to quickly locate / Solve all kinds of problems .
• zabbix from 2 Part of the form ,zabbix server With optional components zabbix agent. adopt C/s Mode acquisition data , adopt B/s Patterns in Web End display and configuration .
• zabbix server Can pass sNMP,zabbix agent,ping, Port monitoring and other methods are provided for remote servers / Monitoring of network status , Data collection and other functions , It can run in Linux Platforms such as .
• zabbix agent It needs to be installed on the monitored target server , It mainly completes the hardware information or memory related to the operating system ,CPU And so on .

1.2 zabbix Monitoring principle

zabbix agent Installed on the monitored host ,zabbix agent Responsible for collecting local data of client regularly , And send it to zabbix server End ,zabbix server After receiving the data , Store data in database , User based zabbix WEB You can see the data showing the image in the front end . When zabbix Monitor a specific project , The project will set a trigger threshold , When the monitored index exceeds the threshold set by the trigger , Will do some necessary actions , Actions include ： Send a message （ mail 、 WeChat 、 SMS ）、 dispatch orders （shell command 、reboot、restart、install etc. ）.

1.3 Common procedures

zabbix Monitoring is deployed in the system , Contains five common programs : zabbix_server、zabbix_agent、zabbix_proxy、zabbix_get、zabbix_sender etc. .

1. zabbix server：zabbix Server daemons , among zabbix_agent、zabbix_get、zabbix_sender、zabbix_proxy All the data will eventually be submitted to zabbix server;
2. zabbix agent： Client daemons , Responsible for collecting client data , for example : collect CPU load 、 Memory 、 Hard disk usage, etc ;
3. zabbix proxy：zabbix Distributed agent daemon , Usually greater than 500 Console host , Distributed monitoring architecture deployment is required ;
4. zabbix get：zabbix Data receiving tools , Commands used alone , Usually in server perhaps proxy The client executes the command to obtain the information of the remote client ;
5. zabbix sender：zabbix Data sending tool , Users send data to server or proxy End , Usually, the user takes a long time to check .

1.4 zabbix Monitoring architecture

Two 、 Deploy zabbix

The server	name		IP Address
zabbix-server	centos7-8	10051	192.168.109.138
zabbix_agent2	centos7-1	10050	192.168.109.131

Turn off the firewall on the server and client

systemctl disable --now firewalld
setenforce 0

2.1 Deploy zabbix Server side

2.1.1 Modify hostname

hostnamectl set-hostname zbx-server

2.1.2 obtain zabbix Download source

rpm -ivh https://mirrors.aliyun.com/zabbix/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm 

2.1.3 Replace zabbix.repo For aliyuan , And download

cd /etc/yum.repos.d
sed -i 's#http://repo.zabbix.com#https://mirrors.aliyun.com/zabbix#' zabbix.repo
yum clean all && yum makecache
yum install -y zabbix-server-mysql zabbix-agent

2.1.4 install SCL

install SCL(Software Collections), It is convenient for later installation of higher version of php, Default yum Installed php Version is 5.4, Version is too low ,zabbix 5.0 Version pair php The minimum version is 7.2.0 edition .SCL You can use multiple versions of software on the same machine , Without affecting the dependent environment of the whole system . The package will be installed in /etc/opt/rh Under the table of contents .

yum install -y centos-release-scl 

2.1.5 modify zabbix-front Front end source , install zabbix Front end environment to scl In the environment

vim zabbix.repo
......
[zabbix-frontend]
......
enabled=1          # Open the installation source 
......

yum install -y zabbix-web-mysql-scl zabbix-apache-conf-scl

2.1.6 install zabbix The required database

yum install -y mariadb-server mariadb
# Boot from boot 
systemctl enable --now mariadb

mysql_secure_installation         # Initialize database , And set the password , Such as 000000（ Go back first , Input again ）

2.1.7 Add database users , as well as zabbix Required database information

mysql -uroot -p000000

CREATE DATABASE zabbix character set utf8 collate utf8_bin;
GRANT all ON zabbix.* TO 'zabbix'@'%' IDENTIFIED BY 'zabbix';
flush privileges;

2.1.8 Import database information

# Inquire about  sql  The location of the file 
rpm -ql zabbix-server-mysql
zcat /usr/share/doc/zabbix-server-mysql-5.0.15/create.sql.gz | mysql -uroot -p000000 zabbix

2.1.9 modify zabbix server The configuration file , Change the password of the database

vim /etc/zabbix/zabbix_server.conf 
......
DBPassword=zabbix					#124 That's ok , Appoint  zabbix  Database password 

2.1.10 modify zabbix Of php The configuration file

vim /etc/opt/rh/rh-php72/php-fpm.d/zabbix.conf
......
php_value[date.timezone] = Asia/Shanghai		#24 That's ok , uncomment , Change the time zone to Shanghai 

2.1.11 start-up zabbix Related services

systemctl restart zabbix-server zabbix-agent httpd rh-php72-php-fpm
systemctl enable zabbix-server zabbix-agent httpd rh-php72-php-fpm

Browser access ：http://192.168.109.138/zabbix

Click next , Set the password of the database zabbix

After installation , The default login account and password are ：Admin/zabbix

Set the file interface ： Click on the menu bar on the left 【User settings】,【Language】 choice Chinese(zh_CN), Click again Update to update .

The following information goes directly to the next step

2.1.12 solve zabbix-server Web Page Chinese garbled problem

# The default is installed 
yum install -y wqy-microhei-fonts

cp -f /usr/share/fonts/wqy-microhei/wqy-microhei.ttc /usr/share/fonts/dejavu/DejaVuSans.ttf

2.2 Deploy zabbix client

zabbix 5.0 Version adopts golang A new version of language development client agent2 .

zabbix Server side zabbix_server By default 10051 port , client zabbix_agent2 By default 10050 port .

2.2.1 Modify hostname

hostnamectl set-hostname zbx-agent01

2.2.2 Both the server and the client are configured with time synchronization

# Default installed 
yum install -y ntpdate
ntpdate -u ntp.aliyun.com
# You can do a planning task 

# The client configures the time zone , Consistent with the server 
mv /etc/localtime{
    ,.bak}
ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# Check the time 
date

2.2.3 Set up zabbix Download source , install zabbix-agent2

rpm -ivh https://mirrors.aliyun.com/zabbix/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm 

cd /etc/yum.repos.d
sed -i 's#http://repo.zabbix.com#https://mirrors.aliyun.com/zabbix#' /etc/yum.repos.d/zabbix.repo

yum install -y zabbix-agent2

2.2.4 modify agent2 The configuration file

vim /etc/zabbix/zabbix_agent2.conf
......
Server=192.168.109.138			#80 That's ok , Appoint  zabbix  Server side  IP  Address 
ServerActive=192.168.109.138		#120 That's ok , Appoint  zabbix  Server side  IP  Address 
Hostname=zbx-agent01			#131 That's ok , Specify the current  zabbix  Host name of the client 

2.2.5 start-up zabbix-agent2

systemctl start zabbix-agent2
systemctl enable zabbix-agent2
# View port number 
netstat -natp | grep zabbix
tcp6       0      0 :::10050                :::*                    LISTEN      54150/zabbix_agent2

2.2.6 Verify... On the server side zabbix-agent2 The connectivity of

# install  zabbix  Active data acquisition command 
yum install -y zabbix-get				

zabbix_get -s '192.168.109.131' -p 10050 -k 'agent.ping'
1

zabbix_get -s '192.168.109.131' -p 10050 -k 'system.hostname' 
zbx-agent01

2.2.7 stay Web Add... To the page agent host

Click on the left menu bar 【 To configure 】 Medium 【 host 】, Click on 【 Create a host 】

【 Host name 】 Set to zbx-agent01

【 Visible name 】 Set to zbx-agent01-192.168.109.131

【 group 】 choice Linux server

【Interfaces】 Of 【IP Address 】 Set to 192.168.109.131

Then click on the menu bar above 【 Templates 】

【Link new tamplates】 Search for Linux , choice Template OS Linux by Zabbix agent

Click on 【 add to 】

https://www.zabbix.com/integrations?cat=official_templates

zabbix Complete collection of monitoring templates , You can download it yourself

2.3 Custom monitoring content

Case list ： Customize the number of login to the monitoring client server

demand ： Limit the number of logins to no more than 3 individual , exceed 3 An alarm message will be sent out

2.3.1 Create custom on the client side key

1. Be clear about what needs to be done linux command

who | wc -l

2. establish zabbix Monitoring item configuration file , For customization key

vim /etc/zabbix/zabbix_agent2.conf
# You can create a customized monitoring item configuration file in  zabbix_agent2.d  Directory 
268 Include=/etc/zabbix/zabbix_agent2.d/*.conf
# The format of user-defined monitoring items is as follows 
291 # Format: UserParameter=<key>,<shell command>

cd /etc/zabbix/zabbix_agent2.d/

vim UserParameter_login.conf
UserParameter=login.user,who|wc -l

systemctl restart zabbix-agent2

3. Verify the newly created monitoring item on the server

zabbix_get -s '192.168.109.131' -p 10050 -k 'login.user'

2.3.2 stay Web Page to create a custom monitor item template

1. Create a template

Click on the left menu bar 【 To configure 】 Medium 【 Templates 】, Click on 【 Create a template 】

【 Template name 】 Set to Template Login User

【 Visible name 】 Set to Template Login User

【 group 】 choice Template

【 describe 】 Customizable

Click on 【 add to 】, At this point, you can 【 Linked templates 】 Search for Template Login User 了

2. Create application set （ Used to manage monitored items ）

Click on the menu bar above 【 Application set 】, Click on 【 Create application set 】

【 name 】 Set to Login User

Click on 【 add to 】

3. Create monitor item

Click on the menu bar above 【 Monitoring item 】, Click on 【 Create monitor item 】

【 name 】 Set to Number of login users

【 Key value 】 Set to login.user # The key value must be consistent with that set in the user-defined monitoring item configuration file

【 Update interval 】 Set to 10s

【 How long the historical data is kept 】Storage period 30d # The retention time can be customized

Click on 【 add to 】

4. Create trigger （ When the monitored item obtains the monitored value, it is compared with the preset value of the trigger , Judge whether to alarm ）

Click on the menu bar above 【 trigger 】, Click on 【 Create trigger 】

【 name 】 Set to Number of login users is greater than 3

【 severity 】 Set to Generally serious

【 expression 】 Click Add ,【 Monitoring item 】 Click Select Number of login users,【 function 】 choice last(),【 result 】 choice > 3, Click on 【 Insert 】 Click on 【 add to 】

5. Create graphics

Click on the menu bar above 【 graphics 】, Click on 【 Create graphics 】

【 name 】 Set to Number of login users

【 wide 】、【 high 】 The default value can be used directly

【 Monitoring item 】 Click Add and check relevant monitoring items Number of login users,【 function 】 choice Maximum , Others can keep the default value , Click on 【 add to 】

6. Associate the host with the template （ A host can associate multiple templates ）

Click on the left menu bar 【 To configure 】 Medium 【 host 】, Click the host you want to associate

Click on the menu bar above 【 Templates 】,【Link new tamplates】 Search for login, choice Template Login User, Click on 【 to update 】

Click here 【 monitoring 】 Medium 【 host 】, Click your associated host 【 graphics 】, You can view the relevant monitoring items and indicators

7. Set mail alarm

Click on the left menu bar 【 management 】 Medium 【 Alarm medium type 】, Click on 【 Create media type 】

【 name 】 Set to qq_Email

【SMTP The server 】 Set to smtp.qq.com

【SMTP Server port 】 Set to 25

【SMTP HELO】 Set to qq.com

【SMTP Email 】 Set to My email address , for example [email protected]

【 Secure links 】 nothing ; If you use SSL, Port is 465

【 authentication 】 choice User name and password

【 User name 】 Set to My email address , for example [email protected]

【 password 】 You can log in QQ Email page , Click on 【 Set up 】–>【 Account 】 Medium 【 Generate authorization code 】, Get the authorization code through SMS

【 describe 】 Customizable

Click on the menu bar above 【Message templates】, Click on 【 add to 】,【Message type】 choice problem , Click on 【 to update 】

Click on 【 add to 】, And test the function

Click on the left menu bar 【User settings】–>【 Alarm media 】, Click on 【 add to 】

【 type 】 choice qq_Email

【 The recipient 】 Set to [email protected]

【 When enabled 】 Set to 1-7,00:00-24:00

Click on 【 add to 】

Click again 【 to update 】

Click on the left menu bar 【 To configure 】 Medium 【 action 】

Select the corresponding action name and click to enter , Click on 【 add to 】

【 type 】 choice trigger ,【 trigger 】 Click Select Nunber of login users is greater than 3

Click on 【 Enable 】

Test email alarm ： Increasing the number of user logins on the test client exceeds the preset value of the trigger , see 【 monitoring 】–>【 The dashboard 】, Acknowledge the alarm