Prometheus deployment

Prometheus Introduce

1、Prometheus Inspiration and Google Of Borgmon, It is mainly used in near real time 、 Based on dynamic cloud environment 、 Container microservices 、 Application monitoring .
Prometheus Focus on all kinds of data that are happening now , Instead of tracking data from weeks ago , Because they think “ Most monitoring queries and alarms are data in a day ”,Facebook Relevant papers have also verified this ：85% The timing query of is 26 Within hours .

In a nutshell ,Prometheus It's a surveillance system , And time series data capability .

2、Prometheus It is a project of the cloud native Computing Foundation , It's a system and service monitoring system . It collects metrics from configured targets at given intervals , Evaluation rule expression , Show results , And trigger the alarm when the specified condition is observed .

take Prometheus Functions distinguished from other indicators and monitoring systems include ：

• Multidimensional data model （ By indicator name and key / Time series defined by value dimension set ）

• PromQL, A powerful and flexible query language , This dimension can be used

• Independent of distributed storage ; A single server node is autonomous

• For time series collection HTTP Pull the model

• Push timing is supported through the intermediate gateway of batch jobs

• Discover targets through service discovery or static configuration

• Multiple graphics and dashboard support modes

• Support hierarchical and horizontal Federation

Architecture Overview

prometheus Monitoring principle

1、prometheus ： Although it is a monitoring platform , But it's actually a database

2、mysql_exporter: It can be understood as a program or software , It works on the target server we want to monitor , Mainly for monitoring mysql The data of .

3、node_exporter: His role is mainly to collect performance test data , Such as cpu、 Memory disk network and other information , Then save the data to prometheus, It is equivalent to storing data into the database .

4、prometheus It can only be used for data storage , Can't show , So we need to use grafana Components .

5、grafana It is mainly used for data display , And it can read data regularly

Prometheus Deploy

Prometheus Download address

Grafana Download address

Basic environment

Host name	IP Address	Installation services
master	192.168.91.137	prometheus
minion	192.168.91.138	node_exporter

preparation

Close the firewall and selinux

[[email protected] ~]# systemctl stop firewalld
[[email protected] ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

[[email protected] ~]# setenforce 0
[[email protected] ~]# cat /etc/selinux/config 
SELINUX=disabled

install Prometheus

//  Download installation package 
[[email protected] ~]# wget https://github.com/prometheus/prometheus/releases/download/v2.31.1/prometheus-2.31.1.linux-amd64.tar.gz

//  Unzip to /usr/local Next 
[[email protected] ~]# ls
anaconda-ks.cfg  prometheus-2.31.1.linux-amd64.tar.gz
[[email protected] ~]# tar xf prometheus-2.31.1.linux-amd64.tar.gz -C /usr/local/
[[email protected] ~]# cd /usr/local/
[[email protected] local]# ls
bin    include  libexec                        share
etc    lib      prometheus-2.31.1.linux-amd64  src
games  lib64    sbin

//  rename 
[[email protected] local]# mv prometheus-2.31.1.linux-amd64/ prometheus
[[email protected] local]# ls
bin  games    lib    libexec     sbin   src
etc  include  lib64  prometheus  share

Start the service

[[email protected] local]# cd prometheus
[[email protected] prometheus]# ls
LICENSE  console_libraries  prometheus      promtool
NOTICE   consoles           prometheus.yml
[[email protected] prometheus]# /usr/local/prometheus/prometheus --config.file="/usr/local/prometheus/prometheus.yml"

Check the port

[[email protected] ~]# ss -antl
State  Recv-Q Send-Q  Local Address:Port   Peer Address:Port Process                                                      
LISTEN 0      128           0.0.0.0:22          0.0.0.0:*                                                                 
LISTEN 0      128              [::]:22             [::]:*                                                                 
LISTEN 0      128                 *:9090              *:* 

web Page access
Browser open 192.168.91.137:9090 The port can be opened Prometheus Its own monitoring page

http://192.168.91.137:9090/metrics View monitoring data

Start monitoring

preparation

//  Close the firewall and selinux
[email protected] ~]# systemctl stop firewalld
[[email protected] ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

[[email protected] ~]# setenforce 0
[[email protected] ~]# vim /etc/selinux/config 
SELINUX=disabled

 download node_exporter Installation package 
[[email protected] ~]# wget https://github.com/prometheus/node_exporter/releases/download/v1.3.0/node_exporter-1.3.0.linux-amd64.tar.gz

[[email protected] ~]# ls
anaconda-ks.cfg  node_exporter-1.3.0.linux-amd64.tar.gz

//  decompression 
[[email protected] ~]# tar xf node_exporter-1.3.0.linux-amd64.tar.gz -C /usr/local/
[[email protected] ~]# cd /usr/local/
[[email protected] local]# ls
bin    include  libexec                          share
etc    lib      node_exporter-1.3.0.linux-amd64  src
games  lib64    sbin

//  rename 
[[email protected] local]# mv node_exporter-1.3.0.linux-amd64/ node_exporter
[[email protected] local]# ls
bin  games    lib    libexec        sbin   src
etc  include  lib64  node_exporter  share

//  Start the service 
[[email protected] local]# cd node_exporter/
[[email protected] node_exporter]# nohup /usr/local/node_exporter/node_exporter

//  View service 
[[email protected] ~]# ss -antl
State  Recv-Q Send-Q  Local Address:Port   Peer Address:Port Process                                                      
LISTEN 0      128           0.0.0.0:22          0.0.0.0:*                                                                 
LISTEN 0      128              [::]:22             [::]:*                                                                 
LISTEN 0      128                 *:9100              *:*

stay prometheus Add a controlled host to the configuration file

[[email protected] local]# cd prometheus/
[[email protected] prometheus]# ls
LICENSE  console_libraries  data        prometheus.yml
NOTICE   consoles           prometheus  promtool
[[email protected] prometheus]# vim prometheus.yml 
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "amu1"
    static_configs:
      - targets: ["192.168.91.138:9100"]

Service to restart

//  Kill process , Restart the service 
[[email protected] prometheus]# pkill prometheus
[[email protected] prometheus]# /usr/local/prometheus/prometheus --config.file="/usr/local/prometheus/prometheus.yml"

revisit web page

install grafana

//  download rpm package 
[[email protected] ~]# wget https://dl.grafana.com/oss/release/grafana-7.3.3-1.x86_64.rpm

[[email protected] ~]# ls
anaconda-ks.cfg
grafana-7.3.3-1.x86_64.rpm
prometheus-2.31.1.linux-amd64.tar.gz

[[email protected] ~]# yum -y install grafana-7.3.3-1.x86_64.rpm

//  Start the service 
[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl enable --now grafana-server.service
Synchronizing state of grafana-server.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable grafana-server
Created symlink /etc/systemd/system/multi-user.target.wants/grafana-server.service → /usr/lib/systemd/system/grafana-server.service.

//  Check the port 
[[email protected] ~]# ss -antl
State  Recv-Q Send-Q  Local Address:Port   Peer Address:Port Process                                                      
LISTEN 0      128           0.0.0.0:22          0.0.0.0:*                                                                 
LISTEN 0      128              [::]:22             [::]:*                                                                 
LISTEN 0      128                 *:3000              *:*                                                                 
LISTEN 0      128                 *:9090              *:*    

web Page access

Browser open 192.168.91.137:3000 Get into grafana Interface
The default username and password are both admin
The first login will prompt to change the password

The following is the reset password interface

This is the login interface

add to prometheus data source

Click gear settings Of “Add data source”

url In frame Input http://192.168.91.137:9090

Dashboards Page selection “Prometheus 2.0 Stats”, Then save

Click on the magnifying glass Search for Click in the first “Prometheus 2.0 Stats” You can see the whole monitoring page