boot actuator - prometheus使用

boot actuator - prometheus - grafana

流程

• boot 应用 提供统计 api,如 http请求
• prometheus 从 boot 收集并存储
• grafana 将 prometheus 数据进行展示

docker prometheus 配置

version: '3.7'

networks:
    monitor:
        driver: bridge

services:
    prometheus:
        image: prom/prometheus
        container_name: prometheus
        hostname: prometheus
        restart: always
        user: root
        volumes:
            - ./prometheus/conf:/etc/prometheus
            - ./prometheus/data:/prometheus
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro
        ports:
            - "9090:9090"
        networks:
            - monitor
        command:
            - '--config.file=/etc/prometheus/prometheus.yml'
            - '--storage.tsdb.path=/prometheus'

    alertmanager:
        image: prom/alertmanager
        container_name: alertmanager
        hostname: alertmanager
        user: root
        restart: always
        volumes:
            - ./alertmanager/conf:/etc/alertmanager
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro
        ports:
            - "9094:9093"
        networks:
            - monitor

    grafana:
        image: grafana/grafana
        container_name: grafana
        hostname: grafana
        user: root
        restart: always
        ports:
            - "3000:3000"
        volumes:
            - ./grafana/data:/var/lib/grafana
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro
        networks:
            - monitor

    node-exporter:
        image: quay.io/prometheus/node-exporter
        container_name: node-exporter
        hostname: node-exporter
        restart: always
        user: root
        ports:
            - "9100:9100"
        networks:
            - monitor
        volumes:
            - /etc/localtime:/etc/localtime:ro
            - /etc/timezone:/etc/timezone:ro

配置 prometheus.yml

配置定时任务

$ cat prometheus/conf/prometheus.yml 
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'codelab-monitor'

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first.rules"
  # - "second.rules"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['192.xx.xx.10:9090']
  - job_name: 'boot-prometheus'           # 配置监听 spring boot
    metrics_path: '/actuator/prometheus'  # spring boot 默认暴露接口
    static_configs:
      - targets: ['172.xx.x.3:9999']

启动 docker-compose up -d

访问 prometheus

http://localhost:9090

grafana 使用

添加数据源 prometheus

问题

“Error on ingesting samples that are too old or are too far into the future”

容器和宿主机时间不不一致，或者相差太多