alertmanager告警配置

本例使用node_exporter的指标项内存占有率为例子，记录alertmanager如何配置内存占用率> 10%报警到腾讯企业邮箱。

docker-compose

version: '3.2'
services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    ports:
    - 9090:9090
    command:
    - --config.file=/etc/prometheus/prometheus.yml
    volumes:
    - ./prom/prometheus.yml:/etc/prometheus/prometheus.yml:ro
    - ./prom/rule.yml:/etc/prometheus/rule.yml
  grafana:
    image: grafana/grafana-enterprise
    ports:
    - 3000:3000
  node_exporter:
    image: prom/node-exporter:latest
    container_name: node_exporter
    command:
      - '--path.rootfs=/host'
    network_mode: host
    pid: host
    restart: unless-stopped
    volumes:
      - '/:/host:ro,rslave'
  alertmanager:
    image: prom/alertmanager
    volumes:
    - ./alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml
    ports:
    - 9093:9093

prometheus配置

• prom/prometheus.yaml

# 告警规则的配置文件
rule_files:
  - /etc/prometheus/rule.yml
# altermanager的服务地址
alerting:
  alertmanagers:
  - static_configs: 
    - targets:
      - alertmanager:9093

• prom/rule.yml

groups:
- name: example
  rules:
  - alert: high_memory
    # 当内存占有率超过10%，持续1min,则触发告警
    expr: 100 - ((node_memory_MemAvailable_bytes{
    instance="192.168.10.64:9100",job="node_exporter"} * 100) / node_memory_MemTotal_bytes{
    instance="192.168.10.64:9100",job="node_exporter"}) > 90
    for: 1m
    labels:
      severity: page
    annotations:
      summary: spike memeory 

alertmanager配置

• alertmanager/alertmanager.yml

如何配置腾讯企业邮箱的密码，我的这篇文章有记录- Jenkins发送email至腾讯企业邮箱 Pipeline方式

global:
  # smtp服务器地址
  smtp_smarthost: smtp.exmail.qq.com:465
  # 发送者邮箱
  smtp_from: <your_email>
  # 发送者邮箱
  smtp_auth_username: <your_email>
  # 发送者密码
  smtp_auth_password: <your_password>
  smtp_require_tls: false

route:
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 1h
  receiver: 'mail'
receivers:
  - name: 'mail'
    email_configs:
     # 接收者邮箱
     - to: '<your_email>'
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'dev', 'instance']