使用 Blackbox Exporter 测试网络连通性

如果你需要监控两个主机、主机与外部服务之间的网络状况，那么就可以试一试本文提到的 Blackbox Exporter。

1. 安装 Blackbox

• 下载二进制包

12345

wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.21.0/blackbox_exporter-0.21.0.linux-amd64.tar.gztar -xzvf blackbox_exporter-0.21.0.linux-amd64.tar.gzmv blackbox_exporter-0.21.0.linux-amd64/blackbox_exporter /usr/bin/mkdir /etc/prometheusmv blackbox_exporter-0.21.0.linux-amd64/blackbox.yml /etc/prometheus/

• 清理安装包

1	rm -rf blackbox_exporter-0.21.0.linux-amd64*

• 新建 Systemd 服务

1	vim /usr/lib/systemd/system/blackbox_exporter.service

新增如下内容:

[Unit]Description=blackbox_exporterAfter=network.target[Service]Restart=on-failureExecStart=/usr/bin/blackbox_exporter \    --config.file=/etc/prometheus/blackbox.ymlRestart=on-failure[Install]WantedBy=multi-user.target

• 启动服务

12	systemctl enable blackbox_exportersystemctl start blackbox_exporter

• 查看运行状态

1	systemctl status blackbox_exporter

2. 测试主机对外访问连通性

blackbox_exporter 默认在本地 9115 端口暴露服务。

实际上，通过拼接 URL 的方式，我们就可以实现主机对任意服务的连通测试，下面是一个实例，只需要提供 target、module 参数即可，其中 target 表示测试的目标，module 表示测试用的模块:

浏览器访问 http://BLACKBOX_HOST_IP:9115/probe?target=google.com&module=http_2xx，需要将 BLACKBOX_HOST_IP 替换为主机 IP。

1 2 3 4 5 6 7 8 910111213141516171819202122232425262728293031323334353637383940414243

# HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds# TYPE probe_dns_lookup_time_seconds gaugeprobe_dns_lookup_time_seconds 0.030818323# HELP probe_duration_seconds Returns how long the probe took to complete in seconds# TYPE probe_duration_seconds gaugeprobe_duration_seconds 0.353982702# HELP probe_failed_due_to_regex Indicates if probe failed due to regex# TYPE probe_failed_due_to_regex gaugeprobe_failed_due_to_regex 0# HELP probe_http_content_length Length of http content response# TYPE probe_http_content_length gaugeprobe_http_content_length -1# HELP probe_http_duration_seconds Duration of http request by phase, summed over all redirects# TYPE probe_http_duration_seconds gaugeprobe_http_duration_seconds{phase="connect"} 0.08580119300000001probe_http_duration_seconds{phase="processing"} 0.201979714probe_http_duration_seconds{phase="resolve"} 0.060847821999999996probe_http_duration_seconds{phase="tls"} 0probe_http_duration_seconds{phase="transfer"} 0.003931112# HELP probe_http_redirects The number of redirects# TYPE probe_http_redirects gaugeprobe_http_redirects 3# HELP probe_http_ssl Indicates if SSL was used for the final redirect# TYPE probe_http_ssl gaugeprobe_http_ssl 0# HELP probe_http_status_code Response HTTP status code# TYPE probe_http_status_code gaugeprobe_http_status_code 200# HELP probe_http_uncompressed_body_length Length of uncompressed response body# TYPE probe_http_uncompressed_body_length gaugeprobe_http_uncompressed_body_length 13645# HELP probe_http_version Returns the version of HTTP of the probe response# TYPE probe_http_version gaugeprobe_http_version 1.1# HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes.# TYPE probe_ip_addr_hash gaugeprobe_ip_addr_hash 4.032438981e+09# HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6# TYPE probe_ip_protocol gaugeprobe_ip_protocol 4# HELP probe_success Displays whether or not the probe was a success# TYPE probe_success gaugeprobe_success 1

在注释中，对这些指标有着很详尽的描述，DNS 延时、响应、协议等。

3. Prometheus 配置

接着，我们可以让 Prometheus 定时抓取 blackbox 的指标。下面是相关的配置文件，需要将 BLACKBOX_HOST_IP 替换为 blackbox_exporter 部署主机的 IP 地址。

1 2 3 4 5 6 7 8 9101112131415161718192021222324252627282930313233

scrape_configs: - job_name: 'blackbox_http' metrics_path: /probe params: module: [http_2xx] static_configs: - targets: ['github.com'] labels: url: 'github.com' - targets: ['dl-cdn.alpinelinux.org'] labels: url: 'dl-cdn.alpinelinux.org' relabel_configs: - source_labels: [__address__] target_label: __param_target - target_label: __address__ replacement: BLACKBOX_HOST_IP:9115 - job_name: 'blackbox_icmp' metrics_path: /probe params: module: [icmp] static_configs: - targets: ['github.com'] labels: url: 'github.com' - targets: ['dl-cdn.alpinelinux.org'] labels: url: 'dl-cdn.alpinelinux.org' relabel_configs: - source_labels: [__address__] target_label: __param_target - target_label: __address__ replacement: BLACKBOX_HOST_IP:9115

4. 配置 Grafana 面板

这里采用的是 https://grafana.com/grafana/dashboards/13587 面板。

稍微进行了调整，最终看起来如下图:

5. 可能遇到的问题

5.1 probe_http_status_code 状态码一直为 0

blackbox_exporter 默认使用的是 ipv6 栈，需要手动指定为 ipv4。

编辑配置文件 /etc/prometheus/blackbox.yml 指定 ipv4 栈。

12345

modules: http_2xx: prober: http http: preferred_ip_protocol: "ipv4"

然后，重启服务:

1	systemctl restart blackbox_exporter

5.2 Grafana 找不到 grafana-piechart-panel

• 安装插件

1	grafana-cli plugins install grafana-piechart-panel

• 在配置文件中指定插件的路径

vim /usr/local/grafana/conf/defaults.ini[plugin.piechart]path = /var/lib/grafana/plugins/grafana-piechart-panel

• 重启 Grafana

1	systemctl restart grafana-server