Testing network connectivity with the blackbox exporter

If you need to monitor two hosts 、 Network conditions between the host and external services , Then you can try the Blackbox Exporter.

1. install Blackbox

• Download binary package

12345

wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.21.0/blackbox_exporter-0.21.0.linux-amd64.tar.gztar -xzvf blackbox_exporter-0.21.0.linux-amd64.tar.gzmv blackbox_exporter-0.21.0.linux-amd64/blackbox_exporter /usr/bin/mkdir /etc/prometheusmv blackbox_exporter-0.21.0.linux-amd64/blackbox.yml /etc/prometheus/

• Clean the installation package

1	rm -rf blackbox_exporter-0.21.0.linux-amd64*

• newly build Systemd service

1	vim /usr/lib/systemd/system/blackbox_exporter.service

Add the following :

[Unit]Description=blackbox_exporterAfter=network.target[Service]Restart=on-failureExecStart=/usr/bin/blackbox_exporter \    --config.file=/etc/prometheus/blackbox.ymlRestart=on-failure[Install]WantedBy=multi-user.target

• Start the service

12	systemctl enable blackbox_exportersystemctl start blackbox_exporter

• View running status

1	systemctl status blackbox_exporter

2. Test the external access connectivity of the host

blackbox_exporter Default is local 9115 Port exposure service .

actually , By joining together URL The way , We can test the connectivity of the host to any service , Here is an example , Just provide target、module Parameters can be , among target Indicates the goal of the test ,module Represents the module for testing :

Browser access http://BLACKBOX_HOST_IP:9115/probe?target=google.com&module=http_2xx, Need to put BLACKBOX_HOST_IP Replace with host IP.

1 2 3 4 5 6 7 8 910111213141516171819202122232425262728293031323334353637383940414243

# HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds# TYPE probe_dns_lookup_time_seconds gaugeprobe_dns_lookup_time_seconds 0.030818323# HELP probe_duration_seconds Returns how long the probe took to complete in seconds# TYPE probe_duration_seconds gaugeprobe_duration_seconds 0.353982702# HELP probe_failed_due_to_regex Indicates if probe failed due to regex# TYPE probe_failed_due_to_regex gaugeprobe_failed_due_to_regex 0# HELP probe_http_content_length Length of http content response# TYPE probe_http_content_length gaugeprobe_http_content_length -1# HELP probe_http_duration_seconds Duration of http request by phase, summed over all redirects# TYPE probe_http_duration_seconds gaugeprobe_http_duration_seconds{phase="connect"} 0.08580119300000001probe_http_duration_seconds{phase="processing"} 0.201979714probe_http_duration_seconds{phase="resolve"} 0.060847821999999996probe_http_duration_seconds{phase="tls"} 0probe_http_duration_seconds{phase="transfer"} 0.003931112# HELP probe_http_redirects The number of redirects# TYPE probe_http_redirects gaugeprobe_http_redirects 3# HELP probe_http_ssl Indicates if SSL was used for the final redirect# TYPE probe_http_ssl gaugeprobe_http_ssl 0# HELP probe_http_status_code Response HTTP status code# TYPE probe_http_status_code gaugeprobe_http_status_code 200# HELP probe_http_uncompressed_body_length Length of uncompressed response body# TYPE probe_http_uncompressed_body_length gaugeprobe_http_uncompressed_body_length 13645# HELP probe_http_version Returns the version of HTTP of the probe response# TYPE probe_http_version gaugeprobe_http_version 1.1# HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes.# TYPE probe_ip_addr_hash gaugeprobe_ip_addr_hash 4.032438981e+09# HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6# TYPE probe_ip_protocol gaugeprobe_ip_protocol 4# HELP probe_success Displays whether or not the probe was a success# TYPE probe_success gaugeprobe_success 1

In comments , There is a very detailed description of these indicators ,DNS Time delay 、 Respond to 、 Agreements, etc .

3. Prometheus To configure

next , We can get Prometheus Grasp regularly blackbox Indicators of . Here are the related configuration files , Need to put BLACKBOX_HOST_IP Replace with blackbox_exporter Deployment host IP Address .

1 2 3 4 5 6 7 8 9101112131415161718192021222324252627282930313233

scrape_configs: - job_name: 'blackbox_http' metrics_path: /probe params: module: [http_2xx] static_configs: - targets: ['github.com'] labels: url: 'github.com' - targets: ['dl-cdn.alpinelinux.org'] labels: url: 'dl-cdn.alpinelinux.org' relabel_configs: - source_labels: [__address__] target_label: __param_target - target_label: __address__ replacement: BLACKBOX_HOST_IP:9115 - job_name: 'blackbox_icmp' metrics_path: /probe params: module: [icmp] static_configs: - targets: ['github.com'] labels: url: 'github.com' - targets: ['dl-cdn.alpinelinux.org'] labels: url: 'dl-cdn.alpinelinux.org' relabel_configs: - source_labels: [__address__] target_label: __param_target - target_label: __address__ replacement: BLACKBOX_HOST_IP:9115

4. To configure Grafana panel

What we use here is https://grafana.com/grafana/dashboards/13587 panel .

Slightly adjusted , Finally, it looks like the following figure :

5. Possible problems

5.1 probe_http_status_code The status code is always 0

blackbox_exporter The default is ipv6 Stack , You need to manually specify ipv4.

Edit profile /etc/prometheus/blackbox.yml Appoint ipv4 Stack .

12345

modules: http_2xx: prober: http http: preferred_ip_protocol: "ipv4"

then , Restart the service :

1	systemctl restart blackbox_exporter

5.2 Grafana Can't find grafana-piechart-panel

• Installing a plug-in

1	grafana-cli plugins install grafana-piechart-panel

• Specify the path of the plug-in in the configuration file

vim /usr/local/grafana/conf/defaults.ini[plugin.piechart]path = /var/lib/grafana/plugins/grafana-piechart-panel

• restart Grafana

1	systemctl restart grafana-server