[CDH] configure CDH components through clouderamanager and collect JMX information with Prometheus monitoring

Preface ：

I need to be in cloudera manager Monitoring community face CDH Components of prometheus monitor . I also refer to many online materials , At present, many of them are modified background configuration files , Now it's tested , If you pass cloudera manager To start the service , I feel that the configuration is not effective . Here we use cloudera manager How to configure parameters in the monitoring interface , Has passed the test . Organize and record here .

Concrete practice ：

Here we use Hbase Service as an example . Let's configure , Refer to other components .

Dependent components

The services needed are

• prometheus
• grafana
• jmx_exporter https://github.com/prometheus/jmx_exporter

Modify the configuration

stay HBase Modify... In configuration **HBase Code snippet for advanced configuration of service environment （ Safety valve ）**:

The configuration contents are as follows: ：

HBASE_MASTER_OPTS=-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=10101 -javaagent:/run/cloudera-scm-agent/jmx_prometheus_javaagent-0.16.1.jar=9213:/run/cloudera-scm-agent/hbase.yml
HBASE_REGIONSERVER_OPTS=-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=10102 -javaagent:/run/cloudera-scm-agent/jmx_prometheus_javaagent-0.16.1.jar=9214:/run/cloudera-scm-agent/hbase.yml

hbase.yml

hbase.yml The contents of the document are as follows ：

---
lowercaseOutputName: true
lowercaseOutputLabelNames: true
rules:

- pattern: Hadoop<service=HBase, name=RegionServer, sub=Regions><>Namespace_([^\W_]+)_table_([^\W_]+)_region_([^\W_]+)_metric_(\w+)
  name: HBase_metric_$4
  labels:
    namespace: "$1"
    table: "$2"
    region: "$3"
- pattern: Hadoop<service=(\w+), name=(\w+), sub=(\w+)><>([\w._]+)
  name: hadoop_$1_$4
  labels:
    "name": "$2"
    "sub": "$3"
- pattern: .+

Have a problem

The debugging has not been able to run before , Various file modification permissions , route , The way of version has been tried , Finally found yml The suffix of the file must be .yml, But not for yaml. do not know why , If you know something, please give me some advice .

summary ：

This has actually been achieved CDH Cluster integration prometheus, The rest is simple , Use prometheus Just listen to the port , This is not our point .

Reference resources ：

• https://community.cloudera.com/t5/Support-Questions/Adding-a-javaagent-in-CDH-5-or-6-2/m-p/304399
• https://community.cloudera.com/t5/Support-Questions/Customize-Hbase-JVM-settings-in-CDH-5-4-2/td-p/30997
• https://zhuanlan.zhihu.com/p/358908161