Netdata mail alarm configuration

Connected to a msmtp Service configuration When it's done , This article continues to configure email alerts

1. To configure netdata Mail to

/etc/netdata/edit-config health_alarm_notify.conf

Command similar vi i Enter edit mode , Edit finish esc Input :wq preservation

modify

DEFAULT_RECIPIENT_EMAIL="root" 

take root Change to email address , Multiple addresses are separated by spaces , Mail can be received |critical Alarm level , Such as

DEFAULT_RECIPIENT_EMAIL="[email protected] [email protected]|critical"

The first email address receives all levels of alarms , The second email address receives critical Level alarm results .

2. To configure sendmail The path of is msmtp The path of

Modify the configuration

/etc/netdata/edit-config health_alarm_notify.conf

take

sendmail=""

Change to

sendmail="/usr/bin/msmtp"

Test send

/usr/libexec/netdata/plugins.d/alarm-notify.sh test 

The first email address can receive 3 Messages , The second email address can receive 2 Messages , One more than the first warning mail .

3. use stress Pressure measurement cpu

install stress

yum install epel-release.noarch
yum install stress

Pressure measurement cpu

stress --cpu 1 --timeout 600

Here we use simulation 1 individual CPU Usage rate 100% Scene

1 yes cpu Check the number , Adjust according to your computer , Change the number of cores to a few , wait for 10 Minutes critical Level alarm .

common problem

1. After ten minutes, no email was sent to find the log

Log error message

msmtp: account default not found: no configuration file available

take .msmtprc File permissions are changed to 644 And cut to /etc/msmtprc Under the table of contents , Don't copy , There will be configuration conflicts during replication

chmod 644 ~/.msmtprc
mv ~/.msmtprc /etc/msmtprc

2. Log error message

msmtp: /etc/msmtprc: line 11: account default was already defined

resolvent ： take /root/.msmtprc File deletion , Only keep /etc/msmtprc file

3. belt |critical The alarm cannot be recovered clear mail

View log messages

touch: /var/cache/netdata/alarm-notify/email/[email protected]/1655974263: Permission denied

If it is docker Environmental Science , take :/var/cache/netdata The directory is mapped out and solved

docker run Command addition

 -v netdatacache:/var/cache/netdata \

Full command

docker run -d --name=netdata  \
  -p 19999:19999 \
  -v netdatacache:/var/cache/netdata \
  -v /proc:/host/proc:ro  \
  -v /sys:/host/sys:ro  \
  -v /var/run/docker.sock:/var/run/docker.sock:ro  \
  --restart unless-stopped  \
  --cap-add SYS_PTRACE \
  --security-opt apparmor=unconfined \
  netdata/netdata

Actual alarm recovery clear The mail is half an hour late ,critical and warning There is no delay in the mail , The specific situation still needs further study .

Be careful ： there  -v netdatacache The use of docker Data volume , Do not use other directories such as ${PWD}/netdatacache Instead of , otherwise docker Can't start

5.docker Log and mail log time is utc

Check the related issue Find out ,netdata Will read /etc/localtime Configuration of , Just do the directory mapping

Add mapping

-v /etc/localtime:/etc/localtime:ro  \

Full command

docker run -d --name=netdata  \
  -p 19999:19999 \
  -v netdatacache:/var/cache/netdata \
  -v /proc:/host/proc:ro  \
  -v /sys:/host/sys:ro  \
  -v /var/run/docker.sock:/var/run/docker.sock:ro  \
  -v /etc/localtime:/etc/localtime:ro  \
  --restart unless-stopped  \
  --cap-add SYS_PTRACE \
  --security-opt apparmor=unconfined \
  netdata/netdata