Supervisor multi process management exception automatic restart visual management

2.006.jpeg

One 、 preface

Supervisor Is a multi process management tool , stay Docker The associated processes in the can pass through supervisor To manage .

Microservice project development stage , It can be used for start-up management of microservice sub projects .

Support web Visual management , It can greatly help developers to monitor and restart the project .

Two 、 Installation and use

（ One ） Installation and configuration

1、 Service installation

Before service installation , Proposed update Python edition , Using a newer version is conducive to service expansion , If managed services depend on newer Python edition , The service needs to be reinstalled again .

yum install -y epel-release
yum install -y supervisor

View version number

supervisord -v

2、 The configuration file

The configuration file path is /etc/supervisord.conf, Which is in English ; Notation . Back up the configuration file , After filtering the annotation configuration, a new configuration file is formed .

#  Backup configuration files 
mv /etc/supervisord.conf /etc/supervisord.example.conf
#  Keep non annotation configuration , Initialize to a new configuration file 
cat /etc/supervisord.example.conf | grep -v '^;' | tr -s "\n" > /etc/supervisord.conf

Use command echo_supervisord_conf View default configuration

[unix_http_server]
file=/var/run/supervisor/supervisor.sock

;  visualization web Monitoring module （ No direct comments are required ）
[inet_http_server]
port=0.0.0.0:9001
username=root
password=root

[supervisord]
logfile=/var/log/supervisor/supervisord.log
;  Maximum single log file size 
logfile_maxbytes=50MB
;  The maximum number of log files to keep 
logfile_backups=10
loglevel=info
pidfile=/var/run/supervisord.pid
;  If it is set as system service , I need to set to false
nodaemon=false
minfds=1024
minprocs=200

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor/supervisor.sock

;  Sub process configuration storage directory 
[include]
files = supervisord.d/*.ini

View the version number with the following command ：

supervisord -v

（ Two ） start-up

1、 Background start

take supervisord Set it to auto start , Ensure that the services entrusted to it can be started normally , It is recommended to use this method to start .

#  Set power on self start 
systemctl enable supervisord

#  Start the main service 
systemctl start supervisord

2、 The front desk starts

Writing Docker Mirror image , You need to manage multiple services in one image at the same time , Need to use The front desk starts .supervisord The default startup method for is daemon, To configure the foreground startup, you need to modify the configuration file /etc/supervisord.conf in nodaemon The property value is true.

#  Replace... With script 
sed -i 's/nodaemon=false/nodaemon=true/g' /etc/supervisord.conf

The front start command is as follows

supervisord -c /etc/supervisord.conf

3、 Launch parameters

Usually, parameters are added to the configuration file , In some cases , Modifying the configuration file is tedious （ For example, the generated image ）, It is convenient to add runtime parameters to the command line .

Parameters	purpose	The default value is
-c	Specify the profile path	/etc/supervisord.conf
-s	supervisord Server listening URL	http://localhost:9001
-u	The user name used to authenticate with the server	user
-p	Password used to authenticate with the server	123

3、 ... and 、 Service management

Service management includes the management of main services and sub services ; Sub services are divided into single management and batch management （ grouping ） management .

1、 View the main service status

If you do not specify a sub service name , By default, you can re view the status of all sub services . Specify the sub service name , Just view the current sub service status .

#  View service status 
supervisorctl status

Master process management

# Common process management commands 
systemctl stop supervisord
systemctl start supervisord
systemctl restart supervisord

2、 Visual interface management

The visual interface adopts different strategies in different stages of software development , Project development and testing phase , To improve development efficiency , Often open the visual interface , When the project is developed and delivered , To ensure the security of the server , Usually close the visual interface .

Turn on Web The visualization service needs to add... In the configuration file inet_http_server modular .

（ One ） Single service management

Single service management refers to the management of a single sub service , All sub services are in the default group , But not shown .

1、 Refresh the sub service list

When adding a newly added sub service , need Refresh list , The main service provider can be included in the management scope .

（1）reload

If you do not specify a sub service name , Restart the list of all sub services by default . Specify the sub service name , Just restart the current sub service , Other services will not be affected .

All sub services refer to whether the configuration is modified or not , Will restart .

#  Refresh the list of services 
supervisorctl reload

（2）update

Restart all sub services with configuration changes （ Including new sub Services ）, The sub service with unchanged configuration will not be restarted .

#  Refresh the list of services 
supervisorctl update

2、 Process management runs

This method manages sub processes with smaller granularity .

#  Start the specified service 
supervisorctl start program_name
#  Stop the specified service 
supervisorctl stop program_name
#  Restart the specified service 
supervisorctl restart program_name
#  Start all services 
supervisorctl start all
#  Stop all services 
supervisorctl stop all

（ Two ） Group management

When there are associated sub Services , Group management can be adopted , Once you set up the grouping and add sub Services , Then the sub service name will change ： From the original program_name become group_name:program_name, such as redis:redis80

Group management needs to modify the main service configuration file .

1、 View the list of grouped sub Services

View the sub service list of the specified group name ,

#  View the list of grouped sub Services 
supervisorctl status group_name:

2、 Group subprocess management

Manage child processes in groups , Contains startup Services 、 Out of Service 、 Restart the service .

#  Start the service under the specified group name 
supervisorctl start group_name:
#  Stop the service under the specified group name 
supervisorctl stop group_name:
#  Restart the service under the specified group name 
supervisorctl restart group_name:

Notice the colon after the group name :.

3、 Group application

Delegate the process to Supervisor management , Grouping is convenient for a group of associated processes , such as Redis Master slave service 、ES colony 、ZK colony 、Kafka colony , They are a set of highly correlated sub services .

Four 、 Write the subprocess running configuration file

supervisor The main process configuration file is /etc/supervisord.conf

In the catalog /etc/supervisord.d Create a new to .ini Configuration file for suffix , Each configuration file represents a child process . Execute the following command , You can add a child process configuration .

Quick script Portal

（ One ） Parameter interpretation

1、directory

When a child process starts, the command cannot start from environment variable Read to , Use this parameter to switch to the specified working directory , Then run the entry command .

2、priority

When priority When the parameter is larger , The lower the priority .

3、environment

If the subapplication cannot get the system environment variable , Then you can explicitly indicate the path to a particular environment .

environment=JAVA_HOME=/usr/local/java

（ Two ） Log management

1、 View the child process log

The subprocess is Supervisor The corresponding operation log will be generated after management , Common are access logs and error logs .

;  Access log 
stdout_logfile=/var/log/park/access.log
;  Error log 
stderr_logfile=/var/log/park/error.log

Add log configuration in the child process configuration file , You can view the child process logs without using the visual interface . visualization Web The interface is convenient for viewing logs , The defect is that you cannot view the error log .

tail -f /var/log/park/access.log

Add parameters to the child process configuration file stdout_logfile and stderr_logfile The log file of will be automatically included in the main process log management , Automatic log rotation , No user intervention .

When the child process does not display the indicated log file path , The default log file exists in /tmp Under the path .

Supervisord Will be based on logfile_maxbytes and logfile_backups Rotation log , The former limits the size of a single log file , The latter limits the number of log backups . This configuration exists with the master configuration file , Non child process configuration file .

（ 3、 ... and ） Common component configurations

1、Nginx

cat <<EOF> /etc/supervisord.d/nginx.ini
[program:nginx]
directory=/usr/local/nginx/sbin
command=/usr/local/nginx/sbin/nginx -g 'daemon off;'
;  When the main service starts, it automatically starts the current sub service 
autostart=true
;  The sub service exits abnormally and restarts automatically 
autorestart=true
;  Sub service start time （ Try to be consistent with the time ）
startsecs=5
startretries=3
redirect_stderr=true
stdout_logfile=/usr/local/nginx/logs/access.log
stderr_logfile=/usr/local/nginx/logs/error.log
EOF

Child process commands must be The front desk operation By , Cannot execute in the background .

2、Redis

cat <<EOF> /etc/supervisord.d/redis.ini
[program:redis]
command=/usr/local/redis/bin/redis-server /usr/local/redis/conf/redis.conf --daemonize no
autostart=true
autorestart=true
startsecs=5
startretries=3
redirect_stderr=true
EOF

Child process commands must be The front desk operation By , Cannot execute in the background .

3、Nacos

cat <<EOF> /etc/supervisord.d/nacos.ini
[program:nacos]
command=sh /usr/local/nacos/bin/startup.sh -m standalone
autostart=true
autorestart=true
startsecs=8
startretries=3
redirect_stderr=true
environment=JAVA_HOME=/usr/local/java
priority=1
EOF

Child process commands must be The front desk operation By , Cannot execute in the background .

4、ElasticSearch

cat <<EOF> /etc/supervisord.d/es.ini
[program:es]
command=/usr/local/elasticsearch/bin/elasticsearch -Enetwork.host=127.0.0.1
user=es
password=es.123.456
umask=002
autostart=true
autorestart=true
startsecs=8
startretries=3
redirect_stderr=true
priority=100
EOF

5、ZooKeeper

cat <<EOF> /etc/supervisord.d/zk.ini
[program:zk]
command=/usr/local/zookeeper/bin/zkServer.sh start-foreground
autostart=true
autorestart=true
startsecs=5
startretries=3
redirect_stderr=true
priority=100
EOF

6、Jenkins

cat <<EOF> /etc/supervisord.d/jenkins.ini
[program:jenkins]
command=/usr/local/jenkins/bin/catalina.sh run
autostart=true
autorestart=true
startsecs=8
startretries=3
redirect_stderr=true
priority=100
EOF

7、Kafka

cat <<EOF> /etc/supervisord.d/kafka.ini
[program:kafka]
command=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
autostart=true
autorestart=true
startsecs=8
startretries=3
redirect_stderr=true
priority=100
EOF

8、Kibaba

cat <<EOF> /etc/supervisord.d/kibana.ini
[program:kibana]
command=/usr/local/kibana/bin/kibana -H 0.0.0.0
autostart=true
autorestart=true
startsecs=8
startretries=3
redirect_stderr=true
EOF

9、MongoDb

cat <<EOF> /etc/supervisord.d/mongo.ini
[program:mongo]
command=/usr/local/mongo/bin/mongod --config=/usr/local/mongo/conf/config.yml
autostart=true
autorestart=true
startsecs=8
startretries=3
redirect_stderr=true
priority=100
EOF

Original address