Summary of the latest remote deployment O & M tools

One ．Puppet 

Reprint https://baike.baidu.com/item/puppet/5109503?fr=aladdin

puppet It's a kind of Linux、Unix、windows Platform centralized configuration management system , Use own puppet Description Language , Manageable profiles 、 user 、cron Mission 、 software package 、 system service etc. .puppet Call these system entities resources ,puppet Our design goal is to simplify the management of these resources and proper Dealing with dependencies between resources .

puppet use C/S Star like structure , All clients interact with one or more servers . Per client cycle （ The default is half an hour ） Send request to server , Get its latest configuration information , Ensure synchronization with the configuration information . Every puppet Client every half an hour ( You can set ) Connect to the primary server , Download the latest profile , And configure the client strictly according to the configuration file . After configuration ,puppet The client can feedback a message to the server . If something goes wrong , A message will also be fed back to the server .

utilize Puppet To automatically generate files defined on the server for the client .

from master End to end multiple Client End distribution files .

Two ．Ducter

Reprint http://www.ducter.net/

Ducter It is device oriented 、 Distributed command platform of the product , Provide command channels for third parties to collect or manipulate information about equipment and products . As an open source product operation and maintenance management system that has been put into use in many companies in China . Ducter It is the product management platform , Realize the management of any product 、 Monitoring and user-defined publishing and rollback , Provide the third party with real-time data of product operation status and event notification .Ducter It integrates device management 、 Product management 、 Service launch and server operation are an integrated operation and maintenance management system . The system is divided into Dcuter-Server/Ducter-Web Two parts .

Ducter The goal is ：

     Product management system

     product -> service -> Service pool （ Service pool properties ）-> Service pool devices -> equipment .

     Automatic product release system

     visualization 、 Customize 、 Concurrent 、 Automatically .

     Applicable to any product type ：c/c++、python、java、php etc. .

     Product control system

     Start of remote operation product service 、 stop it 、 Restart, etc. .

     Real time information collection and notification system for product operation and use resources .

     Collect product operation status and resource usage information in real time . Active real-time event notification when the status changes .

     Distributed command platform .

     Ordered action Implemented by user scripts ：shell、python、ruby etc. .

ducter Provides the environment for the script to run .

Operation and maintenance positioning ：

Relationship with other systems
And puppet、cfengine And so on :
    puppet、cfengine： Configuration oriented
    ducter： Facing interaction , But with puppet Equal integration .
And Mesos The relationship between :
    Mesos： Resource management 、 Scheduling platform .
    Ducter： Distributed task and command platform , Service management platform , Resource and service scheduling is implemented by a third party .

3、 ... and ．Cfengine 

Reprint https://baike.baidu.com/item/Cfengine/1718297?fr=aladdin

cfengine（ Configure the engine ） It's a kind of UNIX Management tools , The goal is to automate simple management tasks , Make difficult tasks easier .Cfengine Suitable for managing various environments , From one host To tens of thousands of hosts fleet All can use . To 2.2 Up to version , The largest installation we know of for General Administration fleet about 20,000 platform .

Cfengine There are many ways to The system configuration And maintenance , Including the following ：

  Complete the post installation task , For example, configure network interface information .

  Edit system configuration files and other files .

  Establish a signal connection .

  test 、 Correct license and ownership of documents .

  Delete useless files .

  Compress the selected file .

  Distribute files across the network .

  Automatically mount NFS file system .

  Check the existence and integrity of important files and file systems .

  Execute commands and scripts .

  Apply security related patches and corrections to similar systems .

  Manage system server processes .

Purpose ： Is to implement policy based Configuration Management . From the perspective of practical application , It means Cfengine It can simplify the system configuration and maintenance tasks to the greatest extent .

Example ： To optimize a particular system , Users no longer need to use Perl Or other user habits shell To compile a program to perform each requirement , In its place , Users can write a simpler policy to describe what they want host How to be configured .Cfengine Based on these descriptions, the software can determine which execution methods and / Or remedy is something that needs to be done . These policy descriptions can also be used to ensure that the system configuration can be as follows System administrator The same is kept as hoped .

Four ．Func  

Reprint ：https://www.cnblogs.com/MacoLee/p/5776869.html

Func , The purpose is to design and develop a basic system management framework to solve this series of unified management and monitoring problems . A tool that can effectively simplify the management of multi server systems , Easy to learn , Easy to use , extensible , And it's powerful .

Func The characteristics of ：

1、Func You can manage any number of servers or any number of server groups at one time on the master computer .

2、Func be based on Certmaster（https://fedorahosted.org/certmaster/） Established Master-Slaves Master-slave SSL Certificate control system , Certificates can be automatically distributed to all controlled servers .

3、Func The command line can directly send remote commands or obtain data remotely .

4、Func Developers have completed the development of most common task modules , Including command execution module 、 File transfer module 、IPtables modular 、 View the hardware information module 、Mount modular 、 Process module 、 Service module 、 Restart the system module .

5、 Can pass Func Provided Python API Easily write extension modules , To achieve specific function expansion . And anything Func What the command line can do , Can pass. API Programming to realize .

6、Func Communication based XMLRPC and SSL Standard agreement .

5、 ... and 、ansible

Reprint http://blog.51cto.com/sofar/1579894

1、 brief introduction

ansible It is a new automatic operation and maintenance tool , be based on Python Development , A collection of operations and maintenance tools （puppet、cfengine、chef、func、fabric） The advantages of , Realized batch system configuration 、 Batch program deployment 、 Batch run command and other functions .ansible It works based on modules , It does not have the ability to deploy in bulk . What really has a batch deployment is ansible The module being run ,ansible It just provides a framework . It mainly includes ：

(1)、 Connect plug-ins connection plugins： Responsible for communication with monitored end implementation ;

(2)、host inventory： Specifies the host of the operation , Is a host that defines monitoring in a configuration file ;

(3)、 Various modules core modules 、command modular 、 Custom module ;

(4)、 With the help of plug-ins to complete logging mail and other functions ;

(5)、playbook： When a script performs multiple tasks , It is not necessary to allow nodes to run multiple tasks at once .

2、 Overall framework

 

Ansible from 5 Component composition ：

    Ansible： The core

    Modules： Include Ansible The core module and custom module  

    Plugins： Complete the supplement of module function , Including connection plug-ins 、 Email plug-ins, etc  

    Playbooks： Script , Definition Ansible Multitask profile , from Ansible Automatic execution

    Inventory： Definition Ansible Manage the list of hosts

3、 characteristic

(1)、no agents： There is no need to install any client on the managed host ;

(2)、no server： Serverless side , Run the command directly when used ;

(3)、modules in any languages： Module based work , Modules can be developed in any language ;

(4)、yaml,not code： Use yaml Language customization script playbook;

(5)、ssh by default： be based on SSH Work ;

(6)、strong multi-tier solution： Multi - stage command can be achieved .

4、 advantage
(1)、 Lightweight , No client installation is required agent, update , Just one update on the operator ;
(2)、 Bulk task execution can be scripted , And it can be executed without being distributed remotely ;
(3)、 Use python To write , Easier to maintain ,ruby Syntax is too complex ;
(4)、 Support sudo.

5、 Task execution process

6、 ... and ．CheungSSH

Reprint ：https://www.cnblogs.com/cheungssh/p/5025854.html

CheungSSH Automatic operation and maintenance tools , The batch management Linux The server , Automatic monitoring server , Automatic deployment Tomcat,Apache Software , management GitHub, Batch execution command , Transfer documents , Fully automated process . Planning task function , Audit of authority , Control of authority , And operation records .

Function is introduced ：

 

Run the architecture ：

7、 ... and ． Open operation and maintenance management system  OSA

Reprint ：http://www.osapub.com/index.html

OSA The open source operation and maintenance supervision and management platform is composed of OSA The development team , For the current server operation and maintenance 、 An open source for monitoring and alarm 、 Easy to use 、 A practical cross platform server operation and maintenance management platform ;OSA The O & M management platform not only meets the O & M needs , It also guides users in knowledge management , Process specification , Daily study, etc .

The main function : Resource monitoring , Original real-time monitoring , Intelligently analyze the memory and memory of the server process CPU Occupancy rate and other information .

Mail alarm : Flexible alarm settings , Service by mail is abnormal 、 Equipment downtime and other alarm notifications .

Single machine operation and maintenance : The routine maintenance and management of the server can be carried out simply , Support login and management through smart phones .

The batch operation : Batch operation combined with planned task function , Make the operation more automatic , One click deployment of thousands of servers .

Fault records : Powerful fault recording and classification function , According to the historical records, the sudden failure can be well analyzed .

The knowledge base : Rich knowledge base content , Help you solve problems , Find answers to questions in daily operation and maintenance .

Operational analysis : Analyze the server graphically , Let operators know the status information of basic equipment like the back of their hands .

Account management : Through a reasonable allocation of authority , Set different roles for different users , Convenient and flexible management .

notes ： Last update time 2012 year 09 month 10 Japan

8、 ... and ． Introduction to domestic open source operation and maintenance management tools TriAquae

Reprint ：https://blog.csdn.net/enweitech/article/details/48177835

IT Batch operation and maintenance management software -- TriAquae  

1. Server group management

2. Batch command execution

3. Batch file transfer 、 receive

4. Server running status fetching 、 analysis

5. Server hardware information collection

6. Scheduled tasks

7. Mass software deployment

8. System password change

reason ： convenient , You can easily make thousands of servers do the same thing as you can manage a local server , such as  uname –a Check the system version , Execute a script to collect logs , Modify a file on all servers ……, Of course it can't do that ……; It's so light , As long as your machine is equipped with Python, Then you can start using it , It does not require you to install a database , No need PHP、 Unwanted JAVA virtual machine ,only  Python. You don't even need to install the client on the client , Just support SSH, Of course you have to know the user name and password ; It's open source. , Free of charge ……

Nine ．SaltStack 

Reprint ：http://www.360doc.com/content/16/0705/10/11991_573220489.shtml ;http://blog.51cto.com/lwm666/2068833

1、 Introduce

SaltStack It is a centralized management platform for server infrastructure , Configuration management 、 Remote execution 、 Monitoring and other functions , It can be generally understood as a simplified version of puppet And the enhanced version of func.SaltStack be based on Python Language implementation , Combine lightweight message queuing （ZeroMQ） And Python Third-party module （Pyzmq、PyCrypto、Pyjinjia2、python-msgpack and PyYAML etc. ） structure , And use RAS Key Way to confirm identity , Transmission uses AES encryption , Higher security .

By deploying SaltStack Environmental Science , We can execute commands in batches on thousands of servers , Centralized management of configuration according to different business characteristics 、 Distribute documents 、 Collect server data 、 Operating system foundation and software package management, etc ,SaltStack It's the operation and maintenance personnel who improve their work efficiency 、 A sharp tool for standardizing business configuration and operation .

2、 Comparison of automation tools

1）Puppet

  advantage ： Old classic automatic configuration software 、 Widely applied 、 Powerful  

  shortcoming ： Cannot batch , Use Ruby and DSL Language development , It is difficult for companies without relevant technology stacks to maintain and redevelop , The threshold is quite high ; Lack of error checking and reporting ; The deadliest thing is Puppet The execution of is unordered , It is difficult to handle some projects with configuration or deployment sequence

 2）Chef

  advantage ： It's also powerful （ There is less information ）

  shortcoming ： The official documents are vague and simple ; Execution is not push Real time execution , It is client Terminal timing acquisition server End data

 3）Ansible

  advantage ： The community is young and energetic ,play-book Use yaml, concise , No client is required client Support

  shortcoming ： Use SSH Configuration management , Theoretically slower than Saltstack, For not demanding execution efficiency ,Ansible It's also a good choice

 4）Saltstack

  advantage ： Functions are not limited to configuration management 、 Use Python Development 、Grain/Pillar Support 、 Customize moudles

  shortcoming ： The community is relatively young , Less documentation （ In fact, the official documents have explained it in depth ...）

3、Saltstack There are three modes of operation of

·  Local

·  Master/Minion

·  Salt SSH

Saltstack The most traditional mode of operation is C/S Pattern , You need to install on the managed node Minion client ; Actually Saltstack Also support SSH The way , No installation required Agent, adopt SSH Implement management

4、 characteristic
(1)、 Simple deployment 、 convenient ;
(2)、 Support most UNIX/Linux And Windows Environmental Science ;
(3)、 Master slave centralized management ;
(4)、 Simple configuration 、 Powerful 、 Extensibility is strong ;
(5)、 Main control end （master） And the controlled end （minion） Certificate based authentication , Safe and reliable ;

(6)、 Support API And custom modules , It can be done by Python Easily expand .

 

5、Saltstack The three main functions at present

·   Remote execution

·   Configuration Management

·   Cloud management

6、SaltStack How it works

SaltStack use C/S Structure to manage the operation and configuration of servers in the cloud ring .SaltStack client （Minion） When it starts , Will automatically generate a set of keys , Include private key and public key . Then send the public key to the server , The server verifies and accepts the public key , To establish a reliable and encrypted communication connection . At the same time through the message queue ZeroMQ Establish a message publishing connection between the client and the server . 

Technical terms ：

Minion yes SaltStack Client installation components that need to be managed , Will take the initiative to connect Master End , And from Master Get resource status information , Synchronize resource management information .

Master As the control center running on the host server , be responsible for Salt Command running and resource state management .

ZeroMQ Is an open source message queuing software , Used in Minion End and Master The end establishes the system communication bridge .

Daemon It's a daemons running in every member , It is responsible for releasing messages and monitoring communication ports .

 

SaltStack Communication schematic diagram

       

Schematic diagram of operation execution

Schematic illustration ：

Minion yes SaltStack Client installation components that need to be managed , Will take the initiative to connect Master End , And from Master Get resource status information , Synchronize resource management information .

Master As the control center running on the host server , be responsible for Salt Command running and resource state management .

Master To execute an instruction on a queue and send it to each Minions To carry out , And return the result .

  SaltStack Deployment architecture

 

Ten ．shipit-deploy

Reprint ：https://cnodejs.org/topic/584545bd4c17b38d354363af

The goal is ： One click deployment of multiple servers ; Rollback multiple servers with one click ; Local operation , You do not need to log in to the server ; Easy to customize and expand , Fully automated processes .

Why? shipit You can operate the remote server without logging in ？

This is because it uses ssh Command to operate the remote server , Students who don't know can feel it by typing an order at the terminal ：[email protected]_host “pwd; ls”.

Why can you deploy to multiple machines with one click ？

simply , Each of the server Will create a corresponding object , Every time a remote command is executed , Will traverse all objects to execute .

Custom extensions ： The code has been uploaded to the server , The next step is to start the project , For example, we use pm2 To start up , Before starting, you need to execute npm install To install project dependencies . Can these processes also be added to automated processes ？

Certainly. ,shipit-deploy A series of tasks will be performed during the execution of , Every time you finish a task , Metropolis emit An event , As long as we monitor this event, we can do what we want to do .

Use with scripts ： that , Is it possible to customize the plug-in without using script files at all , not always , Script files have their benefits , We can also easily add your existing scripts to the automated process .

Since we can use shipit-deploy To synchronize code , Use plug-ins and scripts to complete additional work such as startup , that shipit-deploy It's not just about deploying nodejs Project , Projects in other languages can also be deployed automatically according to this routine . Some people use it successfully shipit-cli/shipit-deploy Implemented a php Automated deployment of the project . In order to minimize the impact of non nodejs Project , It's better to shipit-cli and shipit-deploy Global installation .

11、 ... and .Walle

Reprint ：https://tech.meituan.com/android-apk-v2-signature-scheme.html

Walle One web Deploy system tools , Maybe it's also a continuous release tool , Simple configuration 、 Functional perfection 、 Smooth interface 、 Open the box ！ Support git、svn version management , Support a wide variety of web Code release , Static HTML、 dynamic PHP、 It needs to be compiled JAVA etc. , User authority management , Various custom advanced tasks . One click online 、 Roll back .

Project address ：walle- Tile power , It has been open source for two months , At present, it has supported the online deployment and use of more than ten enterprises , Update one version every week , Continue to bring new features .

walle Component relationships 、 principle ：walle Deployed on a host to provide a web UI, It is convenient for users to update code independently and deploy it to the target cluster .Walle It's a run in LNMP(LAMP) Upper PHP service , The host establishes trust with the target cluster , By manipulating the bash Command to synchronize code 、 Customize advanced tasks .

To configure ： User's items can be customized git/svn Code hosting address 、 The host machine 、 Target fleet 、 Advanced tasks, etc . After the configuration is completed, the current configuration can be copied quickly 、 Check whether the configuration is normal 、 And the audit administrators and members of the project .

walle Continuous deployment and Git Flow Think alike , Deploy different branches in different environments . Divide three environments ： Test environment 、 Pre release environment 、 Production environment . Recommended branch deployment for test environment and pre release environment , Recommended production environment tag Deploy . Of course , This is more appropriate milestone The development way , If you are iterating over multiple versions quickly every day , hit tag The cost is a little high .

Deployment is to pull code from a host , Compile 、 After the configuration , Distribute to the target cluster , Perform relevant target cluster tasks . The deployment process is split into the following 6 A link , among 1-5 For the host computer ,6 Execute on the target cluster .

1. jurisdiction 、 Catalog check , Open up a separate online space for parallel publishing , Prevent code pollution during simultaneous deployment

2.pre-deploy Mission , Some operation tasks before checking out the code , Such as environmental inspection

3. Code from git/svn Check out... From the version library

4.post-deploy Mission , After checking out the code, operate the task , Such as java Of mvn compile ,php Of composer Plug in installation

5. The code kept in the independent space will be synchronized to a version Library of the target cluster

6. Full update ： When all the machines have been distributed , Get started pre-release Mission （java Temporarily Out of Service ）、 Switch version soft chain 、post-release Mission （ java Start the service ）

To ensure the integrity of the service , Synchronize the code first , After switching services . Every time a deployment release is made, the version record will be kept , Once the version goes online , Rollback can be completed instantly . Configurable maximum number of online versions retained , Expired versions will be deleted , At the same time, the deleted version cannot be rolled back . For compilation 、 Customize multi task assistance , Configurable front 、 Post operation custom task ; At the same time, some preset variables are provided （{WORKSPACE} The current independent space of the host 、 Target machine webroot,{VERSION} Version Library Directory ） It is convenient for users to operate custom tasks .

Go online until the target cluster is completed , If any phase fails, it will be rolled back , Need to go online again , At the same time, it will prompt the user with the error message .

 

---------------------
author ： Code help
source ：CSDN
original text ：https://blog.csdn.net/ITLearnHall/article/details/80651209
Copyright notice ： This is the author's original article , Please attach a link to the blog post ！
Content analysis By：CSDN,CNBLOG One click reprint plugin for blog posts