Deep understanding of deepsea and salt deployment tools – storage6

First of all, let's pass the previous articles , A complete set of Ceph colony , To use salt Tool automation colony Know something about , Now we will explain the deployment method in detail

SUSE Enterprise Storage Deployment way

storage4 In the way ：

• ceph-deploy Tools , The standard ceph Script deployment , Suitable for small and medium-sized storage colony
• crowbar Tools , Deploy SUSE Openstack Standard tools for

storage5/6 Adopted by ：

• DeepSea (Salt), Lightweight , agility , flexibility , Elastic deployment

    In the past, we deployed in a community way ceph-deploy or crowbar Tool building , this 2 Each tool deployment has certain limitations , Not suitable for large storage colony Deploy , agility 、 Too much flexibility . So from 2018 Year begins ,SUSE Enterprise Storage 5 Abandoning ceph-deploy / crowbar Cluster deployment tools , Introduction DeepSea How to deploy , This method is more lightweight , High speed intercommunication , agility , flexibility , It is suitable for deploying cluster systems in various scenarios , It's also Ceph The trend of product deployment .

One 、DeepSea brief introduction

      DeepSea Designed to save administrator time , Let them be confident about Ceph The cluster performs complex operations .Ceph It is a highly configurable software solution . It improves the freedom and responsibility performance of system administrators . Lowest Ceph The setting can well meet the purpose of demonstration , But I can't show Ceph Excellent function when dealing with a large number of nodes .DeepSea Will collect and store relevant single The server Relevant data , For example, address and device name . For such as Ceph Of Distributed Storage system , Hundreds of such items may need to be collected and stored . The process of collecting information and manually entering data into the configuration management tool is very energy-consuming , And it's easy to make mistakes . Get ready The server 、 Collect configuration information as well as configuration and deployment Ceph The steps to be performed are roughly the same . however , This approach cannot solve the need to manage independent functions . In daily operation , You must take pains to add hardware to a given function , And calmly remove the hardware .DeepSea These needs are addressed through the following strategies ：DeepSea Multiple administrator decisions can be merged into a single file . These decisions include cluster assignment 、 Role assignment and profile assignment . Besides ,DeepSea Each group of tasks will be collected to form a simple goal . Each goal is a stage ：

  About DeepSea Official information ：

GitHub  link ： https://github.com/SUSE/DeepSea/wiki

Copy

  Two 、SaltStack brief introduction

SaltStack It's a The server Infrastructure centralized management platform , Configuration management 、 Remote execution 、 Monitoring and other functions , It can be generally understood as a simplified version of puppet And the enhanced version of func.SaltStack be based on Python Language implementation , Combine lightweight message queuing （ZeroMQ） And Python Third-party module （Pyzmq、PyCrypto、Pyjinjia2、 python-msgpack and PyYAML etc. ） structure .

By deploying SaltStack Environmental Science , We can execute commands in batches on thousands of servers , Centralized management of configuration according to different business characteristics 、 Distribute documents 、 Collect server data 、 Operating system foundation and software package management, etc ,SaltStack It's the operation and maintenance personnel who improve their work efficiency 、 A sharp tool for standardizing business configuration and operation .

characteristic ：

（1） Simple deployment 、 convenient ;

（2） Support most UNIX/Linux And Windows Environmental Science ;

（3） Master slave centralized management ;

（4） Simple configuration 、 Powerful 、 Extensibility is strong ;

（5） Main control end （master） And the controlled end （minion） Certificate based authentication , Safe and reliable ;

（6） Support API And custom modules , It can be done by Python Easily expand .

SaltStack: Pillar and Grains Detailed explanation

SatlStack Remote execution

 1、 Remote execution

• The goal is （Targeting）
• modular （Module）
• return （return）

2、 The goal is

（1） and Minion ID of , Need to use Minion ID

• wildcard
• Regular expressions
• list

Wildcard mode ：

# salt 'node00[1-3]'.example.com cmd.run 'w'
node002.example.com:
     13:06:12 up  4:05,  0 users,  load average: 0.28, 0.09, 0.02
    USER     TTY      FROM             [email protected]   IDLE   JCPU   PCPU WHAT
node001.example.com:
     13:06:12 up  4:05,  0 users,  load average: 0.23, 0.13, 0.05
    USER     TTY      FROM             [email protected]   IDLE   JCPU   PCPU WHAT
node003.example.com:
     13:06:12 up  4:05,  0 users,  load average: 0.24, 0.09, 0.02
    USER     TTY      FROM             [email protected]   IDLE   JCPU   PCPU WHAT

Copy

Regular expressions

# salt -E 'node(001|002).example.com' test.ping        
node002.example.com:
    True
node001.example.com:
    True

Copy

（2） and Minion ID irrelevant , Not involved Minion ID

• subnet /  IP Address
• Grains
• Pillar
• Compound matchers ( Composite match )
• Node groups ( section �� Group )
• Batching execution （ Batch execution ）

Grains The way ,OS yes SUSE

# salt -G 'os:SUSE' test.ping 
node001.example.com:
    True
node002.example.com:
    True
node003.example.com:
    True
node004.example.com:
    True
admin.example.com:
    True

Copy

obtain Pillar , Then designate pillar The way

# salt 'admin*' pillar.items 
admin.example.com:
    ----------
    available_roles:
        - storage
        - admin
        - mon
        - mds
        - mgr

     .....

    roles:
        - master
        - admin
        - prometheus
        - grafana
    time_server:
        admin.example.com

# salt -I 'roles:grafana' test.ping             
admin.example.com:
    True

Copy

3、 Module details

1000+ Module , At present, it is increasing , saltstack Module link  

admin:~ # salt 'node001*' network.arp  
node001.example.com:
    ----------
    00:0c:29:33:70:2d:
        192.168.2.42
    00:0c:29:33:70:37:
        192.168.3.42
    00:0c:29:ae:44:51:
        172.200.50.39
    00:0c:29:ae:44:5b:
        192.168.2.39
    00:0c:29:d3:ba:15:
        192.168.2.41
    00:0c:29:d3:ba:1f:
        192.168.3.41
    00:0c:29:e0:5c:8c:
        192.168.2.43
    00:0c:29:e0:5c:96:
        192.168.3.43

Copy

admin:~ # salt 'node001*' network.interface eth1 
node001.example.com:
    |_
      ----------
      address:
          192.168.2.40
      broadcast:
          192.168.2.255
      label:
          eth1
      netmask:
          255.255.255.0

Copy

3、 ... and 、 Configure and customize

policy.cfg file

/srv/pillar/ceph/proposals/policy.cfg The configuration file is used to determine the role of a single cluster node . for example , Which node acts as OSD, Or which node acts as the monitor node . Please edit policy.cfg , To reflect the required cluster settings . Paragraphs in any order , But the content of the included line will overwrite the matching key in the content of the previous line .

1、policy.cfg The template of

• Can be in /usr/share/doc/packages/deepsea/examples/ Multiple examples of complete policy files found in the directory .
• Generally, we choose the template based on role definition

# ll /usr/share/doc/packages/deepsea/examples/
total 12
-rw-r--r-- 1 root root 329 Aug  9 16:00 policy.cfg-generic
-rw-r--r-- 1 root root 489 Aug  9 16:00 policy.cfg-regex
-rw-r--r-- 1 root root 577 Aug  9 16:00 policy.cfg-rolebased

Copy

2、 Cluster assignment

To include all controlled ends , Please add the following lines ：

cluster-ceph/cluster/*.sls

Copy

Add a specific controlled end to the white list , Please run the following command ：

cluster-ceph/cluster/abc.domain.sls

Copy

Add a group of controlled terminals to the white list , You can use wildcards ：

cluster-ceph/cluster/mon*.sls

Copy

Add the controlled end to the blacklist , It can be set to unassigned ：

cluster-unassigned/cluster/client*.sls 

Copy

 3、policy.cfg Example

Here is a basic policy.cfg Sample file ：

 1 vim /srv/pillar/ceph/proposals/policy.cfg 
 2 
 3 ## Cluster Assignment
 4 cluster-ceph/cluster/*.sls
 5 
 6 ## Roles
 7 # ADMIN
 8 role-master/cluster/admin*.sls
 9 role-admin/cluster/admin*.sls
10 
11 # Monitoring
12 role-prometheus/cluster/admin*.sls
13 role-grafana/cluster/admin*.sls
14 
15 # MON
16 role-mon/cluster/node00[1-3]*.sls
17 
18 # MGR (mgrs are usually colocated with mons)
19 role-mgr/cluster/node00[1-3]*.sls
20 
21 # COMMON
22 config/stack/default/global.yml
23 config/stack/default/ceph/cluster.yml
24 
25 # Storage
26 role-storage/cluster/node00*.sls
27 
28 # MDS
29 role-mds/cluster/node001*.sls
30 
31 # IGW
32 role-igw/stack/default/ceph/minions/node002*.yml
33 role-igw/cluster/node002*.sls
34 
35 # RGW
36 role-rgw/cluster/node00[3-4]*.sls

Copy

（1） The first 3-4 That's ok ：

• The indication is in Ceph The cluster contains all controlled ends . If you don't want to be in Ceph The cluster contains some controlled ends , Please use ：

cluster-unassigned/cluster/*.sls
cluster-ceph/cluster/node00*.sls

Copy

• Mark all controlled ends as unassigned .
• Coverage and “node00*.sls” Matched controlled end , And assign it to Ceph to cluster around .

（2） The first 7-9 That's ok

• Specify the hostname as admin The host node of has "master" and “admin” role

（3） The first 11-13 That's ok

• Specify to deploy Dashboard Nodes of the visual interface

（4） The first 15-16 That's ok

• The controlled node node001 node002 node003 Set to MON node

（5） The first 18-19

• The controlled node node001 node002 node003 Set to MGR node , This setting must follow MON Same settings

 （6） The first 21-23 That's ok

• Express acceptance fsid and public_network And other general configuration parameters

 （7） The first 25-36 That's ok

• Controlled end “node00*” Will have storage  IGW RGW MDS role

Drive Group

Four 、DeepSea Deployment way

  Through the architecture diagram , We can clearly understand , install Storage6 As long as the management node is installed satl-master and salt-minion, other OSD Node installation salt-minion, And all of minion All point to salt-master IP Address or host name （ Recommended public Segment address ）, And then execute deepsea Of 4 Stage commands “salt-run state.orch ceph.stage.X” It can be easily built .

DeepSea Stage description

Stage 0 — Get ready ： At this stage , All required updates will be applied , And may reboot your system .

Stage 1 — Find out ： At this stage , adopt Salt Installed on the client salt minion, All hardware in the cluster will be detected , And collect Ceph Configure the required information .

Stage 2 — To configure ： You need to prepare the configuration data in a specific format .（ Definition salt Of pillar）

Stage 3 — Deploy ： The creation contains the necessary Ceph Basic of service Ceph to cluster around . List of necessary services

Stage 4 — service ： Can be installed at this stage Ceph Other functions of , for example iSCSI、RADOS Gateway and CephFS. Each of these functions is optional .

Stage 5 — Removal stage ： This stage is not required , During initial setup , This stage is usually not required . At this stage , The role of the controlled end and the cluster configuration will be removed . If you need to remove a storage node from the cluster , You need to run this stage .

DeepSea CLI

DeepSea One is also provided CLI Tools , For users to monitor or run the stage , At the same time, the execution progress is visualized in real time . The following two modes are supported to visualize the execution progress of the phase ：

• Surveillance mode ： Visualize the... Sent in another terminal session salt-run Triggered by the command DeepSea Implementation progress of phase .
• Independent mode ： function DeepSea Stage , And provide corresponding real-time visualization effect when the constituent steps of this stage are executed .

Monitoring mode Monitor Mode

  The program monitoring provides a detailed , Real time visual operation behavior , When running salt-run state.orch when , Monitor what runs during execution

# deepsea monitor

Copy

Independent mode Stand-alone Mode

# deepsea stage run stage-name
# salt-run state.orch ceph.stage.0
# deepsea stage run ceph.stage.0

Copy

Deepsea Help information

# man deepsea-monitor
NAME
deepsea-monitor - Starts the DeepSea stage execution progress monitor.
# man deepsea-stage run

Copy