Return and job management of saltstack

1. SaltStack Of components return

return Components can be understood as SaltStack System execution Minion The returned data is stored or returned to other programs , It supports a variety of storage methods , For example, use MySQL、MongoDB、Redis、Memcache etc. , adopt return We can SaltStack Record each operation of , Provide data source for future log audit . At present, the government has supported 30 Kind of return Data storage and interface , We can easily configure and use it . Of course, it also supports its own definition of return, Self defined return By python To write the . After selecting and configuring the return after , Just in salt Specify after the command return that will do .

Environmental statement ：

Host name	Host type	IP	System	Applications that need to be installed
master	Control machine	192.168.91.135	CentOS8	salt-master salt-minion python3-PyMySQL
node2	Controlled machine	192.168.91.134	CentOS8	salt-minion python3-PyMySQL mariadb-server、mariadb
node3	Controlled machine	192.168.91.138	CentOS8	salt-minion python3-PyMySQL

//  View all return list 
[[email protected] ~]# salt '*' sys.list_returners
[[email protected] ~]# salt '*' sys.list_returners
master:
    - carbon
    - couchdb
    - etcd
    - highstate
    - local
    - local_cache
    - mattermost
    - multi_returner
    - pushover
    - rawfile_json
    - slack
    - slack_webhook
    - smtp
    - splunk
    - sqlite3
    - syslog
    - telegram
node2:
    - carbon
    - couchdb
    - etcd
    - highstate
    - local
    - local_cache
    - mattermost
    - multi_returner
    - pushover
    - rawfile_json
    - slack
    - slack_webhook
    - smtp
    - splunk
    - sqlite3
    - syslog
    - telegram
node3:
    - carbon
    - couchdb
    - etcd
    - highstate
    - local
    - local_cache
    - mattermost
    - multi_returner
    - pushover
    - rawfile_json
    - slack
    - slack_webhook
    - smtp
    - splunk
    - sqlite3
    - syslog
    - telegram

1.1 return technological process

return Is in Master End trigger task , then Minion After accepting the processing task, directly contact return Connect to the storage server , And then the data return Save to storage server . Be sure to pay attention to this , Because this process is Minion End operation storage server , So make sure that Minion The configuration of the client and the dependent package are correct , This means that we will have to be in every Minion Install the specified on return Mode dependent package , If used Mysql As return storage , Then we will be on each Minion Installation on python-mysql modular .

1.2 Use mysql As return storage

In all minion Installation on Mysql-python modular

[[email protected] ~]# yum list all|grep -i 'mysql'|grep python
Failed to set locale, defaulting to C.UTF-8
python3-PyMySQL.noarch                                 0.10.1-2.module_el8.5.0+761+faacb0fb                   @appstream       
python2-PyMySQL.noarch                                 0.8.0-10.module_el8.5.0+743+cd2f5d28                   appstream        
python38-PyMySQL.noarch                                0.10.1-1.module_el8.5.0+742+dbad1979                   appstream        
python39-PyMySQL.noarch                                0.10.1-2.module_el8.5.0+738+dc19af12                   appstream 

[[email protected] ~]# salt '*' pkg.install python3-PyMySQL
node2:
    ----------
    python3-PyMySQL:
        ----------
        new:
            0.10.1-2.module_el8.5.0+761+faacb0fb
        old:
    python3-cffi:
        ----------
        new:
            1.11.5-5.el8
        old:
    python3-cryptography:
        ----------
        new:
            3.2.1-5.el8
        old:
    python3-ply:
        ----------
        new:
            3.9-9.el8
        old:
    python3-pycparser:
        ----------
        new:
            2.14-14.el8
        old:
master:
    ----------
    python3-PyMySQL:
        ----------
        new:
            0.10.1-2.module_el8.5.0+761+faacb0fb
        old:
    python3-cffi:
        ----------
        new:
            1.11.5-5.el8
        old:
    python3-cryptography:
        ----------
        new:
            3.2.1-5.el8
        old:
    python3-ply:
        ----------
        new:
            3.9-9.el8
        old:
    python3-pycparser:
        ----------
        new:
            2.14-14.el8
        old:
node3:
    ----------
    python3-PyMySQL:
        ----------
        new:
            0.10.1-2.module_el8.5.0+761+faacb0fb
        old:
    python3-cffi:
        ----------
        new:
            1.11.5-5.el8
        old:
    python3-cryptography:
        ----------
        new:
            3.2.1-5.el8
        old:
    python3-ply:
        ----------
        new:
            3.9-9.el8
        old:
    python3-pycparser:
        ----------
        new:
            2.14-14.el8
        old:

Deploy one mysql The server acts as a storage server , Right here node2 Deploy on this host

[[email protected] ~]# yum -y install mariadb-server mariadb
 The installation process is omitted ...

//  Start the service 
[[email protected] ~]# systemctl enable --now mariadb
Created symlink /etc/systemd/system/mysql.service → /usr/lib/systemd/system/mariadb.service.
Created symlink /etc/systemd/system/mysqld.service → /usr/lib/systemd/system/mariadb.service.
Created symlink /etc/systemd/system/multi-user.target.wants/mariadb.service → /usr/lib/systemd/system/mariadb.service.

//  Create database and table structure 
[[email protected] ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9
Server version: 10.3.28-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE DATABASE `salt`
    ->  DEFAULT CHARACTER SET utf8
    ->  DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.012 sec)

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| salt               |
+--------------------+
4 rows in set (0.015 sec)

//  Access to database 
MariaDB [(none)]> use salt;
Database changed

MariaDB [salt]> DROP TABLE IF EXISTS `jids`;
Query OK, 0 rows affected, 1 warning (0.016 sec)

MariaDB [salt]> CREATE TABLE `jids` (
    ->  `jid` varchar(225) NOT NULL,
    ->  `load` mediumtext NOT NULL,
    ->  UNIQUE KEY `jid` (`jid`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.019 sec)

MariaDB [salt]> show tables;
+----------------+
| Tables_in_salt |
+----------------+
| jids           |
+----------------+
1 row in set (0.001 sec)

MariaDB [salt]> CREATE TABLE `salt_returns` (
    ->   `fun` varchar(50) NOT NULL,
    ->   `jid` varchar(255) NOT NULL,
    ->   `return` mediumtext NOT NULL,
    ->   `id` varchar(255) NOT NULL,
    ->   `success` varchar(10) NOT NULL,
    ->   `full_ret` mediumtext NOT NULL,
    ->   `alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    ->   KEY `id` (`id`),
    ->   KEY `jid` (`jid`),
    ->   KEY `fun` (`fun`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.030 sec)


CREATE TABLE `salt_events` (
`id` BIGINT NOT NULL AUTO_INCREMENT,
`tag` varchar(255) NOT NULL,
`data` mediumtext NOT NULL,
`alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
`master_id` varchar(255) NOT NULL,
PRIMARY KEY (`id`),
KEY `tag` (`tag`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

MariaDB [salt]> show tables;
+----------------+
| Tables_in_salt |
+----------------+
| jids           |
| salt_events    |
| salt_returns   |
+----------------+
3 rows in set (0.000 sec)

//  Grant access 
MariaDB [salt]> grant all on salt.* to [email protected]'%' identified by 'salt';
Query OK, 0 rows affected (0.006 sec)

MariaDB [salt]> flush privileges;
Query OK, 0 rows affected (0.015 sec)

To configure minion

[[email protected] ~]# vim /etc/salt/minio
mysql.host: '192.168.91.135'	//  Write mysql The host ip
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306

//  Service to restart 
[[email protected] ~]# systemctl restart salt-minion

stay Master The test is stored in mysql in

[[email protected] ~]# salt 'node3' test.ping --return mysql
node3:
    True

Query in database

MariaDB [salt]> select * from salt_returns\G
*************************** 1. row ***************************
       fun: test.ping
       jid: 20211107053320338072
    return: true
        id: node3
   success: 1
  full_ret: {
    "success": true, "return": true, "retcode": 0, "jid": "20211107053320338072", "fun": "test.ping", "fun_args": [], "id": "node3"}
alter_time: 2021-11-07 13:33:23
1 row in set (0.011 sec)

2. job cache

2.1 job cache technological process

return By Minion Interact directly with the storage server , Therefore, it is necessary to install in each Minion Install the module with the specified storage mode on the , such as python-mysql, So can we go directly to Master Store the returned results on the storage server ？

The answer is yes , This is called job cache . It means when Minion Return the result to Master after , from Master Cache the results locally , Then store the cached results to the specified storage server , For example, store in mysql in .

Turn on master Terminal master_job_cache

[[email protected] ~]# yum -y install python3-PyMySQL

[[email protected] ~]# vim /etc/salt/master
#job_cache: True //  Add information after this line 
master_job_cache: mysql	
mysql.host: '192.168.91.134'	//  Write mysql The host ip
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306

//  Service to restart 
[[email protected] ~]# systemctl restart salt-master

Empty the table contents in the database server

MariaDB [(none)]> delete from salt.salt_returns;   
Query OK, 1 row affected (0.001 sec)

MariaDB [(none)]> select * from salt.salt_returns\G
Empty set (0.000 sec)

stay master Test again on the host whether it can be stored in the database

[[email protected] ~]# salt node3 cmd.run 'df -h'
node3:
    Filesystem           Size  Used Avail Use% Mounted on
    devtmpfs             371M     0  371M   0% /dev
    tmpfs                391M   40K  391M   1% /dev/shm
    tmpfs                391M  5.6M  385M   2% /run
    tmpfs                391M     0  391M   0% /sys/fs/cgroup
    /dev/mapper/cs-root   17G  1.9G   16G  11% /
    /dev/sda1           1014M  195M  820M  20% /boot
    tmpfs                 79M     0   79M   0% /run/user/0


MariaDB [(none)]> select * from salt.salt_returns\G
*************************** 1. row ***************************
       fun: cmd.run
       jid: 20211107054058768090
    return: "Filesystem Size Used Avail Use% Mounted on\ndevtmpfs 371M 0 371M 0% /dev\ntmpfs 391M 40K 391M 1% /dev/shm\ntmpfs 391M 5.6M 385M 2% /run\ntmpfs 391M 0 391M 0% /sys/fs/cgroup\n/dev/mapper/cs-root 17G 1.9G 16G 11% /\n/dev/sda1 1014M 195M 820M 20% /boot\ntmpfs 79M 0 79M 0% /run/user/0"
        id: node3
   success: 1
 Omitted after ...

2.2 job management

Get the jid

[[email protected] ~]# salt node3 cmd.run 'uptime' -v
Executing job with jid 20211107054539653392		//  Here is the of this command jid
-------------------------------------------

node3:
     13:45:41 up  2:07,  2 users,  load average: 0.02, 0.04, 0.07

adopt jid Get the return result of this task

[[email protected] ~]# salt-run jobs.lookup_jid 20211107054539653392
node3:
     13:45:41 up  2:07,  2 users,  load average: 0.02, 0.04, 0.07