Ansible learning summary (11) - detailed explanation of forks and serial parameters of task parallel execution

前言

Ansible 执行 task 的时候,will be based on what is specified in the configuration file forks 参数、inventory The host and host group specified in 、as well as the selected target host and defined task The number determines the way of execution.比如 inventory in the host manifest file,记录了 20 台服务器,配置文件中的 forks 参数指定为 5,同时有 2 个 task 需要执行,At this point, it means that the maximum number of pairs is allowed at a time 5 server to operate,If there are not enough servers left 5 台,The action is then performed on all remaining servers.那么对于多个 task And when multiple servers execute,Follow how the target host performs,通常有两种方式：

• 广度优先：将每个 task 先按照 forks 约定的参数,Execute in batches in parallel on all servers,等这个 task 执行完毕之后,Continue to perform subsequent ones in the same manner task.
• 深度优先：serial 指定（通常小于 forks 指定的参数）All are executed in parallel on the agreed number of servers task Then move on to the next set of servers,Do everything in the same way task.

一、forks（广度优先）

依据 forks 参数,Decide how many servers to execute the corresponding in parallel at a time task,对于包含多个 task 的场景,This way will go first 1 个 task After execution is complete on all specified servers,subsequent ones will be executed task.So for one contains multiple task 的 playbook 来说,The state of all servers is incomplete at this point,are all in the middle.But when this contains more than one task 的 playbook 执行完成之后,The status of all successfully executed servers is updated.This method is suitable for scenarios that are not sensitive to the intermediate state of the server,The advantage is that a synchronous configuration operation can be performed on all hosts specified;The disadvantage is that all servers are in an intermediate state with incomplete configuration during the update process.比如下面的场景,有 4 A server needs to be configured（nodeA, nodeB, nodeC, nodeD）,playbook 中定义了 2 个 task,forks 指定的参数是 5,而每个 task The execution time is 5 秒.此时,执行过程如下：

• 第一次执行：由于 forks The number is greater than the number of servers,So all servers will be selected,Execute the first one in parallel task,So the execution time is 5 秒;
• 第二次执行：Also all servers will be selected,and execute the second one in parallel task,Therefore, the execution time is the same at this time 5 秒.
• 最终,这个 playbook After execution is complete on all servers,总的耗时为 10 秒.

示例代码如下：

[email protected]:1st$ vim test_forks.yml
[email protected]:1st$ cat test_forks.yml
---
- hosts: c7u6s1,c7u6s2,c7u6s3,c7u6s4
  gather_facts: false

  tasks:
  - name: echo hostname
    command: hostname

  - name: echo datetime
    command: date
[email protected]:1st$ ansible-playbook -f 5 test_forks.yml

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] >******************************************************************************************> ******************

TASK [echo hostname]  **************************************************************************************************************************
changed: [c7u6s1]
changed: [c7u6s2]
changed: [c7u6s3]
changed: [c7u6s4]

TASK [echo datetime] >**************************************************************************************************************************
changed: [c7u6s1]
changed: [c7u6s3]
changed: [c7u6s2]
changed: [c7u6s4]

PLAY RECAP ************************************************************************************************************************************
c7u6s1                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s2                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s3                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s4                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[email protected]:1st$ 

上述输出结果中,显示每个 task Execute in parallel on all servers,执行完成之后,Subsequent execution will begin task.上面是 forks When the number is greater than the number of servers,对于 forks Scenarios where the number is less than the number of servers,比如有 4 台服务器,playbook 中同样有 2 个task,forks The number is configured as 2,每个 task The execution time of is still 5 秒.此时的执行过程如下：

• 第一次执行：4台服务器中的2station is selected,Execute the first one in paralleltask,So the time taken after the execution is completed is 5秒;
• 第二次执行：剩下的2server is selected,执行第一个task,So it takes time after the execution is completed5秒;
• 第三次执行：4台服务器中的2station is selected,Execute the second one in paralleltask,So after the execution is completed, the time is 5秒;
• 第四次执行：剩下的2server is selected,执行第二个task,So the execution time to complete is 5秒;
• The final total time is 20秒.

示例代码如下：

---
- hosts: c7u6s1,c7u6s2,c7u6s3,c7u6s4
  gather_facts: false

  tasks:
  - name: echo hostname
    shell: hostname; sleep 1

  - name: echo datetime
    shell: date; sleep 1

执行上述的 palybook,具体如下：

[email protected]:1st$ ansible-playbook -f 2 -v test_forks.yml
Using /etc/ansible/ansible.cfg as config file

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s1] => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": true, "cmd": "hostname; sleep 1", "delta": "0:00:01.006387", "end": "2022-06-28 17:10:30.556334", "msg": "", "rc": 0, "start": "2022-06-28 17:10:29.549947", "stderr": "", "stderr_lines": [], "stdout": "c7u6s1", "stdout_lines": ["c7u6s1"]}
changed: [c7u6s2] => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": true, "cmd": "hostname; sleep 1", "delta": "0:00:01.003715", "end": "2022-06-28 17:10:30.557691", "msg": "", "rc": 0, "start": "2022-06-28 17:10:29.553976", "stderr": "", "stderr_lines": [], "stdout": "c7u6s2", "stdout_lines": ["c7u6s2"]}
changed: [c7u6s3] => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": true, "cmd": "hostname; sleep 1", "delta": "0:00:01.004623", "end": "2022-06-28 17:10:31.800163", "msg": "", "rc": 0, "start": "2022-06-28 17:10:30.795540", "stderr": "", "stderr_lines": [], "stdout": "c7u6s3", "stdout_lines": ["c7u6s3"]}
changed: [c7u6s4] => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": true, "cmd": "hostname; sleep 1", "delta": "0:00:01.004561", "end": "2022-06-28 17:10:31.811011", "msg": "", "rc": 0, "start": "2022-06-28 17:10:30.806450", "stderr": "", "stderr_lines": [], "stdout": "c7u6s4", "stdout_lines": ["c7u6s4"]}

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s1] => {"changed": true, "cmd": "date; sleep 1", "delta": "0:00:01.005884", "end": "2022-06-28 17:10:33.050719", "msg": "", "rc": 0, "start": "2022-06-28 17:10:32.044835", "stderr": "", "stderr_lines": [], "stdout": "Tue Jun 28 17:10:32 CST 2022", "stdout_lines": ["Tue Jun 28 17:10:32 CST 2022"]}
changed: [c7u6s2] => {"changed": true, "cmd": "date; sleep 1", "delta": "0:00:01.004152", "end": "2022-06-28 17:10:33.051957", "msg": "", "rc": 0, "start": "2022-06-28 17:10:32.047805", "stderr": "", "stderr_lines": [], "stdout": "Tue Jun 28 17:10:32 CST 2022", "stdout_lines": ["Tue Jun 28 17:10:32 CST 2022"]}
changed: [c7u6s3] => {"changed": true, "cmd": "date; sleep 1", "delta": "0:00:01.004385", "end": "2022-06-28 17:10:34.231918", "msg": "", "rc": 0, "start": "2022-06-28 17:10:33.227533", "stderr": "", "stderr_lines": [], "stdout": "Tue Jun 28 17:10:33 CST 2022", "stdout_lines": ["Tue Jun 28 17:10:33 CST 2022"]}
changed: [c7u6s4] => {"changed": true, "cmd": "date; sleep 1", "delta": "0:00:01.004384", "end": "2022-06-28 17:10:34.236657", "msg": "", "rc": 0, "start": "2022-06-28 17:10:33.232273", "stderr": "", "stderr_lines": [], "stdout": "Tue Jun 28 17:10:33 CST 2022", "stdout_lines": ["Tue Jun 28 17:10:33 CST 2022"]}

from the above output times,可以看出,每个 task 都是分 2 次执行的,因为指定了 forks 数为 2,The actual number of hosts is 4 台.The above is the so-called breadth first,i.e. execute on all hosts first 1 个 task,until all hosts are executed,Subsequent ones are performed in the same manner task.

三、serial（深度优先）

And for the depth-first implementation,It is executed on the specified number of servers playbook 的所有 task 之后,It will continue to execute this on the remaining other hosts playbook 中定义的 task.这是通过在 playbook 中指定 serial 关键字实现的,所以其是在 forks 参数的基础上,Make further arrangements,Thereby, the execution of the specified number of servers is completed playbook 之后,operations that will only be performed on other servers.This approach is similar to rolling updates.比如,此时仍然为 4 台服务器,forks 仍然设置为 5,然后在 playbook 中增加 serial 关键字,并将其值设置为 2,playbook 中仍然有 2 个 task,且每个 task The execution time is 5秒.此时的执行过程如下：

• 第一次执行：从4Pick two out of two servers,Execute the first one in paralleltask,until the first onetask执行完成,Time consuming at this time is 5秒;
• 第二次执行：on both servers,Continue to execute the second one in paralleltask,直至第二个task执行完成,Time consuming at this time is 5秒;
• 第三次执行：在剩下的2台服务器上,Execute the first one in paralleltask,until the first onetask执行完成,Time consuming at this time is 5秒;
• 第四次执行：继续在剩下的2台服务器上,Execute the second one in paralleltask,直至diergetask执行完成,Time consuming at this time is 5秒;
• 至此,4have been executed on each server,总耗时为20秒.

示例代码如下：

---
- hosts: c7u6s1,c7u6s2,c7u6s3,c7u6s4
  gather_facts: false
  serial: 2

  tasks:
  - name: echo hostname
    shell: hostname; sleep 1

  - name: echo datetime
    shell: date; sleep 1

执行过程如下：

[email protected]:1st$ ansible-playbook -f 5 test_serial.yml

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s2]
changed: [c7u6s1]

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s1]
changed: [c7u6s2]

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s3]
changed: [c7u6s4]

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s3]
changed: [c7u6s4]

PLAY RECAP ************************************************************************************************************************************
c7u6s1                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s2                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s3                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s4                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

[email protected]:1st$

上述输出结果,Also verified the execution process,即在 serial Executed on the specified number of servers playbook 中定义的所有 task 之后,This will continue to be executed on subsequent other servers playbook.如果 playbook 中的 serial 值比 forks 的值大,此时以 serial 为准,执行并行操作.修改后的 yaml The files are as follows：

---
- hosts: c7u6s1,c7u6s2,c7u6s3,c7u6s4
  gather_facts: false
  serial: 4

  tasks:
  - name: echo hostname
    shell: hostname; sleep 1

  - name: echo datetime
    shell: date; sleep 1

执行上述 playbook 的时候,指定 forks 值为 2,此时 serial 的值比 forks 值大.此时的执行效果如下：

[email protected]:1st$ ansible-playbook -f 2 test_serial.yml

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s1]
changed: [c7u6s2]
changed: [c7u6s3]
changed: [c7u6s4]

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s1]
changed: [c7u6s2]
changed: [c7u6s3]
changed: [c7u6s4]

PLAY RECAP ************************************************************************************************************************************
c7u6s1                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s2                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s3                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s4                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

可以看出,the above execution process,是 4 Servers execute in parallel,The number of parallels is not affected forks 的数量限制.It can be seen from the above two ways,forks 和 serial Both can determine the number of servers to execute in parallel,当没有在 playbook 中指定 serial 的时候,则以 forks The value is based on the number of servers running in parallel;当在 playbook 中指定了 serial and configured at the same time forks 的时候,则以 serial value as a basis for the number of servers running in parallel.

serial 的其他用法

serial关 In addition to keys can be specified as numbers,Can also be specified as a percentage,Indicates the percentage of the number of hosts for a single parallel execution to the specified total number of hosts.The definition form at this time is as follows：

---
- name: test play
  hosts: webservers
  serial: "30%"

  tasks:
  - name: task1
    command: hostname
  - name: task2
    command: datetime

上述输出表示,Select from all hosts specified each time 30% 执行.还可以将 serial The value is set to a list,Each list item represents the number of servers to execute in parallel.具体如下：

---
- name: test play
  hosts: webservers
  serial:
    - 1
    - 5
    - 10
  tasks:
  - name: task1
    command: hostname
  - name: task2
    command: date

上述代码的含义,when it is executed for the first time,Pick from all servers specified 1 台,执行 playbook 中的所有 task,第二次执行的时候,Select from all servers specified 5 台执行 playbook 中的所有 task,第三次执行,Select from all servers specified 10 台执行 playbook 中的所有 task,If there are still unexecuted servers at this time,则按照 forks The defined number is executed in parallel.在 serial 的列表中,You can also mix numbers with percentages,具体如下：

---
- name: test play
  hosts: webservers
  serial:
    - 1
    - "50%"
  tasks:
  - name: task1
    command: hostname
  - name: task2
    command: date

上述代码的含义,is to select from the specified server 1 台执行 playbook,当执行完成之后,Select from the remaining hosts of the total number of hosts 50% 的主机执行 playbook,If there is still a specified host that has not been executed at this time,则按照 forks 的指定参数,并行执行.代码如下：

---
- hosts: c7u6s1,c7u6s2,c7u6s3,c7u6s4
  gather_facts: false
  serial:
  - 1
  - 50%

  tasks:
  - name: echo hostname
    shell: hostname; sleep 1

  - name: echo datetime
    shell: date; sleep 1

上述 playbook 的执行过程如下：

[email protected]:1st$ ansible-playbook -f 5 test_serial.yml

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s1]

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s1]

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s2]
changed: [c7u6s3]

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s3]
changed: [c7u6s2]

PLAY [c7u6s1,c7u6s2,c7u6s3,c7u6s4] ************************************************************************************************************

TASK [echo hostname] **************************************************************************************************************************
changed: [c7u6s4]

TASK [echo datetime] **************************************************************************************************************************
changed: [c7u6s4]

PLAY RECAP ************************************************************************************************************************************
c7u6s1                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s2                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s3                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
c7u6s4                     : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

从上述输出中可以看出,First pick a server to execute playbook,执行完成之后,Choose the total from the remaining servers 50% 的服务器（总数 4 台,50% 即为 2 台）继续执行 playbook,At this time, there are still hosts that have not been executed,且此时的 forks The number is greater than the number of remaining servers,So directly execute all the remaining servers.