On the quantity control mechanism of swoole collaboration creation in production environment

stay swoole In official documents , It is mentioned that “ Co process overhead ”. A simple reference is as follows ：

• stay  PHP-7.2  The bottom layer of the version will be assigned  8K  Of  stack  To store the variables of the coroutine ,zval  The size is  16 byte , therefore  8K  Of  stack  It can be saved up to  512  A variable . The memory consumption of the coroutine stack exceeds  8K  after  ZendVM  It will automatically expand .

• PHP-7.1、PHP-7.0  It will be assigned by default  256K  Stack memory

This article takes php7.2 For example , Because the author currently uses php The version is also 7.2. 

In order to verify php7.2 in , Whether to create a collaboration by default 8K, You can write a demo Just test it

<?php

$begin_memory = memory_get_usage();
go(function () {
    co::sleep(10);
    echo co::getCid() . PHP_EOL;
});

var_dump(memory_get_usage() - $begin_memory); // Output int(8640), The unit is byte,  Namely 8K about .

Now that you create a collaboration , Default assignment 8K Memory , Suppose that the code we execute in the coroutine uses stack memory , No more than 8K, that 1G About how many coroutines can be created in the memory of ？1G= 1024MB = (1024 * 1024)KB, max_coroutine_num = 1024 * 1024 / 8 = 130072. in other words 1G Memory can create 10w+ The number of trips . Considerable . But the reality , Stack memory may exceed .

I currently have a business scenario , from redis In line pop data , Then open the process to write mysql, We know mysql Your writing speed lags behind redis A lot , When the business peak ,redis When the amount of queue data increases dramatically ,mysql After reaching the bottleneck , The number of collaborative processes will be overstocked . The peak period lasts to a certain extent , In the end, there will be insufficient memory resources , As a result, a new collaboration cannot be created , At this time swoole Will trigger ：PHP Warning: PHP Warning:  go(): exceed max number of coroutine ... 

So here's an experience ： In the production and consumption model , The best of swoole The quantity of collaboration process creation is controlled

swoole Co process control scheme ：

1、 rely on swoole At the bottom max_coroutine To configure . Give an example demo

<?php

set_error_handler(function () {
    throw new Exception("error happen\n", 500);
});
co::set(['max_coroutine' => 1]);
try {
    for ($i = 0; $i < 2; $i++) {
        go(function () {
            co::sleep(10);
        });
    }

} catch (\Throwable $e) {
    if ($e->getCode() == 500) {
        echo " The number of processes exceeds the limit !" . $e->getMessage();
    }
}

2、 Create your own Context Manage and count the number of collaboration processes currently being executed , More than the specified amount , Throw an exception or customize the handling

<?php

define("APP_MAX_COROUTINE", 10);

$co_ctx = [];

go(function () use ($co_ctx) {
    if (count($co_ctx) >= APP_MAX_COROUTINE) {
        // Limit exceeded , sign out 
        return;
    }
    $cid = co::getCid();
    defer(function () use ($cid, $co_ctx) {
        if (isset($co_ctx[$cid])) {
            unset($co_ctx[$cid]);
        }
    });
    $co_ctx[$cid] = $cid;
    
    //begin business ...
});

End of discussion , What's wrong , Your smile ！                     thks!!!