Distributed task distribution framework gearman

Official documents ：http://gearman.org/getting-started/  There are installation methods and examples , You can look at it in detail .

Gearman It's a framework for distributing tasks , It can be used in all kinds of situations , And Hadoop comparison ,Gearman More in favor of task distribution . Its task distribution is very simple , It's as simple as a script .Gearman Originally used for LiveJournal Pictures of the resize function , Due to pictures resize Need to consume a lot of computing resources , Therefore, it is necessary to schedule to multiple backend servers The server perform , After completing the task, return to the front end and present it to the interface .

Gearman What can be done

• Asynchronous processing : The image processing , The order processing , Bulk mail / Notice or something
• Demand is high CPU Or memory processing : Large capacity data processing ,MapReduce operation , Log aggregation , Video coding
• Distributed And parallel processing
• Timing processing : Incremental updating , Data replication
• Rate limiting FIFO Handle
• Distributed System monitoring task  

1, Installation method

# install gearman
yum install gearmand
# install libgearman
yum install libgearman-devel
# install gearman php Expand 
pecl install gearman
# add to gearman.so To  php.ini
echo "extension=gearman.so" >> /etc/php.ini

Copy

  No, pecl If so, pretend

yum install php-pear

Copy

If pecl If it cannot be installed , You can download the source code directly , Use phpize Compilation and installation

cd ~/software
wget http://pecl.php.net/get/gearman-1.1.2.tgz
cd gearman-1.1.2
phpize
./configure
make && make install 

Copy

2, Simple use examples ：（ We use Gearman To asynchronously handle Baidu cloud push service ）

First look at client.php Registration events ,client.php

<?php

//  establish Gearman object 
$client = new GearmanClient();
// addServer The default is localhost, The default port is 4730, If it is not the default, you can adjust 
$client -> addServer();
// $client->addServer("192.168.0.0",4730);

echo "Sending job\n";

$username = "test";
$message = "message";
$data = array(
    "username" => $username,
    "message" => $message , 
);
//  Registration events   as well as   Pass parameters  ,  Multiple parameters use json_encode transformation 
//  Tasks can run in a blocking fashion , Priority can also be specified  ; Of course , It can also be non blocking （ Run without waiting for results ） function 
//  You can refer to php Gearman api file  : doNornal, doHigh, doLow,doBackground
$result = $client -> doBackground("testFunction", json_encode($data));
if ($result) {
    echo "Success: $result\n";
}

?> 

Copy

Look again worker To deal with , In fact, it is to register the event handler , worker.php

<?php

$worker = new GearmanWorker();
$worker->addServer();

//  Register events and event handlers 
$worker->addFunction("testFunction","handler");

//  function worker
while ($worker->work());

function handler(GearmanJob $job) {
    $workload = json_decode($job->workload());
    echo "received: " . print_r($workload,1);
}


?>

Copy

It can be run from the command line php client.php , php worker.php , The best starting sequence here is to start first worker, In this way, the incident can be handled in time ; You can also start client, The event will be in Gearman In line , wait for worker Handle ;

You can also start multiple worker,Gearman Automatic load balancing , Assign to different worker To deal with .

You can explore more application scenarios ...

  for example ： adopt Gearman Realization MySQL To Redis Data synchronization for （ Asynchronous replication ） http://www.linuxidc.com/Linux/2015-01/111380.htm

Gearman Monitoring of ：

have access to supervisor, You can also use gearman manager

Use supervisor monitor Gearman See ：http://www.linuxidc.com/Linux/2015-01/111384.htm