当前位置:网站首页>分布式的任务分发框架-Gearman
分布式的任务分发框架-Gearman
2022-07-03 17:46:00 【星哥玩云】
官方文档:http://gearman.org/getting-started/ 安装方法和示例都有,可以详细看一下。
Gearman是一个分发任务的程序框架,可以用在各种场合,与Hadoop相比,Gearman更偏向于任务分发功能。它的任务分布非常简单,简单得可以只需要用脚本即可完成。Gearman最初用于LiveJournal的图片resize功能,由于图片resize需要消耗大量计算资源,因此需要调度到后端多台服务器执行,完成任务之后返回前端再呈现到界面。
Gearman可以做什么
- 异步处理:图片处理,订单处理,批量邮件/通知之类的
- 要求高CPU或内存的处理:大容量的数据处理,MapReduce运算,日志聚集,视频编码
- 分布式和并行的处理
- 定时处理:增量更新,数据复制
- 限制速率的FIFO处理
- 分布式的系统监控任务
1,安装方法
#安装gearman
yum install gearmand
#安装libgearman
yum install libgearman-devel
#安装gearman php扩展
pecl install gearman
#添加gearman.so到 php.ini
echo "extension=gearman.so" >> /etc/php.ini
没有pecl的话装一下
yum install php-pear
如果pecl没法安装的话,可以直接下载源代码,使用phpize编译安装
cd ~/software
wget http://pecl.php.net/get/gearman-1.1.2.tgz
cd gearman-1.1.2
phpize
./configure
make && make install
2,简单使用示例:(我们使用Gearman来异步处理百度云推送服务)
先看client.php注册事件,client.php
<?php
// 创建Gearman对象
$client = new GearmanClient();
// addServer默认是localhost,端口默认是4730,如果不是默认的话可以调整
$client -> addServer();
// $client->addServer("192.168.0.0",4730);
echo "Sending job\n";
$username = "test";
$message = "message";
$data = array(
"username" => $username,
"message" => $message ,
);
// 注册事件 以及 传递参数 , 多个参数使用json_encode转换
// 任务可以阻塞式运行,还可以指定优先级 ;当然,也可以非阻塞(运行不等待结果)运行
// 可以参考php Gearman api文档 : doNornal, doHigh, doLow,doBackground
$result = $client -> doBackground("testFunction", json_encode($data));
if ($result) {
echo "Success: $result\n";
}
?>
再看worker的处理,其实就是注册事件处理函数 , worker.php
<?php
$worker = new GearmanWorker();
$worker->addServer();
// 注册事件及事件处理函数
$worker->addFunction("testFunction","handler");
// 运行worker
while ($worker->work());
function handler(GearmanJob $job) {
$workload = json_decode($job->workload());
echo "received: " . print_r($workload,1);
}
?>
可以命令行运行 php client.php , php worker.php ,这里启动顺序最好是先启动worker,这样事件可以得到及时处理;也可以先启动client,事件会在Gearman中排队,等待worker处理;
还可以启动多个worker,Gearman会自动进行负载均衡,分配到不同的worker进行处理。
大家可以发掘更多应用场景...
例如:通过Gearman实现MySQL到Redis的数据同步(异步复制) http://www.linuxidc.com/Linux/2015-01/111380.htm
Gearman的监控:
可以使用supervisor,也可以使用gearman manager
使用supervisor监控Gearman任务的例子见:http://www.linuxidc.com/Linux/2015-01/111384.htm
边栏推荐
- [mathematical logic] equivalent calculus and reasoning calculus of predicate logic (individual word | predicate | quantifier | predicate logic formula | two basic formulas | proposition symbolization
- Internet hospital his management platform source code, online consultation, appointment registration smart hospital applet source code
- Electronic Science and technology 20th autumn "Microcomputer Principle and application" online assignment 2 [standard answer]
- TensorBoard快速入门(Pytorch使用TensorBoard)
- When absolutely positioned, the element is horizontally and vertically centered
- Five problems of database operation in commodity supermarket system
- Enterprise custom form engine solution (XI) -- form rule engine 1
- Research on Swift
- MinGW compile boost library
- Servlet specification Part II
猜你喜欢
1164 Good in C
How to install PHP on Ubuntu 20.04
Wechat applet for the first time
Collection of the most beautiful graduation photos in the graduation season, collection of excellent graduation photos
互聯網醫院HIS管理平臺源碼,在線問診,預約掛號 智慧醫院小程序源碼
Leetcode 538 converts binary search tree into cumulative tree -- recursive method and iterative method
Leetcode 669 pruning binary search tree -- recursive method and iterative method
Hongmeng third training
微服务组件Sentinel控制台调用
How to read the source code [debug and observe the source code]
随机推荐
STM32 realizes 74HC595 control
WEB-UI自动化测试-最全元素定位方法
Ml (machine learning) softmax function to realize the classification of simple movie categories
Deops入门
Collection of the most beautiful graduation photos in the graduation season, collection of excellent graduation photos
Create a new file from templates with bash script - create new file from templates with bash script
Golang unit test, mock test and benchmark test
What is the difference between cloud server and cloud virtual machine
VM11289 WAService. js:2 Do not have __ e handler in component:
How to train mask r-cnn model with your own data
微服务组件Sentinel控制台调用
Draw some simple graphics with MFC
IntelliJ 2021.3 short command line when running applications
Managing multiple selections with MVVM - managing multiple selections with MVVM
How to purchase Google colab members in China
QT学习日记9——对话框
Electronic Science and technology 20th autumn "Microcomputer Principle and application" online assignment 2 [standard answer]
Luogu: p2685 [tjoi2012] Bridge
Basic grammar of interview (Part 2)
UE4 official charging resources, with a total price of several thousand