当前位置:网站首页>Rapidly develop GraphScope graph analysis applications
Rapidly develop GraphScope graph analysis applications
2022-07-30 07:36:00 【graphscope】
我们开发了一个 C++ 模板库,Users only need to replace a few functions with custom logic,You can customize the graph analysis algorithm and run it GraphScope 上.
Graphs are an important data source for big data,Various analytical techniques surrounding graphs also have wide-ranging applications.The core of graph computing lies in algorithms,如 PageRank、K-Core Algorithms such as these have shown powerful capabilities in node importance analysis scenarios.GraphScope The graph analysis engine of GRAPE 系统的开源版本,A variety of graph analysis algorithms are built in,can be found in the path、Community discovery and other fields provide one-stop graph analysis solutions;然而,Users often need to customize some algorithms according to their own needs,因此,In order to facilitate users to develop custom graph analysis algorithms,我们提供了一个 C++ 模版库,The next part of this article will introduce how to use the template library.
Step1. 安装 GraphScope
首先,We need to install it in the local environment with the following command GraphScope:
pip3 install graphscope
Step2. Develop algorithms using template libraries
使用浏览器打开template library GitHub 地址,Click to the right of the library Use this template
button to create a code repository of your own,And clone the repository locally with the following command:
# Make sure to replace the <username> and <repo-name> to the right values.
git clone https://github.com/<username>/<repo-name>.git
cd <repo-name>
接下来,我们通过修改 src
目录下的文件,Implement custom algorithm logic,其中:
my_app.h
: This file is used to implement the main logic of the algorithm,即PEval
和IncEval
函数;my_app_context.h
: 该文件用于存储、Define the results of the algorithm when it runs and the data structures used.
Defines the data structures that the algorithm needs to run
To simplify the algorithm logic,Highlight the development process of the algorithm,在本例中,Our algorithm will take care of thatThe sum of the degrees of each node's neighbors in the graph.在这一步中,我们需要在 my_app_context.h
文件中:1) Defines the data structure of the algorithm runtime;2) 在 Init
function to initialize these data structures.
The algorithm runtime data structure is as follows:
变量名 | 类型 | 作用 |
---|---|---|
result | vertex_array_t<size_t> | Store the calculation result of each node |
degree | vertex_array_t<size_t> | 辅助数据结构,Stores the degree of each node |
最终,my_app_context.h
实现如下:
Implement algorithm logic
After the data structure is defined,我们可在 my_app.h
The corresponding algorithm logic is implemented in,即 PEval 和 IncEval 函数.关于 PEval 和 IncEval 函数的定义,可参考 PIE Introduction to the programming model.在这个例子中,
- PEval 函数: Get the degree of each internal node,and send messages along the outgoing edge;
- IncEval 函数: 1) 负责接收、Process messages from the previous round;2) Count the sum of the degrees of each node's neighbor nodes.
最终,完整的 my_app.h
实现如下:
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8WsI3SVy-1658738305019)(https://files.mdnice.com/user/24781/653b7ec0-fca7-46d0-b9f9-3614d52b8f71.png)]
Step3. 构建并测试
Build and test your algorithm locally
使用如下命令,Compile the build algorithm:
make build & cd build
cmake .. && make # compilation algorithm
make package # Package the algorithm as available GraphScope resources running on it (.gar文件)
在 GraphScope Run the algorithm on the cluster
算法开发完成后,We can then assign the corresponding algorithm resources (gar文件) 运行在 GraphScope 集群上,具体步骤如下:
import graphscope
graphscope.set_option(show_log=True)
from graphscope.framework.app import load_app
from graphscope.dataset import load_p2p_network
sess = graphscope.session(cluster_type="hosts")
simple_graph = load_p2p_network(sess)._project_to_simple()
my_app = load_app('<path_to_your_gar_resource>')
result = my_app(simple_graph)
print(result.to_numpy("r"))
结语
本文详细介绍了如何基于 C++ The template library can quickly develop a template library GraphScope Graph analytics algorithms running on the cluster,In the future, we will also share more algorithm cases developed based on this template library in real scenarios,敬请期待!点击链接The source code of the template library can be obtained.同时也欢迎关注 GraphScope 公众号及GitHub,及时了解图计算相关的知识及 GraphScope 的研发进展.
边栏推荐
- wsl2设置静态ip(固定ip)static ip
- 从安装到编译: 10分钟教你在本地使用和开发GraphScope
- 基于STM32F103的消防系统之火焰传感器
- Graph analysis like NetworkX with GraphScope
- 【Exhibition of some projects】
- Biotin-NH2|CAS:111790-37-5(生物素-氨基)是一种生物素化化合物
- 02-Cycript的使用
- Desthiobiotin-PEG4-Acid|脱硫生物素-PEG4-酸| 供应商和制造商
- Vineyard: 开源分布式内存数据管理框架
- libgrape-lite on GPUs: GPU helps accelerate graph analysis tasks
猜你喜欢
[Common usage of markdown]
Biotin-NHS LC(72040-63-2)生物素接头|站点特定探针
VSCode hides the left activity bar
图扑软件数字孪生民航飞联网,构建智慧民航新业态
Unity 渲染流水线
如何将matlab数据导入modelsim仿真
数码管动态显示及模块化编程
QT serial port dynamically displays a large number of data waveform curves in real time (5) ======== "Final perfect solution version"
事件传递和响应者链条
网络协议01 - 基础概念
随机推荐
图扑数字孪生煤矿开采系统,打造采煤“硬实力”
阿里京东“喜提”国电投,顶流互联网和能源大厂为何合作?
com.alibaba.datax.common.exception.DataXException: Code:[ESWriter-03]
The most complete difference between sizeof and strlen, as well as pointer and array operation analysis
js advanced study notes (detailed)
图扑软件数字孪生民航飞联网,构建智慧民航新业态
STM32F103连接L9110S电机驱动控制小水泵
-----博客声明
GCD的定时器
ssh script space character conversion
[Punctuality Atom] Learning and use of IIC (unfinished...)
How to create a shortcut without the "shortcut" suffix?
Explore the efficiency of make_shared
Knowledge of the day: handwritten deep copy and shallow copy (solves the problem of circular references)
单片机之流水灯
题解——汉明距离
如何将modelsim仿真数据存成文件
网络协议01 - 基础概念
wsl2设置静态ip(固定ip)static ip
网络协议03 - 路由和NAT