Interactively compose graphs in GraphScope based on the JupyterLab plugin

目前 GraphScope 发布了 graphscope-notebook 插件,Assist users in an interactive way Jupyterlab The construction and loading operations of graph data are completed in the environment,It greatly reduces the complexity and error rate of loading images.This article will introduce the process of using the plugin in detail.

背景

For any graph computing product,The loading operation of graph data is often the first step,It is also an important and complex step,The main reason for this is the complexity of the graph data itself.因此,In order to improve the user's image loading experience,GraphScope 内置了多种数据集,例如对于 TinkerPop Modorn Graph 图而言,The user only needs one statement to complete the image loading operation:

from graphscope.dataset import load_modern_graph
modern_graph = load_modern_graph()

然而,For user-defined datasets,The process of loading the map needs to define a very long code,我们以 ogbn-mag Take this property graph as an example:

图中包含了 4 types of vertex sums 4 种类型的边,Each of these edges has a label(label),And the labels of the vertices at both ends of the edge are also fixed,Usually each type of vertex、Each side corresponds to one csv 数据文件.

$ tree
├── author_affiliated_with_institution.csv
├── author.csv
├── author_writes_paper.csv
├── field_of_study.csv
├── institution.csv
├── paper_cites_paper.csv
├── paper.csv
└── paper_has_topic_field_of_study.csv

Ultimately for the user,GraphScope The actual load statement required is as follows:

我们可以看到,即便是对于 Python such a high-level language,Contains for the above 4 种 label 的属性图,其 Schema The definition code is also very complex,What's more, there are hundreds of them at every turn label 的顶点、边,Even every kind of vertex、Edges contain thousands of attributes.

因此,In order to reduce the complexity and error rate in the process of user uploading pictures,GraphScope 开发了一款 graphscope-notebook 插件,This plugin is passable交互式的方式,帮助 GraphScope 用户在 Jupyterlab 环境下Complete the loading process of complex graph data.目前,The plugin is already deployed at GraphScope Playground 环境,欢迎大家试用.

接下来,This article will use this property graph as an example,详细介绍如何在 Jupyterlab The loading process of graph data is completed interactively in the environment.

安装插件

The use of the plug-in needs to meet the following conditions:

• JupyterLab >= 3.0
• GraphScope >= 0.12.0

You can install the plugin with the following command:

pip3 install graphscope-notebook

需要注意的是,插件安装完成后,需要重新启动 Jupyterlab,Finally, if the left sidebar is displayed as follows,则代表安装成功;如果安装失败,可在 GraphScope 社区反馈遇到的问题.

使用插件

首先,我们在 Jupyterlab Notebook 中执行如下代码,创建一个 GraphScope Session.

import graphscope
graphscope.set_option(show_log=True)
sess = graphscope.session(cluster_type="hosts")

Session 创建完成后,我们便可以在Resources column on the leftThe resource is monitored,如图所示,点击 “+” 按钮,即可在 Notebook to the right to open the interactive page:

1. 创建顶点

Next, we can complete the construction of graph data vertices according to the prompts on the interactive page,以 “paper” type of vertices for example,Create the vertex page as follows,信息填写完成后,点击 “Create Vertex” The construction of the vertices is completed.

其中,各字段解释如下:

字段	解释
Label	顶点标签(label)
Data Source	Local File Represents a local data file; Online File Represents a web file,例如 OSS 等
Location	数据源路径
Header Row	如果为 true,then the column names will be read from the first line of the source file
Delimiter	数据分隔符,The optional value is a space、“,”、 “;” 和 “\t”
Extra Params	Additional parameters required for data loading,例如 OSS Key, Secret 和 Endpoint
ID Field	Which column in the source file is selected as the point ID,Can be a number of type number eg0,1,2,也可以是string类型的名字
Select all Properties	如果为 true,All properties are loaded,Otherwise, you need to specify the loaded properties

2. 创建边

同"创建顶点"一样,以 “cites” type of edge for example (paper -> cites -> paper),The page for creating edges is as follows,信息填写完成后,点击 “Create Edge” The construction of the edge is completed.

其中,Fields other than vertices are explained below:

字段	解释
Edge Only	如果为 true,it means that the edge is not associated with any other vertex files,Applies to files with only one side,without the vertex file
Src Vertex Label	The label of the source vertex(label)
Dst Vertex Label	The label of the target vertex(label)
Src Vertex Field	Which column in the source file is used as the source vertex ID,Can be a number of type number eg0,1,2,也可以是string类型的名字
Dst Vertex Field	Which column in the source file is used as the target vertex ID,Can be a number of type number eg0,1,2,也可以是string类型的名字

3. Define graph related information

Vertices of all types/After the edge build is complete,We fill in the information related to the diagram,The corresponding fields are explained below:

字段	解释
Name	figure name
OID Type	Graph primitive point type,可选值为 “string” 和 “int64”
Directed	是否为有向图
Generate EID	Whether to generate unique for each edgeid,如果需要使用 GIE Graph query service,则设置为 True

4. Generate image loading statements,Complete data loading

信息填写完成后,鼠标选中 “Notebook Cell”,Click on the interactive page “Generate Code” 按钮,The corresponding image loading statement can be generated,Execute this statement to complete the loading operation of graph data,Finally, we can monitor the corresponding graph resources in the resource column on the left:

至此,我们已经成功使用 graphscope-notebook The plugin completes the loading process of graph data,接下来,便可以参考 GraphScope 文档 Execute the corresponding graph computation task on the loaded graph.

结语

graphscope-notebook 作为一款 GraphScope 适用于 Jupyterlab 环境下的插件,目前具备 1) 监控 GraphScope 运行时资源; 2) Interactive graph data construction and other functions,It can greatly reduce the complexity and error rate in the image loading process.除此之外,We also plan to add interactive mining of graph data in subsequent releases、Visual analysis and other related functions,欢迎大家关注GraphScope.