时序图数据建模与产业链分析

使用图数据分析产业链时序数据

• 数据模型的设计
• 函数与过程功能介绍
• 完整实现 - 构建公司与产品时序图数据
• 查询案例-分析'消费品商贸'产业2020Q2季度总营收
• 其他资料

数据模型的设计

行业、产品、公司相关的产业链数据建模，包含产品层级图谱、行业分层图谱、产业链上下游图谱、公司与产品图谱等。如下主要是公司与产品营收的时序图数据建模实现方案。

函数与过程功能介绍

• HTTP请求

RETURN olab.http.post('api-address','input')

复制

• 重置MAP-移除传入的KEY

RETURN olab.reset.map({map},{keys}) AS value

复制

• 生成JSON-STRING

RETURN olab.convert.json({object})

复制

完整实现 - 构建公司与产品时序图数据

• 在下面的实现中营收相关的时序数据使用JSON格式数据建模存储在关系的属性中

下述实现中集成了GraphQL-API、olab-apoc组件、访问数据库等操作，构建的图数据最终是将时序数据存储在了一个JSON数组中。

CALL apoc.periodic.iterate('CALL apoc.load.jdbc(\'jdbc:mysql://contentdb.crkldnwly6ki.rds.cn-north-1.amazonaws.com.cn:3306/database?user=datalab_dev&password=datalabgogo&useUnicode=true&characterEncoding=utf8&serverTimezone=UTC\', 
\'SELECT calc.company_id AS fromCompanyId,pro.`name` AS `to`,CONVERT(DATE_FORMAT(calc.report_date,\\\'%Y%m%d%H%i%S\\\'),UNSIGNED INTEGER) 
AS report_date,income,profit,product_income,product_income_ratio,product_profit,product_profit_ratio,\\\'id\\\' AS uniqueField,calc.id AS uniqueFieldID,\\\'chinascope_supply_chain.fin_secu_sam_product_calc\\\' AS jsDataApi FROM fin_secu_sam_product_calc calc RIGHT JOIN dict_product_rs pro ON calc.product_code=pro.`code` WHERE calc.update_time>=DATE_SUB(NOW(),INTERVAL 1 DAY)\')',
'WITH REPLACE(\'{"query": "{  horgByName(sourceCode: \\\\\\"company-id\\\\\\", sourceFlag: \\\\\\"chinascope\\\\\\") {    name    hcode  }}","variables": null}\',\'company-id\',row.fromCompanyId) AS query,row WITH apoc.convert.fromJsonMap(olab.http.post(\'http://ip/ongdb/graphql\',query)) 
AS result,row WHERE result IS NOT NULL WITH result.data.horgByName[0].hcode AS fromHcode,result.data.horgByName[0].name AS fromName,row WHERE fromHcode IS NOT NULL WITH fromHcode,fromName,row.to AS toName,row 
WITH fromHcode,fromName,\'HPRDCLS\'+apoc.util.md5([toName]) AS toHcode,row MERGE (from:HORGProductCalc {hcode:fromHcode}) SET from.name=fromName,from:HORG WITH from,toHcode,row MATCH (to:产业 {hcode:toHcode}) 
WITH from,to,row WHERE from<>to WITH from,to,olab.reset.map(row,[\'fromCompanyId\',\'to\']) AS row OPTIONAL MATCH (from)-[r]->(to) WITH from,to,row,apoc.coll.union(apoc.convert.fromJsonList(r.detail),[row]) AS detail 
WITH from,to,row,olab.convert.json(detail) AS detail,SIZE(detail) AS detail_size MERGE (from)-[r:产品]->(to) SET r.detail=detail,r.detail_size=detail_size', {parallel:false,batchSize:100}) 
YIELD batches,total,timeTaken,committedOperations,failedOperations,failedBatches,retries,errorMessages,batch,operations RETURN batches,total,timeTaken,committedOperations,failedOperations,failedBatches,retries,errorMessages,batch,operations;

复制

查询案例-分析’消费品商贸’产业2020Q2季度总营收

• 查询’消费品商贸’行业下级分类中所有相关产品

数据建模时产品与行业有关联关系，因此在这里使用时可以直接得到，关注的某行业下属子子孙孙行业的所有产品。

# 扩展PATH方式MATCH (indus:行业) WHERE indus.name='消费品商贸' WITH indusCALL apoc.path.expandConfig(indus,{maxLevel:3, relationshipFilter:'NEXT>|<BELONG_TO', labelFilter:'+产业|/产品', bfs:false, filterStartNode:true,uniqueness:'NODE_GOLBEL', limit:-1}) YIELD pathWITHpathRETURNpath

# 扩展NODE方式
MATCH (indus:行业) WHERE indus.name='消费品商贸' WITH indus
CALL apoc.path.subgraphNodes(indus,{maxLevel:3, relationshipFilter:'NEXT>|<BELONG_TO', 
labelFilter:'+产业|/产品', bfs:false, filterStartNode:true,uniqueness:'NODE_GOLBEL', limit:-1}) 
YIELD node WITH node
RETURN node

复制

行业与产品关联结构示意图

• '消费品商贸’行业相关的产品【相关指从属关系的划分，这里拿出了下属三层以内的产业相关产品】2020Q2营收分析【案例实现营收加总】

MATCH (indus:行业) WHERE indus.name='消费品商贸' WITH indus
CALL apoc.path.subgraphNodes(indus,{maxLevel:3, relationshipFilter:'NEXT>|<BELONG_TO', 
labelFilter:'+产业|/产品', bfs:false, filterStartNode:true,uniqueness:'NODE_GOLBEL', limit:-1}) YIELD node WITH node
MATCH p=(node)<-[r:产品]-(:HORGProductCalc) WITH apoc.convert.fromJsonList(r.detail) AS detail
UNWIND detail AS map
WITH map WHERE TOINTEGER(map.report_date)>=20200331000000 AND TOINTEGER(map.report_date)<=20200630000000
WITH TOINTEGER(map.income) AS income
RETURN SUM(income)
# 加总后的总营收两万亿
╒═════════════╕
│"SUM(income)"│
╞═════════════╡
│2215100453627│
└─────────────┘

复制

• 产品与公司的样例图结构。

其他资料

• olab-apoc组件

https://github.com/ongdb-contrib/ongdb-lab-apoc

复制

• apoc组件【Neo4j官方开源组件】

https://github.com/neo4j-contrib/neo4j-apoc-procedures

复制