Optimization of ecological | Lake Warehouse Integration: gbase 8A MPP + xeos

In recent days, , Tianjin NANDA General Data Technology Co., Ltd （ hereinafter referred to as ：GBASE Nantah general motors ） Distributed analytical database GBase 8a MPP Completed the cooperation with Beijing Star Trinasolar Technology Co., Ltd （ hereinafter referred to as ：XSKY Star and sky ） Full protocol supported enterprise class object storage XEOS Docking test of , Pass the function test 、 Performance testing , The test results are generally normal and meet the test expectations , The two products have good compatibility , Meet functional requirements .

Data warehouse encounters storage performance and compatibility problems

With the advancement of localization process , Domestic database manufacturers work hard , After years of polishing products , The domestic database system has gradually matured , And accumulated a large number of customer cases . With the promotion of domestic databases and more and more are put into the production environment , The architecture of data warehouse is gradually applied , And related storage performance and compatibility problems follow .
Data warehouse is in the case of a large number of databases , In order to further mine data resources 、 For the sake of decision-making . The purpose of data warehouse construction , It is based on front-end query and analysis , Due to large redundancy , Therefore, the storage required is also large . Data warehouse usually has the following characteristics ：

Too much data ： Data warehouse contains historical data 、 Data composed of detailed data and various types of data . The amount of data is huge , For data management 、 Indexes 、 Persistent preservation is a huge challenge .

There are many file formats ： The data of data warehouse comes from different external systems , Its data is also open to all external application systems . So the data in the data warehouse , It is distinguished according to different themes , According to different topics, a large number of files in different formats will be generated .

Long term retention required ： The data of data warehouse is mainly used for enterprise decision analysis , The data operations involved are mainly data query , Once a certain data enters the data warehouse , In general, it will be retained for a long time , That is to say, there are a lot of query operations in data warehouse , But there are few modifications and deletions , It usually only needs to be loaded on a regular basis 、 Refresh . Data warehouse includes a large amount of historical data .

Files are read and written frequently at the same time ： Because the data warehouse is for each external application system , Facing different business systems , Therefore, the file size varies , And files need to be accessed and read frequently , Frequent file reading and writing requires the ability of the underlying storage system to provide high performance .

Based on the above requirements ,GBASE NTU general and XSKY Star Trinasolar establishes strategic cooperation , This time, depth adaptation , According to various business scenarios , Both parties have conducted a lot of compatibility and performance tests on the products ,GBase 8a MPP Database side reference standard AWS S3 The agreement is standardized for commercial development and transformation ,XEOS Object storage perfectly supports this Protocol , After several rounds of testing and verification by technicians of both parties , Achieve the expected goals of performance and compatibility .

XEOS Object storage and GBASE Database docking instructions

This docking test is formulated according to the actual needs of the cloud data warehouse project of China Mobile headquarters , Provide decision-making reference and support for later platform construction . This test includes function test 、 Performance testing , The test results are generally normal and meet the test expectations .

A functional test ： Coverage criteria SQL、 Data loading 、 Table data storage and distribution 、 Development interface 4 In terms of , common 15 Test items , All functional test items fully support , All pass , Specific test results , The details are as follows ：

Through this function and performance test , Have achieved the expected goal ,GBase 8a MPP Database and XSKY Enterprise class object storage XEOS Fusion , The system operates reliably and stably , Excellent performance .

XEOS Advantages of object storage products

XEOS Object storage , yes XSKY The company has independent intellectual property rights 、 Independently develop enterprise level software definition storage products for multi business scenarios , It is based on domestic CPU、 The domestic operating system and its application have been deeply adapted and optimized , Through to AWS S3、HDFS、NFS And S3 Ecosystem and other agreements , The stability of 、 reliability 、 Ease of use and performance meet the needs of various complex applications .

GBase 8a MPP Product advantage

GBase 8a MPP yes GBASE It is independently developed by Nanjing University General Motors , A high-performance domestic analytical database product for massive data query and analysis applications , It is used to meet the increasing data query in various data intensive industries 、 Data statistics 、 Data analysis 、 Data storage such as data mining and data backup 、 Manage and process requirements , It can be used as a data warehouse system 、BI System and decision support system . It's mainly used in finance 、 telecom 、 Energy and other industries with massive business data have been applied on a large scale , Outstanding performance in business analysis and business intelligence market .