[teacher Zhao Yuqiang] Alibaba cloud big data ACP certified Alibaba big data product system

Alibaba's big data product system is based on the data processing services of Alibaba cloud Apsaras platform . It is mainly divided into Alibaba cloud big data infrastructure products and Alibaba cloud digital plus platform , The product architecture is shown below ：

One 、 Alibaba cloud big data infrastructure products

1、 Cloud database ——RDS（ApsaraDB for RDS For short ）

• Stable and reliable 、 Scalable online database services
• Open and use ,DMS Visual interface
• compatible MySQL,SQL server,PG Relational database
• Provide database online expansion , Backup rollback , Performance monitoring and analysis functions
• Read only instances and temporary instances

advantage ：

• hot standby —— Second switching , Service availability reaches 99.5%
• Safety protection —— prevent DDOS attack ,SQL Injection warning , Multiple backup of data
• Simple and easy to use —— One click data migration , Visual management operations

2、 Distributed relational database ——DRDS（Distributed Relational Database Service）

• be based on RDS Distributed data storage and retrieval products
• Horizontal split can be expanded smoothly
• Solve user list RDS Unable to support the suffering of the business
• Reduce the difficulty of using distributed database

advantage ：

• Simple and easy to use —— compatible MySQL( Interaction protocol 、SQL)
• Stable and reliable —— Share Alibaba components
• Distributed —— Horizontal split , The capacity is 100 times that of a single node
• Scalable —— Increasing or decreasing nodes has little effect on the application , Efficient data migration

3、 Form storage ——TS（Table Store）

• Built on Alibaba cloud Apsaras distributed system NoSQL Data storage services
• Massive structured data storage and real-time access
• Flexible resource reservation
• Real time monitoring display

advantage ：

• Stable —— Automatic fault detection and recovery , System availability 99.9%
• Security —— User level data isolation 、 Access control and rights management , Data redundancy backup
• On a large scale —— From a single watch to a hundred TB Level data storage
• High performance —— Millisecond level single line read and write latency , One hundred thousand QPS

4、 Analytical database ——ADB（Analytic DB）

• Cloud computing platform for real-time and high concurrency online analysis of massive data
• Free computing and query capabilities
• High availability and high security
• Full compatibility MySQL agreement

advantage ：

• A high degree of computational freedom —— adopt SQL Flexible multidimensional analysis 、 Pivoting 、 Data filtering, etc
• Rapid response time —— Millisecond level of 100 billion level data perspective , Millisecond level large table Association computation
• Easy to use —— standard SQL, Support standards MySQL agreement , Built in a variety of cloud platform data input and output
• Rich features and functions —— High performance automatic indexing , The rapid export of massive data, etc

5、 Big data computing services ——MaxCompute

• in the light of TP/BP Level data 、 Real time requires less distributed processing power
• Big data computing power
• Open the box
• Data security

advantage ：

• Distributed —— Distributed cluster architecture , Flexible extension
• Security —— Automatic storage fault tolerance mechanism , All the calculations are done in the sandbox
• Ease of use —— Full support based on SQL Data processing of , Provide standards API, Data upload and download with high concurrency and high throughput
• Management and Authorization —— Multi user management and collaborative data analysis , There are many ways to manage user rights , Flexible data access control decisions

6、 Data integration （Data Integration）

Data integration is a stable and efficient service provided by Alibaba group 、 Elastic and scalable data synchronization platform , Offline services for alicloud big data computing engine （ Batch ） Data access .
advantage ：

• many ： There are many kinds of data sources , Multiple data channels , Complete data transmission mode , Rich data processing plug-ins ;
• fast ： Efficient way to call , Strong transmission speed , Powerful throughput ;
• good ： Robust transmission channels , Intelligent error detection , Automatic transmission recovery ;
• province ： Open the box , Dynamic allocation , Stretch , Apply on demand , Pay as you go ;

7、 Object storage （Object Storage Service, abbreviation OSS）

• For amounts 、 Security 、 Low cost 、 Highly reliable cloud storage services ;
• Open and use , Storage clusters with infinite space ;
• adopt API/SDK Interface or OSS Migration tools are convenient to move massive data in or out ;
• Storage object operations are atomic , Strong consistency ;

advantage ：

• reliable ： Service availability 99.99%, Data persistence 99.999999999%（9 individual 9）, Multiple backups , Scale automatically ;
• Security ： User level resource isolation , Disaster tolerance in other places , Enterprise level multi-layer security protection , Multiple authorization mechanisms ;
• Low cost ： Multi line GBP Backbone network , No bandwidth limitation , Upstream traffic is free ;
• Multiple types of data processing capabilities ： The image processing 、 Audio and video transcoding 、 Content accelerated distribution 、 Yellow service , Archiving services, etc .

Two 、 Alibaba cloud digital plus platform

Alicloud's data products are integrated into the data plus platform , The positioning of alicloud's public cloud plus platform ： One stop data platform （ Integration includes from basic data analysis and application to big data development 、 Dispatch 、 Operation and maintenance , To machine learning and so on ）; Three layers of service （ Underlying computing 、 Data platform analysis tools 、 Application layer service ） Industry solutions .

1、DataWorks（ primary Data IDE）

Data workshop DataWorks（ The original big data development kit Data IDE） Is based on MaxCompute As a computing and storage engine, massive data offline processing and analysis platform for workflow visualization development and hosting scheduling operation and maintenance .
advantage ：

• major ： Ali for many years DW/BI Experience precipitates , Full link solution , High efficiency and low cost ;
• Powerful ： Integrated component services , Multiple heterogeneous data sources support , Multi person collaborative code development , Perfect version management , minute 、 Hourly scheduling 、 Drag and drop data analysis and visualization algorithm modeling ;
• Big data processing capabilities ： Perfect fusion Max Compute, Support the orderly operation and management of 100000 level tasks .

2、Quick BI

Provide massive data real-time online analysis services , Supports drag and drop operation , It provides rich visualization effects , It's easy to do data analysis , Business data exploration , Report making, etc .
advantage ：

• Low threshold ： Drag and drop , Simple and easy to use ;
• Strong function ： A variety of solutions , Rich means of presentation ;
• Big data processing capabilities ： Data analysis , Powerful data processing capabilities

3、 machine learning PAI

• be based on MaxCompute、GPU colony , Support MR、MPI、SQL、BSP、SPARK And so on ;
• Built in Ali 、 Distributed algorithm of ant precipitation , Support 10 billion level data training ;
• WEB Interface , By dragging 、 PULL 、 Drag and other ways to complete the complex data mining process ;

advantage ：

• Provide a one-stop platform service from data preprocessing to model evaluation , Significantly reduce the threshold of big data algorithm modeling ;
• Support custom algorithms and components , Flexible and open personalization , It greatly improves the modeling efficiency ;
• Provide rich distributed algorithms , Improve model accuracy , Help mining business value from massive data .