Trend of the times - the rise of cloud native databases

Preface

• Talking about databases , In the early days when we deployed the project , Enterprises need to deploy special machines as special database servers , Whether it is for customer deployment or internal needs , This undoubtedly increases the operation and maintenance cost , You will also encounter many unexpected problems , For example, the customer's machine cannot be connected to the Internet , A series of problems such as machine selection need us to spend some time to solve .
• With the development of the Internet , Cloud computing , Cloud primitives appear one after another , Traditional databases can't meet the needs of enterprises , So the cloud native database was proposed .
• The user base of the Internet is increasing step by step , The traditional database has gradually become a little difficult , The emergence of cloud native database completely breaks the bottleneck of traditional database , More and more enterprises are turning to cloud native databases .
• So , Amazon cloud native database has launched a series of activities for us , Amazon cloud technology provides 100 Free packages for more than products . among , Computing resources Amazon EC2 First year 12 It's free for three months ,750 Hours / month ; Storage resources Amazon S3 First year 12 It's free for three months ,5GB Standard storage capacity ; Database resources Amazon RDS First year 12 It's free for three months ,750 Hours ;Amazon Dynamo DB 25GB storage capacity Free forever . Direct activities

The pain points of traditional databases

Availability and data reliability issues

1. Deployment phase
   You need to purchase servers offline , Then manually build the traditional database , The operation configuration is complex , The deployed server needs to be placed in the company's computer room , The database deployment phase requires a certain amount of human and material resources .
2. Maintenance phase
   It is not only necessary to arrange personnel to ensure the safe and stable operation of the server , Also set up master-slave replication by yourself , build by oneself RAID redundancy , Self built monitoring , Purchase exclusive load balancing equipment offline and manually configure load balancing strategy . Because most of the machines are deployed in the company , If the server is powered off due to physical factors 、 Uncontrollable reasons such as man-made damage , It will cause great losses to the enterprise .

System security issues

1. Cost
   For database security , If you want to ensure the security of data , Need to purchase relevant safety software and hardware to maintain the security of data , Not only the cost of previous data servers , It also increases the cost of safety , Undoubtedly, it increases the cost of the enterprise .
2. Maintenance
   Not only the software and hardware purchased by ourselves need to rely on the operation and maintenance personnel to configure and maintain data security , Developers also need to abide by the corresponding code of conduct to avoid the security problems caused by data , When there is a data problem , We should also repair the vulnerability of database security by ourselves .

Resource utilization

1. The impact of time
   In the process of purchasing servers , Not only in the enterprise, the approval process is cumbersome , The process is slow , Unable to expand the number of data servers in time , It will cause certain losses to the enterprise .
2. Impact in use
   Due to different usage scenarios , The data server needs to consider different carrying capacity , If it is a self built traditional database, the peak value should be considered , Make some redundancy in resource utilization , Ensure safe operation , Because if the settings are less , Expansion is also troublesome , Different services have peak and low peak periods , In the low peak period, idle resources cause waste . Like some special scenes of the secsha , Need to do some redundancy , Thus, the data server cannot be fully and effectively used in the low peak period .

The hosting problem of the system

1. Traditional databases cannot be managed automatically , In the deployment , A series of problems in operation and maintenance require special technicians to operate and maintain , In addition to spending a lot of hardware costs 、 It also costs personnel 、 Time cost .

The problem is coming. , What is the high-profile cloud native database ？

Cloud native database is essentially built through visual pages on the cloud platform 、 distribution 、 A series of activities to deploy corresponding services . Users who use the cloud native database can store data on the corresponding cloud platform 、 Data management 、 Operations related to data extraction , If we want to define it , Then we may think that it is a database that can make full use of the advantages of cloud Nativity . If you want to experience cloud native database for free , Amazon cloud native database provides free resources and tutorials for Xiaobai carefully , Database free trial links and tutorials .

Cloud native database solution

Aiming at a series of problems of traditional database , Cloud native database puts forward corresponding solutions

Availability and data reliability solutions

1. Compared to traditional databases , Offline purchasing servers are no longer needed , Manual deployment is no longer required , It can be operated and managed through the visual panel , Easy to deploy globally , Ensure global customer availability , Because the physical location of the server can be different , The influence caused by physical factors can also be ignored .
2. Cloud databases generally provide active and standby nodes , At the same time, two nodes can be in different time zones in the same region . Cross region active and standby can provide greater availability , The generated database is a highly available architecture , High availability 99.9999%, Amazon cloud service provider provides automatic backup , Fast fault recovery , On the other hand, cloud database can enable read-write separation for free to realize load balancing , Read write separation, easy to use , Such as Amazon cloud Amazon RDS Will create and save SQL Server Automatic backup of instances .

System security solutions

1. Compared with traditional database, it faces security problems , The cloud native database comes with an anti DDos attack , Flow cleaning and other functions .
2. Compared with the traditional database built on the self purchase server , Special technicians are needed to maintain data security vulnerabilities , For cloud native databases , You can repair database security vulnerabilities by yourself , It saves the cost of technicians .
   Support multiple regions 、 The master replicate more , And provide complete data supervision with multiple security levels , Includes network isolation and end-to-end encryption .AWS Databases provide the high availability required for business critical enterprise workloads 、 Reliability and safety .
3. AWS Databases are built for critical business workloads . You can build extensible 、 reliable 、 Secure enterprise applications , At the same time, in the high security data center AWS Protect your data behind the infrastructure .

Resource utilization solutions

1. Compared with the traditional database built on the server, we need to consider not only the peak , In the low peak period , Resource utilization is very low , For cloud native databases , Customers can purchase according to their actual needs , Can achieve better resource utilization , Of course , If you encounter peak or low peak , You can easily choose to expand or reduce the number of servers through the cloud platform , The elasticity of cloud native database can facilitate the change of specific business . such as ： double 11, JD.COM 618 Some high peak scenes .
2. It meets the requirements of different applications in server performance 、 Data access 、 Effective use of resources 、 Cost and other scalability requirements , Improve resource utilization 、 Save space cost and energy consumption . such as ： The capacity of the database is insufficient , It can be expanded separately ; Insufficient computing performance , It can also be expanded separately CPU, Fully improve the utilization of resources .

System hosting solution

1. In the amazon 2014 In, the cloud native relational database was released Amazon Aurora, Release the first Serverless Function computing services Amazon Lambda, Subvert the application operation mode , Reduce the cost of operation and maintenance personnel , Cloud database can reduce the risk of the whole enterprise from many aspects . Providers of cloud native databases can deploy secure servers through automated methods , Reduce errors caused by operation and maintenance personnel , This is the main reason for software downtime . Amazon's fully hosted service allows you to get rid of the complexity of database and data warehouse management . No server function can automatically increase or reduce throughput according to demand . Use Amazon RDS Innovate and build new applications , Don't worry about managing your own database – Self management can take time 、 Complex and costly .

If you want to catch up with the trend of cloud native database , Come and swim on the cloud with me ！ Cloud native database online conference

Cloud native database service type

For different types of data , The cloud native database makes different adaptation schemes .

Relational database

Concept ：

• Relational databases use predefined schemas and the relationships between them to store data . These databases are designed to support ACID Business , Maintain the integrity of references and strong data consistency , For traditional relational databases , similar Mysql,Sqlserve, Amazon cloud native database is provided Amazon Aurora,Amazon RDS,Amazon Redshift.

Use scenarios ：

• Because its internal relationship model refers to the two-dimensional table model , The internal connection is a data organization composed of two-dimensional tables and the connections between them , So there are social platforms in daily use scenarios 、 Community forums and other websites , Enterprise Resource Planning (ERP)、 Customer relationship management (CRM), Such as the CSDN Blog background database , JD.COM , Taobao and other background relational databases

Key database

Concept ：

• The key value database is optimized for common access patterns , Usually used to store and retrieve large amounts of data . These databases can provide fast response time even when the number of concurrent requests is very large , Amazon cloud native database provides Amazon DynamoDB, This type of database can be obtained quickly for any scale 、 Flexible and serverless NoSQL database , To support keys / Values and document workloads .

Use scenarios ：

• Because it uses a simple key value method to store data . The key value database stores data as a collection of key value pairs , Where the key is the unique identifier . Keys and values can be anything from simple objects to complex composite objects with high traffic Web Applications 、 E-commerce system 、 Game app .

1. Session storage
   A session oriented application （ Such as Web Applications ） Start the session when the user logs in , And remain active until the user logs off or the session times out . in the meantime , The application stores all session related data in main memory or database .
2. The shopping cart
   During the holiday shopping season , E-commerce websites may receive billions of orders in a few seconds . The key value database can handle a lot of data expansion and extremely high state changes , At the same time, it provides services for millions of concurrent users through distributed processing and storage .
   

In memory database

Concept ：

• In memory databases are used for applications that need real-time access to data . By storing data directly in memory , These databases provide microsecond latency for applications that are not satisfied with millisecond latency . similar Redis, Amazon cloud native database provides Amazon ElastiCache,Amazon MemoryDB for Redis, This kind of database is used to provide compatible and persistent memory database services with ultra fast performance .

Use scenarios ：

• Because of its internal structure, it adopts memory storage , Higher efficiency than disk , The application scenario is hot data , Common usage scenarios are as follows , For example, live leaderboard 、 Game leaderboards 、 cache 、 session management 、 Geospatial applications and after user login Token Token, etc .

1. Quick access to retail customer data
   With microsecond read and millisecond write latency , Provide a personalized customer experience , manage user profile 、 Preferences and inventory tracking and fulfillment .
2. Develop online games
   For large-scale 、 Game applications with low latency and high concurrency for real-time updates , Build player data storage 、 Conversation history and leaderboard .
   

Document database

Concept ：

• Document databases are designed to store semi-structured data as similar JSON Documents . These databases can help developers quickly build and update applications . As we often use it MongoDB, Amazon cloud native database provides Amazon DocumentDB（ compatible MongoDB）, Use with MongoDB Compatible enterprise level document database services can be easily extended JSON The workload .

Use scenarios ：

• The document database supports flexible indexing 、 Powerful temporary query and document collection analysis , For example, there is content management 、 Catalog 、 User profile .

1. Content management
   Document databases are a great choice for content management applications , For example, blogs and video platforms . Through the document database , Each entity tracked by an application can be stored as a single document .
2. Catalog
   Document databases are efficient and effective in storing catalog information . for example , In e-commerce applications , Different products usually have different quantities of attributes . It is inefficient to manage thousands of attributes in a relational database , And reading performance will be affected . Using the document database , The attributes of each product can be described in a single document , To facilitate management and speed up reading .
   

Wide column database

Concept ：

• Wide column storage is NoSQL A kind of database . It uses tables 、 Row and column , But unlike relational databases , The name and format of the column may vary from row to row in the same table . Amazon cloud native database provides Amazon Keyspaces, In scalable 、 Run on highly available managed wide column database services Apache Cassandra The workload .

Use scenarios ：

• Because its structure has a wide range of usability 、 Expansibility 、 High performance and high availability distributed NoSQL database . It is often used for equipment maintenance 、 Large scale industrial applications for queue management and route optimization .

1. Application data storage
   Use Amazon Keyspaces Storage and Internet of things (IoT) Information about device applications or gamer profiles .
2. Build applications with low latency requirements
   High speed data processing for applications that require a single digit millisecond delay , Such applications include industrial equipment maintenance 、 Trade monitoring 、 Fleet management and route optimization .
   

Graphic database

Concept ：

• For applications that need to browse and query the relationship between highly interconnected graphic datasets and realize millisecond delay on a large scale , You can use the graphic database . Amazon cloud native database provides Amazon Neptune, Use fast 、 Reliable graph database services build applications that handle highly interconnected datasets .

Use scenarios ：

• Because the graph database uses nodes to store data entities , Use edges to store relationships between entities . An edge always has a start node 、 The end node 、 Type and direction , One side can describe the relationship between father and son 、 action 、 Ownership, etc . Common scenarios include fraud detection 、 Social networks 、 Suggestion engine .

1. Fraud identification
   Figure database can be used for complex fraud prevention . With the help of graphic database , You can use relationships to process financial and purchase transactions in near real time . Through quick Graphic Query , You can detect , for example , The e-mail address and credit card used by potential buyers are the same as those contained in known fraud cases .
2. Recommendation engine
   Figure database is a good choice for recommended applications . Using the graphics database , You can take the customer's interest 、 The relationship between friends and information categories such as purchase history is stored in the graph .
   

Time series database

Concept ：

• Time series databases are used to efficiently collect data from time-varying data through queries across time intervals 、 Merge and refine insights . Amazon cloud native database provides Amazon Timestream, Use fast 、 Scalable and serverless time series database services store and analyze trillions of events every day .

Use scenarios ：

• Because the structure of time series database is suitable for data with time tags, that is, time series data . This type of database is often used in industry , Like the Internet of things (IoT) Applications 、 Development and operation 、 Industrial telemetry .

1. Internet of things applications
   Amazon Timestream Enables you to use built-in analysis functions （ Like smooth 、 Approximation and interpolation ） Quickly analyze the time series data generated by the Internet of things application .
2. Develop O & M applications
   Amazon Timestream It is an ideal choice for developing operation and maintenance solutions , It is used to monitor the operating conditions and usage indicators , And analyze data in real time to improve performance and availability .
   

Ledger database

Concept ：

• The ledger database provides a centralized, trusted and authoritative way to maintain scalability for each application 、 immutable 、 Encrypted and verifiable transaction records . Amazon offers Amazon Ledger Database Services (QLDB), Provide transparency using fully managed ledger database services 、 immutable 、 Encrypted transaction log .

Use scenarios ：

• Because of its special structural type , This type of database is commonly used in business , System records 、 Supply chain 、 register 、 Banking affairs . Such as Zilliant Seamlessly record audit logs that can be encrypted and verified , Implement the recording system application to create a complete 、 Centralized employee details record , For example, the payroll 、 Bonuses and benefits .

1. Realize the centralization of data records
   Implement the recording system application to create a complete 、 Centralized employee details record , For example, the payroll 、 Bonuses and benefits .
2. Maintain claim history
   Follow up throughout the life cycle of the claim , And verify the data integrity by encryption , So that the application can resist data input errors and human manipulation .
   

The development trend of Amazon cloud native database

The following is a summary of the development of Amazon cloud native database in the past ten years , Fully demonstrate the deep capabilities of Amazon cloud database .

Successful business cases

Intuit Mint Use RDS for MySQL Reduce the cost of database 25%

Cathay Pacific Airways from the local Oracle Migrate to RDS Improved performance 20%

Samsung Use AWS DMS Put it 11 Billion Users from Oracle Migrate to Amazon Aurora

summary

• Traditional database is coming to an end , Cloud native databases follow , For a series of problems of traditional database , Amazon cloud native database has provided solutions for different databases through its long-term development and exploration ,Amazon Relational Database Service（RDS） Is a collection of managed services , It can simplify setting up in the cloud 、 The process of operating and expanding databases . Contains the following popular engine choices Auroda,MySQL、MariaDB、PostgreSQL、Oracle as well as SQL Server – And use it locally Amazon RDS on AWS Outposts Deploy .
• Cloud native database is an inevitable trend for database providers to enter the cloud market , It can also effectively reduce the operation of enterprise customers 、 Operation and maintenance 、 Integration and many other aspects IT cost . The author thinks , Cloud native database will undoubtedly be the next battlefield of the database industry . Let's wait and see .

Learning platform

besides , Amazon cloud technology has created a variety of learning platforms for developers ：

• Getting started Resource Center ： from 0 To 1 Easy to get started with cloud services , Content covered ： Cost management , Start training , Development resources .
• Architecture Center ： Amazon cloud Technology Architecture Center provides cloud platform reference architecture chart 、 Reviewed architectural solutions 、Well-Architected Best practices 、 Pattern 、 Icon, etc .
• Builder Library ： Learn how Amazon cloud builds and operates software .
• Toolkit for developing and managing applications on Amazon cloud technology platform .

Exclusive benefits for fans

When you browse here , It shows that you have certain needs , To help you better use cloud database , The following benefits are only for fans .

1. Fukuichi ：100 Free packages for more than products . among , Computing resources Amazon EC2 First year 12 It's free for three months ,750 Hours / month ; Storage resources Amazon S3 First year 12 It's free for three months ,5GB Standard storage capacity .
2. Welfare II ： The latest discount gift bag ,200$CountAccording\; to\; theAndbranchAnalysisOffsetbuckleCoupon,200$ Machine learning voucher ,200$ Micro service and application development deduction voucher .
3. Welfare three ： Solution CloudFormation One click deployment template library