当前位置:网站首页>Aurora system model of learning database
Aurora system model of learning database
2022-07-06 13:23:00 【Li Suqing】
overview
Aurora yes AWS Provided Relational Database Service One of , oriented OLTP Business scenario .
The core idea
The performance bottleneck of data processing has shifted from storage and computing to the network .
Solution
take RedoLog Push down from the kernel to Aurora Multi tenant scalable storage services .
Introduce
Deposit is separate
In modern distributed cloud services , elastic and Extensibility Mostly through Deposit is separate Realization , The upper layer is stateless , Lower level shared distributed storage .
The introduction of distributed storage makes IO Operations can be executed concurrently , But there are advantages and disadvantages , Because of the local IO To the network IO The transformation of , The system performance will be limited by the network environment .
besides , Traditional database architecture 2PC、3PC Distributed transaction protocol , Due to the high requirements for the network environment , Will no longer apply to shared distributed storage architectures .
Kernel disassembly
Aurora The same is to use the storage and accounting separation architecture , In order to solve the above problems , take RedoLog Push down to the storage tier , The upper layer only needs to write to the storage layer RedoLog, Reduce network IO, also RedoLog Relevant functions of, such as fault recovery 、 Backup restore , Handled asynchronously by the storage layer .
WAL Benefits of falling into the storage tier ?
- Better fault tolerance .
- IOPS A lower .
- Asynchronous replication and recovery .
Persistence of large-scale systems
Aurora use Quorum Realize the persistence of storage layer , Set the total number of nodes to T, Every time you write N Nodes , Each read M Nodes , Yes Quorum Constraints ,N+M>T
establish , And because each write needs to be aware of the last write , therefore N>T/2
establish .
Think about ,Quorum It ensures that the written data will be read .
Area and availability area
The computer rooms of cloud computing companies are generally divided into several regions (Region), For example, a computer room in North China , A machine room in East China . Each region is divided into several isolated availability areas such as power and network (AZ), identical Region Of AZ It can be interconnected through intranet .
Nodes are deployed in the same AZ in , Lower network latency ; Deploy in different AZ in , Better disaster tolerance . stay Auraro in , Use 3 individual AZ, Every AZ hold 2 Copies , Write 4 read 3 The plan .
Segmented storage
If an error occurs , And another error occurred before the error was fixed , that Quorum It is likely to be broken , Breaking can be reduced by reducing the time to fix errors Quorom Probability ,Aurora Solve this problem by segmented storage .
Split the database volume into several 10GB Data segment size , Each paragraph has 6 Copies , Call this 6 Copies are one Protection Group, Average distribution in 3 individual ZA On . After segmentation, the space of each segment is smaller , Data recovery is also faster .
The log is the database
As mentioned above Quorum+Group Model ,IO Will be copied and enlarged , At the same time a lot of IO There will be synchronous waiting , Even with chain copying , We also have to face the problem of synchronization delay .
Traditional database writing will modify the data page first , Then record WAL, And in the Aurora in , Data pages will not be changed at the database level , Only records WAL, The log application will be applied asynchronously in the background WAL Generate data pages .
In the structure above , The primary instance logs to the storage layer , wait for 6 One copy of 4 Responses succeeded , The copy is responsible for applying the log to the data page . Although this is the same 6 Times magnified the write operation , But it greatly reduces the network load .
Principle of consistency
Because of the log , So there's no need for 2PC Let's do consistency , Instead, state consistency can be maintained asynchronously . stay Aurora in , Each copy maintains two values :VCL and CPL, Respectively represents the maximum applied log serial number , And the agreed maximum log serial number . And when recovering, it is more than VDL( Less than or equal to VCL One of the biggest CPL) Log is truncated .
I can't help thinking of Raft, This discord lastCommitted and lastApplied What is the same ?
About whether the latest data can't be read by the read operation ,Auraro Also associated with a read operation LSN( Log serial number ), Ensure that the read data pages are up-to-date through the sequential application of logs .
End
This article mainly talks about share disk、6 copy 、WAL These things , Not very detailed , It's mainly because I'm afraid of writing more and making more mistakes , For more details, please see the original . Non professional articles , If there is any mistake, please point it out .
边栏推荐
- Application architecture of large live broadcast platform
- First acquaintance with C language (Part 2)
- Dark chain lock (lca+ difference on tree)
- 7.数组、指针和数组的关系
- View UI plus released version 1.3.1 to enhance the experience of typescript
- Implement queue with stack
- 165. Compare version number - string
- List set map queue deque stack
- 一文搞定 UDP 和 TCP 高频面试题!
- MySQL 30000 word essence summary + 100 interview questions, hanging the interviewer is more than enough (Collection Series
猜你喜欢
抽象类和接口
阿里云微服务(三)Sentinel开源流控熔断降级组件
Alibaba cloud microservices (I) service registry Nacos, rest template and feign client
View UI plus released version 1.3.0, adding space and $imagepreview components
Design a key value cache to save the results of the most recent Web server queries
TYUT太原理工大学2022数据库大题之数据库操作
TYUT太原理工大学2022软工导论大题汇总
TYUT太原理工大学2022数据库大题之概念模型设计
E-R graph to relational model of the 2022 database of tyut Taiyuan University of Technology
图书管理系统小练习
随机推荐
Experience summary of autumn recruitment of state-owned enterprises
Share a website to improve your Aesthetics
7.数组、指针和数组的关系
Branch and loop statements
All in one 1405: sum and product of prime numbers
List set map queue deque stack
E-R graph to relational model of the 2022 database of tyut Taiyuan University of Technology
TYUT太原理工大学2022软工导论考试题型大纲
凡人修仙学指针-1
Record: solution of 404 error of servlet accessing database in dynamic web project
Redis介绍与使用
(超详细二)onenet数据可视化详解,如何用截取数据流绘图
阿里云微服务(三)Sentinel开源流控熔断降级组件
阿里云微服务(四) Service Mesh综述以及实例Istio
Alibaba cloud microservices (III) sentinel open source flow control fuse degradation component
MySQL backup -- common errors in xtrabackup backup
Questions and answers of "signal and system" in the first semester of the 22nd academic year of Xi'an University of Electronic Science and technology
arduino+DS18B20温度传感器(蜂鸣器报警)+LCD1602显示(IIC驱动)
初识C语言(下)
分支语句和循环语句