Aurora system model of learning database

overview

Aurora yes AWS Provided Relational Database Service One of , oriented OLTP Business scenario .

The core idea

The performance bottleneck of data processing has shifted from storage and computing to the network .

Solution

take RedoLog Push down from the kernel to Aurora Multi tenant scalable storage services .

Introduce

Deposit is separate

In modern distributed cloud services , elastic and Extensibility Mostly through Deposit is separate Realization , The upper layer is stateless , Lower level shared distributed storage .

The introduction of distributed storage makes IO Operations can be executed concurrently , But there are advantages and disadvantages , Because of the local IO To the network IO The transformation of , The system performance will be limited by the network environment .

besides , Traditional database architecture 2PC、3PC Distributed transaction protocol , Due to the high requirements for the network environment , Will no longer apply to shared distributed storage architectures .

Kernel disassembly

Aurora The same is to use the storage and accounting separation architecture , In order to solve the above problems , take RedoLog Push down to the storage tier , The upper layer only needs to write to the storage layer RedoLog, Reduce network IO, also RedoLog Relevant functions of, such as fault recovery 、 Backup restore , Handled asynchronously by the storage layer .

WAL Benefits of falling into the storage tier ？

Better fault tolerance .
IOPS A lower .
Asynchronous replication and recovery .

Persistence of large-scale systems

Aurora use Quorum Realize the persistence of storage layer , Set the total number of nodes to T, Every time you write N Nodes , Each read M Nodes , Yes Quorum Constraints ,N+M>T establish , And because each write needs to be aware of the last write , therefore N>T/2 establish .

Think about ,Quorum It ensures that the written data will be read .

Area and availability area

The computer rooms of cloud computing companies are generally divided into several regions （Region）, For example, a computer room in North China , A machine room in East China . Each region is divided into several isolated availability areas such as power and network （AZ）, identical Region Of AZ It can be interconnected through intranet .

Nodes are deployed in the same AZ in , Lower network latency ; Deploy in different AZ in , Better disaster tolerance . stay Auraro in , Use 3 individual AZ, Every AZ hold 2 Copies , Write 4 read 3 The plan .

Segmented storage

If an error occurs , And another error occurred before the error was fixed , that Quorum It is likely to be broken , Breaking can be reduced by reducing the time to fix errors Quorom Probability ,Aurora Solve this problem by segmented storage .

Split the database volume into several 10GB Data segment size , Each paragraph has 6 Copies , Call this 6 Copies are one Protection Group, Average distribution in 3 individual ZA On . After segmentation, the space of each segment is smaller , Data recovery is also faster .

The log is the database

As mentioned above Quorum+Group Model ,IO Will be copied and enlarged , At the same time a lot of IO There will be synchronous waiting , Even with chain copying , We also have to face the problem of synchronization delay .

Traditional database writing will modify the data page first , Then record WAL, And in the Aurora in , Data pages will not be changed at the database level , Only records WAL, The log application will be applied asynchronously in the background WAL Generate data pages .

In the structure above , The primary instance logs to the storage layer , wait for 6 One copy of 4 Responses succeeded , The copy is responsible for applying the log to the data page . Although this is the same 6 Times magnified the write operation , But it greatly reduces the network load .

Principle of consistency

Because of the log , So there's no need for 2PC Let's do consistency , Instead, state consistency can be maintained asynchronously . stay Aurora in , Each copy maintains two values ：VCL and CPL, Respectively represents the maximum applied log serial number , And the agreed maximum log serial number . And when recovering, it is more than VDL（ Less than or equal to VCL One of the biggest CPL） Log is truncated .

I can't help thinking of Raft, This discord lastCommitted and lastApplied What is the same ？

About whether the latest data can't be read by the read operation ,Auraro Also associated with a read operation LSN（ Log serial number ）, Ensure that the read data pages are up-to-date through the sequential application of logs .

End

This article mainly talks about share disk、6 copy 、WAL These things , Not very detailed , It's mainly because I'm afraid of writing more and making more mistakes , For more details, please see the original . Non professional articles , If there is any mistake, please point it out .