How to improve your system architecture?

source ： Amoy Technology

System design and architecture theory , The answer is very broad , Basically all technical theories can cover . As a code fast 10 Back end technologists in 2000 , Give me a brief comment .

System design and Architecture , It is still closely related to the business type of the system , For example, traditional business systems mainly focus on domain modeling and design , High concurrency 、 High availability 、 Data consistency and other systems , When designing, it will be quite different from the business system , So this is for different types of systems , Let's briefly introduce some difficulties and solutions in design .

Background conventional business system design is the key —— Domain model

The key of business system design is how to define the model of the system and the relationship between the models , The main part is the definition of domain model , When we determine the model , The relationship between models will also be clear .

Model design can refer to the classic books of domain model 《Domain-Driven Design》 A Book , Through this basic, we can define the domain 、 Anticorrosive coating 、 There is a clear understanding of concepts such as anemia model .

The domain model system in a single application also needs to pay attention to domain layering , As a developer, have you seen 、 Refactored a lot Controller-Service-DAO Layered design of style code ？ It often makes people vomit blood when doing reconstruction .

For better domain design, here is a layered suggestion ：

▐   The interface layer Interface

Mainly responsible for interacting with external systems & signal communication , For example, some dubbo service 、Restful API、RMI etc. , This layer mainly includes Facade、DTO Some more Assembler.

▐   application layer Application

The main component of this layer is Service service , But pay special attention to , This layer Service It's not simple DAO Layer packaging , In the framework of Domain Driven Design ,Service The first floor is just one floor “ thin ” A layer of , It doesn't implement any logic inside , Just responsible for coordinating and forwarding 、 Delegate business actions to the lower domain layer .

▐   Domain layer Domain

Domain Layer is the core of domain model system , Responsible for maintaining object-oriented domain model , Almost all business logic will be implemented in this layer . The interior mainly includes Entity（ Entity ）、ValueObject（ The value object ）、Domain Event（ Field events ） and Repository（ Storage ） And many other important domain components .

▐   Infrastructure layer Infrastructure

It is mainly Interfaces、Application and Domain Three layers provide support . All related to specific platforms 、 Framework related implementations will be implemented in Infrastructure Provided in the , Avoid three layers, especially Domain Layer doping into these implementations , thus “ Pollution ” Domain model .Infrastructure One of the most common facilities in is the concrete implementation of object persistence .

High concurrency system design

Are you often asked a question in an interview ： If your system traffic increases N How do you redesign your system ？ This highly concurrent problem can be solved at all levels , such as

▐   The code level

• Lock the optimization （ Adopt lockless data structure ）, Mainly concurrent Below the package is about AQS Some contents of the lock

• Database cache design （ Reduce the pressure of database concurrent contention ）, There will be another cache 、DB Data inconsistency , In practical use , High concurrency systems and data consistency systems adopt the opposite strategy .

• Merge update is adopted for data update , You can update and merge in the application layer , The same Container There will only be one at a time DB Update request .

• Others are based on BloomFilter Space for time 、 Reduce processing time through asynchronization 、 Concurrent execution through multithreading and so on .

▐   database level

• Different storage models are selected according to different storage demands , From the early RDBMS, Until then NoSql（KV Storage 、 Document database 、 Full text indexing engine, etc ）, To the latest NewSql（TiDB、Google spanner/F1 DB） wait .

• Table data structure design , Field type selection and difference .

• Index design , We need to pay attention to the principle of cluster index and the elimination of overlay index sorting , As for the leftmost matching principle, it is common sense of rotten Street , Advanced indexing eliminates some sorting mechanisms, and so on ,B+ Trees and B The difference between trees .

• The last conventional means ： Sub database and sub table 、 Read / write separation 、 Data fragmentation 、 Hot data splitting and so on , High concurrency tends to divide data into buckets , There's a lot more in it , For example, how to initialize a bucket 、 Routing rules 、 How to merge the data in the final stage , The classic way is to divide the bucket into a main bucket +N A bucket .

▐   Architecture design level

• The distributed system is service-oriented

• Stateless supports horizontal elastic expansion and contraction

• Business logic level failfast Fast failure

• Call link hotspot data

• Multi level cache design

• Advance capacity planning, etc

High availability system design

For systems with very high availability requirements , We usually say a few 9 The availability of , such as 99.999% etc. .

In the face of high availability system design, it can also be analyzed from all aspects

The code level ： Need to focus on distributed transactions ,CAP Theory is the routine of interview

Software level ： Applications support statelessness , Multiple modules deployed are completely equivalent , The processing results of the request in any module are completely consistent => The module does not store context information , Process only according to the parameters carried by the request . The purpose is to expand and contract quickly , Service redundancy . Common examples are session Problems, etc. .

▐   Load balancing problem

After deploying multiple copies of the software , How to ensure the system load ？ How to choose to call the machine ？ That is, load balancing

• In a narrow sense, load balancing can be divided into these types ：

1. Hardware load ： such as F5 etc.

2. Software load ： such as LVS、Ngnix、HaProxy、DNS etc. .

3. Of course , There is also load balancing on the code algorithm , such as Random、RoundRobin、ConsistentHash、 Weighted rotation training and other algorithms

Load balancing in a broad sense can be understood as the ability of load balancing , For example, a load balancing system needs the following 4 Ability ：

1. The faulty machine finds out automatically

2. The fault service is automatically removed （ Service failure ）

3. Request automatic retry

4. Service recovery auto discovery

▐   Idempotent design problem

When load balancing is mentioned above , Generalized load balancing needs to complete the automatic retry mechanism , So in business , We must guarantee idempotent design .

Here you can go from 2 Consider at three levels ：

• Request level

  Since the request will be retried, idempotent must be done , You need to ensure that the request is repeated and the result is exactly the same . Idempotent design at the request level requires idempotent design at the data modification level , That is, the data access layer read request is naturally idempotent , Writing a request requires idempotent . Read requests are generally naturally idempotent , No matter how many times you query, the returned results are consistent . The essence of this is actually a distributed transaction problem , Here, let's introduce in detail .

• Business level

  Non idempotent can cause problems such as multiple rewards 、 Repeat orders and other very serious problems . Idempotence at the business level is essentially a problem of distributed locking , Later on . How to ensure that there are no repeated orders ？ Here, for example token Mechanism and so on . How to ensure the goods are not oversold ？ Such as optimistic lock, etc .MQ How to guarantee the power of the consumer is a common question in the interview .
  

▐   Distribution contraction

Idempotent design at the business level is essentially a distributed lock problem , What is distributed lock ？ Globally unique resource of lock in distributed environment , Serialize the request , Actually represent mutex , Solve the business layer idempotent problem .

The common solution is based on Redis The cache setnx Method , But as a technician, it should be clear that there are still single problems 、 Unable to renew the lease based on the timeout 、 Asynchronous master-slave synchronization, etc , Deeper ,CAP theory , One AP The system is essentially unable to implement a AP demand , Even if it's RedLock Not good either. .

How do we design a distributed lock ？ Strong consistency 、 High availability of service itself is the most basic requirement , Others, for example, support automatic renewal , Automatic release mechanism , Highly abstract and easy to access , visualization 、 Manageable, etc .

Reliable storage tier based solutions such as ：

• zookeeper

  CP/ZAB/N+1 You can use ： Based on temporary node implementation and implementation Watch Mechanism .

• ETCD

  CP or AP/Raft/N+1 You can use ： be based on restful API;KV Storage , Strong consistency , High availability , Reliable data ： Persistence ;Client TTL Pattern , Need heartbeat CAS Unique voucher uuid.

▐   Fusing of services

After micro service , System distributed deployment , Between systems through RPC Communications , The probability of failure of the whole system increases with the increase of system scale , A small fault is amplified by link conduction , It may cause greater failure . I hope that when calling the service , When the service quality of some non critical path services decreases , Choose to shield the impact as much as possible .

Most of them return to the default value null, You can also customize it ,RPCClient Native support is best , The business party should not change the code （ Where the fuse is placed ）, When entering the fuse , Print fuse log , At the same time return to Exception（ The business party customizes the circuit breaker method ）, A service governance platform is needed , You can see the status of the service 、 Whether to downgrade 、 Is it fused 、 It can issue threshold configuration in real time .

▐   service degradation

The overall service load exceeds the preset upper limit , Or the upcoming flow is expected to exceed the threshold , In order to ensure the normal operation of important or basic services , Refuse some requests or delay or suspend the use of some unimportant and non urgent services or tasks .

The main means are as follows ：

• Service layer degradation , The main means

1. Reject part of the request （ Current limiting ）, For example, cache request queue , Reject some long waiting requests ; according to Head, To reject non core requests ; There are other current limiting algorithms, such as token bucket 、 Leaky bucket algorithm and so on .

2. Shut down some services ： Such as double 11 Great promotion 0 Point will close the reverse refund service and so on .

3. Graded downgrade ： For example, autonomous service degradation , From gateway to service to DB According to the interception 、 Business rules gradually reduce downstream requests , Reflected in the gradual decline of processing capacity from top to bottom .

Data layer degradation

For example, when the flow is large , Update requests are cached only to MQ, Read request read cache , When the flow is small , Make up operation ( If the general data access layer is downgraded , There is no need to do it in the data layer )

• Flexible availability strategy

For example, some current limiting tools that specify the maximum flow , Or according to CPU Load current limiting tools, etc , It needs to be ensured that it opens automatically , Do not rely on labor .

▐   Availability problems caused by publishing method

Publishing method is also a point that affects high availability , ha-ha , I have also experienced some cases of online direct shutdown release before （ Bank internal system ）, But as a tall Internet , It mainly adopts these publishing methods ： Grayscale Publishing 、 Blue green release 、 Canary release and so on .

Data consistency system design

Generally, some financial 、 The accounting system will be very strict on this part , The following mainly introduces the transaction consistency involved 、 Consistency algorithm, etc .

▐   Business Consistency issues

stay DB level , Usually by Rigid business To achieve data consistency , Mainly through Prewrite log (WAL) The way to achieve ,WAL(write ahead logging) The way of pre writing logs . Is all the modifications to the data file , You must write a log first , such , Even if it crashes while writing data , It can also be recovered through log files , Traditional database transactions are based on this mechanism （REDO The data of the submitted transaction also needs to be changed UNDO Rollback of uncommitted transactions ）.

In addition to this way , Another is through Shadow data block For data backup , Record the pre modification status of the modified data block in advance , Back up , If you need to roll back , Just overwrite the backup data block directly .

Others are based on two-stage submission XA Model 了 .

But the current Internet system , Distributed deployment mode has been widely adopted , Traditional rigid transactions cannot be realized , therefore Flexible transaction has become the mainstream of distributed transaction solution , The main modes are as follows ：

• TCC Pattern / Or call it 2 Stage mode

  stay try Pre deducted resources at this stage （ But don't lock resources , Improve usability ）, stay Confirm perhaps Cancel Data submission or rollback in phase . It is generally necessary to introduce coordinators , Or transaction manager .

• SAGA Pattern

  Each participant in the business process commits a local transaction , When a participant fails, it compensates the previous successful participant , Support forward or backward compensation .

• MQ The business news of

  Is to start first halfMsg, After processing , Send again commit perhaps rollback Msg, then MQ Will ask regularly producer ,halfMsg  Can you commit perhaps rollback, Finally achieve the final consistency of the transaction . In fact, the compensation action is entrusted to RocketMQ.

• Segment things （ Asynchronously ensures ）

  Based on reliable information + Local transaction message table + Message queuing retry mechanism . At present, this is also the mainstream scheme of some large factories , The interior is generally called segmented things .

Flexible transactions are basically implemented based on final consistency , So there must be compensate The action is inside , Before reaching final consistency , General presentation to users Soft state .

One thing to note is that , Not all systems are suitable for introducing a data consistency framework , For example, users can modify their requests at any time , for example , The merchant sets up the background system , The merchant will modify the data at any time , If consistency is involved here , The introduction of a consistency framework will lead to compensation actions before reaching final consistency , The resource lock will block the user's subsequent requests . Resulting in poor experience . In this case, it is necessary to ensure data consistency through other means , Such as data reconciliation .

▐   Consistency algorithm

From the early Paxos Algorithm , And then to the later derivative zab agreement （ Reference resources ：A simple totally ordered broadcast protocol）, It provides a reliable distributed lock solution at present . And then later Raft Algorithm （In Search of an Understandable Consensus Algorithm）, They are also some knowledge points that need to be learned in distributed system design .

Last

Here is a brief introduction to some difficulties faced in the design of different systems , Basically every point inside , It is the predecessors' continuous exploration on the road of solving various difficult problems , Finally, these industry solutions , In front of everyone , As a technician , It's only a matter of time to learn these technical points , But this discovery is problematic 、 Face the problem 、 When it comes to problem-solving ability and spirit, we are most worthy of learning , It is also a necessary ability to be a system designer or architect .

---

“ Technology leadership ” Video Number opened ,K Brother Arrange a live broadcast every week , The topics we talked about before are ： How to deal with the midlife crisis in the workplace 、 How to make money from sidelines 、 Technology management practice ...... Pay attention to , Don't miss K Every time my brother's live broadcast .

Author's brief introduction ：Mr.K,“ Technology leadership ” official account   author , From an ordinary programmer to a Technical Executive of Unicorn . Published many best sellers , Write multiple articles 10W+ article . Share ： Industry trend 、 Workplace experience 、 Team management 、 Topics such as business general education .

notes ： Dispatch 24 You can reprint it in hours , Reprint please indicate ： author 、 source 、 The authors introduce . Once the non-standard reprint is found , Will be permanently disqualified from reprinting .

 -END- 

---

5 Ten thousand yuan 《 The Handbook of making money in sidelines 》, Free ！
Focus on What about sidelines , reply ：fuye

---

tencent / Google 《 Meta universe research report 100 page 》 download

Focus on AI Professor Panda , reply ：yyz666

---

Everyone was watching ：

1.39 Year old Ali P9 unemployment , assets 1.5 Billion ！

2. Grieved ！B standing 25 Year old employee died suddenly at night

3. Postponement of lifting the ban on didi shares , My loss 6000 ten thousand

4. How to bring down a team with agility ？

5. Why? CTO Don't write code , It's still so awesome ？

6. How to quickly reduce the enthusiasm of an employee