System integration under microservice architecture

source ： Outside the yard

author ： Ma Guangguang  

Advantages of microservice architecture over single architecture , You can list many ： Service individuals are smaller , More cohesive , Clearer business responsibilities , More reusable , You can deploy and publish independently, and so on ; From the perspective of software development , Flexibility and efficiency will be greatly improved ……

However , Microservice architecture is essentially a distributed system architecture , Each service needs to cooperate to complete the product requirements , The dispersion of business data makes it necessary for services to complete collaborative work through integration , So here comes the question , How to integrate ？ What principles should be followed ？ How to design to ensure the consistency of data between different services ？

 

One 、 Chaotic reality

Recommended for microservice architecture REST As a way of synchronous communication between services , For non real time requirements , Asynchronous collaboration can be implemented based on events . This principle is very simple , It's also very easy to achieve , However, it is difficult for us to follow the path of micro service only by following this principle .

Some systems called microservice Architecture , At first, the service split was not too big a problem , But as the logic expands , The increase of cross service data exchange scenarios , This simple principle is difficult to guide daily decision-making , It even caused some new problems , Take a few examples .

1.1 Service cycle dependency

In some scenarios, services A dependent service B, The service is called B Of API, And in other scenarios , service B Services are also required A The data of , Also by calling the service A Of API To achieve . From the implementation level alone , Get data on demand , Implementation is very convenient , Can complete the effect of the business ; But in the long run , The coupling between the two services is getting tighter and tighter , In the future, the realization cost of new demand will be higher and higher , Become a technical debt .

1.2 Third party systems are integrated into business services

In some business scenarios , The business data required by the current product needs to be obtained from several third-party systems and integrated, or even used after some calculation , Some previous projects were implemented in the early stage , After the data is obtained from the third-party system , After processing, it is directly written to the business database , The integrated code is written directly to the business service , It doesn't seem like a big problem , Just isolate the code .

But actually , Some thorny problems occurred in the later stage, so we had to make changes ：

• Some timing triggered integration tasks , It will only run on one service instance at a time , There may be a lot of data reading during operation 、 Calculation 、 to update 、 Insert, etc , It will occupy a large amount of resources of the current service instance for a short time , In severe cases, the current service instance can't even provide services normally .

• Similar integrations are integrated into business services in the same way , For example, product inventories of different brands will be obtained from multiple third-party systems , Business services become bloated .

• Some changes in the integrator have a direct impact on business services , such as API The upgrade , This change can only be completed by re upgrading and deploying business services .

1.3 The integration interface is not idempotent

Whether in the process of third-party system integration and internal service invocation , If the interface is not idempotent, it will cause the problem of data inconsistency . The most typical scenario is service A Call the service B Interface to update or write data , service B Handle a successful , But because of the Internet , service A No service received B Response , service A Retry calling interface , here , service B An unexpected response was returned because the interface is not idempotent , The business process cannot continue .

The ideal is full , The reality is skeletal , Most projects begin with the passionate dream of a clean Architecture , The development work is not as smooth as expected , With the replacement of project personnel 、 High delivery pressure 、 Insufficient personnel capacity, etc , Architecture began to move in an unexpected direction , Technology debt is high , While the siege lions are struggling to catch up with the progress , I can only look at the debt platform and sigh .

 

Two 、 The principle of integration

When problems happen , Architects can stand up for the first time and say that the design is too bad , How can it be done like this ; Hindsight is an important means for us to accumulate experience , From the high-tech debt platform , From the online problems tired of fighting fire , We can't find a reasonable solution after receiving the new demand …… We have been summing up experience , To avoid falling in the same place again , What architectural design principles can go first ？

2.1 One way dependency

At the beginning of micro service splitting, the upstream and downstream relationships of each service are defined , We can define the dependencies of upstream and downstream services as follows ：

• Downstream services can directly rely on upstream services , Downstream services can be provided through upstream services API Read and write data of upstream services synchronously .

• Upstream services do not rely on downstream Services , If the upstream service data status changes, it will affect the downstream service , You can publish domain events , The downstream system listens to events and makes corresponding operations .

2.2 Only redundant reference information between services

There are various relationships among various domain entities of microservices , When domain entities A Dependent domain entities B Information time , You usually need to create entities in the domain A Retain some domain entities in B Copy information for , What information should this copy contain ？

for instance , Domain entity A It's an order , Domain entity B It's the customer , Every time the client queries the order information , All require to find out the customer's name at the same time 、 Gender and cell phone number , Will this 3 A key information redundancy can easily meet the demand in the service of the order , Everything sounds so beautiful .

However , this 3 If a key message can be changed , So here's the problem , If domain entity B The information in has been updated , Domain entity A Do you want to update the information in , If not updated, the data will be inconsistent , If the update adds complexity for no reason , And it is likely to produce cyclic dependency .

therefore , When the accuracy of data changes cannot be determined , It is safest to keep only the reference information in the copy , When complete information is needed, you can query relevant information by reference .

2.3 Build appearance services for third-party systems

Facing third-party system integration , In the process of system integration, we should consider more comprehensively , In general , We have no control over third-party systems , In case of integration problems , Need to communicate solutions , Scheduled development , Joint commissioning test, etc , The repair cycle is very long .

Another problem is that the technology stack or integration mode commonly used by third-party systems may be completely different from our system . For example, it may be a very old system , The interface provided is based on XML Of SOAP; Another example is that it may not provide API, You can only export files to a SFTP Or send mail ……

In order to minimize the impact of third-party system integration , We tend to build facade services to hide the details of third-party system implementation , Packaging the functions of the third-party system through appearance service , Provide a more consistent integration mode with the current system . We can regard the appearance service and the third-party system as a whole , Then the integration of the third-party system is important for the services in the current system , Integration between services can be handled just like internal services . For the third-party system integration, the relevant details are isolated in the appearance service , Changes in third-party system integration , Just modify the appearance service .

2.4 Idempotency should be considered in the implementation of integration interface

This principle is very simple , But the implementation process is very easy to be ignored , If not in the designed test case , It is also difficult to find problems during the test ; In most cases, it's online , The user will find the problem only in the real scene , This has had a business impact .

Therefore, in the interface involving Integration , Pay special attention to , Whether the idempotency of the interface is required in the business , Whether the business can flow normally when the interface is not idempotent .

2.5 Contract test

The interface provided by a service usually has more than one consumer , Different consumers often care about different information , As the number of services increases , Without contract testing , This is likely to happen because a requirement modifies the implementation of the current service interface , And affect other existing functions . And often this problem can not be fully discovered until the full return , It may even happen because the scope of change is very small , Return is not enough , It's hard to find before going online .

Contract testing can solve this problem well , On the one hand, test the front , Provide feedback early , On the other hand, it also plays the role of architecture guard .

 

Summary

System integration is a problem that must be discussed in distributed systems , And it's a big problem , Because it directly affects the architecture of the current system , A trivial change is likely to undermine the principles of the entire system architecture , Over time, the principle will be in vain .

Microservice architecture is no exception , In the absence of architectural constraints , A quick implementation will often ruin the advantages of microservices , A small unreasonable change will gradually destroy the whole architecture building , The so-called thousand mile dike , That's the reason why you're in an ant's nest .

therefore , At the beginning of microservice architecture design , We need to establish some principles within the team , Specify some specifications that need to be followed for inter system integration , And regularly in practice review, If necessary, you can use some auxiliary tools for architecture guard , To protect the health of the architecture .