Philosophical enlightenment from single point to distributed

1. Introduction

I saw a technical article today , Speaking of from mysql Migrate to tidb, To avoid hot issues , take auto_increment Self increasing ID The primary key is changed to pre segmentation RowID The technique of , Thus, we can make full use of the advantages of distributed resources .

On the surface, it seems to be a small skill to deal with small problems , But if we discuss it in depth , But it can trigger the migration from single point to distributed , Including methodological thinking of data and application migration .

2. Thinking about translation and reconstruction

With the deepening of digital transformation , The increasing demand for online data , And the evolution trend of business sensitivity brought by scenario , In recent years , From single point technology system to distributed technology system , It has formed an irreversible upsurge . This involves many specific technical issues , Including the stability of the migration process 、 Uniformity 、 Continuity, etc , But the inevitable core work is to integrate data and Application “ Moved to ” In the distributed architecture . In order to pursue the efficiency of migration , The most convenient and effective way is to try to achieve “ translation ”.

In practice , There are not a few organizations that use this set of ideas to guide their work , That is, invest a certain amount of energy to do “ translation ” Tools 、 Try to choose traditional technology tools “ compatible ” Products, etc , To achieve the goal of running business on a distributed architecture soon . However , Start with the above tips , You can ask a question ： Is such a strategy right ？

Similar to the self increase above ID The needs of , During the entire migration , There will undoubtedly be many, many . Overall adoption “ translation ” The strategy of , Plus a lot of skill summary , It seems to be a feasible solution . But you should see , The primary key of the database table mentioned above ID Adaptation skills for resource dispersion , In fact, it can be seen as the adjustment of table structure , Further talk , It's a change in the model . This leads to the thinking of this article ： Migration from single point to distributed , Whether the overall model reconstruction should be the fundamental strategy , Not big “ translation ” Skill positioning under the idea ？

In order to deepen our understanding of , Strengthen the impression , Another example ： stay NewSQL Before the popularity of distributed databases , Migration from traditional databases to big data platforms is very popular , There are still many such needs today , The purpose is clear , It's also cost 、 Scalability and efficiency considerations . In this project , There are also data migration problems of traditional relational databases . Naturally, , For convenience and workload , Big data platforms try their best to provide “ compatible ” The function of , Try to shift the work as far as possible . But this has caused many problems , namely ： Tables from relational databases , Migrate to big data platform , Change the original table structure or model completely , Or try not to change , Realization “ translation ”, It will make the inherent advantages of big data platform difficult to play to a large extent . The big data platform sacrifices consistency 、 The strictness of the model paradigm and the requirements of low redundancy are in exchange for the performance improvement , In traditional relational databases, according to the Paradigm Model 、 Design concept of main and foreign key model , It is not applicable to big data platform ： A simple “ Student -- Course selection ” Relationship , In the standard relational model , Three tables are needed to build ER chart , Student table 、 Curriculum and related tables , Then perform data operations through table Association ; In the big data platform , You may need to combine these three tables into one table （ Or in a document database MongoDB Class used in JSON hierarchical model ）, This table not only logically organizes all the information together , The physical storage mode has also changed as a whole , Redundancy is not considered , Regardless of paradigm , Don't consider schema Constraints , Focus entirely on expansion and efficiency , This is in line with its fundamental design concept . in other words , In the project of migrating to big data platform , Most use “ translation ” No case of strategy should be considered successful , Never understand or combine the essential advantages of big data platform . The right thing to do is obviously ： From the beginning, from the overall strategy 、 Fundamentally reconstruct the model , Even if it seems to take a little more time at first .

thus , I believe your understanding is better than the previous ” Self increasing ID modify ” The simple example of goes deeper . Again , burgeoning NewSQL Distributed database , Although it is indeed much friendlier than the big data platform for traditional relational libraries , But the fundamental idea of its design is also different ; The same is true of distributed applications with microservicing as the core . This naturally requires , When migrating from single point to distributed , It should start from , Fundamentally and thoroughly respect the concept of distributed design , Reconstruct the overall model , Rather than “ translation ” For strategy . That's the only way , Can we really make data 、 Make application and distribution integrate organically , Instead of seeming together .

Someone will say , Then I'll use “ translation ” Strategy , Go online as soon as possible , And then transform , Would you? ？ The point here is not to suggest . Because in that case , Many problems will be hidden , Always lurking , It will even cover most of the key issues ; And may encounter more workload in the later stage , Even lead to the failure to implement in the mechanism . Only at the beginning , In an all-round way “ restructure ” For guiding strategy , In order to drive people to fundamentally combine the distributed technology concept to reconstruct and design the original system .

Draw a little deeper , Again , Cloud native migration , It's the same thing .

As a saying goes ：“ Ordinary people change the results , Good people change reasons , And the best people change the model ”.

3. A little philosophical enlightenment

Come here , The point of view should be relatively clear , But this article also wants to further introduce , Do some philosophical thinking from single point to distributed .

Distributed architecture allows many single points to do one thing in parallel at the same time , Its essence is the improvement of resource utilization . If a single point system has the same level of resources as a distributed system , Such as calculation and storage , that , Theoretically speaking , If the resource utilization of a single point system can be the same as that of a distributed system , The efficiency of both should be the same , Distributed has no advantage . however , However, the distributed system allows users to flexibly increase or decrease their constituent cells , The single point system can only add or subtract a whole , Obviously , From the perspective of elasticity , Only distributed .

that , If you put a pile MySQL Pile up , Is it distributed ？ Strictly speaking , not . Because this pile MySQL, There is no external integrity , Thus greatly increasing the complexity of the problem . Distributed systems need external integrity , in other words , For users , It should be no different from the commonly understood single point .

Look at it this way , By distributed system, we mean ： Open to the outside world the monomer system of its constituent cells . In terms of the ultimate goal , What it provides to users is actually similar “ Single point ”, Not distributed , But it is a complex system with open structure . It should be emphasized that , there “ to open up ”, Including the opening of cells , The openness of structural construction , Not just the source code .

therefore , This paper holds that , The technical nature of distributed systems , To a large extent, it is open , And distributed systems can be more dynamic than single point systems 、 More advanced , There may also be advantages of architecture , It also has the advantage of functional design , But its most fundamental advantage is openness . Distributed way , It represents the road of technological openness , It indicates the future direction of technological development , It must be more and more open , This also includes open source .

Therefore, it brings a little philosophical enlightenment ： Maybe , Openness is the fundamental premise and essential feature of the sustainable development of all systems .