From programmers to large-scale distributed architects, where are you (I)

    Write this article in order to know more about your technical ability , And share it with everyone , See where your technical level is .

　 Limited personal ability , Based on what I understand ： From programmers to large distributed architects , Where the hell are we .

Please point out the improper description , Brother, you can go to a higher level ！！！

I just drew it before Microservice system architecture diagram Let's explain one by one .

Original image address ： Microservice system architecture diagram | ProcessOn Free online drawing , Online flow chart , Online mind mapping |

From the perspective of programmers to the perspective of large distributed architects ……

1. Ignorant i

     I just started to learn Java When programming , Just write a simple formatted output statement , Print it out in the development tool ; Simply read and write file data ; Later, I began to contact the database , Study JDBC, Learned to simply add, delete, modify and check ; Then the pre-school three swordsmen , Study jsp, Learned to write a simple page . Here I have experienced , Ordinary jar Applied to the web application . But at that time, I looked confused , I don't know what I can do , Even development web Application is used HTML+ajax Development or use jsp+ajax To develop , Because I didn't know the difference between dynamic pages and static pages .

     Because the circle , So study . For better and faster development , I began to touch frame 、 Contact MVC Three layer architecture 、 Contact JQuery. The interaction between application and database does not need to be encapsulated by itself JDBC 了 、 With norms, it's not messy to write 、 No need to write native js So boring . At that time, I was also ignorant , Know how to use it , I don't know where my technology is , I don't know how to grow .

     Later contact Spring, at that time SpringBoot Is not popular , But with a very popular framework SSM(Spring,SpringMVC,MyBatis) The whole project is simple and developed quickly ,copy The configuration file can be used under configuration . But the front end was still used jsp,el expression ,ajax. At this time, the system is still an ordinary JavaWeb project . At the beginning, a management system can still be used , Ha ha ha . I learned later tomcat、linux、 The front frame bootstrap、easyUI、maven, Come into contact with Fore and aft end separation .

    SpringBoot 2.0 After the release of , Just started to use SpringBoot Development . The front frame vue  It's getting popular , And began to learn vue, Learn new ui The frame is like ：elementui. Learn the separation and deployment of front and rear ends , The front-end compilation package is deployed in nginx, Back end compilation and packaging tomcat Deploy .

summary

     Technology learned here , Mastered jar application 、 Integrated web Separate from the front and rear ends web Development of , But it's still a stand-alone project , On the whole front end 、 Back end and database , Located in the micro service system “ Red box ”.

    ​ Although it is a drop in the ocean , But it is firmly rooted . This is just the basis of business code , It's also the work of many programmers .

    ​ Here we need to understand a little ： Under the large-scale distributed architecture platform , Programmers use different technologies in the platform to develop applications to solve the growing business .

2. Cultivate hard and become strong

In order to deal with all kinds of complex business , Need to keep learning all kinds of new technologies . Next, we will briefly describe the common technologies one by one .

2.1. Task scheduling technology

     Also known as scheduled tasks , Regularly execute some programs to complete the functions required by the business . Such as ： Regular day end reconciliation .

Common task scheduling technology

2.2. Cache technology

    ​ To put it bluntly, caching is to operate data in memory , Fast reading and writing , It also supports cache data persistence in files . Here, we will focus on the use scenario of Technology , With representative caching technology Redis For example , Common scenarios are ： Hot data cache , Data sharing , Distributed lock , overall situation ID, Shopping cart and so on .

2.3. Message queuing technology

    ​ Common message queues RabbitMQ、RocketMQ、Kafka. Such as ： For log collection and data collection scenarios Kafka. Common application scenarios are shown in the figure

2.4. File storage technology

    ​ It is mainly used to save business data files and file transfer , You can modify the storage , The tree structure of file storage and the way of path access Convenient user understand 、 Memory and access . Such as ： Some business interactions need to be through ftp File transfer to complete .

2.5. Object storage technology

    ​ It is mainly used to save data that is not modified in storage . Convenient computer software Access the file storage system . Such as ： picture , Video etc. .

　 About how to choose file storage or object storage , It mainly depends on whether support is needed Modifying data on a storage device , Data like pictures do not need to be modified directly on the storage device . A very important point ： The data stored in the object is the whole data , modify , Put it back , Direct modification is not supported on .

    ​ Technical solution ：MinIO

    ​Minio It's based on Golang Write an open source object storage Suite , be based on Apache License v2.0 Open source licenses , With high performance 、 Scalable 、 Support cloud native 、 Amazon compatible S3 Cloud storage service interface and other advantages .

2.6. Search engine technology

It is mainly used for fast data retrieval .

Technical solution ：Elasticsearch、Solr

Common use scenarios ：

2.7. Database sub database and sub table technology

    ​ If you still use the old way , To query the data of a table, you need to visit different libraries many times to query the data . After the database is divided into databases and tables, new technologies need to be introduced to facilitate the operation of data . There are two options ： Client solution and middleware （ Middle layer ） programme

    ​Sharding-JDBC The client solution is to introduce new jar, Access the database through the client , Agent free layer , There is no need to change the deployment architecture ;

    ​MyCat The middleware solution is to add a new layer of agent , Access the database through a proxy , There is no code intrusion on the back end , Direct access to the proxy layer .

2.8. Cluster technology

    ​ Some middleware naturally supports clustering , Others need to build their own cluster deployment environment .

A combination of commonly used software load balancing technologies ：

All three use reverse proxy to realize load balancing .

    ​nginx More widely used , It can not only be used to build load balancing , It can also be used to deploy front-end systems ,nginx Support for higher concurrency 、 Compress and transfer the transferred files .

2.9. Intranet private library

     Some companies You can only use Intranet development or Self developed the framework technology , At this time, use the library of the external network （maven library 、Gradle Library or npm library ） Can't solve the problem , At this time, the company must build a private database to provide services uniformly , Upload the self-developed framework to the private database or intranet service construction environment, and take the intranet private database .

     Use Nexus  Technology to build a private library , Support Maven Warehouse 、Gradle Warehouse and npm Warehouse .

​

summary

     The first 2 The main point is about the commonly used middleware technology . So far, the commonly used development technologies have basically come into contact with （ Framed red part ）. Learning to this level has been very good , Don't learn about e-commerce systems all day long 、 Build a complete set of micro service architecture , It is the king's way to learn these technologies well .

     Let's talk more here , The construction of microservice governance architecture will not improve your programming ability , How many people have access to the micro service architecture of the production environment ？ To put it bluntly, what is the significance of building these ？ Don't they all serve the business in the end ？ Growing business , The system is getting more and more complicated , Development and maintenance are becoming more and more complex , We need to build a governance environment to better develop and deploy , Monitoring operation and maintenance . Purpose , We should learn the relevant technology of service governance after laying a solid foundation .

Continue to explain in the second part ： Distributed service governance scheme 、DevOps( Integration of development, operation and maintenance )、 Big data access 、 Intranet DNS wait .

Second articles ： from ​ Programmers to large distributed architects , Where on earth are you （ Two ）

  Reprint, please name the source ！！