Study notes of grain Mall - phase I: Project Introduction

One 、 Project brief introduction

1. Project background

On the market 5 A common e-commerce model B2B、B2C、C2B、C2C、O2O

• B2B Pattern (Business to Business), It refers to the business relationship established between businesses . Like Alibaba
• B2C Pattern (Business to Consumer), We often see that suppliers directly buy goods to users , namely “ business-to-consumer ” Pattern . That is commonly known as commercial retail , Sell products and services directly to consumers , Such as Suning Tesco , JD.COM , Tmall , Millet shopping mall
• C2B Pattern (Customer to Business), That is, consumers to enterprises . First there is consumer demand, and then there is enterprise production , That is, the first consumer demand , Later, the production enterprise organizes production according to demand
• C2C Pattern (Customer to Consumer) , That is, customers sell their own things online , Such as Taobao 、 Salted fish
• O2O Pattern (Online To Offline), It will also combine offline business opportunities with the Internet , Let the Internet become the front desk of offline transactions . Online quick payment , Online quality service , Such as ： Are you hungry , Meituan , Panning ticket , Jingdong home

Grain mall is a B2C E-commerce platform based on mode , Sell self operated goods to customers , The project has the following characteristics ：

• Separate the front and back development , And develop based on vue The background management system
• SpringCloud New solutions
• Application monitoring 、 Current limiting 、 gateway 、 Fuse degradation and other distributed schemes involve all aspects
• Thoroughly explain distributed transactions 、 The difficulty of distributed system such as distributed lock
• Analyze the coding method of high concurrency scenarios , Thread pool , Asynchronous orchestration, etc
• Stress testing and performance optimization
• The difference and use of various cluster technologies
• CI/CD Use
• …

2. The basic concept of distributed

Before introducing the complete structure of the grain mall project , We first need to understand some basic concepts of distribution

1. Microservices

Microservice architecture style , It's like developing a single application into a set of small services , Each small service runs in its own process , And use lightweight mechanisms to communicate , Usually Http API, These services are built around business capabilities , And deploy independently through a fully automated deployment mechanism . These services are written in different programming languages , And different data storage technologies , And maintain a minimum of centralized management

In short , Reject large single application , Based on the business boundary, the exhibition business is micro split , Each service is deployed and operated

2. colony & Distributed & node

• colony It's a physical form , Distributed It's a way of working

  • Distributed It means distributing different businesses in different places
  • colony It refers to putting several servers together , Achieve the same business

• Distributed systems It's a collection of independent computers , These computers are like a single related system to users

  Distributed systems (distributed system) It's a software system based on the network

for example ： Jd.com is a distributed system , Many businesses run on different machines , All businesses form a large business cluster . Every small business , Such as user system , When the access pressure is high, one server is not enough . We should deploy the user system to multiple servers , That is, each business system can also be clustered .

• Every one in the distribution node , For example, user service , You can do it. colony , Clusters are not necessarily distributed
• node ： A server in a cluster

3. The remote invocation

In distributed systems , All services can be on different hosts , But services inevitably need to call each other , We call it remote call

• SpringCloud Use in Http+Json To complete the procedure call

  

4. Load balancing

Distributed system ,A The service needs to call B service ,B Services exist on multiple machines ,A Calling any server can complete the function .

In order not to be too busy or too busy for every server , We can load balance each server , Improve the robustness of the website .

See the load balancing algorithm ：

• polling ： Select the first back-end server in the health pool for the first request , And then, in order to select , Until the last one , Then the cycle .
• The minimum connection ： The number of preferred connections is the least , The back-end server with the least pressure , In the case of a long conversation, you can consider this approach .
• hash ： According to the... Of the request source IP Hash of （hash） To select the server to forward . This method can ensure that specific users can connect to the same server to a certain extent. If your application needs to process state and requires users to connect to the same server as before , Consider taking this approach .

5. Service registration / Find out & Registry Center

A The service call B service ,A The service didn't know B Which servers does the service currently have , What's normal , Which services have been offline . To solve this problem, we can introduce the registry ;

If some services go offline , The rest of us can perceive the status of other services in real time , So as to avoid calling unavailable services

6. Configuration center

The configuration center is used to centrally manage the configuration information of services

Each service eventually has a large number of configurations , And each service may be deployed on multiple machines . We often need to change the configuration , We can let each service get its own configuration in the configuration center .

7. Service failure & service degradation

In the microservices architecture , Microservices communicate with each other through the network , There is interdependence , When one of the services is unavailable , It may cause avalanche effect . To prevent such a situation , There must be a fault tolerance mechanism to protect Services .

For example, downstream Services C Becomes unavailable for some reason , There's a huge backlog of requests , service B The request thread of is also blocked . Thread resources are running out , Make service B And it's not available . Followed by service A It also becomes unavailable , The whole call link is dragged down

Service failure ： Set the timeout for the service , When the called service often fails to reach a certain threshold , We can turn on the circuit breaker protection mechanism , Subsequent requests do not call this service anymore . The default data is returned directly locally

service degradation ： During the O & M period , When the system is at its peak , System resource shortage , We can downgrade non core businesses . Downgrade ： Some services don't handle , Or simply deal with （ Throw exceptions 、 return NULL、 call Mock data 、 call Fallback Processing logic ）

8. API gateway

In the microservices architecture ,API Gateway As an important component of the overall architecture , It abstracts the public functions needed in the middle of micro Services , At the same time, it provides client load balancing , The service fuses automatically , Grayscale Publishing , Unified certification , Current limiting flow control , Rich functions such as log statistics , Help us solve many problems API Management problems

3. Project architecture

Next, we will give a complete introduction to the grain mall project from the two aspects of micro service architecture and micro service division

1. Microservice Architecture

The above is the architecture diagram of grain Mall for micro Services , The project is developed by separating the front end and the back end , It is divided into internal network deployment and external network deployment ：

• Internet deployment is the deployment of front-end projects , It's for public access , Like mobile phone. APP, Computer web page .
• Intranet deployment is to deploy the whole background service cluster .

A complete request call chain flow is as follows ：

• Users send requests through any client , Request goes first Nginx colony .
• Nginx Forward the request to API gateway , The gateway here uses springcloud gateway. The gateway can dynamically route to the specified service according to the current request , For example, do you want to call Commodity services or shopping cart services or retrieval services , If there are many requests routed , The gateway can also call one of the service cluster servers to handle routing in a load balanced manner , If some servers have problems, the service can also be blown or degraded at the gateway layer , For example, using springcloud alibaba Provided sentinel Components , Besides , The gateway also has other functions, such as authentication and authorization 、 Current limiting （ Only part of the request is released to the background , Prevent the server from collapsing ） wait .
• When the request finally reaches the business cluster through the gateway , The server processes the request , These services adopt springboot The micro services written , Services and services may call each other , We use feign Component to realize the call between services .
• Some requests may be processed only after logging in , So we also set up a system based on OAuth2.0 The certification center of , In addition to general login , It is also integrated based on OAuth2.0 Social login .
• Security and permission usage in the whole backend springSecurity Control .
• Besides , The service may store some data , Here we use the method of passing mysql Clusters do persistent storage , It can be separated from reading and writing , You can also divide databases and tables ; Of course, some data may need to be cached , We use redis colony , It can be divided + The way sentinels are clustered .
• We also use message queues to accomplish asynchronous decoupling between individual services 、 Consistency of distributed transactions , Here the RabbitMQ.
• in addition , Some services may require Full-text Retrieval , It's used here ElaticSearch To achieve .
• And some services may need to access some pictures during operation 、 Video and so on , We use Alibaba cloud's object storage service OSS.
• When the project goes online , In order to quickly locate some problems in the operation of the project , We use ELK Processing logs , Use LogStash Collect various logs in the business and store them in ES in , And then use Kibana Visualize pages from ES Retrieve relevant log information , Help us quickly locate the problem .
• Besides , In distributed systems , Because each service can be deployed on many machines , Services and services may call each other , We need to know where we are , So you need to register all services in the registry , Then the service can find the location of other services from the registry , The registration center here adopts springcloud alibaba Of Nacos.
• Again , There are many configurations for each service , In order to change one configuration, all the same configurations are changed synchronously , We need a configuration center , It's also used here springcloud alibaba Of Nacos As configuration center , All services are dynamically configured from the configuration center .
• Besides , The service may have problems during invocation , We can use the service tracking call chain to see where there is a problem , It's used here springcloud Provided Sleuth + Zipkin + Metrics, Give the information of each service to open source Prometheus Do aggregate analysis ,
  Again by Grafana Make a visual presentation , Finally through Prometheus Provided AlterManager Get the warning information of the service in real time , This alarm message can be written by SMS / Inform service developers or operation and maintenance personnel by email .
• Last , The project also provides continuous integration and continuous deployment . After the project is released , Because there are so many micro Services , It is too troublesome to package and deploy every service to the server , After continuous integration , Developers can submit the modified code to github, Then the operation and maintenance personnel can use automated tools Jenkins Pipeline from github Get the code in and package it into docker Mirror image , In the end, we can use k8s Integrate docker service , Will serve with docker The way the container works .

2. Micro service partition diagram

The above is the micro service division map of cereal mall , It reflects the microservices and related technology combinations that need to be divided in the whole project .

The project is based on the separation of front and rear development , Front end projects are divided into ：

• admin-vue： Background management system for staff
• shop-vue： For public access web Website
• There can also be public app End and applet end （ Unrealized ）

Back end services are roughly divided into ：

• Goods and services ： The addition, deletion, modification and search of commodities 、 On and off shelves of commodities 、 Goods details

• Payment services

• Preferential service

• Customer service ： User's personal Center 、 Shipping address

• Warehousing services ： Inventory of goods 、 Which warehouse is there

• Seckill service

• Order service ： Order addition, deletion, modification and query 、 User order list, etc

• Retrieval service ： Commodity search

• Central certification services ： Sign in 、 register 、 Single sign on 、 Social login

• Shopping cart service ： Add, delete, modify and check shopping cart products 、 Check out, etc

• Background management system ： For staff

  These services will rely on some third-party services during operation , For example, logistics information retrieval 、 Text messaging 、 The payment remittance related to finance declined and rose 、 User authentication , These services are not written by us , We can call some interfaces of the three parties .

During the operation of these many micro Services , How to manage them so that they can run in an orderly way , We also need to match the following technologies ：

• Use Nacos As a registry 、 Configuration center
• Use Seata Implementing distributed transactions
• Use Sentinel Implement service fault tolerance 、 Downgrade 、 Current limiting
• Use Feign Solve the problem of remote invocation and load balancing between services
• Use API The gateway filters 、 A series of operations such as routing
• Use Sleuth + Zipkin Realize visual tracking of services
• Use Prometheus + Grafana Monitor the status information of the whole application

Last , The data support layer of the whole project adopts the following technologies ：

• Use Redis Do the cache
• Use MySQL Data persistence , Subsequent approval ShardingSphere Yes MySQL Carry out the operation of sub warehouse and sub table
• Use RabbitMQ Do message queuing
• Use ElasticSearch Do full-text search
• Use alicloud OSS Store image 、 Video and other files