Module 5 (microblog comments)

Computational performance prediction

2020.9 Live every month 5.11 Billion , Diurnal activity 2.24 Billion （ Reference resources 《 Microblogging 2020 User development report 》）, Refer to the teacher's previous analysis of microblogging , The daily sending volume of Weibo is 2.5 Billion bars ,TPS=10k/s. Microblog comments mainly focus on big V Or stars , So suppose that one day Weibo corresponds to 10 comments , The number of comments is ：2.5 Billion *10 = 25 Billion , The time period of most comments coincides with that of tweeting , Then it's TPS=10*10 = 100k/s.

High performance architecture for non hot events

analysis

【 Business characteristic analysis 】 

This is a write operation , But it doesn't need to be shown immediately after writing  , Its timeliness is not as good as microblogging , After writing at the same time , It can be considered that it will not be modified , And make sure you don't throw it away , It is better to use cache , Asynchronous processing   At the same time, because there are a lot of comments in a day , Load balancing is required

【 Architecture analysis 】

  Over 100 million users , You should use a multi-level load balancing architecture , Cover  DNS -> F5 -> Nginx ->  Multilevel load balancing of gateway .  use kafka The cluster does write caching , Write comments to cache , fifo , Asynchronous processing ,TPS by 10 ten thousand /s,  Therefore, the capacity of the buffer queue can also be set to 10 Wan to 12 ten thousand

【 Architecture design 】

 1.  Load balancing algorithm selection   Writing comments depends on login status , The login status is generally saved in the distributed cache , Therefore, sending a request to any server can , Choose here “ polling ” perhaps “ Random ” Algorithm .

 2.  Estimate the number of business servers   Write comments and a few key treatments ： Content review （ Rely on the audit system ）、 Write data to storage （ Dependent on storage system ）、 Data write cache （ Rely on the cache system ）, because   This is processed per second per service 500 To estimate , complete 100K/s Of  TPS, need 200 Servers , Plus a certain amount of reserve ,250 This server is almost

• The time period for writing comments is the same as that for tweeting in most cases , But sometimes it doesn't overlap , Sometimes it may take several hours for Weibo to send out , Even after a few days to write a comment . A micro blog , There may be tens of thousands of people reading , One , Twothousand people commented , The number of comments and microblogs , There is a big difference in the number of times you read Weibo ,
  So you need to set the comment as a separate service

Multilevel load balancing architecture

High availability computing architecture for hot events

【 Business characteristic analysis 】

Hot events , Most of the comments are written under the hot microblog , The comments just written may be brushed to the back by the people behind , Can't see the , Therefore, its importance and responsiveness are relatively small , But make sure you don't leave comments ,

Therefore, current limiting can be considered , Using funnel algorithm

. The size of the write cache when it is not a hotspot is 10 To 12 ten thousand , At this point it can be expanded 10 times , To 100 Wanor 120 ten thousand .

Methods of current limiting ：

• Hot events occur , And the judgment may affect the system availability , Can be manually , Give flow instruction , Restrict the interface to receive user comments .

• stay kafka When writing comment cache queue , If the cache is full , Subsequent write operations can be discarded