Why is count (*) slow

The first thing you need to be clear about is , In different MySQL In the engine ,count(*) There are different implementations .

MyISAM The engine stores the total number of a table on disk , So execute count(*) It will return this number directly , It's very efficient ;

and InnoDB The engine is in trouble , It performs count(*) When , You need to read the data line by line from the engine , And then the cumulative count .

What needs to be noted here is , What we discuss in this article is that there are no filtering conditions count(*), If you add where conditions ,MyISAM The watch can't go back so fast .

Thinking questions ： Why? InnoDB Not with MyISAM equally , Save the numbers, too ？

Because even multiple queries at the same time , Due to multi version concurrency control （MVCC） Why ,InnoDB surface “ How many lines should be returned ” It's also uncertain .

InnoDB It's the index organization table , The leaf node of the primary key index tree is data , The leaf node of the common index tree is the primary key value . therefore , A normal index tree is much smaller than a primary key index tree . about count(*) This kind of operation , The result of traversing which index tree is logically the same . therefore ,MySQL The optimizer will find the smallest tree to traverse . On the premise that the logic is correct , Minimize the amount of data scanned , It is one of the general principles of database system design .

To sum up ：

1.MyISAM Although the table count(*) Soon , But it doesn't support transactions ;

2.show table status The command returns quickly , But it's not accurate ;

3.InnoDB The table directly count(*) Will traverse the entire table , Although the results are accurate , But it can lead to performance problems .

If you now have a page that often displays the total number of operation records of the trading system , What should I do ？

Save the count with the cache system

1. There is data loss problem

2. There are data inconsistencies

chart 1

chart 2

If we put this count directly into a separate count table in the database C in , What will happen ？

You would say , Is this different ？ It's just to put the picture 2 Chinese vs Redis The operation of , Changed to a counter C The operation of . As long as the picture appears 2 This execution sequence , This problem is still unsolvable ？

This problem is really not unsolvable .

The problems we are going to solve in this article , It's all because of InnoDB To support transactions , Which leads to InnoDB A watch can't put count(*) Save it directly , Then, when querying, it directly returns the formed .

Let's take a look at the implementation results now . Although the conversation B The read operation of is still T3 Executive , But because the update transaction has not been committed at this time , So the count plus 1 This operation is very important to the conversation B It's not visible yet .

therefore , conversation B What you see is , Look up the count and “ lately 100 Bar record ” Results seen , Logically, it's consistent .

---

count(*) usage

cont principle
count() It's an aggregate function , For the returned result set , Judge line by line , If count The argument to the function is not NULL, Add... To the cumulative value 1, Otherwise, we will not add . Finally return the cumulative value .

therefore ,count(*)、count( Primary key id) and count(1) All indicates that the total number of result sets satisfying the conditions is returned ; and count( Field ）, It means to return the data line that meets the conditions , Parameters “ Field ” Not for NULL Total number of .

about count( Primary key id)

InnoDB The engine will traverse the entire table , Put each line of id Take out all the values , Return to server layer .server Get the floor id after , Judgment cannot be empty , Just add up by lines .

about count(1)

InnoDB The engine traverses the entire table , But no value .server Layer for each row returned , Put a number “1” go in , Judgment cannot be empty , Add by line .

Just look at the difference between these two usages , You can compare ,count(1) Perform better than count( Primary key id) fast . Because back from the engine id It's going to involve parsing the data lines , And copy field values .

about count( Field )

If this “ Field ” Is defined as not null Words , Read the field line by line from the record , Judgment cannot be null, Add by line ;

If this “ Field ” The definition is allowed to be null, So when it comes to execution , To judge that it might be null, We need to take out the value and judge again , No null Just add up .

That's the first principle ,server What fields does the layer want ,InnoDB What fields are returned .

however count( * ) It's an exception , It doesn't take all the fields out , It's optimized , No value .count( * ) Definitely not null, Add by line .

In order of efficiency , Field < Primary key id<1≈*

reflection ： Because transactions can ensure that intermediate results are not read by other transactions , Therefore, the order of modifying the count value and inserting new records does not affect the logical results . however , From the perspective of concurrent system performance , You think in this transaction sequence , The operation record should be inserted first , Or should we update the counter first ？

From the perspective of concurrent system performance , The operation record should be inserted first , Update the count table .

Because updating the count table involves row lock competition , Inserting before updating can minimize the lock waiting between transactions , Increased concurrency .