MySQL combat optimization expert 06 production experience: how does the production environment database of Internet companies conduct performance testing?

1、 After applying for the machine , You do Java Architects need to know

In the last article, we talked about real projects , The first thing is to apply for a database machine , Generally speaking, we need to apply 8 nucleus 16G perhaps 16 nucleus 32G The height of Configure the machine , Even all the machines should be equipped SSD Solid state disk , And then let DBA Brother installs and deploys one on the applied machine MySQL, At the same time to start MySQL database .

Of course, how to install and deploy MySQL, And how to start MySQL, It's very simple , A random search on the Internet will see a large number of similar things , that Not what we are going to talk about in this column . then MySQL The adjustment of various complicated advanced parameters in the production environment , For the time being, we won't immediately involve , those It belongs to MySQL DBA Things that need to be done .

But we will follow up with the progress of the column , Will explain part MySQL Optimization and configuration of high-order parameters in production environment , Some are developed with us Java Department of application The system is closely related , We as developers , Also need to understand MySQL Optimization of some high-order parameters , Sometimes when we optimize the system performance , You may need to talk to DBA Work together to tune .

But in short , We are the core of a project Java Engineers even Java Architects , You must choose which configuration machine your database uses , Have a big heart To understand the database deployed by this configured machine , It can roughly help us resist the number of concurrent requests per second .

For example, you apply for 8 nucleus 16G To deploy MySQL, Then you as a project Java Architects , I should know that your database will resist one every second Twothousand requests are still ok , If you apply for 16 nucleus 32G Machine , Then you know in your heart that toto can resist twoorthree per second , Even threeorfour thousand requests , Your heart You can count it in , This is what you want to do

2、 Give the machine to a professional DBA, Let him deploy MySQL

The second thing you need to know is , After you apply for a machine to come down , Then this machine is in a company with a certain scale , It must be handed over to the company's professional DBA To install 、 Ministry Deploy and launch MySQL Of ,DBA This time will be based on his past experience , Using one's own MySQL Production tuning parameter template , Put directly MySQL In go to , And then use A parameter template to start this MySQL, Many parameters here are often tuned .

and DBA It might be right linux Some of the machines OS Adjust the kernel parameters , For example, parameters such as maximum file handle , These parameters are often required To adjust .

Then when DBA After finishing the database on this machine , I'll leave it to you to use , You know the address, user name and password of this machine , Then Java The system is It can be directly connected , You can perform various SQL Statement to realize business logic .

3、 With the database , Pressure measurement is also required first

When you have a usable database at hand , You think you can develop it directly based on him Java Is the system working ？

It's not like that ！ Doing so in an Internet company tends to appear more amateur , Because you have to make a basic benchmark for this database first measuring .

in other words , You have to simulate a system based on some tools 1000 Requests to the database , Watch him CPU load 、 disk IO load 、 The Internet IO load 、 Memory complexity , Then whether the database can handle this every second 1000 A request , Or can we only process 500 A request ？ This process , Pressure measurement .

You don't just use tools to send every second 1000 A request , It can also simulate sending per second 2000 A request , even to the extent that 3000 A request , Gradually test it out , This database is in the project Under the previous machine configuration , How about his general load pressure , How is the performance , The maximum number of requests per second .

Some people may ask questions , He would say ： teacher , Why do we need to do a benchmark pressure test on the database at the beginning ？ You can wait Java The system has been developed 了 , And then just let Java System connection MySQL database , And then directly to Java Pressure measurement of the system ！

If someone asks this question , Then I don't know , The pressure test of the database and the above Java Pressure measurement of the system , It's actually two things , First you have to know your number According to how much pressure the library can resist , Then you go to see your Java How much pressure can the system resist .

Because one possibility is , Your database can resist every second 2000 A request , But your Java The system can only resist 500 A request , It's also possible . the So you can't just aim at Java System to conduct pressure measurement , Before that, you have to do a pressure test on the database , Have a number in mind .

4、 Stupid is not clear ：QPS and TPS What's the difference ？

Since the pressure test is needed , Then you must understand something first , Our pressure test database , Finally, I want to see the database under the existing machine configuration , It can resist more per second One less request ？ How many requests are resisted per second , In fact, there are professional terms , Namely QPS and TPS.

Just QPS for , His full English name is ：Query Per Second.

In fact, the literal meaning of English has been very clear ,QPS That is to say , How many requests can your database handle per second , You can roughly understand it as , One request It's just one. SQL sentence , That is to say, how many databases can be processed per second SQL sentence .

about QPS for , In fact, some of your Java When the system or middleware system is under pressure test , You can also use this indicator , in other words , Yours Java Every time the system How many requests can be processed per second .

Then another term is TPS, His full English name is ：Transaction Per Second. In fact, it is the number of transactions that can be processed per second , This TPS Often used in There are more in the database , In fact, it can be seen from the literal meaning , It means how many transaction commits or rollbacks the database will handle per second .

Because everyone should have a basic understanding of database , Is what his business is ？

Simply speaking , A transaction will contain multiple SQL sentence , these SQL It is better to either commit the transaction , Everyone succeeded together , Or the best transaction rollback , Big The family failed together , This is the business .

therefore TPS It often refers to how many transactions are completed in a database per second , Transaction commit or rollback is the completion of transaction execution , therefore TPS Measured Is the number of transactions processed by a database per second . Some people tend to put TPS It is understood as the number of requests processed by the database per second , In fact, this is not very rigorous Of .

5、IO Relevant pressure measurement performance indexes

Then I'll tell you a few things to pay attention to during pressure measurement IO Related performance indicators , Everyone should also know about him ：

（1）IOPS： This refers to the random of the machine IO The ability of concurrent processing , For example, the machine can achieve 200 IOPS, It means that you can execute... Every second 200 A random IO Read write request .

This indicator is very critical , Because we explained in the principle of database architecture before , Dirty database that you update in memory , In the end, the backstage IO The thread is inaccurate Fixed time , Brush back to the disk , This is random IO The process of . if IOPS The index is too low , That will lead to the efficiency of brushing dirty data in your memory back to the disk It will not be high .

（2） throughput ： This refers to how many bytes of data can be read and written per second in the disk storage of the machine

This indicator is also very critical , Because everyone knows through previous study , We usually carry out various SQL At the time of statement , When you commit a transaction , All of them are A lot of people can write redo log Such logs , These logs will be written directly to disk files .

So how many bytes of data can a machine read and write per second , It determines how much he can put every second redo log Such logs are written to disk . One Generally speaking, we write redo log Such logs , Are written to disk files in sequence , That is, write line by line , Don't say random reading and writing , that Generally, the throughput of sequential writes of ordinary disks can reach 200MB about .

So in general , The disk throughput of the machine is enough to carry high concurrent requests .

（3）latency： This indicator refers to the delay in writing a piece of data to disk .

This indicator is also important , Because we carry out SQL Statement and commit transaction , All need to be written in sequence redo log Disk files , So at this time, you write a log to Go to the disk file , What is the delay 1ms, Or delay 100us, This is for your database SQL Statement execution performance is affected .

Generally speaking , Of course, the lower the read-write latency of your disk , The higher the performance of your database , You perform every SQL Statements and transactions will be faster .

6、 Other performance indicators that should be paid attention to during pressure test

Other than that QPS、TPS、IOPS、 throughput 、latency Beyond these indicators , Other performance indicators of the machine need to be paid attention to during pressure measurement .

（1）CPU load ：CPU Load is a very important performance index , Because suppose your database pressure measures processing per second 3000 Request the , Maybe other performance indicators All normal , But at this time CPU The load is very high , It also means that your database cannot continue to test higher QPS 了 , otherwise CPU It's too much to eat .

（2） Network load ： This mainly depends on your machine bandwidth , A certain value is measured at the pressure QPS and TPS When , How much does the network card of the machine input per second MB data , How much will be output MB data , Because it is possible that your network bandwidth can transmit at most per second 100MB The data of , Then maybe yours QPS To 1000 When , network The card is full , Has been transmitted per second 100MB The data of the , At this time, even other indicators are normal , But you can't continue the pressure test

（3） Memory load ： This is to see when the pressure is measured under certain conditions , How much memory does your machine consume , If the memory consumption of the machine is too high , Description also Can't continue the pressure measurement

7、 The following practical instructions of pressure measurement

Next article , I will install one on my own computer MySQL database , Then I'll show you how to use the convenient pressure measuring tool , Put... Into the database Do a certain pressure test , At the same time of pressure measurement , What convenient tools should be used , Observe the machine performance and various indicators in the pressure measurement process .

8、 Today's question

Today, I want everyone to think about it QPS and TPS Two terms , I want you to talk about yourself before reading today's article , I'm right QPS and TPS How to understand , stay Does your company have relevant systems and databases QPS/TPS The statistics of ？ After reading this article today , You are right about QPS and TPS Is there any difference in the understanding of the two terms Change ？

in addition , I'll give you another question to think about , Suppose you are in charge of a trading system , For this trading system , He split it into many services , A deal Completion requires multiple services to complete , In other words, a transaction request needs to call multiple services to complete .
So you think for every service , The number of requests he processes per second is QPS still TPS Well ？ For the whole trading system , The number of transactions he processes per second yes QPS still TPS Well ？ Please share your views .