Advanced performance test series 3. performance index, reliability test, capacity test and performance test

Catalog

• One 、 review
  • 1. Performance testing
  • 2. The load test
  • 3. Pressure test
• Two 、 Performance indicators
  • 1.tps
  • 2. throughput
  • 3.rps
  • 4.qps
  • 5.hps
• 3、 ... and 、 Reliability test and capacity test
  • 1. Reliability test
  • 2. Capacity test
• Four 、 Performance testing
  • 1. Build yourself
  • 2. Independent environment
  • 3. Does the ECS use a wireless network ？

One 、 review

1. Performance testing ： Through the tool , Simulate a certain number of concurrent users , Request to the server , Get performance indicators .

I don't know how many concurrent users the tested system needs , So guess the number of concurrent users , And then do , In this way, a performance index can be obtained .

For example, guess one 50 Number of concurrent users , To do performance testing will also get performance indicators . Guess one 80 Number of concurrent users , To do performance testing will also get performance indicators . The obtained performance index ： response time ,tps, Resource utilization and so on .

Which number of concurrent users does the performance test to get the real server performance index ？ We don't know . So we will do a load test before we guess the number of concurrent users .

2. The load test ： Gradually increase the number of concurrent users , Request to the server , To obtain the inflection point interval of the maximum number of concurrent users .

Each time the number of concurrent users is increased, there will be an interval , For example, by 0, Each time 10 personal , Gradually increase , Keep increasing .

2.1 How to judge the inflection point interval ？

1） See if there's a mistake . For example, add to 80 The number of concurrent users is normal , Add to 90 The number of concurrent users , It is found that some of the requests in the response result are consecutive and report errors .

stay 90 The number of concurrent users has exceeded the maximum number of concurrent users that the server can accept . So this inflection point range is 80-90 There should be a maximum number of concurrent users between .

Then do another load test , Reduce the interval . from 80 Start , Add one... At a time . The interval increased each time is a concurrent user number .

If 83 The number of concurrent users is normal ,84 There is a continuous error message when the number of concurrent users is , The maximum number of concurrent users is 83.

Next, put 83 The number of concurrent users is put into the performance test , Then run it continuously for a period of time to get the value of the performance index , Then write it in the report , This completes the performance test .

Sometimes, there is a situation that the error will not be reported continuously , It depends on whether the average response time exceeds 1.5s, To determine the inflection point range .

2） Does the average response time exceed 1.5s. from 0 Start , Each time 10 individual , Up to 100. Found in 70 The number of concurrent users , Its average response time is 1.5s following .

stay 80 The number of concurrent users , If the average response time has exceeded 1.5s, It can be said at this time that 70-80 The maximum number of concurrent users that can be accepted has appeared .

stay 70-80 There should be a value between , Its average response time is close to 1.5s.

So from 70 Begin to increase gradually , Each time 1 Or 2 Number of concurrent users , Until 80. So do a load test again to find a specific value , It happens that the average response time of this value is 1.5s.

Here 1.5s Time before , Is the maximum number of concurrent users we can accept . Then take the number of concurrent users , To do a performance test , Get relevant performance indicators .

3）tps falling .

As the number of concurrent users increases , After the number of concurrent users increases , If the server can handle , Then the frequency of requests will be very high , The total number of requests will also be higher , The total number of requests that can be processed per second will also be higher .

As the number of concurrent users increases ,tps The value will rise . But it exceeds the maximum number of requests the server can handle , If it exceeds the maximum processing capacity of the server ,tps The value will drop , This time is also the inflection point .

At this time, the maximum number of concurrent users is obtained , When you bring it to the performance test, you can get the maximum number of concurrent users , Server performance indicators .

3. Pressure test ： Use Certain amount Number of concurrent users , It lasts for a long time Request to the server , Check the server's stability .

That is to say, the purpose of the stress test is to test the stability , Not to get the performance index . Performance index is what performance test should do .

The load test is to get the inflection point interval and the maximum number of concurrent users .

3.1 How to understand a certain amount ？

The range must be less than the maximum number of concurrent users , No request will be made to the server with the maximum number of concurrent users . Generally, the maximum number of concurrent users is used 20% or 80%.

1） Why is there such a saying ？

For example, the maximum processing capacity of the server is 83 Number of concurrent users ,20% Namely 16 It looks like about two . Then use 16 About concurrent users , Always make requests to the server , Look at stability .

Or is it 83 Of 80%, Probably 60 The number of concurrent users , Always make requests to the server , Look at stability .

2） Why use first 20% Reuse 80% Well ？

I can use 83 Of 80% That is to say 65 Individuals continue to make requests to the server , At this time, the total number of requests I receive and the corresponding logs will be greater than 83 Of 20% The number of concurrent users and the number of logs and requests are higher .

Through the log 、 Analyze the data volume , Of the maximum number of concurrent users 20% The amount of data , It's easier to analyze . So first use the maximum number of concurrent users 20%, And the maximum number of concurrent users 80% To do it .

Of the maximum number of concurrent users 20%, Requests may be slower , The total number of requests per second may be less , It may take a little longer to accumulate , I also have enough time to observe the use of resources .

Generally speaking , To test this stability , First, it will continue to run for a slightly longer time with a relatively low number of concurrent users , Then run it for a long time with a large number of concurrent users .

Relatively speaking , The maximum number of concurrent users 80% The maximum number of concurrent users 20% When , It will take less time .

3） For a long time ： Few hours , A few days .

Load tests and performance tests typically run for several minutes , About ten minutes or half an hour .

It takes a long time to run , If some resources are not released in time , Will accumulate , I haven't been released enough for a long time , It is possible that more and more accumulated resources are not released, resulting in insufficient resources in the end .

Insufficient resources may lead to service downtime . It does not necessarily mean that the operating system is down , It's just a service that hangs up , This is also the instability of the service .

A service is deployed on this machine , All the resources of this machine used , This causes the machine to run out of resources , Your machine may be down .

More often, the service is down .

Two 、 Performance indicators

1.tps: The number of transactions processed by the server per second , It reflects the comprehensive processing capacity of the server .

The server will have cpu、 Memory 、 disk 、 The Internet 、 Applications, etc. constitute servers .cpu Is a resource provider , Memory is a resource , Disks are also resources ,io It's also a resource , Get these resources , Integrated to handle a request .

For example, a request to log in is initiated , When the server processes , It needs to cpu To calculate , Memory resource exchange is required .

There must be an account and password when logging in , Data must first enter the memory through the network , I got the login account and password in the memory , In the memory, the login calculation should be carried out .

Calculate whether the user exists , There is no registration , Is the account password correct . So a simple login needs to consume memory space 、cpu The calculation of 、 Database query data 、 Read and write of disk .

To log in, you need to transfer to my network card through the network , The data of the network card is converted into the data of the operating system . The data of the operating system should enter the memory , These data are stored in memory .

When checking, you should enter cpu,cpu When checking, you get the data from the disk and compare it with the memory .

So we need the network io, On disk io, Memory ,cup resources , Therefore, it is the embodiment of the overall resource capability .

2. throughput ： The Internet The number of transactions that can pass per second .

It's just a passage , Equivalent to a water pipe , The thicker the water pipe , The greater the flow of water that can pass through at this time . The thinner the water pipe , At this time, the water flow that can pass through is smaller .

The thicker the pipe , Water can flow through , The more powerful the server is . If the server has strong processing power , But the water pipe is very thin , Then the water will not flow out , It will also block the performance of the server's processing capacity .

When the network has no bottleneck , All the processing of the server can flow out through the network . This is the time , The number of transactions processed by the server per second will be numerically equal to the number of transactions on the network , But they measure from different angles , One is to measure the server , One is to measure the network .

3.rps： request , Per second user Request rate , The initiator. （ No matter how the user initiates ）.

For example, now we use jmeter Tools , simulation jmeter Call this request , How many requests per second .

If you click with your hand , Is to simulate the click event of real people on the page , How many requests per second . This is measured from the perspective of the user as the initiator .

The server has no bottleneck , Every second 100 A request , The server can handle . The server could have handled 300 A request , You are now sending messages to the server every second 100 individual , The server processes every second . In this case ,rps Hui He tps Numerically equal .

rps Is the number of requests per second by the initiator . because rps too low , Does not reflect the server tps The ability of .

4.qps： Query rate per second , The server Query for .

Some interfaces need to query the database , Some interfaces do not need to query the database . A transaction may call multiple queries .

So a transaction may have to be queried many times , There may be a tps Corresponding multiple qps.

For example, landing. , First, query whether the user exists in the database , If it doesn't exist, you don't need to verify the user name and password .

It is found that this user exists , Then I will check whether the combination of user and password is consistent . Consistent combination , I think you can login successfully .

The combination is inconsistent , Login failed . You have successfully logged in , You need to tell the user the result of successful login . Then you have to query the relevant information of users .

therefore , This is just querying the database , It is possible to query three times .

In the enterprise , If there is no strict distinction , It's a 1 One transaction , As a query only 1 Time , but , The actual request may be 1 Business : n A query .

5.hps： hitpers. Per second user The click through rate of .（ There is such a concept only when the page is clicked ）

lr There are hps The concept , however jmeter There is basically no hps That's the concept .jmeter The interface is used to send a request to the server , No longer through html The page of initiated the request .

Concurrent ： Requests are initiated at the same point in time every second .

rps Is the rate of requests initiated by users per second . Now there is 10 Individuals make requests to the server together , The number of concurrent users is 10.

If the user can request per second 30 Time , So this 1 Seconds 10 In total, individuals will initiate 300 Requests . that 1 There will be... In seconds 300 concurrent .

If you add it to 20 Individuals to initiate requests , If its frequency is still 30, that 1 Seconds to launch 600 A request , that 1 Second concurrency is 600.

3、 ... and 、 Reliability test and capacity test

1. Reliability test ： Given a certain amount of business pressure , Run for a while , See if the system is stable .

key word ： whether Stable , A certain amount of business pressure .

Of the maximum number of concurrent users 20% Do reliability tests .

2. Capacity test ： In a certain soft 、 Under hardware conditions , In the database Different orders of magnitude Data volume , Read in the system \ Write more business to test , So as to obtain the performance index values under different data levels .

In the performance test , If the data magnitude of the database is inconsistent , Performance index values may also vary . During the performance test , The data magnitude of the database must be consistent .

For example, the data level of the production database is millions , But the environment for testing was built by myself , When I set up my own database, it was an empty database. I started to test the performance and create data , The result may be dozens of data , Hundreds, thousands , No more than 10000 .

Is there a big performance difference in this case ？ It's possible . For example, open a file with only oneortwo rows of data excel form , It was soon opened . Open a file with 10000 rows of data excel form , The opening speed is very slow .

The less data the open table has , The faster it opens , This is from the perspective of data volume .

A small notebook at hand is used for writing and drawing , It will not list the directories in the front . But a dictionary , If you want to look up a word in a dictionary , Then the table of contents in front of the dictionary is very useful .

When there's a lot of data , It will be much faster to search through the directory . The whole dictionary is equivalent to the data in a table in the database , This directory is equivalent to the index in the database .

If your production is in the millions , The data level of this table in the database of your performance test environment should also be guaranteed to be millions .

If the data of the company is millions , As the time goes by , The data magnitude will increase , One day it will reach tens of millions , Billion level , What about the performance ？

The performance test is to predict in advance at the level of ten million , Hundreds of millions of people , What will the performance index value change .

The performance index values vary with different data levels .

When there is no special instruction for capacity test , To ensure that the data level of the database in the performance test environment is consistent with that of the production database .

Four 、 Performance testing

1. Build yourself ： That is to say, performance testing needs to build its own performance testing environment , And it's an independent environment .

Performance testing , You cannot use the functional test environment 、 Automated test environment 、 check before acceptance 、 The production environment cannot be used , You need to build your own independent environment .

therefore , You should know the structure of the company's products , Have the ability to build the environment independently .

2. Independent environment ： A real performance testing environment , The resource allocation of the machine is exactly the same as that of the production （ The hardware configuration is the same 、 The quantity is the same 、 The Internet is the same 、 The architecture parameters are the same ）.

The production environment of the company has 20-30 Servers , Let me build an independent environment , Do I have to 20-30 A machine ？

Finance 、 Telecommunications companies , These companies can really give you enough machines to do performance tests . General company , I can't give so many machines , however You can't even use one server .

2.1) Never use a production environment for performance testing , If you use highly concurrent users to make requests to the server , The response time is long . It happens that some users are also using , The response time is long , The user experience is very poor .

Now you've hung up the server , The user suddenly finds that the server is down , I'm going to call to complain ：“ The server is down , What about my data ？”. This is a very serious customer complaint .

If you do performance tests in a production environment in the middle of the night , Very few people use it at threeorfour in the morning , But the data of your registration and order is also written in the database of the production environment .

These data in the production database become Dirty data . Dirty data appears in the database of a production environment , This dirty data cannot be cleaned up .

In case of production environment problems caused by this dirty data , Accountability is troublesome .

Of course, someone has built a shadow database in the production database , When performing performance tests, write the data to the shadow database , But there is still dirty data in the log . So it is also unreasonable , Try not to use a production environment .

Functional test environment 、 Automated test environment 、 The acceptance environment cannot be used , This is because there will be a lot of concurrency during performance testing , This causes the server to become unstable .

If he is doing a functional test , When you click zhenghi on the page , The server is down , A little wrong . I thought it was bug 了 , Relevant personnel come together to locate the problem , The result is that the server is down .

Because you're doing a performance test , This matter is hard to explain .

2.2) The hardware configuration should be the same ：

Suppose the production server is 8 nucleus cpu,16g On top of that , As a result, the configuration of the server is not so high ,cpu Can you give it 8 nucleus , But memory can only be given to 8g, Is this ok? ？

Can not be ！ Memory by 16g become 8g, Will the resource utilization rate be halved ？ It can't be calculated so simply . Theory is divorced from practice .

2.3) Number （ Yes, we can give less machines ）： The production environment has 30 Servers , The hardware configuration is 8 nucleus 16g, It costs too much money to give so many machines , Can you reduce the number of sets ？

This is OK . For example, the production environment has 30 Servers , Performance test to 3-5 The platform is OK . Can you give me 30 Taiwan is the best , be unable to give 30 platform , At least give me some .

But only one is not enough . Because it is necessary to form a cluster , Minimum need 2 Servers .

2.4) The Internet ： Network architecture , Network foundation , The performance test cannot use wireless network , And don't use v** Equal bridging , Use LAN whenever possible .

such as a and b There is no direct data exchange between two LANs , Need a bridge , From one network to another .

There are many ways , for example v**. The company's intranet is connected to the company's production environment server network .

Because the wireless network is unstable , Insufficient bandwidth will lead to congestion and packet loss . Unless you're doing a telecom network test project , Then there's no way .

Out of commission v** Equal bridging , Because when you do the performance test , A lot of data will be transmitted through the network . Because the purpose of these bridging networks is to ensure the security of data .

On the premise of ensuring data security , Some problems of speed and stability may be sacrificed . It's easy to disconnect at any time .

Performance testing ： First of all, you should be able to build a test environment .

A firewall ： When our services can be accessed externally , You must open a port in the firewall , I can access this service through this port . Must be able to configure this firewall .

3. Does the ECS use a wireless network ？

The vast majority of servers used in enterprises will be cloud servers , But ECS is not a wireless network .

Your computer is plugged into the company's limited network --- Switches connected to the park --- Switch into telecom --- The national group stem network --- Switches in the corresponding regions of the ECS service provider --- Then enter the network where the region is located .

Also through a limited network , Not over the wireless network .