Ten reasons for system performance failure

Many software systems fail because of performance problems , At every stage of the development life cycle and the performance test life cycle, there are reasons for performance failure . occasionally , Performance issues are uncontrollable , It's not in the project manager 、 Within the control of the technical architect or performance engineer . From a business and personal perspective , Most system performance failures are simply the result of performance engineers 、 Developer 、 DBA、 There was a lack of communication between the business team and stakeholders from the beginning , This leads to many other problems , These problems will directly affect the performance and ROI. For any application / The only goal of effective performance testing of products is to achieve a satisfactory return on investment . Performance testing and software engineering are risky , And always need to start from the early stages of development , Do a lot of trial and error .

The failure of system performance must be handled similarly to other business problems . Understand what the problem is , Why is there a problem , And how to prevent . In most scenes , Everyone needs to know / Understand the performance challenges in an end-to-end lifecycle implementation . His shan zhishi , According to the experience of old farmers , Summarizes a list of reasons for system performance failure .

1. Turn a deaf ear to end-user feedback

As end user , Will be aware of existing potential performance problems . In order to understand the existing performance problems in the production system , Need to get continuous feedback from end users about how the application works under different expected load conditions . There are always many users using a function in the production environment , Even if this feature does not meet their expected performance , They will not question it , And will assume that it is correct , When users can access from multiple locations at the same time , It could be a big problem . therefore , If you want to improve the performance of your application , End users must be involved , For continuous feedback on the performance of an application or system in a production environment . Of course , Interaction with end users takes time and effort , For all that , In order to make the product / Applications provide the best performance , It's definitely worth it .

2. Not focusing on performance goals

Setting goals is one of the most important aspects of determining system performance . Many teams often fail to achieve their performance goals in order to improve , So it took a lot of time to fix the existing and hidden performance problems in the system . The perfect goal of performance testing should be defined under the most realistic conditions 、 Design and execution , For example, real browsers 、 Devices and multiple geographic locations . Identify the right metrics to monitor , Define the minimum threshold for each metric , Perform performance tests to get baseline results , All these numbers are necessary to determine what changes can create performance improvements . It is a good practice to start performance testing early in the software development lifecycle , You can eliminate bottlenecks first , And make sure to constantly check the performance of the application under heavy user load .

3. Unclear and incomplete non functional requirements

Collecting complete nonfunctional requirements is more complex than functional requirements , Because they are regarded as the second or even third kind of requirements . therefore , They are often misunderstood and ignored , Only a few organizations treat non functional needs as first-class citizens . This will be in the system architecture / Cause serious problems in the design , Often leads to project crashes and website crashes , Make the system unusable . in the majority of cases , Incomplete non functional requirements document 、 atypism , Or it doesn't exist in most unsuccessful projects . The first step in performance testing is to test the application / The system carries out feasibility analysis , And create a clear set of non functional requirements . A reliable nonfunctional requirements document will determine the product / Apply all standards with the best performance . Besides , It also needs to be :

• For the product / Applications and systems establish clear performance goals and expectations
• We must let everyone ( The development team 、QA、 Management team 、 DBA、 Stakeholders and business teams ) To reach an agreement .
• In the technical team 、 Establish communication between the project team and the business team , To understand the actual performance problems of end users in the production environment
• Using analysis tools , Obtain production flow statistics , To create an appropriate workload model
• When collecting non functional requirements , Understand the application / Product architecture 、 Design 、 Problems and existing performance problems
• Non functional requirements should be discussed from the beginning of the software development process and throughout the life cycle ; If the application is new , Baselines and benchmarks are necessary for performance testing .
• Get complete information about all involved internal and external components , And learn how they communicate (CDN、 A firewall 、 DNS、 Load Balancer 、 The server 、 Networks and systems 、 Cache, etc )
• Understand application memory footprint and third-party architecture limitations
• Talk to stakeholders and business teams , To understand the goals and determine what is fundamental 、 Existing legacy system performance issues 、 Platform constraints and competitors .
• It is necessary to record everything , And let the business and other stakeholders participate in the meeting , To determine whether existing non functional requirements are appropriate , And on the defined SLA To reach an agreement .

4. Bad architecture design

first , Poor architectural design can only lead to minor problems , These problems will be less at the beginning , But it will gradually accumulate . Simple maintenance is a challenge , Any change in one area can destroy the rest of the application . If an inappropriate decision is made during the architecture design phase , application / The system may suffer serious performance degradation , Causes excessive network latency and other problems . Due to lack of understanding of clearly defined system architecture , There is a high risk of too much uncertainty and complexity in the load test execution phase , This can cause unexpected performance problems for performance testing and engineering teams . In the application design and development phase of the software development life cycle , Due to performance challenges , Software release may be delayed .

5. Lack of foresight about technology dependence

Dependency is to allow more connections between applications and functional components . Specific operating system version 、 application server 、 Database server or Java virtual machine 、 The common language runtime and framework are examples of dependencies . However , Some dependencies are more complex , For example, by Linux、 Java The dependencies of various packages in , as well as Python and Ruby And other scripting languages . Understand the design and infrastructure dependencies of each technology on each component , What technologies are used , And what frameworks and tools are used to develop applications , For system performance , It is essential to complete the performance test with the desired results .

6. Excessive expansion of new functions

Excessive new function expansion is a major obstacle that software developers often encounter . An effective way to deal with this situation is to hold regular user experience oriented meetings and discussions , Every team member is involved , To verify each function , And make sure it meaningfully solves the problem of setting . The performance test team must start with a schedule for planning the release , And should take the initiative to announce the time needed , In case any new features are added at the last minute before release . If the project has a fixed deadline , You need to plan your environmental needs in advance , To ensure that unexpected environmental delays do not affect the progress of performance testing . If you continue to add new features at the last minute , The quality probability of delivery will be affected . Final , The customer may reject the final deliverable , Resulting in rework and shortage of additional resources .

7. Praise the great achievements

In the initial version of the performance test execution , Focus directly on the goal SLA To reach acceptable limits may be unrealistic . Performance testing is an iterative process , A lot of continuous performance testing is required to identify and eliminate all performance bottlenecks . It takes extra time to optimize every line of code and component , To improve the system / Application performance . In performance testing , Every SLA and KPI It's all necessary , And only through continuous performance testing 、 The code analysis 、 Memory analysis 、 Performance engineering 、 Monitoring and client-side and server-side tuning to get the required response time 、 throughput 、 Network latency and resource utilization , This sometimes takes a long time . Analyze all performance results and degradation , And from the user level 、 Operating system level 、 The system level 、 Use appropriate metrics to collect data at the network and server levels , Analysis of the root causes of all performance problems is critical .

8. Lack of capacity planning

Many infrastructures fail to implement effective planning , In short , The capacity planning process is not straightforward . We can create a scene 、 Add traffic 、 Evaluate the results 、 Solve performance problems , Then repeat , Until you are satisfied , But the actual problems are often accompanied by poor capacity planning . Poor capacity planning increases the likelihood of performance loss , The risk will be completely exposed , It eventually leads to failure . All of these can be properly addressed through careful capacity planning . System engineers from the infrastructure field 、 From the database field DBA And programmers from the field of application development are the three categories of people who most need to participate in the effective capacity planning process . Many people sometimes confuse capacity management with capacity planning , Inability to accurately predict and mispredict future workloads . Need to ensure that accurate resource requirements are identified (CPU、 Memory 、 Disk space and network bandwidth ) , To support current and future increased workloads , To meet business needs and avoid capacity planning failures . Using the right metrics for continuous monitoring will help us make effective capacity planning , It also helps handle unexpected workloads when traffic increases .

9. The performance problem has not been completely solved

When the number of users of the application increases , You will often see more performance problems . as time goes on , Hidden performance problems and known performance problems in the system are the main reasons for continuous performance degradation . Each bottleneck identified must be discussed with each team member in the project , To successfully ensure that customers SLA Performance of . When it comes to performance , Every second counts , If you ignore existing performance problems , The system will slow down , Even worse . for example , Some services may stop running on severely overloaded servers , This makes the application inaccessible . Find out the monitoring data , Check the health of the service , In general, you can find out the common causes of performance problems .

10. Lack of methodology

Lack of appropriate methods to establish performance testing strategies and their coverage , It will be difficult to obtain effective performance test results . Understanding performance testing methods and processes will help each engineer in the team , Especially when performance problems occur , Provide the correct fix for each problem bottleneck . The performance testing process should be well planned and defined , And documented . Good documentation can be found in the developer 、 DBA and QA Establish effective communication between . As software becomes more complex and diverse , And there are more and more platforms and locations to test , A strong performance testing method is needed , To ensure that the software system under development has been fully tested , To ensure that they meet specific business requirements , And be able to run with high performance under all expected load conditions and environments .