High quality defect analysis: let yourself write fewer bugs

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

author |  Song Hua

Reading guide ： Well done defect analysis ,bug Write less . Ali senior technical experts share with you how to carry out high quality defect analysis , Sum up 5 A point , Eliminate all kinds of blind spots in development through defect analysis , Build a learning team .

Defects in software development are of great value , But many organizations simply tolerate the costs and consequences of defects , But let the value slip away .

The value of a defect is the opportunity it triggers to learn and grow . Grasp the learning opportunities brought about by defects , Can quickly improve the ability of the organization , The future has fewer flaws , A lower cost , It's easier to succeed . But at the same time , Effective defect analysis and tracking actions require effective methods and appropriate organizational support .

Defects imply high value

Recently we did a workshop on Defect Analysis .

“ Is it a good thing to have defects ？” At the beginning of the workshop , I asked the students involved . “ That's a bad thing, of course .” “ Whether it's good or not , It's right there . I don't think it matters , It's a normal thing .”  “ That seems to be true , But the defects are troublesome , I can't like flaws .”

Yes , No one likes flaws , It costs research and development , Affect the development cycle , But at the same time , Defects go hand in hand with software development , No matter how many , All the time . Why is that ？

Look at the picture below ：

Software development is the process of eliminating uncertainty

Software development is totally different from industrial production . Industrial production by eliminating the variability in the process , Can gradually approach the goal of zero defect . therefore , Six Sigma method is very effective in industrial production .

The process of software development is the opposite . Every development , It's all uncertain , We tend to be near the end of a project , The problems and details of the whole project become clear . Under this assumption , Instead of pursuing zero defects , Rather, we should seek to reduce the impact of defects , such as , The first time a defect is created （ Injection time, even before injection ） It's a defect —— Because at this time the cost of the defect is almost zero , This can be equivalent to “ Zero defect ” Is that right .

If the Six Sigma method in industrial production comes from the building of production system , that , In software development ,“ Zero defect ” What is the corresponding system ？ It certainly contains software development processes and tools , however , in my opinion , The most important , It should be the core subject of building software —— people . Continuous learning through defect analysis , In order not to waste the cost of defects .

Why do you step on the pit repeatedly

There are a lot of teams that have defect analysis . I once carefully analyzed a team's defect cause analysis , The following high-frequency words for the causes of these defects have been found ：

• Coding problem , Think more carefully next time you write code .
• Code review is not good . The next code review should be full .
• Business scenario analysis is not comprehensive , The next analysis will be more comprehensive .
• ......

I Believe , Write down the reasons for the above analysis of the students , The heart must be very sincere , I really feel that I didn't write good code at that time , Business scenario analysis is not comprehensive , Not enough code reviews . however , The action that this analysis brings , But it is often impossible to achieve . Is it really not careful , Or just can't think of ？ This review is not good , Will it be ready next time ？ This scene analysis is incomplete , So how can we be more complete ?

The cause analysis is too broad , So that it is difficult to produce practical and effective improvement actions , Next time, I often fall in the same place —— Let's just look at the previous cause analysis , How many times have similar answers ？ Every time I repeat , It's a new step in the pit .

There is also a kind of cause analysis , On the contrary , Too specific again , To the level of no learning value . for example , It was not designed at that time ,A Services should not call B service ,A Service should take into account B Exception scenarios in service invocation , wait . ok , The defect has now been fixed ,A The service call B Service exception scenarios have been fixed in the code , Next time if it's C The service call D What to do with the exception scenario of the service ？

The most appropriate cause of defects should be based on such criteria ： These causes need to be systematic and actionable . The standard test is ： Next time, if something happens , Whether our response plan is effective ?

Do a good job in defect analysis 5 A point

In practice , We summarized 5 A point , To maximize the practice of defect analysis for learning purposes . They are ：

• Summarize in time , Set the checkpoint
• Pair analysis , Group summary
• Total analysis supported by negative list
• Operational results
• Group learning , Mechanism construction

Summarize in time , Set the checkpoint

“ Defect analysis is very important , But R & D students are too busy , How about we do it once every two months ？”

Don't be so nervous —— Just in time is the most economical way . To be free from busyness , Every time you spend 15 minute , Do an effective defect analysis , The time is right .

The best time to complete the defect analysis is to fix it . The memory is the freshest at this time 、 You can also get early profits . If a defect is two months old , Then the cost of defect analysis becomes higher , We have to get back to the original memory and context , Whether this memory is accurate or not is another matter .

How can we ensure that we do it in time , In order to ensure that these important and not urgent things happen ？ A more effective way , It is to set the checkpoint in the process ： When a defect is set to a fixed state 、 Or when it is turned off , Force defect analysis as a process checkpoint , In this way, a better driver can be formed .

Pair analysis , Group summary

Who is responsible for the defect analysis ？ It's for the students with the specific defect to do , Or bring the whole team together ？

Bring the whole team together to do defect analysis , Sometimes it's too expensive . Even if we only analyze the later online problems , Even if each defect only analyzes 15 minute ：100 A flaw , Each team 8 personal , The product is 12,000 minute , close 200 Hours , It's also an amazing number , Input output is out of proportion .

The students who solve the problem really understand the defect best . however , Will this form “ A single point of the problem ”, Reduce the effectiveness of problem analysis ？

Our plan is ：

1. Take pair analysis as a system

Let the students who solve the defects be in charge , With a little partner . Pairing forms a complementary knowledge , To some extent, the blind spot of thinking has been eliminated , It also forms a deeper discussion through pairing , The results are also carried out ahead of time “ check before acceptance ”, Improve the quality of analysis . If necessary , Groups in pairs can decide whether to bring in other people .

2. The team discusses learning on a regular basis

The team regularly discusses important defect analysis results , It is the acceptance of the team's achievements , More conducive to the formation of communication among team members , learn from each other .

Total analysis supported by negative list

The purpose of defect analysis is to improve , therefore , Focus on solving those “ Unknown unknown unknown ” The problem of . Obviously, not every defect should be analyzed in depth . however , If we define for each defect whether it should be analyzed , It will also lead to high decision-making costs , And the quality is unreliable . therefore , Our practice is based on the default full quantity , Use negative lists to filter . Everything that doesn't exist on the negative list , We do defect analysis . The negative list is team level . Each team should maintain its own list , for example ：

• Occasional problems
• Problems that have been listed in the improvement list （ Keep expanding ）

This thing is similar to gold rush , Make sure what you don't want , It's more efficient to avoid drowning out what's really worth doing . in fact , Every defect analysis expands the length of the negative list , The amount of defect analysis required will be less and less , The problem is getting more and more focused , Time is also being saved .

Operational results

Defect analysis should produce valuable insights , Enough depth is the point . There are many mature ways to generate deep insights , The most typical is 5 Whys, In addition, there are famous tools such as fishbone diagram . In order to control the length , The introduction of these methods is omitted in this paper , Only one example is given to illustrate that in actual defect analysis , How we generate deep insights .

A defect describes the following scenario （ This example makes a moderate abstraction without affecting the problem description ）：

The user owns a virtual device , The device has some ancillary resources , When the user removes the device , The attached resources of the device should be released . however , Found out in a special situation , This ancillary resource has not been released .

The code is as follows ：

void releaseResources (resoure_id){    
    if (failedOfHardwareResourceRelease(resource_id)){        
        writeLog("resource release failed");    
    }
}

The following is a dialogue about this question ：

“ What's the reason ？” “ We did not consider this release failure scenario in the requirements analysis phase .” “OK. Demand analysis is the problem , This is an improvement .—— But more importantly ： What is the most direct opportunity to find out this problem ？” “ When writing code .” “ Did we notice that when we wrote the code ？” “ I noticed , So I wrote log, But I didn't think about what to do with it . This shows that our responsibilities for this code are not clearly defined .” “ Maybe we can add one to the programming specification ： When an exception occurs, you should not only record log, Instead, you should clarify the scenario and processing plan with the person in charge . some time , When there's just a writing error log, But when there's no other treatment , We can notice that .”

Enough depth to test , The most direct indicator is whether the result is “ Actionable ”. If a result is not actionable , It often means that there is not enough depth or abstraction .

Group learning , Mechanism construction

Learning organizations are not always easy to build . In addition to the above mental model and operation method , Organizational mechanisms are often the focus of success . We summarize the following points ：

• It's a long-standing team
• Building a mental model of continuous learning
• Maintain and utilize the intellectual assets of the organization on an ongoing basis

There seems to be no need to say much about these points . But about intellectual assets , I still need to emphasize more ： The result of the analysis may be process improvement 、 Programming habits and specifications 、 Checklist for code review 、 The improvement of design ability 、 Some new engineering practices are introduced to standardize the requirements , There are two kinds of ：

1. Short term action

For example, introduce the practice of instantiation requirements 、 Build automatic testing mechanism, etc . For such specific actions , To define the responsible person and end date , And manage them as well as work items like management requirements , Make sure it happens .

2. Long term rules

These are things that need constant attention , For example, a list of frequently asked questions about code reviews 、 Adopt some design idea, such as contract design 、 Defensive programming, etc . For this kind of problem , Because of the need for sustained attention , They need to be maintained , And make them part of the team's assets . Yes, of course , If it's technically feasible , We should make some of them into tools as soon as possible , Reduce the burden of memory , Improve operability .

The more such assets are maintained , You'll find that the fewer defects you need to continue to analyze in the future —— Yes, of course , That's what all assets have in common .

summary

The reality is complicated , A unified approach often doesn't exist . But mental models and certain rules 、 Ideas still exist . This article focuses on learning through defect analysis .

By proper means , It can be invested in a controlled amount of time , Accumulate valuable wealth for the organization , And get several times in future development 、 Tens of times, hundreds of times . Busy is not the reason , A new one is missing in the future bug, It's earned back .

Through defect analysis , We can produce the following output ：

• Build a team about requirements analysis 、 software design 、 Programming 、 test 、 Common mind in operation and maintenance
• Form a checklist of frequently asked questions
• Adopt or develop new tools
• Improve existing processes
• Develop action plans for specific issues

The most important , By eliminating all kinds of blind spots , Our abilities are getting stronger and stronger , Development is getting smoother and smoother , Distance from zero defect target , It's getting closer .

“ Alibaba cloud native Focus on microservices 、Serverless、 Containers 、Service Mesh And other technical fields 、 The trend of primary popular technology of focus cloud 、 Large scale practice of cloud original , Official account of cloud developers .”