Rob Kraft's Software Development Blog

Software Development Insights

Use Root Cause Analysis for Defect Prevention in your Software Development Process

Posted by robkraft on February 5, 2012

Recently I decided to review all the bugs we fixed in our last release to determine if they originated from a few common causes.  In software development, there is not a lot of data that can be mined that may provide insight into actions to take to improve software development, but I thought this might be one.  We fixed about 40 bugs in the last release, and I can tell you right now that my analysis has not lead to any significant changes in the way we develop software.  This is not because we didn’t find areas that could benefit from improvement, it is just because, in our case, attempting to make improvements to reduce the source of the bugs is not currently more valuable than other tasks we can be working on.

Of the bugs we found, I considering them to be “coding flaws”,  “configuration flaws”, “design flaws”, “process flaws”, or “requirements flaws”.  I realize that my technique is crude and I have very little data to draw definite conclusions from, but this is an exercise that anyone on a project team can perform if you are tracking the defects fixed in a release.  You may find patterns that help you identify aspects of software development most in need of improvement.  An excellent paper based on a lot of software development data at NASA is online here:  The goal of the study in the paper, and my goal is well, was to use this “root cause analysis” to determine ways to prevent and reduce defects in the software.  Given that excellent article provides much more detail than I enter into here, I recommend that you read it if you are really interested in improving your development processes through root cause analysis of your bugs.

I will add here, a list of reasons that I came up with for the sources of bugs in our code:

18% – Coding Flaws.  The programmer did not think the issue was a bug, or the programmer did not test the issue thoroughly, or the programmer felt it was insignificant and lacked the time to address the problem.

39% – Coding Flaws.  The programmer should have noticed the problem.  But perhaps senior developers or business analysts should have provided a junior developer more training and input so that the junior developer would have recognized this as a problem.

3% – Coding Flaws.  The bug is a bug in 3rd party software we use (in this case Microsoft), that we need to wait for them to fix, or rewrite the feature.

5% – Coding Flaws.  The programmer should have just provided an error message to the user that is easier to understand.  Perhaps the message provided should always be determined by a business analyst.

5% – Coding Flaws.  A known bug was released.  The bug had minor impact and the cost to fixing it was high, as was the cost of delaying the release.

8% – Configuration Flaws.  We use code generation for much of our code.  The business analysts made mistakes that led to improperly generated code.

5% – Design Flaws.  The design did not allow for a feature to work as it needed to in some situations.

8% – Process Flaws.  We don’t test 100% of the User Interface possibilities in our application because the time required makes it impractical.  However, such testing is unnecessary for most of the code as long as we follow our processes for development.  In a few cases, we failed to follow the correct processes.

3% – Requirements Flaws.  The developer was not aware that the feature was desired.

3% – Requirements Flaws.  The requirements provided to the developer were incorrect.

3% – Requirements Flaws.  We removed a feature from the software that we thought was not being used, but it was and we had to re-add it.

We also improved performance in the last release.  Another way to phrase that is that we fixed performance problems.  It is difficult to decide if improving performance is fixing a bug or not when it doesn’t violate a clear service level agreement.  A code change is not always objectively a bug fix or an enhancement.  If the client believes it was in the original specification, the client will say the missing feature is a bug; but if the developer thinks the feature was not in the specification, then the developer will consider the addition of it in the next release to be an enhancement.  Just part of the joys of software development.


10 Responses to “Use Root Cause Analysis for Defect Prevention in your Software Development Process”

  1. Your articles contain valuable information for me, many thanks for this essential article. I will bookmark this page and I will tell other people to visit your website, so they can learn lots of new useful information than anybody else.

  2. PM Hut said

    HI Rob,

    We have published a very elaborate article on root cause analysis in project management (you can find it here). The article was published back in October. We really hope to see some feedback from you on the PM Hut site on this topic! (maybe a comment on that article if you get the chance?)

  3. Although you decided to not to change the way you develop your software, I get the impression that doing the Root Cause Analysis has given much insight in the causes and quality. This I assume will help you to improve your product, and your programming skills, which is valuable.

    • robkraft said

      That is correct Ben! Though we have not taken significant action yet, the Root Cause Analysis was valuable and enlightening. I suggest other development teams do a similar analysis because there is a chance you will discover a persistent problem that is easily correctable.

  4. […] Data obtained from: Use Root Cause Analysis for Defect Prevention in your Software Development Process […]

  5. Davis Tan said

    Thanks for the analysis report. I’ve added a visual graph from your data. Cheers!

  6. Qurat-ul-ain said

    hello every one:
    i need your help regarding how defects can be classified

    • robkraft said

      What sort of help with classifying defects are you looking for? I agree that it is not clear how to group them. I recommend that you make broad groups at first and start assigning the defects to the groups. If some groups of defects are particularly large, then consider if it is possible to break the large groups into smaller groups. So you if have 5 groups of defects, and the number in the groups is 4, 5, 4, 28, 6 – I would examine the group of 28 to see if it can be narrowed down. Don’t try to identify all your groups at the start, that will come over time.

  7. Excellent

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: