Misconceptions About Software Testing – How Undervaluing Testing is Costing You on Projects

Written by Ashe Prone.

Testing is an afterthought to many companies.

This is a bold statement but hear me out.

“But…!” I can hear you exclaim. “Testing is the most critical part of our project! We always include time for testing.”

This statement is generally true. However, I have found companies harbour one or more misconceptions about testing that lead to a “cutting corners” approach that they may not be aware they are employing, and it ends up costing more in the long run.

And truly – I do understand. Projects are finite; funding, scope, time, and resources are constrained, and choices must be made. Unfortunately testing often comes last in the order of priority. Perhaps this is because the test phase occurs later in delivery and the frenzy of activity at the start of a project causes focus to shift to the most immediate actions, but I also believe it happens because the purpose and value of testing is not generally understood, and so it gets relegated to the back of the line.

Testing becomes a footnote on the project schedule. Something to worry about when the time comes, right?

This mentality hurts projects and reduces quality in all aspects of delivery. Plus, it costs more money.

Please Don’t Make These Mistakes

The following misconceptions are the most common situations I have encountered on projects and in most cases caused the project to suffer as a result.

1. We don’t need a Test Manager/Analyst until we’re ready to start testing

I see this one the most. Often, test resources get brought in weeks, sometimes days, before the project needs to begin a testing phase. In the short timeframe leading up to kick-off, a Test Manager must quickly get up to speed on system and design documentation, build relationships with stakeholders, organise and train testers, and map out an efficient, impactful test strategy and plan (an experienced Test Manager will have this capability because it is such a common practice in projects, this does not mean this should be the expectation). Right out of the gate, this approach increases expense to the project.

According to the International Software Testing Qualifications Board (ISTQB), the expense of fixing failures is greater when detected during acceptance testing or in production than during earlier stages of the project.¹ Bringing in a skilled tester earlier in the project is one way to achieve a lower cost of quality, preferably during requirements gathering, so they can participate in the creation and review of documentation. Testers can help prevent failures in later testing phases by identifying inconsistencies in requirements and defects in documentation prior to design and implementation, which can be costly in expense and customer trust down the road.

Including a Test Manager during the planning stages of a project can help lower the cost of quality by:

Identifying risks earlier.
Highlighting areas of testing that may require more resources or time to prepare (i.e., performance testing).
Establishing effective test practices from the start.
Providing guidance for the project plan, including informing critical path decisions.
Generally, just allowing more time and space to plan testing – think active, not reactive!

Project initiation should include risk and cost-benefit analysis to determine when to bring in skilled test resources. Not all projects require a Test Manager, but at the least, each project should have a skilled tester who is included from the start of the project as they contribute significantly to quality.

2. We don’t need a Test Analyst – Developers / Business Analysts / Business Users can be the testers

Developers can fix defects early but are at the lowest level of test independence
Firstly, Developers testing their own code during unit testing is fine, that is standard practice. For higher levels of testing such as System and Acceptance Testing, it is recommended to have some level of independent testing as independence contributes to quality by reducing bias and expanding the scope of testing (for example, developers tend to focus on positive test cases or may not have the ability to perform end-to-end testing).

Developers testing their own code is the lowest level of independence available in the range of options, with external test teams sitting at the highest level.² If a project team decides only the Developers will test, then it is recommended to have clearly defined test coverage criteria and include an auditor to review the test products for thoroughness.

Business Analysis is fundamentally different from Test Analysis
Secondly, Test Analysis is a skillset and a full-time role, just as the other major roles are on a project. Unless hired specifically as a BA/QA, a Test Analyst is not going to be hired to fill a Business Analyst role, so it is strange when a test role is expected to be fulfilled by a Business Analyst.

Does this mean a Business Analyst cannot be a Test Analyst or vice versa? No, the two roles are not mutually exclusive. However, they are fundamentally different roles with distinct perspectives and objectives. This can lead to variable levels of focus, motivation, and testing skills. Because these disciplines differ, context switching is required between activities, so when one person must embody both roles it can reduce the effectiveness of both functions. It can be done though!

If the project team decides the Business Analyst will function as the Test Analyst, ensure the BA is properly supported by:

Allowing time to solely focus on test activities
Providing basic training in test analysis
Nominating a Business Analyst backup during formal test phases

Business Users have their place
Lastly, Business Users have their own unique place in testing and should be involved in testing – especially if they are the end-user – as they have business process knowledge and are closest to the customer. However, like Developers, Business Users will have their own biases and ideas about test scope, which may not align to defined requirements or provide adequate test coverage. In contrast to Developers, there is a greater level of independence when Business Users test so contribution to quality should be higher.

Additionally, Business Users may not be familiar with project delivery and test practices and may need training to be effective testers. Project leadership should recognise these limitations when deciding on this approach to testing – perhaps invest in business test champions to support Business Users through the testing process.

Testing “sweet spot” is somewhere in the middle
External test teams provide the highest level of independence compared to Developers, Business Analysts, and Business Users. External teams also have their place in the testing space, but they require excellent documentation and clearly defined test objectives, which projects often lack. The testing “sweet spot” therefore lies between the extremes of independence, where the delivery team includes embedded testers familiar with the product, business, and test practices.

3. Documentation? But we’re agile!

It is every Test Manager’s worst nightmare to join a project with disorganised documentation. If the delivery team cannot clearly show what is being delivered, how can you expect a Test Manager or Analyst to properly organise testing?

Documentation is tedious – we get it! However, if you want to set up your test efforts for maximum success, please ensure that documentation quality is high. Quality can be improved by establishing governance processes that include versioning history, template structures with a clear numbering system, and review systems that ensure all team members adhere to documentation processes. It will help your test analysts and your final decision-making if the project clearly establishes traceability from requirements to design products to test products.

You want your test efforts to be spent on testing, not going on hapless adventures trying to figure out what needs to be tested.

4. We’ll test in production

Okay, this one is a joke. But if anyone ever says this seriously, just laugh and move along.

Conclusion

Simply scheduling time to test your product barely scratches the surface of the depth of value that can be realised from understanding and implementing an effective testing capability on projects. Test Analysts and Managers add value by bringing a unique skillset to the project that can contribute significantly to product quality in the form of greater product stability, risk reduction, cost savings, and increased customer trust. Additionally, establishing appropriate documentation standards early in a project will help test efforts later – not only in savings but your Test Manager will thank you, too.

References