So you are the person in charge of a software project. It may be something small, an MVP (Minimum Viable Product) or a large enterprise project.

You want it quickly, cheap, and without bugs. I’m not even going to say the “pick two” cliche (there, I said it), but the truth is that too often in the industry, not even two of those boxes are being ticked.

There are many variables affecting the quality of the delivered product. Testing, which is my focus here, is one of the important ones.

How much effort should be spent on testing? There is no silver bullet answer.

How much effort should be spent on testing by the project team? While more testing is usually better than less, it comes with a cost. While the cost increase is roughly linear with the increase in effort, the benefit curve will flatten at some point: more testing, while continuing to add to costs, will not bring any significant increase in quality.

There isn’t a silver bullet answer that matches every project, and the right answer has to take into account both costs and risks.

When thinking “risk”, two main components have to be taken into account:

  • What is the probability for an undesired event to happen?
  • What are the consequences of such an event taking place?

The right answer will always depend on multiple factors such as:

  • Business maturity – early startup, mature startup, enterprise, etc.
  • Industry – financial or health care will have a different risk profile compared to NGOs
  • Project type – more risk is acceptable for an application developed for in-house use compared to an app that will be used by external customers

The two ends of the spectrum are:

A. Startups

Startups often need to spend a lot of their early efforts validating their business idea with real users. Given the high quality expectations users have in today’s technology ecosystem, releasing a buggy app just for validation purposes is not a good option. It may be a better idea that early validation is done using a clickable mockup solution instead of implementing the functionality in an app. This is fast to create, as there are quite a few viable solutions on the market (e.g. InVision, Framer, Marvel, Origami, Moqups, etc.), and requires almost no testing at all.

The next step would be to implement an MVP based on the feedback received during the previous phase. While testing is needed in this phase, cost can be kept under control by focusing on developing only features that have been identified as relevant/important to early adopters.

Once past the MVP stage, testing needs to become an integral part of the development process.

B. Enterprises

Both expectations and stakes (financial and reputational) are usually high for enterprise level projects. Corners cannot be cut, and applications need to be tested to the extremes for full compliance with both functional (what the application should do) and non functional (security, performance, scalability, etc.) requirements. This should always be done, and is usually possible as enterprise budgets are more generous than startup ones. The QA team need to work closely with the business in order to develop and maintain a test plan based on relevant use cases.

At times, clients may be tempted to reduce costs by doing the testing themselves. It doesn’t work.

At times, clients may be tempted to reduce costs by doing the testing themselves. It doesn’t work. Even without realizing, they are not ready to receive untested functionality, and no matter how many times the development team will reiterate that the functionality has not been tested, the client will be left with a feeling that what they received was of poor quality. You need to let the team test their work before they deliver it to you. It’s a product, let the team developing it finish it properly before you receive it, and it’s not really finished until it’s been properly tested and any issues found during testing were fixed.

Would you ask the maker of your car to skip quality control for a reduction in price?

When you buy a car, you don’t ask the maker to skip quality control for a reduction in price.

Testing should not be thought of as being separate from development. It should be part of the development process at (almost) every step, and coding itself should be done with testing in mind. The way an application is written should facilitate testing as early as possible, with Test Driven Development being a very good example.

To summarize, how much testing you need will depend on your particular project, but you do need testing! You should discuss it before project start with the development team, and it should depend a lot on priorities and on the amount of risk you can afford to take. It is important that you communicate from the very start all the specifics and agree on a plan that will maximize the chances of project success.