Automation Testing in a Risk-Obsessed World

Recently I was talking to a PM friend of mine asking how his projects were going and his response was interesting. “I want to automate all of our testing. My architect and developers are telling me we can but the testers are saying ‘No’ due to risk to the customer! It’s driving me crazy.”

This made me think:

  • Are us testers too protective of our roles on a project that we don’t want to embrace a change, which means there may be a need for fewer testers on a project? Or,
  • Do other project members have a different understanding of what risk means?

We continued our conversation over a couple of beers and a pizza and I explained my take on where risk and automation fit in a project: I call it the Fast Food vs Luxury Dining Concept.

In this concept we – that is, you and I – are the customer and the fast food outlet and restaurant are the vendors.

SCENARIO 1

As the customer, I just want to go to a fast food outlet and I know exactly what I want off the menu. Therefore, I accept the risk that I cannot change the food. There may be some slight variations offered but, on the whole, it’s an ‘off-the-shelf’ order. I accept the risk that it may be tasty or it may not. I also accept the risk that it may not be exactly what I want but it fills a need and I’ll come back time after time if I just want a burger.

OUTCOME: The vendor knows what it has to offer, does nothing bespoke for the customer, turn around time is very short – as you don’t want to keep the customer waiting – and delegates all the risk from the vendor onto the customer.

If that is the case for your project, then yes – Automate as much as you can and deliver a cost-effective product, repetitively, that hits the mark 70% of the time. That is, quantity over quality.

SCENARIO 2

As the customer, I don’t know what exactly it is I want for my meal but the restaurant provides me with a menu with multiple options that I can pick and choose from. On top of this, I can customise my chosen option to meet my taste.

I expect the restaurant to provide exactly what I order and for them to hold the risk that the quality should meet my expectations. However, if the meal is faulty in any way, I can hand it back and get them to fix it or replace it at their cost. That said, I accept that the impact of this is a higher price of the meal compared to the fast food outlet and the longer wait time for the meal to be plated, due to it being created from scratch.

OUTCOME: The risk remains with the vendor to deliver the order to the customer’s specific requirements, the vendor is prepared to accept changes and accommodate them, however the customer also accepts the consequence for all these changes made – such as the long waiting time and higher cost.

If this is the case for your project, then Automate some areas but realise that the customer is expecting quality over quantity, so continued use of manual testing or explorative testing is essential.

So, I said to my PM friend, “understand your customers’ needs, requirements and expectations. Make sure you have explained and communicated all of this to your team.”

If the customer wants a ‘Fast Food Solution’ and they accept the risks, then the testers will alter their approach to the project. If they want a “Luxury Dining Solution” then the risk sits with the project and testing will cost more.

Now he understands…