Enhance Hackathon 2019: How to manage complex test cases

THE PROBLEM

One of the problems that Enhance faces as a company is:

“What are methods to manage complex manual test cases by using test management tools and automation?

During the Hackathon Enhance members split into teams to investigate and formulate ideas and solutions around how to handle this problem in our day-to-day work.

GETTING STARTED

The group discussed how we wanted to approach the problem and came with a list of key considerations that we think the solution should cover, which were as follows:

  • A tool that can import and export test cases using common file formats.
  • User friendly interface for building test cases.
  • Supports a BDD style approach.
  • Tools that can be integrated with other third-party tools.
  • How to handle a client’s existing manual test cases.
RESULTS OF THE RESEARCH

As a team we investigated tools such as Zephyr, HipTest and Test Rail. Ultimately, for this specific scenario we found HipTest to be the best tool for the task at hand and decided to use this as a baseline for our “tool to beat”.  The ability to refactor features, the auto completion of steps and the reusability of steps between the multiple tests were what stood out in this tool and led us to select HipTest for further exploration.

MAKING IT WORK

In the limited time we had on our hackathon day we focussed on HipTest and quickly discovered how easy it was to use because of all the tutorials and documentation HipTest has.

HipTest as a tool is user friendly and easy to use. It gives several options when creating a new project. We explored importing test cases from an Excel sheet, importing a zipped Java automation project and exporting tests into Java code.

Like any other tool, HipTest has its own set of rules for importing. For importing .csv or .xlsx (Excel) files, a template is provided to build your own cases in and you are given details on the rules the file must follow when importing it. We successfully imported cases using this template.

We also imported a zipped Java automation project which contained our custom framework and existing BDD feature files. The outcome was surprisingly good and HipTest was able to auto generate the test cases and steps based on the feature files and step definitions of the project.

Finally, we created test cases within HipTest itself and exported them as a Java/Cucumber project. This produced a sample automation project with feature files based on our tests and the associated step definitions pre-generated.  There are also many other programming language/framework options available when exporting your test cases.

CONCLUSIONS

Our results and recommendations were as follows:

  • For projects that are just starting out we found HipTest easy to use to create test cases directly. If a project has a significant number of manual test cases defined already it would be easier to import these using the template Excel file, although there would be a degree of effort in formatting the existing cases to fit the import template.
  • Transferring existing test cases is a tedious task, which is why we also recommend only moving the core functional test cases as this will provide the most value to the team in the short term. Additional cases may be imported if time and budget allow.
  • The management and updating of BDD test cases to and from automated code is simple and can work alongside the management of manual-only test cases within the same project. Manual test cases written in a BDD style may also have additional comments inserted which will allow them to be used as a comprehensive test suite without impacting the BDD flow of automated tests.
  • We also recommend taking advantage of the third-party applications that HipTest can integrate with such as JIRA to ensure the test efforts are linked back to the development tasks and bugs.

Ultimately, HipTest is not the only solution to the problem, but it was a good solution to this specific problem. We found out that most of the tools could be used based on client situations and we must adjust to them accordingly. In the future maybe we will find an application that can be used across any kind of client with any kind of situation, a one size fits all solution. But for now, we think that HipTest is one of the best for this problem.