Test Management Blog

The ability to add parameters to testcases enables you to write testcases once and then run different instances multiple times with different parameters. It’s a key capability in test management that saves time, enables you scale up your testing more efficiently and helps avoid mistakes when writing repetitive testcases.

It does add a degree of complexity to your test management process though. Managing large volumes of parameter data can become difficult. Sharing parameters across different test cases is not straightforward. And the point at which you define the parameters (e.g. at run time) can be critical to the success of this approach.

With QAComplete we’re going to see how to create parameters in testcases, how we define the parameters and what happens at run time when we create the different permutations of the tests.

Key points to understand with QAComplete are:

1. the parameters (or tokens as they are called in QAComplete) are defined in an external CSV file and have to be uploaded. This makes it slightly more cumbersome having to modify and then upload the parameter file each time. Having said that it usually far easier to manipulate the data in Excel in the first instance.

2. A parameter file has a one-to-one relationship with a testcase. So if you need to use the same parameter file for different testcases you’ll end up having to update and manage multiple instance of the parameter file. This common across most test management tools so isn’t a probably that’s unique to QAComplete.

3. QAComplete automatically creates the permutations of testcases at run time. There is no capability to modify or adjust the parameters when come to start the test run. Having said that you can modify the testcase when as you execute it (assuming you have the privileges set to do so).

Using parameters is a very useful feature to have in a test management tool. It simplifies managing permutations of test cases. Having said that it does bring with it it’s own set of issues and complexities. So long as you understand these points, like how easy is it to manage the parameter data, it’s usually a far simpler way to manage an otherwise complex part of your test management process.

It’s important to understand if and how version control of your QA artifacts is relevant to your test management process. If you work in a highly regulated environment then it likely that version control is very important. If you work in an environment where documentation and traceability aren’t so important then this may not be so relevant. Either way it’s useful to trace the history and modifications to your test artifacts. As most tools will do this for you in the background it’s usually worth starting off with tracking the history of changes at first.

To understand the different approaches to version control read this previous blog post on the Three Approaches to Test Case Management Version Control.

So at a basic level version control may just mean having your test management tool track all changes that you make to a testcase. At a more advanced level version control may mean:

1. having a specific version identifier for a record every time it changes
2. tracing the version of a testcase that is used in execution
3. tracing the version of a testcase that is in development
4. implementing the process of checking in and checking out changes to records
5. having baseline sets of artifacts that are clearly
6. having formal sign off for reviewed and approved records

Needless to say different test management tools deliver different capabilities when it comes to version control. In the following example we look at how QAComplete implements version control and how the QA team can use these capabilities.

With QAComplete each testcase in the library is version controlled. Any change to the record will result in the version identifier being automatically updated. Testcases are grouped into sets. Whilst a set links to the testcases it doesn’t link to specific versions of the testcases. This means that only at testcase execution time does QAComplete’s version control come into play. When you run the cases in a set QAComplete takes a copy of the latest versions of the testcases. This means two things…

1. the tests in the library can be updated without affecting the those already being run. This is important because tests already run should not be modified if the library testcase is updated. If the test already run were updated then this would invalidate the run results. This is a concept that many low end test management tools don’t deal with very well.

2. tests in the run can also be updated without affecting the library testcases. Having said this there is the ability in QAComplete to change the testcase run time. If you do this then QAComplete can give you the option to propagate that change back to the library (and in doing so incrementing the version of the testcase in the library).

There is no concept of a baseline or check in/out in QAComplete. Although, there is the capability to setup a review process so that tests can be signed off and/or approved by users.

Version control in test case management can lead to all sorts of complexities in the QA process. If you keep it at it’s most basic it can be just about tracking changes and the modification history to a record (which is pretty easy for a tool to do in the background). However, for some companies it’s more important than this and there is a requirement to trace where different versions of a testcase are executed. Add to this the process of checking in and checking out testcases for modifications, along with the need to baseline collections of testcases and you have a vastly more complex test management process.

All test management tools have the concept of assigning testcases. Some provide more capabilities than others. Certain tools provide just about every capability you can imagine but do this with a level of complexity which can be quite difficult to manage. Others have limitations that may or may not impact your ability to implement the process you need. In this article we look in detail at how your test management tool implements assignment at the set, case and step level.

Key points you should consider in regards to how a test management tool implements the concepts of assignment are:

1. At which level do I want to carry out assignment (e.g. step, case and set).
2. Do I need to be able to define different assignments of individual test steps within a case.
3. Do I need to be able to define assignment of individual testcases within a test set.
4. At run time how does assignment of the same test set to different configurations work.
5. Can assignment of these different components be changed at run time.

In the following example we look at how QAComplete and ALMComplete implement the concept of assigning testcases and sets. Cases are contained within a Library where we carry out the development and store our testcases ready for use in a set. Each set will contain multiple testcases and can then be executed against a release and a configuration. It should be noted that assignment can not be set at the step level or for test cases within a set. All the steps within a testcase are assigned at the testcase level. And all the tests within a set are assigned at the set level. With the test management tool QAComplete, default fields include both owner and assigned to fields for tracking assignment.

Both cases and sets have an owner (the person who created the record) and the person it is assigned to (the person developing or executing). Within the library individual tests may be assigned to someone. The owner may be the person responsible for maintaining the set and the assigned to the person responsible for executing the set. If you want to have different people run the same set against different configurations or releases then you have to change the assignment prior to each run of the set.

The concepts around assiging testcases to a tester sound simple. In practice though there are many ways to approach this and there are many ways the different tools provide this capability. The key is understanding these different approaches. Then making sure you select and set up your test management tool to cover the right process flow for you and your team.