When evaluating test management applications it’s easy to get carried away with the focus on the technology. Technology isn’t every thing though. Especially these days when the different products all seem to offer a similar feature set. Let’s face it most of the solutions these days cover pretty much the same ground.
You may even consider the price more important than the technology. Even so you’re still missing one very important factor. Probably the most important factor. That factor influences the success of your project like no other. That factor is the choice you make on who implements the product.
Most companies have requirements to integrate their test management solution into their existing process and environment. This means configuring the tool to match their workflow. And integrating with other tools like defect tracking and source code control tools. This is no simple matter for established QA teams.
Configuring the work flow can cause problems in it’s own right. Often it’s the first time a company sits down and actually thinks through their workflow. Couple this with a new test management tool (which you’re still learning) and you’ve got lots of opportunities to get it wrong.
Once you’ve defined the work flow incorrectly it can be difficult to put right. For example a good vendor will likely take you down the ‘less is more’ path. It’s easier to start out with fewer configurable fields and add more later. It’s more difficult to start out with lots of configurable fields, tracking every last bit of info you think you need. Then when you come to remove the unwanted fields later you find they’re locked in to some aspect of our work flow. The temptation to add more custom fields and extra work flow steps is always high. For someone in your team who’s been left to setup the tool with little experience you’ll end up tracking everything. An experienced implementation vendor will keep things lean and efficient for you right from the start.
Integration with existing tools the other big item to consider. Rarely these days will a test management tool sit in isolation. Often they will need to integrate with existing defect management applications, possibly requirements tracking applications and maybe even with source code control tools. Add to that integration with automation environments and you have a situation which no out of the box solution is going to work with. An implementation vendor will have seen this and managed this type of setup many times. The time they save getting this setup will out weigh the cost of bringing on board help many times over.
A good decision made on choosing the technology can very quickly be undone by choosing the wrong implementation vendor. Test management applications no longer live in a vacuum where you are setting up the process from scratch. They have to mould to your existing process and integrate with existing applications. The way to extract maximum benefit from your test management tool is to choose a supplier that understands this. So separate the selection of the technology from the selection of the implementation partner.