Test Management Blog


Setting Up Automated Tests in QAComplete

November 21st, 2014 by admin

We’re going to continue with our look at how to combine manual and automated tests with QAComplete and TestComplete. We’ve already seen how to install and configure the Test Agent which connects QAComplete and TestComplete. Now we look at how to setup automated tests and execute them.

You will need to configure the Test Manager Agent on the TestComplete clients machines before you follow these steps. More on this in Bringing Manual and Automated Tests Together

Checking That Automation Hosts Are Available

At this point we should have the agent installed and running on the TestComplete client machine. This process should show in Task Manager on the TestComplete machine as

        TestManagerAgent     -      Test Manager Agent

You can also check in QAComplete that the TestComplete host is available by viewing the ‘Test Hosts’ records.

Show Hosts In QAComplete

Note that if the host isn’t listed at all you may need to enable the ‘Show Inactive Test Hosts’ option.

Show Active Hosts

If the host isn’t active then you’ll need to start the service on the TestComplete machine. On the TestComplete machine Press Ctrl+Shift+Esc to display task manager and then click on the ‘Services’ tab followed by the ‘Services’ button. In the Services window click on the ‘TestManager Agent’ service and start the service.

Starting the test agent for TestComplete

If you have a lot of virtual machines running TestComplete (along with the ‘TestManager Agent’) then this ‘Test Hosts’ view in QAComplete is a really useful view. It lets you to see which virtual machines are active and available for you to run your tests on.

Creating An Automate Test

Now that our Test Hosts are ready we can start to add automated tests to QAComplete and then execute those tests from QAComplete. This is a 4 stage process.

  1. Package up the TestComplete project suite
  2. Define the Automated Test in the QAComplete Test Library
  3. Execution of Automated Tests – standalone
  4. Exeuction of Automated Tests – as part of a Test Set

Package up the TestComplete Project Suite

Key to this step is packaging up the TestComplete project suite. Note that this works only at the project suite level. You can’t zip up just the projects. It must be the whole project suite. To zip the project suite up follow these steps:

i. make sure you’ve defined the ‘Test Items’ you need and enabled them within your TestComplete project(s).

Defining Test Items In TestComplete

ii. find the location of your TestComplete project suite on the file system of your test complete machine.

Project Suite Location In TestComplete

From here you’ll be able to see where TestComplete is storing the project on your file system.

Project Suite Location (2) In TestComplete

iii. On the file system (or in TestComplete) remove the log files. You may want to back these up or copy them to a different directory if you need to keep them. For the purpose of creating this zipped up project suite we don’t want Mb’s of log files which are of no use in the zipped project.

Delete Log Files In TestComplete

You will need to do this for all the projects in the project suite and the project suite log files.

iv. At the project suite level on the file system find the folder containing your project suite and zip up this project suite.

Zip up Project Suite In TestComplete

Note the name of the zip file (e.g. ProjectSuite2.zip) and remember where you’ve created this zip file.

Once we’ve zipped up the TestComplete project suite we’re in a position to create the automated test in QAComplete.

Define the Automated Test in the QAComplete Test Library

In this step we create the test case in the ‘Test Library’ area of QAComplete and then attach the zipped up TestComplete project suite to this test case.

v. First we need to create a new test. Navigate to the Test Management Library area in QAComplete and select ‘Add New’.

Add New Test Case in QAComplete

Then we need to define the usual meta data required to create the test case (e.g. Title, Description, etc). A couple of fields that are important though:

  • Execution Type: set this to Automated
  • Default Host Name: set this to the host that will be used by default to execute the automated test

Assuming you selected Execution Type = Automated then when you save the test case you’ll be taken to the ‘Automations’ tab for the test case. Here you’ll need to click ‘Add New’ to add a new TestComplete Project Suite.

Add New Automation in QAComplete

When adding a new TestComplete project suite to QAComplete you’ll be presented with the following 6 fields to complete:

  1. Title: either leave this blank and QAComplete will give this automated test the same name as the TestComplete project or define your own name
  2. Time Out: this is how long it should take to run the test. If it goes past this time out value then the test runner will stop running the test and move on to the next one.
  3. Entry Point: use this to identify a specific test or project to run. If you leave it blank the whole project suite will be run. Specify a specific test case to execute an individual test case or a specific project to run only one project.
  4. Agent: at the moment QAComplete only supports one type of test agent which is TestComplete/TestExecute. Other types of test agent are in the pipeline.
  5. Web Site Address or UNC Path: you can place the zipped up project suite file on a shared drive. In which case you’ll define the path to that location and the file name here. Alternatively…
  6. File Attachments: attached the zipped up project suite file to the QAComplete test case and upload the file to QAComplete

A completed record with a project suite zip file uploaded looks like this (the entry point in this example is at the Test Case level)…

Test Case Record in QAComplete

A completed record with a UNC path looks like this (The Entry point in this example is at the project level)…

Test Case Record (2) in QAComplete

At this point in time you can only add one automation project suite to a single QAComplete automated test case. In the future it should be possible to add multiple project suites to one test case. Each ‘automation’ instance then calling different test cases or projects within the project suite for execution. This is in the pipeline though.

Once we’ve defined the automated test in QAComplete we’re ready to execute the test, either as a standalone test or as part of a test set.

Standalone Execution of Automated Tests

To run the automated test case in isolation you click the run now button on the Test Case Library list view.

Running tests from the test library in QAComplete

If you don’t see the run now button then you’ll need to grant the neccessary security privileges in the Security Groups area under setup:

Setting Security Privileges in QAComplete

When running in Standalone mode you will be presented with this Test Runner window:

Standalone run mode in QAComplete

1. The ‘Run By Host’ setting will default to the ‘Default Host Name’ you define in the Test Case. If you want to change this you can click on the current run host name and you’ll see a drop down where you can change the host.

Before you run the test make sure that TestComplete/TestExecute is closed down on the host machine. If TestComplete/TestExecute is already running on the host machine the test run will fail.

2. Click the run button in the Test Runner window.

Not a lot appears to happen at this point in QAComplete but if you go to the run history area in QAComplete you’ll see a new run entry and the status of the run.

Run Status in QAComplete

If you have access to the TestComplete/TestExecute host you should see TestComplete/TestExecute start up and the test project suite run.

Exeuction of Automated Tests as part of a Test Set

At this point you may have a number of different automated test cases defined in the test library. We can group these into Test Sets and run all of these automated tests in one run.

One quick way to create a Test Set is to search for the test cases in the Test Library and then click ‘Create Test Set’.

Create Test Set in QAComplete

This gives you a new Test Set with your automated test cases already included in the Set. Give the new Test Set a name and enter any other meta data that’s important to you. Click submit.

Create Test Set (2) in QAComplete

Now when you go to the Test Set area in QAComplete you should be able to find your new Test Set and view the Test Cases included in the set.

Create Test Set (3) in QAComplete

Create Test Set (4) in QAComplete

You have two options at this point:

1. Run The Test Set Immediately

You can either run the test set immediately by clicking on the ‘Run Now’ button:

Run Test Set in QAComplete

In which case you’ll need to specify the release and configuration you want to run these automated tests on.

Select Release in QAComplete

Then select the hosts to run the tests on follow by ‘Run’.

Select Host in QAComplete

2. Setup a Test Schedule

The second option is to setup a test schedule.

Create Test Schedule in QAComplete

Where you define the Host Name you want the schedule to run on, Run days, Start time, End Time, etc.

Define Test Schedule in QAComplete

The important part here is the link to items. This is where you define the Test Sets and Test Cases that you want this schedule to execute.

Link to Test Items in QAComplete

Once you’ve defined the schedule you can leave it to QAComplete to kick off the automatoin runs on the required hosts at the defined times.

Don’t forget that TestComplete/TestExecute needs to be closed down on the host machines in order for the automated tests to run. It’s also worth pointing out that you can mix automated and manual test cases in a test set. If you do this then an automation run will only execute the automated test automatically (obvious really) and then you’ll have to go into the test run to run the manual tests yourself.

Next up, run history, traceability and reporting. We looked at how combining manual and automated tests in QAComplete helps you with traceability to other artifacts like requirements and reporting on combined execution runs.

Bringing Manual and Automated Tests Together

November 7th, 2014 by admin

Bringing manual and automated tests together to deliver combined reporting and traceability is an important goal for many QA teams. Usually this means linking an automated testing process into a manual test tracking tool. With the new 9.9 release of QAComplete and release 10.0 of TestComplete implementing an integrated solution becomes possible without investing vast amounts of money or lots of time and effort.

In this post we’ll be looking at how to setup QAComplete and TestComplete integration. Then we’ll see how this can really benefit your QA efforts as we look at combined reporting and traceability in the follow up post.

The high level architecture here revolves around having a central test management capability (delivered by QAComplete) and a distributed test automation capability (provided by multiple instances of TestComplete running on multiple hosts). The key being to install and configure a test agent on the TestComplete machines. These test agents then communicate with QAComplete to link both tools.

The first step in all of this is to setup these test agents on the TestComplete host machines. This is a 3 step process.

Step 1 – Setup a QAComplete User

From within QAComplete we need to setup a new user that will act as the account accepting all the automated test run results. This can be done from the Setup tab in QAComplete.

Adding a User In QAComplete

The important thing to remember here is that this user should be assigned to a ’security group’ that has privileges for the following areas:

  • Test Automations
  • Test Hosts
  • Test Schedules

This can be seen in this screen shot:

Security Admin Settings in QAComplete

Step 2 – Install the Test Agent

From your TestComplete machine login to QAComplete with a browser and download the TestAgent.

Downloading the Test Agent in QAComplete

When you run the Test Agent installer you’ll be guided through a few dialogue boxes covering warnings and licence information. Once you’re past these you’ll hit the dialogue box that covers the connection settings to QAComplete:

Connecting the Test Agent in QAComplete

If you’re running with the SaaS QAComplete service then you’ll need to enter the following Web service URL:


     SaaS Web service Url: https://qacomplete.smartbear.com/rest-api/service

If you’re running with your own On-premise install of QAComplete then your Web Service URL will be in the following format (clearly you’ll need to adjust this to point at your server)


     On-premise Web service Url: https://qacomplete./rest-api/service

To make sure that your TestComplete client machine has access to the server you can just take this URL and paste it into the address bar in a browser.


     https://qacomplete.smartbear.com/rest-api/service

You should see a list of the Rest Opperations:

Rest Operations List in QAComplete

This proves that connectivity to the QAComplete server is okay.

Then enter the user and password that you created in Step 1. Note that the User should be the email address of the user you setup.

At this point when you click the next button the agent will attempt to check the connnectivity and account that you’ve specified. If this succeeds you’ll get the next dialogue box that covers the windows account that the agent will use on this TestComplete host machine:

Windows User Account in QAComplete

In here you’ll need to add the domain, user name and password you use to log in to Windows and run TestComplete.

At this point the installer will configure and setup your new Test Agent. Once that’s finished your instalation should be complete and you’ll be presented with the completion dialogue box:

Agent Installed on TestComplete Machine

Step 3 – Check Hosts

If you log into QAComplete now, select ‘Test Management’ and then ‘Test Hosts’ you should see your TestComplete host listed.

Test Agent List in QAComplete

Now we’re ready to configure our host in QAComplete, setup TestComplete and start running our automated tests. We’ll cover this in the next post.

Looking for help implementing a scalable and maintainable test automation solution? Talk to us about our Accelerator package. Download the Pdf brochure here

Often when you start out with TestComplete you’ll find yourself bewildered by the complexity of identifying and naming the objects in the application you’re testing. TestComplete has a lot of powerful features to help with identifying objects in applications but when you’re starting out you’ll want to get to grips with the basics. The next few posts look at good approaches for mastering Name Mapping in TestComplete.

If you want to understand ‘why’ we have name mapping then this into article is a good place to start. If you want to understand ‘what’ name mapping is then read on. Or if you just want to find out ‘how’ to get started then we’ve got another post coming up soon.

What Is Name Mapping?

When we start out we’ve got a system that looks like this to TestComplete:

Test Complete Object Browser

What we need is a view or perspective on this that focuses on what we’re interested in for our tests. Something like this would be much easier to work with:

TestComplete Alias View

The Object browser

What’s in the Object Browser is exactly what TestComplete sees it at the system level. If you really wanted to you could even use the full system names to identify the objects in your tests. So for example if you use the object spy to identify an object you can see the full system name for the object.

You can then select the ‘Show in Object Browser’ button and you’ll see the object in the objects list in TestComplete. It’s this “Sys.*” name that you can then use in your test steps if you want to:

Sys.Browser("iexplore").Page("https://www.zoho.com/crm/lp/login.html").Panel(0).Frame("zohoiam").Table("outertable").Cell(1, 0).Panel("loginform").Form("login").Table("inntbl").Cell(0, 0).Table(0).Cell(2, 1).EmailInput("lid")

It’s not normally wise to use this though. The problem with the full system name is that it changes. It’s also difficult to understand because it’s so long. There are much better ways of identifying objects too.

Just bear in mind that this is where it all starts. It’s the foundation that the Name Map is built on. Think of the Object browser as the lowest layer. Then think of the TestComplete Name map and mapped objects as a layer on top of the Object Browser. When the Name Map is laid over the Object Browser it hides a lot of the chafe and complexity that you’re not interested in.

Mapped Objects

When you’re writing automated tests all you want to focus on are the objects in your application that you’re interacting with. So that you can focus on just the objects you’re interested in, the Name Map ‘Mapped Objects’ are picked out of system Object map. What you see then in a cut down list of mapped objects which are the objects you’re interested in for your testing.

So for example when you record a test, TestComplete will populate the name map with all the objects that you’ve interacted with during that recording. All the other objects on the system are ignored. Thus you get a clearer, more focused list of objects that you’re testing with.

That’s not to say TestComplete will give you a completely clear and focused list. TestComplete takes it’s best guess to identify the objects you interact with and map them. It’s not a mind reader though. It will map absolutely everything you interact with not just the objects you’re really interested in. TestComplete will also record the full paths to these objects which can be a little unwieldy. For example:

NameMapping.Sys.browser.pageSignInToYourZohoCrmAccount.
frameZohoiam.formLogin.buttonSignIn

Not exactly easy to type if you have to refer to this object many times in your scripts. Having said that it’s a big improvement over the “Sys.*” object name. In fact TestComplete has taken some steps to reduce the complexity of the naming by removing some of the container references which helps a bit too (e.g. levels like Table(”inntbl”) have been removed).

Where does this leave us? Well we started out with this….

Sys.Browser("iexplore").Page("https://www.zoho.com/crm/lp/login.html").Panel(0).Frame("zohoiam").Table("outertable").Cell(1, 0).Panel("loginform").Form("login").Table("inntbl").Cell(0, 0).Table(0).Cell(7, 1).Button("submit_but")

And now we’re down to this…

NameMapping.Sys.browser.pageSignInToYourZohoCrmAccount.
frameZohoiam.formLogin.buttonSignIn

Notice also that the reference starts out with the string “NameMapping.*”. Mapped objects start with “Sys.*”. The “NameMapping.*” naming convention can be used now in your scripts to identify that you’re refering to objects that are listed in the Name Map.

To make our lives even easier, and to avoid these long “Sys.*” and “NameMapping.*” naming conventions you can also make use of Aliases.

Aliases

Aliases allow us to put another layer on top of the Object Map and the Name Map. So this final layer on top simplifies things even further.

This simplifies things for 3 reasons….

1. You can change the name of objects to make them more meaningful

So for example you can take the name “emailinputLid” that your developers used in the application and change it to “EmailTextBox”.

2. You can manually remove layers of the heirarchy/path to simplify the path name.

So for example you might have a long path in the Name map and then modify this in the aliases to shorten it to something more logical. So you’d go from …..

Aliases.browser.pageSignInToYourZohoCrmAccount.frameZohoiam.
formLogin.EmailTextBox

to

Aliases.browser.pageSignInToYourZohoCrmAccount.EmailTextBox

3. You can rename items in the hierarchy/path

So these parts of the path that are left still look a little unweidly. So we can manually go in and rename parts of the path. At this point we’ll take this..

Aliases.browser.pageSignInToYourZohoCrmAccount.EmailTextBox

and rename to give us

Aliases.browser.SignInPage.EmailTextBox

Far more readable and meaningful when used throughout out keyword and scripted test cases.

Summary

We’ve set out with a system name of….

Sys.Browser("iexplore").Page("https://www.zoho.com/crm/lp/login.html").Panel(0).Frame("zohoiam").Table("outertable").Cell(1, 0).Panel("loginform").Form("login").Table("inntbl").Cell(0, 0).Table(0).Cell(7, 1).emailinputLid

Then we’ve reduced it down with Name Mapping and Aliasing to end up with..

Aliases.browser.SignInPage.EmailTextBox

It’s not difficult to see just how much easier it will be to write and read our tests with the Alias naming convention rather than the full System naming convetion.

Up next: Using the name map properties pane to identify the objects reliably.

NameMapping – Conditional Mode

October 7th, 2013 by admin

We’ve mentioned in previous posts about Name Mapping in our software test automation tool TestComplete. Whilst creating a Mapped name for an object we select a few properties which we use to identify that object. This is the ‘Basic Mode’ of Name Mapping. In some cases though, the value of the selected property may differ. Perhaps in a different environment the properties are different.

For example, a window may have different class name in different OS. Sometimes you may want to give the value of the property within some range or with some condition. For example, Childcount > 34, or Name should start with the string “save”. To create name mapping with these conditions we have to use the Conditional Mode of Name Mapping.

By default, while creating a Mapped name for an object, TestComplete opens the Object Name Mapping window in Basic mode.

ObjectNameMappingwindow

However, you can switch to conditional mode. You just need to click the Conditional Mode button at the bottom of the window. The conditional mode window will then look like this.

object-name-mapping

From this you can see that you can give combinations of properties with extended conditions.

Let us say there is a log-in window in a windows application. This window has a different class name in each OS as shown below.

OS
Class Name
windows 2003 windClass3
windows 2008 class8Wind
windows XP claswindXP
Windows Vista claswindvis
Windows 7 claswind7wind

So we might be working on XP and while creating the Mapped name for this window we select the following properties.

wndClass claswindXP
wndCaption New User

selected-properties-in-windows-xp

You may also use the following script to refer to this log-in window.

mapped-name-of-login-window

After developing the script for XP, if you try to execute the same code in Windows 7 it will not recognize the window object. TestComplete will log an error message “Object does not exist”. This is because in Windows 7 this window has the wndClass value as claswind7wind. This values is not equal to the value selected in Name Map. So by default our software test automation tool fails at this point of object identification.

To avoid this issue you could create multiple mapped names for this object on each OS. However, you would end up with a very long name map object tree. With conditional mode in NameMapping, you can simplify this and mention all the class names in a single mapped name (as shown below).

conditional-mode-of-NameMapping

If we provide the conditions above then name mapping will work in any OS. So, the above mentioned script will then work in all the operating systems. The following section explains how to name map for this login window in conditional mode.

Steps to Create NameMapping in conditional mode:

  1. Open the Object Name Mapping window of that object. Click on Conditional Mode button.
  2. ObjectNameMapping-1-1

  3. The Object Name Mapping window will be displayed in conditional mode as in the image below, with the selected properties.
  4. property-selected-inobjet-name-mapping

  5. Select the row of a particular property and click AND or OR button. (For above example select the row of wndClass and click OR)
  6. select-property

  7. TestComplete will show a new row as in the screenshot below.
  8. new-row-OR-condition-added

  9. In this new row, select the cell in the Property column and click on the ellipse button.
  10. select-this-cell

  11. Our software test automation tool will invoke the Edit Property window. Select a Property from the Type combo box and click OK button. (From the above example select wndClass property)
  12. edit-property

  13. Select a condition type in the Condition column. (From the above example, select Equals)
  14. select-condition

  15. Select the cell in the ‘Value’ column and click on the ellipse button.
  16. select-value

  17. This will invoke the Edit Value window. Select or enter the appropriate values in each field and click the OK button. (From the above example, use the values as in the image below)
  18. adding-value-to-each-field

  19. In this example, we have added two wndClass values with the OR condition. Now this name mapping will work in XP and Windows 7.
  20. two-values-added-for-wndClass

  21. In the same way you can add multiple values for wndClass with the OR condition.
  22. windows-class-for-each-os

Now in the above example, you can use the same code in all the operating systems. You will not need to make any changes to your code to manage the different operating systems. Our software test automation tool will recognize the window in all the OS using the single mapped name.

So the conditional mode reduces our effort and increases the testing productivity. We can make our script more adaptable. And since we no longer need to make any changes in the script, maintenance of our code becomes simpler.

TestComplete Project Test Items

October 7th, 2013 by admin

In Testcomplete the Test Items page within your project is used to organize, manage & execute the test cases that you’ve automated. For example we might have developed test automation scripts for thousands of test cases. However, each time we want to create a software test automation run we may want to execute only a selection of the testcases. We may also want to execute some multiple times with different test data. The Test Items feature allows us to perform all of these tasks simply.

  1. Organise Testcases
  2. Execute only selected Testcases.
  3. Execute a testcase multiple times
  4. Executing a testcase with different test data

A blank Test Items page will look like this:

TestItems

So lets work with this and consider the following software automation script.

automation-test-script

This TestComplete project has keyword and script routines. Let us say each keyword test represents a different testcase. Also each method in Unit1 is a testcase. As shown in the image the script routine ‘Test4’ requires two parameters (name and age) in order to be executed.

If we add every method & keyword test as a Test Item, we can easily execute the required test cases alone. You can see this in the image below.

run-the scripts

If we click the Run button this will execute only the selected Items. We can add any type of Test items (Script routines, Keyword Test, Load Test, Low level script unit etc) here in order to execute the items in the desired order. Also, if we need to we can pass parameters between the items we’re executing.

Before we look at passing parameters lets see how we can add a Test Item.

  1. Open the Test Items page of the Project. To do this,
    • Double Click on Project Folder in the Project Explorer panel.
    • double-click-project-folder

    • It will open the project Editor window. Click on the Test Items option which is available at the bottom.
    • tes-project-folder-select

  2. Click on ‘New Test Item’ button. It will add a New Test Item row.
  3. New-Test-Item

  4. In the Name column of the New Test Item row, enter a name for this Test Item.
  5. enter-name-test-item

  6. In the Test column, select the Test item to be executed. A Test item may be a script routine or keyword test etc. To select a Test item do the following steps.
    • Click on the ellipse button in Test column.
    • select-test-column

    • It will open Select Test window. Here select a Script routine or Key word test and click OK button. (Here we select the script routine Test4)
    • Select-new-Test

    • Now in the Test column it will show the selected Test item.
    • selected-test-routine-displayed

  7. In the Count column enter the number of times the Test item is to be executed.For example If we enter 3, this Test item will be executed 3 times. For now let us keep the value as 1.
  8. enter-number0of0itens-test-execution

  9. The Timeout column shows the maximum execution time allowed for that particular Test Item. If we set this value as 20, after 20 minutes if the execution of the test item is not finished, it stops executing this test item and moves to next Test item.
  10. set-time-out

  11. In the Parameters column, you can set the values for the parameter of that Test item.
    • If the Test Item has no parameter, the value will be displayed as [none]
    • Parameters

    • If the Test Item has any parameters, we should pass values for those parameters. To pass values do the following.
      • Click on the ellipse button in parameters column.
      • Test-Parameters-window

      • It will open Test Parameters window. This window will show the list of parameters available for that Test Item.
      • test-Parameters-window1

      • For each parameter select the value Type and Enter (or select) the Value. Then Click OK button.
      • select-type-and-value-in-test-parameter

  12. In the Description column, enter a description for this Test Item.
  13. enter-description-value

Now we have added a Test Item. The Check box in the Name Column is used to enable or disable the Test Item. If it is not checked, the Test Item will not be executed.

enable-disable-test-item

With this, we can add the Test items we need and organise our test execution. All this without having to modify any of our coded scripts or our keyword tests. This approach to structuring your automation delivers a level of flexibility that makes organising your testing far simpler.

Copyright ©2014 - Traq Software Ltd - All Rights Reserved.