Text Recognition for Test Automation

August 23, 2013

Sometimes with test automation tools it’s difficult not to jump to the conclusion that they aren’t suitable for the job you want them to do. Usually this because of the steep learning curve and the difficulties associated with working out how best to approach the problem you are trying to resolve. One of the most important aspects with any tool is it’s ability to identify objects in the application you’re trying to automate. When you can’t identify the object you’re stuck.

With Test Complete you may come across instances where it won’t recognize the control or window in your application. This is because Test Complete may not have in-built support for those objects. So you may think it is impossible to write a reliable test automation script. Part of the learning curve of working out a new tool is to find other ways to approach these problems. We have other approaches we can use to resolve these type of issues. One of the best solutions is the Text Recognition feature in Test Complete.

Consider the following example with this test automation tool. Here I want to write a script to click Sent Items folder in Outlook. So I am trying to get the properties of Sent Items folder object, using testcomplete’s Object spy. Unfortunately this object is not recognized. It takes the entire Folder Window as an object.

Entire window_Single_Object

If you open the properties on this Folder Window in the object browser it shows as in the following screenshot.

Child_Count_0

You can see that this Folder Window has 0 child objects. So you may think that it is not possible to write a script to click the Sent Items folder.

If you try Record & Playback, it will give you the following script.

recordedd_Script

Here it records the coordinates of the point we clicked on when we clicked the Folder within the pane. If you change the position of Sent Items folder, we need to change the coordinates in the script. We all know that it’s very difficult to identify the coordinates of a point in a window. This doesn’t make for a good approach to implementing a test automation tool.

As a smart automation engineer, you may search for some other way to create a reliable script. Instead of coordinates, if we identify the folder object using some unique name, the script will be far more reliable.

Using the Text Recognition capabilities in a test automation tools is a good solution in this situation. If you enable the Text Recognition option for a window, Test Complete reads the Text on every controls in that window and creates Text objects for each. For example we’ll get: TextObject(“Inbox”), TextObject(“Sent Items”) etc. Using this Text object, we can access a particular control on that window and write the script we need.

How do we enable the Text Recognition option for a window?

  1. From the object browser, open the properties of that window (or Using Object spy open it’s properties). Note down the value in the WndClass property. (For Folder Window Object the value is NetUIHWND)
  2. Windclass_Property

  3. From the Main Menu of Testcomplete, Click Tools >> Current Project Properties option.
  4. Text_Recognition

  5. In the Current Project properties window, Go to Open Application >> Text Recognition Option.
  6. In this Window Click the Add Button and enter the WndClass value of the Window for     which you want to enable the Text Recognition capability. (In the above example its   NetUIHWND). Also Check the Check box for this added Item.
  7. Add_className

  8. Click Save Button.

Now you’ve enabled the Text Recognition option within the test automation tool for your window. In the object Browser check the properties of this window.

Add_className

The Window has child objects. It has created child objects using the Text on every control.  You can access any of these child objects using the name TextObject (<Text on that control>). For the above example you can modify the script as shown below.

Modified_Script

This script will click the Sent Items folder, even when the position changes.

In my experience, many windows applications have table objects, tree objects etc. Sometimes though they are not open to Test Complete. You can identify the table or tree objects as a whole, but you can’t identify every cell or node within the objects. If you enable the Text Recognition capability for those windows, it will create Text Objects for each cell or node and you can access those objects. Once you can access the objects you can use this test automation tool to write scripts to perform actions on the objects.