Archives: Lessons

Versions & Components

The other major features in Jira that lets us group and categorise our issues are Versions and Components. Both of these features provide us with useful ways to visualise the status of our projects and the work/issues that we need to be focusing on. Each Project you define in Jira can contain many 'Versions' and many 'Components'. Before we look at what Versions and Components are though lets just recap on what a project is. A project in Jira is just a container for a lot of issues. It's the top level container. Every issue must be assigned to one (and only one) project. With a project we can also create Component and Version identifiers. The issues within the project...
Read more...

Workflow & Status

Workflow & Status If you're looking at tracking work tasks the simplest way of categorising the pieces of work you have is with a few simple status values like 'To Do', 'In Progres' and 'Done'. In Jira terms you'd have your issues tracking those pieces of work. Each issue then would be set with a status value of 'To Do', 'In Progress' and 'Done'. When you haven't yet started a piece of work the status is set to 'To Do' on the issue. When you pick up the piece of work and start working on it you move the status value for the related issue to 'In Progress'. And obviously once you've finished the work you move the status value...
Read more...

Labels & priorities

In the last module we saw how to create and structue our Jira Projects. In this module our objective is to understand why we need labels and priorities. We'll also look at setting project priority values and label values. Finally we'll see how we can search for issues based on labels and make bulk updates to our Jira issues.  Why Do We Need Labels and Priorities? There are two other properties of an issue that will help you organise and track your issues. One is the ability to give an issue a priority category. The other is to give your issues one or more labels. Assigning lables and priorities helps you look, for example, for all your high priority...
Read more...

Projects

We’ve looked what an issue is in Jira and how to create them. It’s great to be able to track our work as issues in Jira but the real power comes from Jira’s ability to group, organize and categorize these issues. When you start to get into 100’s, 1000’s or even 10,000’s of an issue the ability to focus just the issues you need becomes absolutely critical. Each Jira issue records a number of key pieces of information that allow us to categorize the issues. Those pieces of information can include the following: Project Components Releases Priority Status Labels Assignment So, for example, you might have an issue that is part of a particular software project, that is related to...
Read more...

Issues and what issues are

We’ve put together a series of 6 training modules that make up an introduction to Jira course. This course is focused on the basics of Jira. We’re covering the core concepts, the building blocks, that makeup Jira. This isn’t focused on any particular development methodology (like Agile) and we don’t go into configuring/using Jira to manage Agile projects here. We’re leaving that for a separate course. What we are doing is going through the core components of Jira and showing you what Jira is built on. It’ll give you the knowledge you need to deal with Issues and see how those issue records can be managed. We’re going to start out looking at what an Issue is (not any specific...
Read more...

Tracking Configurations and Releases

April 4, 2019
It’s well known that keeping track of results from testcases against builds, configurations, environments, etc can become quite difficult to manage. It’s one of the reasons we usually implement test management tools. These tools enable us to log our results against these different aspects of our testing and then quickly produce traceability reports to show our coverage. It’s just that the permutations and combinations can become quite difficult to track, even with the right tools. If we have just 1 testcase, which we need to run against 2 platforms, 2 operating systems along with three configurations relevant to our application we already have 12 permutations. That’s 12 permutations for just 1 testcase. And with this example we’re not even considering...
Read more...

Managing Retests

Re-running tests to check that bugs are fixed is an essential part of the test management process. Teams grow, the amount of data generated by completed testcases increases and the number of raised defects expands too. As a result it becomes more difficult to keep track of the retesting. There are several different ways to approach this. So at some point in the test management process you’ll need to re-test fixes that have been made to resolve issues. This re-test process needs to be approached from two angles. Firstly, identifying failed test cases from previous test runs and then re-running those test cases in a subsequent cycle. Secondly, identifying defects that have been resolved and running tests to confirm that...
Read more...

Test Result Aggregation

No test team can run every single test case on every single release of a product. So being able to aggregate results from different views or areas in your test management tool is essential when it comes to seeing the whole picture. However, how your test management tool deals with that aggregation might not be quite so simple. This partly depends on how the QA team have set things up and partly on how well the tool you use deals with this. In many cases we’ll need our test management tool to aggregate and report on results from various perspectives. For example we may need to aggregate across different projects, application modules, runs or cycles. Whilst this sounds simple in...
Read more...

Managing a Library of Test Cases

Your ability to manage a library of testcases in your test management tool depends a lot on the relationship between the different instances of the testcases. The instances that resides in the library and the instances that are created to be executed and run. At a basic level you you will have two instances of a testcase.. 1. The instance that resides in the library. This is the master copy of the testcase and will not usually have a result record associated with it. 2. The instances that have been used at run/execution time. These instances will all have a result (pass, fail, not yet run, etc) value against them. They may differ from the instance that resides in the...
Read more...

Using Test Parameters

The ability to add parameters to testcases enables you to write testcases once and then run different instances multiple times with different parameters. It’s a key capability in test management that saves time, enables you scale up your testing more efficiently and helps avoid mistakes when writing repetitive testcases. Scaling up your test cases effectively usually involves the practice of parametrization. That is, the concept of having one testcase and then feeding in parameter values to create variations of the test case at run time. So essentially you have the same testcase which you execute repeatedly with different data values. Defining parameters in Quality Center is done at the step level where each step is fed parameter values from a...
Read more...

Version Control of Test Cases

It’s important to understand if and how version control of your QA artifacts is relevant to your test management process. If you work in a highly regulated environment then it likely that version control is very important. If you work in an environment where documentation and traceability aren’t so important then this may not be so relevant. Either way it’s useful to trace the history and modifications to your test artifacts. As most tools will do this for you in the background it’s usually worth starting off with tracking the history of changes at first. To understand the different approaches to version control read this previous blog post on the Three Approaches to Test Case Management Version Control. So at...
Read more...

Assignment of Test Cases

At some point in the process we need to assign a test to a tester. The question is are you looking to assign it at the step, case or set level? Do you need to allocate different tests within a set to different qa engineers? How do you track the difference between assignment for writing and executing the testcase? All of these points complicate a seemingly simple concept. When you look at the different ways different tools support this capability you’ll find it’s not quite as straightforward as you might first think. All test management tools have the concept of assigning testcases. Some provide more capabilities than others. Certain tools provide just about every capability you can imagine but do...
Read more...

Conclusion

When we started this module we had everything running in our automation framework. Jenkins could install the application under test, run the Selenium tests, run the SoapUI tests and execute some performance tests. What we didn’t have was control over the source code that was created for these Selenium, SoapUI and Jmeter tests. None of our tests were stored in a central location and none of them version controlled. In this module we showed you how to setup a central Git server, commit our test files to this Git source code repository and then configure our Jenkins jobs to use the test files stored on this Git server. Finally we looked at how you can develop on one machine and...
Read more...

Part 9: Modify and Commit our Selenium Scripts from another server

At this stage then, maybe another tester decides our Selenium script needs a little updating (more comments perhaps). On our Windows master machine we can check out the scripts, make our modifications and then push the mods back to the repository. Next time we run our Jenkins ‘RunSeleniumTests’ job we should see those changes in the execution of the Selenium script. We’ll see this in action as we complete the next few steps where we make changes to the SelLoginTest.py script on the Windows master machine. We’ll then push those changes to our Git repository. When our Jenkins job runs on the Windows client machine we should see those changes incorporated in that test run. So back on the Windows...
Read more...

Part 8: Updating our Jenkins Job to Use the Git Repository

With our Selenium source code safely stored in our Git Repository all we need to do is make sure that our Jenkins job, that executes the Selenium scripts, pulls the latest source from the repository before it starts. To do that we just need to make a few updates to our ‘RunSeleniumTests’ job. TODO: put the schematic diagram in here Configure the ‘RunSeleniumTests’ job Click on the configure menu option for the ‘RunSeleniumTests’ job on the Jenkins home page: http://testmanagement.com/wp-content/uploads/2019/04/BTS-module6-img10-configure-selenium-job.png" alt="" /> Change the ‘Source Code Management’ option In the ‘Source Code Management’ section change the option from ‘None’ to ‘Git’   Enter details for the Git Repository We need to point this job at our new Git repository residing...
Read more...

Part 7: Commit our Selenium Script to our Git Repository

In the previous parts we initialised the Git repositories on the Linux Ubuntu machine and we installed the Git client application on our Windows machines. From here we need to use that Git client on our Windows client machine to “add” our Selenium code to the Git repository on the Linux Ubuntu machine. Not surprisingly we’ll be using the Git ‘add’ command along with the Git ‘commit’ command. We’ll be doing all of this with the Git command we just ran from the Windows command prompt. Open an RDP session to the Windows Client machine Then open an RDP session to the Windows client machine     You’ll need to go to your AWS console and get the Public DNS/IP...
Read more...

Part 6: Commit Our Selenium Source Code to the Git Server

First then we need a Git client running on our machines where we currently have our source code. For example we have developed our Selenium scripts on our Windows client machine. We’ll need to install a Windows Git client on our Windows master machine and our Windows client machine. That Git client can then commit our Selenium scripts to our Git server. Then all our machines (e.g. our Jenkins slave machines) will have access to this source when they need it. Let’s install this Git client then. These steps need to be repeated BOTH on your Windows Master machine AND your Windows Client machine.     In IE download Git Open IE and go to this Url https://git-scm.com/download/win Fight your...
Read more...

Part 5: Configure the Git Server

Git is already installed on our Ubuntu Linux server (we set this up earlier). We just need to run a handful of commands to configure Git as we need it. We can do this from either of the SSH shells we have access to. Either the Putty SSH shell running on the Windows Master machine or the SSH shell provided as part of the AWS management console. I’m going to use the SSH shell from Putty on our Windows master machine. Connect to the Linux Ubuntu machine using Putty On the windows master machine from the Putty saved sessions select ‘Unix-Git’ This should take you straight into an SSH terminal with no authenticating required. And if you type ‘whoami’ you...
Read more...

Part 4: Configure the Git User Account

To configure our Git server we’ll need to run through a few steps. configure a Git user set up ssh for that user create and store this users SSH key pair copy the private key pair somewhere safe install the private key on the Windows Master machine install the private key on the Windows Client machine What are going to have is a user defined on our new Git server. This user (called ‘ae’ for automation engineer) will be setup with SSH (Secure Shell) access. We’ll be able to have all our clients of this Git repository running with the SSH private key so that they can login to this Git server without having to authenticate with username and password...
Read more...

Part 3: Install Git

We have our server up and running with an SSH shell connection open. Just need to install Git now. Pretty straightforward. Just run this command: sudo apt-get install git Select all the defaults as you are prompted.     This should complete cleanly having installed all the required packages:     And that’s it. Simple. Just need to configure a Git user account and set the Git server up.
Read more...

Part 2: Setup the Security Group and SSH Connection

Now we have our Unix-Git Source Code Control (SCC) machine running we need to make sure the AWS security settings will let us connect to it and configure an SSH terminal connection. First then the AWS Security Group Configuration. Set up the security group by first checking that this linux machine is has the right security group assigned to it. You can do this by selecting the host in the AWS EC2 console.     Click on the ‘Unix-AUT’ link which will take you through to the security groups page for this specific group. Then click on the ‘Inbound’ tab.   At this point we can ‘edit’ the security group and add a new rule:     This rule we’ll...
Read more...

Part 1: Sart a Unix Ubuntu Git SCM Server

April 3, 2019
First we need a Linux AWS server that will run our Source Code Management (SCM) tool Git. We’ve done this a few times before now so we’ll step through the AWS Linux server configuration quickly. The other Linux machines we’ve setup in this course are designed to be started automatically by Jenkins on demand. Slightly different with this Linux machine. We need a machine that’s not started by Jenkins, that's always on, has ??? storage (not emphiperal?) and is protected from being shut down. In the Amazon AWS interface launch a new instance (click on a Launch Instance button). Select the ‘Free tier only’ option and configure this new AMI with the following parameters: STEP 1 : Amazon Machine Image...
Read more...

Prerequisites

Make Sure you have your Private key (.pem file) Back in Module 1 we created our public and private key pairs? Well at that stage you should have saved your private key pair .pem file (e.g. FirstKeyPair.pem) file. You’ll need this private key when configuring Jenkins later. If you don’t have have this private key you can go back and create a new key pair. Much easier if you can find the one you created in Module 1 though. If you’ve followed upto Module 4 so far you should already have your Amazon Virtual machine environment up and running along with Jenkins, Selenium and SoapUI. This existing setup gives us the 2 machines we’ll need to use in this module....
Read more...

The Tools We’ve Chosen

We’re testing a browser based application so it seems sensible to choose Selenium as the tool to write our automated test cases in. We can get Selenium drivers for a range of different browsers so we’ll be able to provide test coverage against IE, FireFox and Chrome (we could do more but this’ll be adequate for now). It’s important to point out though that this isn’t a Selenium course. So the Selenium scripts we’re using here are simple, linear and, well basic. This gives the added advantage that it makes things easier to understand. You just need to be aware that in order to scale the actual test cases you’ll probably want to look at improving the way these test...
Read more...

What You’ll Learn

The aim is to pull all our test files together and manage them effectively from one location. That means pushing any changes to this central location or “repository” as it’s better known. This means getting our Jenkins jobs to automatically use the test files stored in this repository. And it means learning to collaborate on changes to test files by making those changes easily accessible to everyone in your team. Part 1: Sart a Unix Ubuntu Git SCM Server Part 2: Setup the Security Group and SSH Connection Part 3: Install Git Part 4: Configure the Git User Account Part 5: Configure the Git Server Part 6: Commit Our Selenium Source Code to the Git Server Part 7: Commit our...
Read more...

Overview

In this final module we’re looking how we can best control all of our test resources and files. We’ve created Selenium, SoapUI and JMeter tests. The files for all of these tests are now scattered all over our distributed test automation environment. Not great for colloaboration, maintaining versions and backups. Down right dangerous really. What we need to do is pull all of our files together into one central repostiory. Well, with the tool we’re using, Git, it’s more a central distributed repostiory. ‘Central distributed repository’ sounds like a bit of a contradiction. We’ll explain that contradiction as we go through this. Anyway, we’ll be running up an Amazon Liniux instance and installing the source code management tool, Git. Then...
Read more...

Conclusion

Once you start to get a feel for Jenkins building up each of the blocks to create an end-to-end test run covering Gui functional, Gui Api and performance tests isn’t too difficult. Admitidly we’ve only touched the surface of these tools like Selenium, SoapUI and JMeter. With the basics though it’s easy to start building out a lot of these tests and increase the test coverage significantly. The key from here though is making sure the build and the test run are run on a regualr basis. Either configuring the build job for Rocket Chat to run every night or triggering it from source code check-in monitoring (a topic for another course). Kick everything off automatically on a regular basis...
Read more...

Part 8: Check the Performance Job and Configure in the Whole Build Chain

Everything is configured now. We need to check our ‘RunPerformanceTests’ job in isolation and then link this job into the whole build chain. From the Jenkins home page click on the ‘build now’ icon for the ‘RunPerformanceTests’ job:   As the performance reports are ‘trending’ reports you may want to run this job two or three times. That way when you view the results we’ll have some trend data to view. So click the build job icon again. Once the job has completed for the 2nd or 3rd time check the build job results by clicking on the job name:   From here you’ll see the Performance Trend graphs:   And if you click on the ‘Performance Trend’ links you...
Read more...

Part 7: Configure the Jenkins Job to Deploy our Performance Tests

As mentioned previously our performance test files ought to be stored in a source code control system (possibly Git or Svn or something similar). That way when we develop our JMeter tests on our Windows machine we’d check them into our source code control system. Then we’d configure our Linux performance host to check out these tests from our source code control system before running them. We’ll look at doing this the in a later stage but for now we’ll just get Jenkins to copy the JMeter configuration files on to the Performance Server when we need them. First then, we’ll copy the JMeter ‘.jmx’ file to our Jenkins master machine in the ‘userContent’ directory. With the file in this...
Read more...

Part 6: Add and configure the Jenkins Performance Plugin

In Jenkins, in order to capture reports from JMeter we’ll add the ‘Performance Plugin’. Once this is installed and configured we’ll be able to chart the trend of performance results from one build to the next. It’s this trend that’s key to us spotting potential performance issues before it’s too late. Add the ‘Performance plugin’   Once this is installed you should see the plugin listed on the ‘Installed’ tab in the Plugin Manager page on Jenkins (you might need to apply the filter to see this amongst all the other plugins installed) We have our ‘Performance Linux Client’ configured. We have the Jenkins plugins installed that we need. Now we’re in a position to create the Job to run...
Read more...

Part 5: Parameterise our JMeter Project

We’ve configured our JMeter project and we’re ready to set this up as part of our Jenkins jobs now. Only slight problem is that the hostname of our Rocket Chat server is going to change everytime we re-build and install Rocket Chat. That means the hard code Server Name we’ve entered in JMeter won’t work. So we need to parameterise this value in our JMeter project. The aim is to pass the Rocket Chat hostname as a command line argument to JMeter when we kick off the performance test. To do this we need to update our JMeter ‘HTTP Request Defaults’ properties. In JMeter open the ‘HTTP Request Defaults’ config element   Update the Server Name field with this string...
Read more...

Part 4: Configuring our Linux Performance Test Server

We’ve developed our tests using the JMeter GUI on a windows machine. That’s fine and makes our life easier on the development side of things. However, to deliver our distributed and scalable performance capacity we’ll deploy these tests on an AWS Linux instance that’s run up automatically. Normally we’d have an SVN or Git source code repository that we’d check our tests into (all our Selenium, SoapUI and JMeter). During development of the tests we’d check our JMeter tests in to this source code repository from the Windows machine. At excution time we’d have our Linux machine check out the latest version of these tests. For the purpose of this course though we’re just going to take our JMeter configuration...
Read more...

Part 3: Configuring Our Performance Tests

In SoapUI we created a test case that logged in and then passed the userId and authToken to the subsequent test cases (API calls). We need to create a similar setup in JMeter. The main difference this time round being that we’ll have to simulate multiple users and track multiple userID and authToken sessions. We do this in JMeter by configuring Threads, where a single thread represents a single user. First we need to create a new test plan in JMeter   We can leave all the default settings for the test plan as they are, although you can change the name if you need to. Next we can add a new thread group. Right click on the test plan...
Read more...

Part 2: Initial Configuration of JMeter

Initially we just need to setup JMeter and check that we can send a single request and get a response. We’ll configure a Thread Group (users) against the Rocket Chat /api/version end point. Once we can get a response from this end point we’ll move on to configure a more realistic set of load scenarios. Right click on ‘Test Plan’ and select ‘Thread Group’ so that we can define the number of users we need to simulate   Update these settings for the Thread Group: Number of Threads (users): 10 Loop Count: 20 This way we’ll simulate 10 user and repeat our test 20 times.     Right click on the ‘Thread Group’ node and select ‘Add’ followed by ‘Sampler’....
Read more...

Part 1: Install JMeter

We’re going to use our existing Windows Master machine for this. We’ll install JMeter on this machine so that we can develop our performance tests on this Master test machine. We just need to open the RDP session, download JMeter and install it. On the Windows Master machine follow these steps: Open a browser Enter the following address: http://jmeter.apache.org/download_jmeter.cgi Download the Apache JMeter Zip’ package Open the download folder and extract the JMeter folder from the zip file… Open the folder that contains JMeter. Before you can run JMeter we need to configure the PATH environment variable so that it contains the path to our Java install. Configure the environment variable by ‘right clicking on Computer’. Then selecting ‘Properties’ followed...
Read more...

Prerequisites

If you’ve followed Modules 1, 2 and 3 you should already have your Amazon Virtual machine environment up and running along with Jenkins and Selenium. This setup gives us the 3 machines we’ll need to use in this module. Windows Master machine: this is running Jenkins and controlls all our other machines (including the installation of the AUT and the exection of our Selenium tests). This machine will be responsible for kicking off our SoapUI API tests Linux Client machine: this Ubuntu linux machine is run up on demand by Jenkins and then has the AUT (Rocket Chat) autoamtically installed on it. This machine provides the web interface for the Rocket Chat application and the API for the Rocket Chat...
Read more...

Running Our Performance Tests with JMeter

Overview In this Moudule we’re focusing on running our performance and load tests. We’ll create some simple scripts in JMeter and link the execution of these scripts into our build process with Jenkins. Once our Selenium functional tests and our SoapUI REST API tests are complete we’ll kick off these JMeter tests. The setup of this will cover these topics: 1. Install JMeter 2. Configure and Create Performance Tests 3. Add and Configure Jenkins Performance Plugin 4. Deploy and Run all our test jobs   To keep things simple we’ll use JMeter to test the performance using the Rocket Chat Rest Api. This will allow us to focus on the key concept of building the automation framework rather than getting...
Read more...

The SCM Tool We’ve Chosen

As we’ve already mentioned we’ve chosen Git. Git isn’t an “unpleasant or contemptible person” as the dictionary definition points out. Git is version control system that will store all our test artifacts or files. Git maintains a changes to those files over time so that we can revert to previous versions if we need to. All our changes are tracked so that we can see what changes were made, when and by who. Why’s this important? Well take that scenario where your colleague makes a small innocuous change to a Selenium script. Nothing radical but when you come to run the latest version of the this automation script nothing works anymore. Well with Git we can see exactly what the...
Read more...

Conclusion

Well nothing too difficult on the Jenkins side of things this time round. By now you should be experienced enough to configure the Jenkins build job easily to handle the execution of the SoapUI test run. The only new thing we introduced here for Jenkins is the capability to process JUnit test results and display them. A key capability for tracking test results in many other tools controlled by Jenkins too. We delved into a lot of SoapUI features to create a handful of REST based Api test cases. The only real complexity here being the passing of parameters between test cases. Once you’ve seen how to do this it not difficult to repeat the process and build up a...
Read more...

Part 7: Reporting the SoapUI Test Results in Jenkins

What we want to do now is see the individual test case results displayed in Jenkins. SoapUi can save the test results in a JUnit style report format. Then we can get the Jenkins ‘xUnit’ Plugin to display the test results. Lets install the plugin first. 1. Install the xUnit Jenkins Plugin Just go to the Jenkins ‘Manage Plugins’ page, select the available tag and search for “xUnit”. Click install without restart. Once the plugin is installed we can configure our SoapUI project to save the test results in JUnit format. 2. Modify the Jenkins ‘RunApiTests’ Job Back on the Jenkins home page configure the ‘RunApiTests’ job. What we need to do now is configure our SoapUI command line so...
Read more...

Part 6: Running the SoapUI Test Suite from Jenkins

First we’ll need that SoapUI command from the previous step. So you’ll have something like this… cmd.exe /C “C:\Program Files\SmartBear\SoapUI-5.2.1\bin\testrunner.bat” -ehttp://<your-aut-host>:3000 C:\Users\Administrator\REST-Project-1-soapui-project.xml Back on to our Windows Master machine that’s running Jenkins (open the RDP session if neccessary). Then bring up the Jenkins home page. From here we need to create a new job / new item: We’ll make this a ‘Freestyle Project’ and call the job ‘RunApiTests’ We have a few key things to configure on this new job: 1. Restrict where this project can be run: SeleniumTestClient 2. Build Triggers: Build after other projects are built Projects to watch: RunSeleniumTests Trigger only if build is stable Then we need to add 3 new build steps. To create each...
Read more...

Part 5: Running the SoapUI Test Suite from the Command Line

There is an easy way to work out how to run this Test Suite from the command line. You can initiate a command line run from the SoapUI gui and then copy the command line that the GUI gives you. 1. Launch the TestRunner To do this, from the ‘Project’ menu select ‘Launch TestRunner’:     This will give you the test runner dialogue box. One the ‘Basic’ tab all we need to define is the path to the TestRunner. When we run these tests from the command line we’re basically running a windows batch file called ‘testrunner.bat’. You just need to set the path to this file to: TestRunner Path: C:\Program Files\SmartBear\SoapUI-5.2.1\bin The version number in the directory may...
Read more...

Part 4: Converting the REST Requests to Tests

From here then we have 4 REST requests that all give us valid responses. With SoapUI the next step is to convert them to tests and create the assertions that validate the responses. Two ways you can do this: 1. Create individual test cases for each Request NOTE: you can do this one by one (as listed below) or you can jump to section 2 below and convert the whole lot in one go. The choice is yours. i. Right click on the ‘request’ ii. Select ‘Add to TestCase’     iii. Create the Test Suite (just give it a test suite name) iv. Create the Test Case (just give it a test case name) v. Accept the default options...
Read more...

Part 3: Creating Some REST Requests

Now we have access to the Rocket Chat API we can start creating some REST requests. These requests will be written in SoapUI and executed on the Windows Client machine. Eventually we’ll have these tests kick off automatically from Jenkins. For now though lets write some requests in the SoapUI gui and get them working. We need to create a new REST project and … 1. create a new REST project and enter the Rest URI and then enter the URI for your Rocket Chat Rest API and click ok http://<your-amazon-aws-instance >:3000/api/version This should give you something like this… The navigator section here showing you the hierarchy of project, end point, resource, method and finally requet. We’ll look at each...
Read more...

Part 2: Connect to the Rocket Chat API

Part 2: Connect to the Rocket Chat API So Rocket Chat, our AUT, comes installed with a REST Api. This API is documented here: https://github.com/RocketChat/Rocket.Chat/wiki/REST-APIs On your install of Rocket Chat you should be able to connect to the Api directly with a browser here… http://<your-amazon-aws-instance >:3000/api/ Remember earlier we said you could get your host name from the publicHostname.txt file on the Windows Jenkins master machine. So if you replace the <your-amazon-aws-instance> text with your public hostname you should be able to connect to this Api from any remote machine. You’ll see something like this: If you have trouble connecting from outside your Amazon Virtual Private Cloud you’ll need to check your Security Group settings in your AWS control...
Read more...

Part 1: Install SoapUI

We’re going to use our existing Windows client machine for this. So once we have the RDP session open we can download SoapUI and install it on our Windows Server 2012 R2 64 bit machine. Only a few steps to follow here then, all to be carried out on the Windows client machine: 1. Open a browser 2. Enter the following address: https://www.soapui.org/downloads/open-source.html 3. Download the ‘Windows Installer (64 bit)’ package 4. Run the installer, accept the agrements and accept all the default options… Once you’ve finished you should be presented with the SoapUI user interface: Now we have SoapUI running we can check the connection to the Rocket Chat Rest API.
Read more...

The Tools We’ve Chosen

We’re using JMeter mainly because of it’s popularity and it’s wide use within the industry. Mind you that’s no good if it doesn’t support our technical requirements. We need to create REST Api requests, listen for the REST responses and then store the test results. Another reason for using JMeter is that there is a Jenkins ‘Performance’ plugin that supports JMeter. This plugin allows us to process the log files created by Jenkins and create some neat charts. You can never have too many charts! It’s also worth mentioning that JMeter runs (under Java) on both Windows and Linux platforms. We can create tests using the GUI on our Windows Master test machine. We can also run these tests from...
Read more...

Prerequisites

If you’ve followed Modules 1, 2 and 3 you should already have your Amazon Virtual machine environment up and running along with Jenkins and Selenium. This setup gives us the 3 machines we’ll need to use in this module. 1. Windows Master machine: this is running Jenkins and controlls all our other machines (including the installation of the AUT and the exection of our Selenium tests). This machine will be responsible for kicking off our SoapUI API tests 2. Linux Client machine: this Ubuntu linux machine is run up on demand by Jenkins and then has the AUT (Rocket Chat) autoamtically installed on it. This machine provides the web interface for the Rocket Chat application and the API for the...
Read more...

Selenium Browser Automation Conclusion

So we’ve configured Jenkins to start a slave Windows machine that will run our browser tests. We’ve install the software we needed to run our browser tests (e.g. a set of different browsers, Python and Selenium). We’ve created our first Selenium script and checked that this runs against our application under test. With the automated tests in place we’ve then configured Jenkins to run these scripts auotmatically on our Windows client machine. We’ve worked out how to run RDP/RDC terminal sessions on our Master Jenkins machine so that these Selenium tests can run in a real environment (not just some headless mode). In order to do this we’ve gone through some quite complex configuration steps in this module. Most of...
Read more...

Part 12: Running the AUT build followed by the Selenium Tests

We now have the 4 things we need in place to get this up and running fully. We have the Jenkins project that installs and starts the application under test. We have the project that opens the client RDP terminal session on the master machine. And we have the project that starts and runs the Selenium tests on the client. Lastly we have the project that closes the client RDP terminal session. All we need to do now is string them together. First we’ll link the RDP session opening (RunClientRDPSession) to the Selenium test execution (RunSeleniumTests) and the RDP close session (CloseClientRDPSession). Then we’ll link in the application under test install at the start of that (BuildRocketChatOnNode). 1. Modidfy the...
Read more...

Part 11: Closing the RDP/RDC Session on our Master Jenkins Machine

Bit simpler this bit. Once we’ve opened the RDP session and then run our Selenium tests we’ll want to close the RDP session. We’ll need to close it because if we run the process again we don’t want to end up with 2 RDP sessions open. 1. So at this stage you should be able to create a new project by clicking ‘New Item’ on the Jenkins home page. 2. Then use the Item name: Item Name: CloseClientRDPSession 3.We don’t need to restrict where this project can be run as we want it to run on the master machine that has the RDP terminal session open. So all we need to set is the batch command to run. 4. Then...
Read more...

Part 10: Running an RDP/RDC Session on our Master Jenkins Machine

The next challenge (and it’s kind of a big one) is to get our master Jenkins machine to open up an RDP/RDC session to our client windows machine that runs our Selenium scripts. We are going to run GUI tests on a jenkins windows slave with a remote desktop connection running on the Jenkins master machine. We want this because the Selenium scripts are designed to run on a terminal with the browsers running. We could run some browser tests in a headless mode but this comes with it’s own set of issues and isn’t really a close enough representation of what the end user will be doing (in my opinion). So we need to get Jenkins to create an...
Read more...

Part 9: Writing Some Selenium Tests for Rocket Chat

Now we’re not going to get into how to develop good Selenium scripts in this module. Plenty of good resources on the web that already cover that. What we’ll do is write a login/logout script and run that for each browser type. This will give you enough to expand on as you develop your own environment. We’re going to take an example directly from the Selenium Python Bindings documentation (found below) and modify this for our purposes: http://selenium-python.readthedocs.org/getting-started.html 1. Open notepad with the sel.py script and replace the contents with the following script (making sure you get the indentation correct): from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions...
Read more...

Part 8: Checking we can Access Rocket Chat from our Windows Client

What we need to do now is run up our application under test, Rocket Chat. Now we already have that scripted in Jenkins and designed to run on a dedicated AWS Linux instance. So lets start out by running this up. 1. Kick off the ‘BuildRocketChatOnNode’ job in Jenkins So click on the ‘build now’ icon for the Rocket Chat job. This might take a few minutes to complete but you should see this in Jenkins: 2. Make sure your Windows Selenium test client/slave is running If you check on your Jenkins dashboard and you see the Windows Client as offline like this… Then you can bring it back online automatically by running this Jenkins job too So we’ve started...
Read more...

Part 7: Running the Selenium Script from the Jenkins Master Machine

Lets setup the Jenkins master machine to kick off this selenium script on the Slave machine. The Selenium scirpt is configured to fail at the moment so we should see this fail the job first time round. Then we’ll update the Selenium script to fix it and see the job pass. That’ll just confirm for us that we’ve configured all of this correctly. 1. Create a new Job/build task in Jenkins ** We’re back on the Master Jenkins Windows 2008 machine now ** 2. Give the new job an item name and select Freestyle project Item Name: RunSeleniumTests Freestyle project: <selected> 3. Configure the job with the following settings: Restrict where this project can be run: <checked> Label Expression: SeleniumTestClient...
Read more...

Part 6: Create Our First Selenium Script

So at this point we have everything we need on this Windows 2012 slave machine. We just need to create a few Selenium tests and run them locally. Once we have them running locally we can start triggering their execution from the master Jenkins machine. To begin with we’ll check Python and Selenium can drive all our browsers. We’ll great a short script to open each browser in turn. 1. Open notepad and enter the following script from selenium import webdriver driver = webdriver.Firefox() driver.get("http://www.google.com") driver.quit() driver = webdriver.Ie() driver.get("http://www.google.com") driver.quit() driver = webdriver.Chrome() driver.get("http://www.google.com") driver.quit() Save this script to the desktop with the file name ’sel.py’ 2. Open a command prompt and run this script C:\Users\Administrator\Desktop\sel.py Which should be...
Read more...

Part 5: Installing the Software we need on the Windows Slave

We have the slave Jenkins machine running and under the control of our main Jenkins machine. Now we can deploy some of the software we need to this slave machine so that it can run our Selenium tests. Software packages we’ll need will include: Chrome Firefox Python 3.5 1. Open an RDP session on the Windows 2012 slave machine i. login to your AWS management console ii. go to your AWS EC2 dashboard iii. select ‘Instances’ in the left hand menu iv. click on the entry for the new Windows Server 2012 instance v. right click on this instance and select ‘Connect’ vi. click ‘Download Remote Desktop File’ 2. On your Windows 2012 Server client machine download and install the...
Read more...

Part 4: What to do when you Can’t Connect the Windows Slave

The connection between the Jenkins master machine and the Jenkins windows slave is established using Windows Remote Management (WinRm). This is installed and running by default when we create our Windows slave instance using the Windows Server 2012 R2 Base AMI. However, sometimes you may find you run into problems connecting the two machines using WinRm. To see if the connection issue is down to WinRm configuration you can try this on the Jenkins Windows Master machine. In your AWS account find out what the ‘Internal IP address’ is of the “Client/Slave Windows Machine”. You’ll also need the Administrator password for this machine too. On the Jenkins master machine open a command terminal Run this command in the terminal window...
Read more...

Part 3: Completing the Windows Client Configuration

With the windows client machine started we still need login to it manually with an RDP session, setup some additional configuration manually and then update the Jenkins configuration. Once we’ve done this Jenkins will have complete control over this Windows client machine. 1. Connect Using RDP to the Windows Client machine Going back to our AWS management console we need to start an RDP session on our Windows Client machine. Follow these steps to open the RDP connection: i. login to your AWS management console ii. go to your AWS EC2 dashboard iii. select ‘Instances’ in the left hand menu iv. click on the entry for the new Windows Server 2012 instance v. right click on this instance and select...
Read more...

Part 2: Configuring Jenkins to Start A Windows Slave Machine

We’ve already configured Jenkins to run up one slave machine. This slave machine runs our Ubuntu Linux environment that hosts our Application Under Test (AUT) – Rocket Chat. Now we need to add to this environment and create a connection to the Windows slave machine that we’ll run our Selenium browser testing from. First step is to configure this machine in Jenkins and make sure we can connect to it. We can complete all of this from the Manage Jenkins / Configuration page in the Jenkins Gui. 1. Open up Internet Exporer and type in this URL http://localhost:8080/ 2. In the Jenkins home page click on Manage Jenkins -> Configure System 3. At the bottom of the configuration page we...
Read more...

Part 1: Setup Our Windows Test Machine

Create a Windows 2012 Server Instance in AWSFour things we need to create this Windows instance: Microsoft Windows Server 2012 R2 Base – ami-83a5bce2 t2.micro Name: Windows-Client Security: default Back to our EC2 dashboard then: i. login to your AWS management console Step 1: go to your AWS EC2 dashboard Step 2: select the correct region Step 3: click ‘instances’ in the side menu Step 4: select the AMI listed above (Microsoft Windows Server 2012 R2 Base ) Step 5: select the Type listed above (t2.micro) Step 6: accept all the ‘Instance Config’ defaults Step 7: accept all the ‘Storage’ defaults Step 8: [optional] add a tag for the Name if you like (e.g. Windows-Client) Step 9: for the security...
Read more...

Prerequisites Selenium Browser Automation

Right, we need to make sure we have everything setup correctly before we start. First we need to be sure we’ve launched our Windows master machine and that we have an RDP session open. Make Sure your Windows Master EC2 Instance is Running If you don’t have your Windows master machine up and running yet then you’ll need to revisit Module 1 and take a look at the section on Running up the Windows Instance. With the Windows 2008 master machine configured then we’ll need make sure the instance is running and that we have an RDP session open. You can complete these steps with 1. Open your AWS management console, and view your list of Instances (making sure you...
Read more...

The Tools We’ve Chosen

We’re testing a browser based application so it seems sensible to choose Selenium as the tool to write our automated test cases in. We can get Selenium drivers for a range of different browsers so we’ll be able to provide test coverage against IE, FireFox and Chrome (we could do more but this’ll be adequate for now). It’s important to point out though that this isn’t a Selenium course. So the Selenium scripts we’re using here are simple, linear and, well basic. This gives the added advantage that it makes things easier to understand. You just need to be aware that in order to scale the actual test cases you’ll probably want to look at improving the way these test...
Read more...

Jenkins Conclusion

Building on our AWS environment we’ve configured Jenkins so that we can control a slave machine (an Ubuntu Linux machine) and build and install the application under test. We’ve also factored in a few small checks to make sure everything is up and running. The key take away here is that Jenkins is not just for building software. It’s not even just for implementing continuous integration. Both of these are great goals. However, on it’s own it’s just a great tool for testers. It allows us to automate a lot of our repetitive tasks like running up test environments and installing the software we need to test. Then as we add to this the capability to kick of automated tests...
Read more...

Part 12: Starting Subsequent Jenkins Builds

Now at this point, do you remember this setting in your Amazon Ec2 configuration… Idle termination time: 30 Well your Linux slave machine has probably been idle for 30 minutes now. As a result it’s probably shut itself down. Then again maybe you went through this really quickly and it’s still running. Either way, depending on the state of this slave machine Jenkins will do one of two things when we start a new build: a. If it has shut itself down: When we kick off a new build it will start from the beginning and create a new EC2 instance. NOTE: if the node has been shutdown and then a new node run up for a subsequent build then...
Read more...

Part 11: Jenkins Post Build Actions and Smoke Tests

Once we have a setup that will build and install our application it’s worth configuring Jenkins to run a few checks and tests just to confirm that things completed without any issues. We’ll use the Post-Build Actions feature to carry out two actions: a) check the logs for successful install messages b) use ‘wget’ to check that the Rocket Chat home page is displayed We can set these up by following these steps: 1. go back to the Job configuration by selecting configure on the home page.   2. Scroll to the bottom of the configuration page to find the Post Build Actions section 3. Click the ‘Add post-build action’ button and select the ‘Post Build Task’ option Check the...
Read more...

Part 10: Setting Jenkins up to install the AUT

So the last bit to configure is the download, build and install of the application under test (AUT) on our slave machine. The AUT we’re going to use is a chat application called Rocket Chat. https://rocket.chat Now it’s worth mentioning here that this is just an example app we’re using for this course. We’ll go through the steps to build and install this app but you DON’T need to understand the scripts needed to build and install Rocket Chat. These are specific to Rocket Chat. In practice, when you’re testing your own applications, you’ll need to replace the Rocket Chat build and install scripts with your own scripts. This will mean talking to your developers or build engineers. Then adding...
Read more...

Part 9: Jenkins Slave Machine

At this stage we can use Jenkins to start up a slave test machine automatically. There are a few points worth mentioning about how we can control this slave machine now. Once the slave is running Jenkins runs a ‘Slave Agent’ on this machine. It’s this agent that allows the slave to talk to the master Windows Jenkins machine. It’s this slave that gives Jenkins full control of jobs and actions that can be carried out on the slave. Slave Idle Termination Time For example in our configuration for the Amazon EC2 slave (Manage Jenkins -> configuration) we had a setting called ‘Idle termination time’   This tells Jenkins that if the slave is inactive for more than 30 minutes...
Read more...

Part 8:Instance Initialisation Script

 13. Set Up the Instance Initialisation Script The last thing we need to setup during the run up of this instance is the installation of a few additional software packages. These packages aren’t installed on the AMI we’ve started out with. To get round this Jenkins has a field in the EC2 configuration called “Init Script” To update this we’ll need to: return ot the main Jenkins dashboard (click the ‘Jenkins’ link top left) click the ‘Manage Jenkins’ link click the ‘Configure System’ link scroll to the bottom of the page where we configured our Amazon EC2 setup the last section on the page should be the ‘Init script’ field So we’ll need to add some Unix shell commands...
Read more...

Amazon Web Services Conclusion

So we’ve started with a new AWS account. Then we went through the sign up process to create a new account. And with this new account we've made a good start on building out our test automation framework with the core Windows and Linux machines that we need. We’ve learnt about the AWS fundamentals covering Virtual Private Clouds, Elastic Cloud Computing and storage. From here we created our first Windows and Unix instances from Amazon Machine Images. To finish off we set up our servers so that we have RDP and SSH access. In short we started out with nothing and now we have our own cloud environment with virtual machines and storage. In the next module we’ll be installing...
Read more...

Part 11: How to Check Your AWS Spend

Within the AWS management console you’ll find a ‘Billing and Cost Management’ option: In here you’ll see what your current balance is (should remain at $0.00 if you stay within your free tier constraints). The most important section though is the ’Top Free Tier Services by Usage’ section. Monitor the stats here to see how close you are to going over your free tier allocation. If you’ve started with a clean AWS account for this course you shouldn’t go over your free tier allocation. IT IS YOUR RESPONSIBILITY TO MONITOR THIS. CHARGES INCURED ARE YOUR CHARGES TO PAY. If you want to be absolutely sure that you don’t go over the free tier allocation then I would recommend reading this…....
Read more...

Part 10: The Difference Between AWS Terminate and AWS Stop

You’ll notice in the management console that you have 2 options for bringing your servers down (both for Windows and Unix). Stop: when you stop an instance the instance is shutdown. If it has EBS storage (like our Windows server does) the data on this storage is maintained. If you have SSD storage (like our Unix server does) the data on this storage is NOT maintained. When an instance is shutdown you can restart the instance when you need it again. Things like instance ID, EBS storage, private DNS and private IPs are maintained and restored. Things like Public DNS and Public IPs may change (they will on our setup). Note that when an instance is in a ‘Stopped’ state...
Read more...

Part 9: Connecting to the Unix Client Machine

April 2, 2019
Three very straight forward steps to getting connected to our Unix machine. Very important we set it up correctly though. First, because it makes our life easier (quickly creating a terminal session on the unix machine) and secondly because later our Jenkins setup will connect automatically from this Windows machine to the Unix machine using SSH. SSH is a key component in our process of automating everything. Step 1. Convert our .pem key So the .pem private key Amazon gave us for connecting to our Windows and Unix machines isn’t supported by default by Putty. But it’s easy to convert it to the right format. With the FirstKeyPair.pem file (or whatever you named the .pem file you downloaded) follow these...
Read more...

Part 8: Installing Putty and SSH on the Windows Machine

I'm not sure if you’ve come across Putty yet but this is a great little (actually not that little if you consider how much has gone into this suite of tools) application for connecting via a secure shell from a windows machine to a unix machine. If you set it up correctly you can open a Shell session (command prompt) on the Unix machine at a click of a button without even entering a password. Here’s how: Open Internet Explorer within the RDP session on your Windows server Either search for Putty or enter this URLhttp://www.chiark.greenend.org.uk/~sgtatham/putty/download.html (MAKE SURE YOU GO TO THE ‘chiark.greened.org.uk’ SITE) Download the Putty Installer: putty-0.67-installer.msi (0.67 or later) Run the installer selecting all the defaults At...
Read more...

Part 7: Connecting to the Windows Master Machine

Before we can start building anything in our Amazon Virtual Private Cloud we need to connect to the Windows Master machine. That means decrypting our password and connecting with an RDP session. Pretty straight forward to connect to the Windows machine. Just follow these steps: In the Amazon EC2 Management console right click on the 'Windows' machine in the list and select ‘Get Windows Password’ Select the path to the ‘Private Key’ you saved earlier Click the ‘Decrypt Password’ button Write down the Windows Admin password and the Public DNS host name IMPORTANT: If you don’t get a public DNS host name you’ll have to complete these steps and change this setting: Click ‘Services’ (top menu bar) Select ‘VPC’ (in...
Read more...

Part 6: Running up the Ubuntu Unix Instance (VM)

With the Windows Instance running up we're on to creating our Linux instance. Pretty similar process, just need a few different parameters. There’s four things we need to create our Unix Instance AMI – Ubuntu Server 14.04 LTS (HVM), SSD Volume Type – ami-9abea4fb* Instance Type – t2.micro Security Group – Unix-AUT (we created this earlier) Private Key – .pem file (from our key pair created earlier) * – again you need to be aware that Amazon will change the AMI id from time to time. If you can’t find this ami id then look for an AMI in the free tier for Ubuntu Server 14.04 LTS (HVM), SSD Volume Type. Next then, go back to the EC2 console/home page...
Read more...

Part 5: Running up the Windows Instance (VM)

At this point we're ready to start running up our virtual machines (or rather 'Instances' in AWS terms). First though we need to know what type of machine we need to run up. There’s four things we need to create our Windows Instance: AMI – Microsoft Windows Server 2008 R2 Base – ami-7d15fa1d * Instance Type – t2.micro Security Group – Windows-Master (we created this earlier) Private Key – .pem file (from our key pair created earlier) On the EC2 console/home page click “EC2 Dashboard”. You should see a resource list like this…. From here you click the ‘Launch Instance’ button. You should be able to follow the steps using the ‘four’ items of info listed above to create your...
Read more...

Part 7: Starting the Linux Amazon Instance Slave

5. On the ‘AMI’s line click the ‘Add’ button 6. Enter the following details for your AMI instance Description: RocketChat-Server AMI ID:<see below> So the AMI id is the Amazon Machine Image number that correlates with our Linux Ubuntu machine that we want to run up. So it’s the same Id as the Linux Ubuntu machine instance that we already have configured in our AWS account. We find this ID with … i. login to your AWS management console ii. go to your AWS EC2 dashboard iii. select the correct region iv. click ‘instances’ in the side menu v. in the list of instances select your Unix-client instance vi. in the instance details panel (bottom of page) look for this:...
Read more...

Part 6: Amazon AWS Integration

 Back on the Jenkins home page you’ll need to: 1. click on the ‘Manage Jenkins’ link again     2. click the ‘Configure System’ option   There are a lot of options for configuring Jenkins. All we need to concern ourselves with is the ‘Cloud’ section (at the bottom of the page). The Cloud section is where we define our Amazon EC2 instance start up details. 3. Scroll to the bottom of the Jenkins ‘Configure’ page where you should find the ‘Cloud’ section. Then click ‘Add a new cloud’ and select the ‘Amazon EC2’ option   From here we can enter all of the details needed to run up our EC2 Ubuntu instance. Before we can complete this though...
Read more...

Part 5: Configuring Jenkins

Now for the interesting part. We need to configure Jenkins so that the tasks we need automating are setup to be run by Jenkins. There are four key parts to this configuration. Firstly we need to set Jenkins up so that it is integrated with our Amazon AWS service. The ‘Amazon EC2 Plugin’ we’ve installed needs to have the ‘Amazon Access Key Id’ installed so that it has permissions to drive AWS. Secondly, we need to configure Jenkins so that it will start our Linux Ubuntu machine on demand. When Jenkins starts that machine it needs to specify the Amazon AMI to use and install other software packages that we’ll need for our build/instll. Thirdly, we’ll also need to setup...
Read more...

Part 4: Installing Plugins

 Let’s install the plugins we need for this project. We’re going to need these: Amazon EC2 Plugin https://wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin With this plugin we can use our Jenkins machine to run up an Amazon EC2 instance automatically. We’ll be running up a Linux Ubuntu instance that will then run a Jenkins client application. This Jenkins client application will then be responsible for downloading the AUT source code, building the AUT and running the AUT. SSH Slaves Plugin https://wiki.jenkins-ci.org/display/JENKINS/SSH+Slaves+plugin If you remember from our first Module we connected to our Linux Ubuntu client machine using SSH. This plugin gives Jenkins the capability to use our SSH private key to connect to the Amazon EC2 client machine. Once Jenkins has a connection to...
Read more...

Part 3: Jenkins’ Plugins

A fundamental concept within Jenkins is it’s plug-in module capability. There’s a plug in module to help you automate just about any task you need to automate. A full list of the modules can be found here.. https://wiki.jenkins-ci.org/display/JENKINS/Plugins These plugins are broken into a number of different types that deliver different types of capabilities: Source Code Management: Whilst Jenkins supports source code tools like Subversion and CVS out of the box there are other source code tools you might want to work with. Source Code Management plugins support all sorts of other source code tools. We’ll be working with Git and GitHub so we’ll need to add a plugin for GitHub in a minute.   Build Triggers: When do you...
Read more...

Part 2: Jenkins Configuration

We need to achieve three things with our configuration of Jenkins 1. Start a slave machine: we’ll get Jenkins to automatically start our Linux Ubuntu machine when we’re ready to build and install the application under test on this Linux machine 2. Download, Build and Install the application under test: once the client Linux machine is running we’ll download the source code for the AUT, build the AUT and install it on our Linux client machine. 3. Run some Smoke Tests Against the AUT: once the AUT is installed and running we’ll run a couple of very simple tests to make sure that it is up and running. We look at the main configuration areas of Jenkins and see how...
Read more...

Why use Jenkins

On the Jenkins web site they don’t talk about Continuous Integration or Dev Ops or use any other popular buzz words. They simply refer to Jenkins as the “Open Source Automation Server”. And that’s the key. Jenkins isn’t just an app for developers. It’s an automation tool for testers too. Don’t think of Jenkins as an app for developers or build engineers. Don’t think of it as an app for continuous integration or for building software. Yes it does all of that. But there's far more you can do with Jenkins. Think of Jenkins as a "process automation tool". It’s just the best app for automating a load of the daily or weekly tasks that your test team have to...
Read more...

Part 4: Creating a Security Key Pair

Now we're on to creating a security key pair that will allow us to (1) decrypt our Windows password and (2) setup our SSH (secure shell) between the Windows AWS instance and the Linux AWS instance. Again, make sure you have the right region selected (select this from the management console top right). Key pairs are create FOR each region and can't be used across regions. The private key you obtain from this process you'll need to keep safe. We'll need it later in the process. It'll be needed to decrypt the windows password and to login to the Unix machine using SSH/Putty. To create your key pair follow these steps: 1. on the EC2 Dashboard page click on "Key...
Read more...

Part 3: Configuring Security Groups

At this point we need to configure our AWS security groups so that we restricted unfettered access and allow access between our different machines. And of course we need to provide for our own access to our AWS machines too. First make sure you select the right region (select this from the management console top right). Security Groups are created FOR each region and can't be used across regions. Once you've selected the region we need to create 2 security groups. The first will be used for our Windows master machine. The second security group for our Unix machine that is running Rocket Chat (the application under test). You can configure these two security groups by following these steps.... 1....
Read more...

Jenkins Prerequisites

A few things we need to make sure we have setup correctly before we start. Make Sure your Windows Master EC2 Instance is Running First, if you don’t have your Windows master machine up and running yet then you’ll need to revisit Module 1 and take a look at the section on Running up the Windows Instance. With the Windows 2008 master machine configured then we’ll need make sure the instance is running and that we have an RDP session open. You can complete these steps with 1. Open your AWS management console, and view your list of Instances (making sure you have the correct Region selected). If you're Windows Master machine is in a 'stopped' state then start it....
Read more...

Part 1: Installing Jenkins

At this point we’re ready to visit the Jenkins-ci.org web site and download the Windows install package. Follow these steps to complete the install on your Windows 2008 server master machine.   1. Open up Internet Explorer and type in this URL http://jenkins-ci.org 2. On the home page for Jenkins you’ll see a download link or download button. Download the LTS release for Windows. So click this link.   3. Open the download folder and drill down into the Jenkins zip file. Extract the file in the zip file by dragging it to the desktop     4. Run the installer accepting all the default install options. Nothing very exciting seems to happen on completion of the install. Except that...
Read more...

An Introduction to Jenkins

We have the test environment in place and we know enough to be dangerous with Amazon Web Services. Next step is probably the most important. We need to get Jenkins installed, configured and running. Jenkins has to be one the most useful tools for testers. It’s not just handy for kicking off automated tests it’s indispensable for automating all sorts of processes you have to complete day in day out. We’re going to cover 4 main areas here… 1. Jenkins Installation 2. Installing Jenkins Plugins 2. Jenkins Configuration 3. Deploying and Building the Application Under Test By the time we’ve completed this module we will have Jenkins running on our Windows Master machine. The Jenkins configuration will be completed so...
Read more...

7. Tracking Configurations and Releases

It’s well known that keeping track of results from testcases against builds, configurations, environments, etc can become quite difficult to manage. It’s one of the reasons we usually implement test management tools. These tools enable us to log our results against these different aspects of our testing and then quickly produce traceability reports to show our coverage. It’s just that the permutations and combinations can become quite difficult to track, even with the right tools. If we have just 1 testcase, which we need to run against 2 platforms, 2 operating systems along with three configurations relevant to our application we already have 12 permutations. That’s 12 permutations for just 1 testcase. And with this example we’re not even considering...
Read more...

6. Managing Retests

Re-running tests to check that bugs are fixed is an essential part of the test management process. Teams grow, the amount of data generated by completed testcases increases and the number of raised defects expands too. As a result it becomes more difficult to keep track of the retesting. There are several different ways to approach this. So at some point in the test management process you’ll need to re-test fixes that have been made to resolve issues. This re-test process needs to be approached from two angles. Firstly, identifying failed test cases from previous test runs and then re-running those test cases in a subsequent cycle. Secondly, identifying defects that have been resolved and running tests to confirm that...
Read more...

5. Test Result Aggregation

No test team can run every single test case on every single release of a product. So being able to aggregate results from different views or areas in your test management tool is essential when it comes to seeing the whole picture. However, how your test management tool deals with that aggregation might not be quite so simple. This partly depends on how the QA team have set things up and partly on how well the tool you use deals with this. In many cases we’ll need our test management tool to aggregate and report on results from various perspectives. For example we may need to aggregate across different projects, application modules, runs or cycles. Whilst this sounds simple in...
Read more...

4. Managing a Library of Test Cases

Your ability to manage a library of testcases in your test management tool depends a lot on the relationship between the different instances of the testcases. The instances that resides in the library and the instances that are created to be executed and run. At a basic level you you will have two instances of a testcase.. 1. The instance that resides in the library. This is the master copy of the testcase and will not usually have a result record associated with it. 2. The instances that have been used at run/execution time. These instances will all have a result (pass, fail, not yet run, etc) value against them. They may differ from the instance that resides in the...
Read more...

3. Using Test Parameters

The ability to add parameters to testcases enables you to write testcases once and then run different instances multiple times with different parameters. It’s a key capability in test management that saves time, enables you scale up your testing more efficiently and helps avoid mistakes when writing repetitive testcases. Scaling up your test cases effectively usually involves the practice of parametrization. That is, the concept of having one testcase and then feeding in parameter values to create variations of the test case at run time. So essentially you have the same testcase which you execute repeatedly with different data values. Defining parameters in Quality Center is done at the step level where each step is fed parameter values from a...
Read more...

2. Version Control of Test Cases

It’s important to understand if and how version control of your QA artifacts is relevant to your test management process. If you work in a highly regulated environment then it likely that version control is very important. If you work in an environment where documentation and traceability aren’t so important then this may not be so relevant. Either way it’s useful to trace the history and modifications to your test artifacts. As most tools will do this for you in the background it’s usually worth starting off with tracking the history of changes at first. To understand the different approaches to version control read this previous blog post on the Three Approaches to Test Case Management Version Control. So at...
Read more...

The Seven Complexities

Below we set out and introduce each of the seven complexities that we'll examine throughout this course. 1. Assignment: Assignment of Test Cases in Test Management Tools At some point in the process we need to assign a test to a tester. The question is are you looking to assign it at the step, case or set level? Do you need to allocate different tests within a set to different qa engineers? How do you track the difference between assignment for writing and executing the testcase? All of these points complicate a seemingly simple concept. When you look at the different ways different tools support this capability you’ll find it’s not quite as straightforward as you might first think. 2....
Read more...

1. Assignment of Test Cases

At some point in the process we need to assign a test to a tester. The question is are you looking to assign it at the step, case or set level? Do you need to allocate different tests within a set to different qa engineers? How do you track the difference between assignment for writing and executing the testcase? All of these points complicate a seemingly simple concept. When you look at the different ways different tools support this capability you’ll find it’s not quite as straightforward as you might first think. All test management tools have the concept of assigning testcases. Some provide more capabilities than others. Certain tools provide just about every capability you can imagine but do...
Read more...

Part 2: AWS Fundamentals

These fundamentals are all you’ll need to complete all of the modules in this course. It’s all you’ll need to be productive with your testing on AWS. Trust me, despite all the different services and options, it’s easy to get started with AWS. You just need to grasp the following: Services: Amazon provide a range of different services. A service can be thought of as the type of work a particular cloud resource provides. These services are grouped into categories like Compute, Storage, Database, Networking and others. As an example, in the ‘Compute’ category, we have the EC2 (Elastic Compute Cloud) service along with other services like VPC (Virtual Private Cloud). In the ‘Storage’ category we have the EBS (Elastic...
Read more...