We have the test environment in place (if you don’t check here) and we know enough to be dangerous with Amazon Web Services. Next step is probably the most important. We need to get Jenkins installed, configured and running. Jenkins has to be one the testers most useful tools. It’s not just handy for kicking off automated tests it’s indispensable for automating all sorts of processes you have to complete day in day out.
We’re going to cover 4 main areas here…
1. Jenkins Installation
2. Installing Jenkins Plugins
2. Jenkins Configuration
3. Deploying and Building the Application Under Test
By the time we’ve completed the module we will have Jenkins running on our Windows Master machine. The Jenkins configuration will be completed so that it will carry out six main actions…
i. Run up a new/clean EC2 Ubuntu instance (client)
ii. Install the Jenkins client on the Ubuntu client
iii. Download the AUT from GitHub to the Ubuntu client
iv. Build the AUT on the Ubuntu client machine
v. Install and start the AUT running on the client machine
vi. Run some simple post build checks
What You’ll Learn
In this module we’ll go through the 12 parts listed below looking at the practical aspects of setting up Jenkins. By the ended of the module you will understand the following concepts:
- Part 1: Installing Jenkins
- Part 2: The Main Jenkins Configuration Areas
- Part 3: Jenkins’ Plugins
- Part 4: Installing Plugins
- Part 5: Configuring Jenkins
- Part 6: Configuring Jenkins – Amazon AWS Integration
- Part 7: Configuring Jenkins – Starting the Linux Amazon Instance Slave
- Part 8: Configuring Jenkins – Instance Initialisation Script
- Part 9: A Few Points about our Jenkins Slave Machine
- Part 10: Setting Jenkins up to install the AUT
- Part 11: Jenkins Post Build Actions and Smoke Tests
- Part 12: Starting Subsequent Jenkins Builds
When we’re finished we’ll have this setup:
Note that we’re building on top of the AWS infrastructure we built in the first module by adding Jenkins and a bit of configuration. Whilst we created a client Ubuntu machine in in the first module we’re not actually going to be using that machine. What will happen is that Jenkins will fire up a completely new instance of the Ubuntu machine automatically whenever it’s needed. This is an ideal use case for bringing up Amazon Instances as and when we need them. More on how we do this in a minute.
Why Use Jenkins
On the Jenkins web site they don’t talk about Continuous Integration or Dev Ops or use any other popular buzz words. They simply refer to Jenkins as the “Open Source Automation Server”. And that’s the key. Jenkins isn’t just an app for developers. It’s an automation tool for testers too.
I personally love Jenkins. I don’t think of it as an app for developers or build engineers. I don’t think of it as an app for continuous integration or for building software. Yes it does all of that. But! It’s just the best app for automating a load of the daily or weekly tasks that the test team have to perform. Whether that’s installing the application you want to test or setting up your test environment. It’s just the best way to save yourself loads of time and effort on repetitive tasks.
A few things we need to make sure we have setup correctly before we start.
Make Sure your Windows Master EC2 Instance is Running
First, if you don’t have your Windows master machine up and running yet then you’ll need to revisit Module 1 and take a look at the section on Running up the Windows Instance.
With the Windows 2008 master machine configured then we’ll need make sure the instance is running and that we have an RDP session open. You can complete these steps with
1. Open your AWS management console, and view your list of Instances (making sure you have the correct Region selected).
2. Once started open an RDP session on this Windows machine.
Then enter the password to open up the desktop session (you may need your .pem private key file to decrypt the password if you’ve forgotten it) .
Create and Save Your Amazon ‘Access Key Id’
Second we need to get our Amazon ‘Access Key ID’. We’ll need this later as we’re configuring Jenkins to start our client/slave machines. For other systems or programs to connect to your AWS account they need an access key (like a password but for an application not a person). Jenkins needs this access key if it’s going to be allowed to start EC2 instances.
To get this Id follow these steps:
1. Login to your AWS account https://aws.amazon.com
2. Click on your user account drop down and select ‘Security Credentials’
3. Expand the ‘Access Keys’ section and click on ‘Create New Access Key’
4. You can either download or show the access key. Either way you need to record these two parts of the key
Access Key Id
Secret Access Key
NOTE: once you’ve downloaded or viewed the secret access key you CAN NOT obtain it again. If you lose it you’ll have to create another access key (no big deal but it’s easier if you just look after it now).
Make Sure you have your Private key (.pem file)
Third, remember back in Module 1 we created our public and private key pairs? Well at that stage you should have saved your private key pair .pem file (e.g. FirstKeyPair.pem) file. You’ll need this private key when configuring Jenkins later.
If you don’t have have this private key you can go back and create a new key pair. Much easier if you can find the one you created in Module 1 though.
Terminate any Running Unix Client Instances
Lastly, we need to make sure we don’t have any Linux Ubuntu clients running. When Jenkins creates and starts new instances of the Linux clients it checks to see that no existing instances are running OR in a stopped state. So we need to TERMINATE any running instances. To do this follow these steps:
i. login to your AWS management console
ii. go to your AWS EC2 dashboard
iii. select the correct region
iv. click ‘instances’ in the side menu
v. in the list of instances select your Unix-client instance
vi. right click the Unix-client entry, select ‘Instance State’ followed by ‘Terminate’
vii. in the list of instances you should now see this:
With our Amazon Access ID, Key pair private key, our Linux machine terminated, our Windows master machine running and a desktop session available we’re ready to start installing and configuring Jenkins.
Part 1: Installing Jenkins
At this point we’re ready to visit the Jenkins-ci.org web site and download the Windows install package. Follow these steps to complete the install on your Windows 2008 server master machine.
1. Open up Internet Explorer and type in this URL http://jenkins-ci.org
2. On the home page for Jenkins, right hand side, you’ll see the Download Native packages for Windows section. Click this link.
3. Open the download folder and drill down into the Jenkins zip file. Extract the file in the zip file by dragging it to the desktop
4. Run the installer accepting all the default install options. Nothing very exciting seems to happen on completion of the install. Except that after a while a new tab should open in Internet Explorer with the following:
If this page doesn’t open automatically you can type in this Url:
This is the default location and port that Jenkins runs on. This’ll be just fine for what we need.
And that’s it. That simple. Jenkins installed and ready to go. Next step configuration and creating your first job.
A Quick Overview of Jenkins
Before we start configuring Jenkins lets take a look at some key features and concepts. As we’ve already mentioned Jenkins is an automation server that gives us the capability to automate many process within our development and test projects.
Part 2: The Main Jenkins Configuration Areas
We need to achieve three things with our configuration of Jenkins
- Start a slave machine: we’ll get Jenkins to automatically start our Linux Ubuntu machine when we’re ready to build and install the application under test on this Linux machine
- Download, Build and Install the application under test: once the client Linux machine is running we’ll download the source code for the AUT, build the AUT and install it on our Linux client machine.
- Run some Smoke Tests Against the AUT: once the AUT is installed and running we’ll run a couple of very simple tests to make sure that it is up and running.
Part 3: Jenkins’ Plugins
A fundamental concept within Jenkins is it’s plug-in module capability. There’s a plug in module to help you automate just about any task you need to automate. A full list of the modules can be found here..
These plugins are broken into a number of different types that deliver different types of capabilities:
Source Code Management: Whilst Jenkins supports source code tools like Subversion and CVS out of the box there are other source code tools you might want to work with. Source Code Management plugins support all sorts of other source code tools. We’ll be working with Git and GitHub so we’ll need to add a plugin for GitHub in a minute.
Build Triggers: When do you want, or need, to build and test the AUT? Well plugins for Build Triggers allow you to monitor external activities and trigger a build of your AUT when certain conditions are met. For example a developer may check in some code and you’ll want to trigger a build off that check in.
Cluster Management and Distributed build: Typically with lots of automation activities being carried out, and many builds to process, you’ll distribute these activities to other client machines. You won’t run everything on your main Jenkins server. Distributed build plugins allow you to run up, communicate and control other machines. We’ll be using an Amazon EC2 Plugin that runs up and controls our Amazon EC2 Linux Ubuntu slave machine.
Slave Launchers and controllers: Depending on the type of slave machines you run up you’ll want to find different ways of communicating with those slaves. You’ve already seen how we’ll be communicating between our Windows master machine and Ubuntu client machine with SSH. So we’ll be installing the SSH Slaves plugin so that our Windows master Jenkins machine can communicate and control the Linux Ubuntu slave machine over SSH.
Other Post Build Actions: Once the build is complete you may want to complete other actions like running Windows Power shell scripts or Unix shell scripts. Things like this can be setup with ‘Post Build Action’ plugins. We’ll be using Hudson Post build task’ plugin to run some install completion scripts.
There are many other categories of plugins, including Build notifiers, Artifact up-loaders and build reporting. There are hundreds of plugins so chances are any automation task you need to complete will be covered either by features native to Jenkins or a plugin. If it’s not you can always write your own plugin too (but that’s a topic for another day).
Part 4: Installing Plugins
Let’s install the plugins we need for this project. We’re going to need these:
Amazon EC2 Plugin
With this plugin we can use our Jenkins machine to run up an Amazon EC2 instance automatically. We’ll be running up a Linux Ubuntu instance that will then run a Jenkins client application. This Jenkins client application will then be responsible for downloading the AUT source code, building the AUT and running the AUT.
SSH Slaves Plugin
If you remember from our first Module we connected to our Linux Ubuntu client machine using SSH. This plugin gives Jenkins the capability to use our SSH private key to connect to the Amazon EC2 client machine. Once Jenkins has a connection to the client machine it can install the Jenkins client application on this machine and then control this machine.
For this project our AUT is an application called Rocket.Chat. This project’s source is hosted on GitHub. With the Jenkins Github plugin we can connect to GitHub and download the source automatically.
Post Build Tasks
Once we’ve completed the AUT build and install we’ll want to run a few tasks to check that everything is okay. With this plugin we’ll be able to check the logs for successful install messages and run a few checks to make sure the AUT is running correctly.
How do we install and/or update these plugins? Well Jenkins makes is really easy. Just follow these steps:
1. Click ‘Manage Jenkins’ in the control panel
2. Click ‘Manage Plugins’ in the control panel
On the Plugin Manager screen you should see a list of plugins grouped by tabs that cover; Updates, Available, Installed and Advanced tabs. First we’ll need to update one plugin.
3. On the ‘Update’ tab find the ‘SSH Slaves plugin’, check the ‘install’ check box and click the ‘Download now and install after restart’ button.
5. Let the install and upgrade complete so that ‘Download Successfully’ is displayed. Click the ‘Go back to the top page’ link.
So that’s one plugin updated just another 3 to install now. To install these plugins..
- Select the ‘Available’ tab and search for each of these plugins in turn, then checking the check box:Amazon EC2 plugin
GitHub PluginHudson Post build task
Then click the ‘Download now and install after restart’ button again.
At this point Jenkins should start installing the plugins and all the dependency plugins. So you’ll see a lot more than just 3 plugins installed.
Once you see all of the ‘Downloaded Successfully’ messages you can go ahead and restart Jenkins.
6. Restart Jenkins just by clicking on the ‘Restart Jenkins when installation is complete…’ check box
At which point you should see this restart message:
With all our plugins installed we’re ready to start configuring Jenkins.
Part 5: Configuring Jenkins
Now for the interesting part. We need to configure Jenkins so that the tasks we need automating are setup to be run by Jenkins. There are four key parts to this configuration.
Firstly we need to set Jenkins up so that it is integrated with our Amazon AWS service. The ‘Amazon EC2 Plugin’ we’ve installed needs to have the ‘Amazon Access Key Id’ installed so that it has permissions to drive AWS.
Secondly, we need to configure Jenkins so that it will start our Linux Ubuntu machine on demand. When Jenkins starts that machine it needs to specify the Amazon AMI to use and install other software packages that we’ll need for our build/instll.
Thirdly, we’ll also need to setup an initialisation script for the Linux Ubuntu instances that we start via Jenkins. This script will be responsible for updating existing software packages on the Linux client and adding some additional software that isn’t included in the AMI we’ve used.
Lastly, we need to configure the ‘job’ that kicks of the build and install of our AUT. Jobs in Jenkins are just automated tasks that you configure Jenkins to run. For each Job you configure you can run that Job (e.g. build now), track the changes you’ve made to it and see all the run results for that job.
Hot tip: this setup is designed so that the client/slave machine that we install our application under test on gets shut down and everything deleted on a regular basis. This means any manual configuration you do, or data you add, on this slave machine gets deleted too. This is by design. By good design.
When you know your test server will get deleted and everything removed on a regular basis it FORCES you to make sure EVERYTHING is in the automation scripts. Every actions that you need to build the test environment is scripted. This means everything is documented and repeatable.
Part 6: Configuring Jenkins – Amazon AWS Integration
Back on the Jenkins home page you’ll need to:
1. click on the ‘Manage Jenkins’ link again
2. click the ‘Configure System’ option
There are a lot of options for configuring Jenkins. All we need to concern ourselves with is the ‘Cloud’ section (at the bottom of the page). The Cloud section is where we define our Amazon EC2 instance start up details.
3. Scroll to the bottom of the Jenkins ‘Configure’ page where you should find the ‘Cloud’ section. Then click ‘Add a new cloud’ and select the ‘Amazon EC2’ option
From here we can enter all of the details needed to run up our EC2 Ubuntu instance. Before we can complete this though we’ll need to get our Amazon ‘Access Key ID’ which you should have from going through the Prerequisites Section of this module.
4. Enter the following details for your Amazon EC2 instance
Access Key ID: <key created in prerequisite stage of module>
Secret Access Key: <key created in prerequisite stage of module>
Region: <select your region>*1
EC2 Key Pair’s Private Key: <copy in the contents of your .pem file>*2
*1 – make sure you select the same region that your Windows 2008 server machine is running in. Login to the AWS management console and check the region if you’re not sure. It’s critical that you select the right region.
*2 – in the Prerequisite section in this module we talk through getting your .pem private key file that you created in module 1. You’ll need to open this .pem file in a text editor and paste the contents in to the Jenkins ‘EC2 Key Pair’s Private Key’ text box.
Now we’re connected to our AWS account we’re ready to start configuring the details for the Linux Ubuntu slave machine we need to start up.
Part 7: Configuring Jenkins – Starting the Linux Amazon Instance Slave
5. On the ‘AMI’s line click the ‘Add’ button
6. Enter the following details for your AMI instance
AMI ID:<see below>
So the AMI id is the Amazon Machine Image number that correlates with our Linux Ubuntu machine that we want to run up. So it’s the same Id as the Linux Ubuntu machine instance that we already have configured in our AWS account. We find this ID with …
i. login to your AWS management console
ii. go to your AWS EC2 dashboard
iii. select the correct region
iv. click ‘instances’ in the side menu
v. in the list of instances select your Unix-client instance
vi. in the instance details panel (bottom of page) look for this:
From here you can pick out the ‘ami’ which should be something like:
Then we just need to click on the ‘Check AMI’ button. This should give you confirmation that AWS can find the image you need. It should look like this:
- Complete the rest of the AMI details
Instance Type: t2.micro (IMPORTANT*)
EBS Optimized: Un-checked
Availability Zone: <leave blank>
Use Spot Instance: Un-checked
Security Group Names: default, Unix-AUT (see below)
Remote FS root:<leave blank>
Remote user: ubuntu
AMI Type: unix
Root command prefix:<leave blank>
Remote ssh port: 22
Usage: Utilize this node as much as possible
Idle termination time: 30
Init script:<leave blank>
*-make sure you select ‘t2.micro’ here as this is the only Instance Type on the free tier. Select anything else and you’ll be charged for it!
The security groups you configure here will be determined by what you’ve setup in your AWS account. To find the right value:
i. login to your AWS management console
ii. go to your AWS EC2 dashboard
iii. make sure you have the correct region selected
iv. click ‘Security Groups’ in the side menu
v. in the list of security groups find your Unix-AUT group name
vi. find the ‘Group Name’. You’re looking for this:
This group will be specific to you. You may have used the same name as me but it’s best to check using the steps above. You’ll need to use both the Unix security group (Unix-AUT) and the default security group (default)
That should give us everything we need for this initial stage. Once you have the settings looking like this click the ‘Apply’ button.
Now we’re ready to check that Jenkins can run up an Amazon EC2 Linux Ubuntu instance for us automatically.
8. Return to the Jenkins home page (click ‘Jenkins’ link top right)
9. Click the ‘Build Executor Status’ link on the home page
This will take you to the ‘Nodes’ page. On this page you’ll have a list of nodes that can act as build machiens. You’ll already see your Windows 2008 Master machine listed. You won’t see a slave machine in the list yet because it’s not running yet. However, we can run the salve machine up from here.
10. Click the ‘Provision via RocketChatServer’ button and select the AMI we defined earlier
At this point you should see a message from Jenkins that ‘This node is being launched’
From here we can do two things:
- check the Jenkins log to make sure the node is launched correctly
- check our AWS account to see the running node
11. Check the Jenkins log by clicking ‘See log for more details’
At this point Jenkins is logging all the actions carried out as it brings up this Ubuntu instance. What you need to check for is this…
This just tells us that the instance was started and that Jenkins successfully logged in using our private key under the account ‘ubuntu’. You may see some warning messages but as long as the machine is running and the login worked we’re on the right track.
12. Check the AWS management console
On the AWS management console we shold be able to see our new instance running now.
i. go to your AWS EC2 dashboard
ii. make sure you’ve selected the right region
iii. click ‘Instances’ in the side menu
iv. in the list look for Unix-client (or similar)
v. select the Unix-client in the list and check it’s running
At this point you should see something like this…
We’ve successfully run up our Linux Ubuntu instance automatically from Jenkins. Great!
A few things worth mentioning…
Firstly, you may have noticed that one of the parameters we configured in Jenkins for this instance was:
Idle termination time: 30
This means that if the instance hasn’t done anything for 30 minutest Jenkins will automatically shut the machine down for us. No need to do anything ourselves we can just leave it to Jenkins to shut the system down when it’s not needed.
Secondly, you may have noticed, in the Jenkins logs, that there were some error messages around the installation of Java. Something along the lines of:
WARNING: Failed to download Java
We’ll solve that in the next step when we get Jenkins to run some initialisation scripts on the server. We do need Java though as the Jenkins Java client needs to be installed on this Ubuntu machine in order for Jenkins to have full control over it.
Part 8: Configuring Jenkins – Instance Initialisation Script
13. Set Up the Instance Initialisation Script
The last thing we need to setup during the run up of this instance is the installation of a few additional software packages. These packages aren’t installed on the AMI we’ve started out with. To get round this Jenkins has a field in the EC2 configuration called “Init Script”
To update this we’ll need to:
- return ot the main Jenkins dashboard (click the ‘Jenkins’ link top left)
- click the ‘Manage Jenkins’ link
- click the ‘Configure System’ link
- scroll to the bottom of the page where we configured our Amazon EC2 setup
- the last section on the page should be the ‘Init script’ field
So we’ll need to add some Unix shell commands to this Init script field. The purpose of these commands is to:
a. update software already on the system
b. install Java (Jenkins needs Java to run it’s remote client app)
c. install npm (more on this later)
You don’t really have to understand this script (it’s just a unix shell script) but as long as you understand why it’s there that’s the important part. So copy the following into the Init Script field:
#!/bin/sh # Update existing packages and install Java export JAVA="/usr/bin/java" if [ ! -x "$JAVA" ]; then sudo apt-get update sleep 10 sudo apt-get install -y openjdk-7-jre ca-certificates-java tzdata-java libcups2 libcups2 libjpeg8 icedtea-7-jre-jamvm openjdk-7-jre-headless openjdk-7-jdk git npm sleep 5 fi # Add Swap Space SWAP=/mnt/swap1 if [ ! -f $SWAP ]; then sudo dd if=/dev/zero of=$SWAP bs=1M count=2K sudo chmod 600 $SWAP sudo mkswap $SWAP # add new swap to config and start using it echo "$SWAP none swap defaults 0 0" | sudo tee -a /etc/fstab sudo swapon -a fi
You should have this now:
14. Return to the Jenkins home page (click ‘Jenkins’ link top right)
15 .Click the ‘Build Executor Status’ link on the home page
16. Click the ‘Provision via RocketChatServer’ button and select the AMI we defined earlier
At this point you should see a message from Jenkins that ‘This node is being launched’. This time round when you click on the ‘See log for more details’ you should see a bit more going on. In summary the log file should show you …
i. the Instance launching (INFO: Launching instance: i-1ebaa1c7)
ii. the ssh connection (INFO: Connected via SSH)
iii. execution of the Init Script (INFO: Executing init script)
iv. updates to existing software packages
v. installation of Java (INFO: Verifying that java exists)
vi. launching the Jenkins slave (INFO: Launching slave agent)
And then the final entry in the log…
Slave successfully connected and online
17. Go back to the Nodes dashboard (click ‘Jenkins-> Nodes’ in the top menu bar)
In the node list we should now see our RocketChat-Server node up and running:
At which point we have a fully configured Jenkins slave machine, running all the software we need that’s ready for us to start installing our AUT on. Next we’ll look at how to configure Jenkins to get that AUT built and installed automatically.
Part 9: A Few Points about our Jenkins Slave Machine
At this stage we can use Jenkins to start up a slave test machine automatically. There are a few points worth mentioning about how we can control this slave machine now.
Once the slave is running Jenkins runs a ‘Slave Agent’ on this machine. It’s this agent that allows the slave to talk to the master Windows Jenkins machine. It’s this slave that gives Jenkins full control of jobs and actions that can be carried out on the slave.
Slave Idle Termination Time
For example in our configuration for the Amazon EC2 slave (Manage Jenkins -> configuration) we had a setting called ‘Idle termination time’
This tells Jenkins that if the slave is inactive for more than 30 minutes to ‘Terminate’ this slave Linux EC2 instance. Great for keeping control of our Amazon charges (don’t want to be spending money on machines that are standing idle). Not great if we have data on that machine that we need as we’ll lose it all when it’s terminated. For us though we’re not worried about losing data so this’ll be just fine.
Managing Slaves from the Jenkins Master Machine
Return to the Jenkins home page and click on the link for our slave.
Viewing details for our slave you’ll see a number options in the left hand menu. These options allows us to:
Delete Slave: select this and you’ll Terminate the Amazon EC2 instance for this slave.
Configure: from here you can view and modify some of the configuration parameters for this slave
Build History: details about builds we’ve run on this slave.
Load Statistics: stats on how hard this machine is having to work.
Script Console: feature allowing us to run Groovy script on the slave machine from our Jenkins master machine
Log: details about the running up and install activities that took place when we created this instance
System Information: full list of system properties
Disconnect: the ability to disconnect Jenkins from the slave but leave the instance running still (you can reconnect later if you need to)
Part 10: Setting Jenkins up to install the AUT
So the last bit to configure is the download, build and install of the application under test (AUT) on our slave machine. The AUT we’re going to use is a chat application called Rocket Chat.
Now it’s worth mentioning here that this is just an example app we’re using for this course. We’ll go through the steps to build and install this app but you DON’T need to understand the scripts needed to build and install Rocket Chat. These are specific to Rocket Chat.
In practice, when you’re testing your own applications, you’ll need to replace the Rocket Chat build and install scripts with your own scripts. This will mean talking to your developers or build engineers. Then adding the build/install steps for your app into Jenkins.
So from this you just need to understand how you configure Jenkins. You don’t need to understand the Rocket Chat build/install process. In real life you’ll have your own applications build/install process to add to Jenkins.
Jobs and Node Labels
Jenkins is all about configuring and running jobs. Those jobs are typically build jobs but they can include any task you need to automate. For example jobs that run tests and jobs that install applications. Anything you need to automate Jenkins can be configured to run as a job.
Jobs need to run on nodes or slaves. Jenkins has some pretty clever features for working out the best places to run jobs (e.g. on the master machine or a particular slave). We’re going to keep it simple though. We going to give our slave/node a label (e.g. RCBuildDeployServer). Then when we configure the Job we’ll say “the job must be run on a slave/node that has label RCBuildDeployServer”. This way we tell Jenkins not to run everything on the Master machine (which is the default) but to run up an instance of the slave that we need (if it’s not already running) and run the job there.
So first up we to make sure we’ve configured our node/slave label.
To give our Linux Ubuntu instances a lable we need to go back to our Jenkins coniguration page.
1. On the Jenkins home page click on ‘Manage Jenkins’
2. On the Manage Jenkins page click on ‘Configure system’
3. Scroll to the bottom of this page and find the ‘Amazon EC2’ section
4. In the ‘Labels’ field enter ‘RCBuildDeployServer’
5. Click the ‘Save’ button
Setup the Job
Now we can specify where we want to run a Job (on the slave/node labled ‘RCBuildDeployServer’) we can configure a new job
1. On the Jenkins home page click on ‘create new jobs’
2. Enter a Job name (e.g. ‘BuildRocketChatOnNode’), select ‘Freestyle Project’ and click OK
3. On the Job configuration page set these key values
Restrict where this project can be run: RCBuildDeployServer
Source Code Management: None
Build Triggers: Build periodically
No need to add a schedule here yet. We’ll kick our jobs off manually for now.
4. Click on the ‘Add Build Step’ button and select ‘Execute Shell’
5. In the command text box we need to add our build script. Remember you don’t need to understand the detail here. Just that for our demo app these actions build and install the app. You’ll have a completely different set of actions for your application.Copy and paste all of the scripts below into the ‘Command’ field:
#Clean up if any left overs from last build SCREEN_RUNNING=’/usr/bin/pgrep SCREEN’ ifSCREEN_RUNNING fi NODE_RUNNING=’/usr/bin/pgrep node’ ifNODE_RUNNING fi if [ -f master.zip ]; then rm -f master.zip fi INSTDIR=./Rocket.Chat-master if [ -dINSTDIR fi MONDIR=/home/ubuntu/db if [ -d $MONDIR ]; then rm -Rf /home/ubuntu/db fi pwd #Install packages we need for the build sudo apt-get install unzip curl https://install.meteor.com/ | sh sudo npm install -g n sudo n 0.10.40
#Configure Mongo Database MONDIR=/home/ubuntu/db if [ ! -d $MONDIR ]; then mkdir /home/ubuntu/db fi pwd
#Build and install Rocket Chat export PUBLIC_HOSTNAME=”$(curl http://169.254.169.254/latest/meta-data/public-hostname 2>/dev/null)” wget https://github.com/RocketChat/Rocket.Chat/archive/master.zip unzip -o master.zip cd ./Rocket.Chat-master meteor build –server $PUBLIC_HOSTNAME –directory . cd ./bundle/programs/server npm install cd ../..
#Make sure processes continue to run when Jenkins script completes echo -e “\n” ps -ef export BUILD_ID=dontKillMe echo -e “\n”
#Start mongo DB and Rocket Chat screen -d -m /home/ubuntu/.meteor/packages/meteor-tool/.1.1.10.ki0ccv++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/mongodb/bin/mongod –bind_ip localhost –dbpath /home/ubuntu/db/ –smallfiles sleep 10 export MONGO_URL=mongodb://localhost:27017/rocketchat export ROOT_URL=http://localhost:3000 export PORT=3000 screen -d -m node /home/ubuntu/workspace/BuildRocketChatOnNode/Rocket.Chat-master/bundle/main.js sleep 10
# list system details and what's running on the system before we complete script echo -e "\n" pwd echo -e "\n" env echo -e "\n" ps -ef
So at this stage we should have the Build Execute script entered as shown
Like I’ve said this little lot is particular to the Rocket Chat app we’re using for this example. When you come to implement this in the real world you’ll work with your developers to determin what goes in here.
We can then click on the ‘Save’ button
6. On the Job summary page click the ‘Build Now’ button
7. Jenkins will go through a few steps now. First it will run up a Linux Ubuntu EC2 slave machine. This is the slave machine Jenkins needs to buid and install the application on.
If you check your AWS dashboard you should see this instance initializing. It may take a while as it has to run through all the initialisation scripts we defined too.Then, once the slave is ready, we’ll see the build start.This is our first build, as indicated by the ‘#1’ tag. We can view the detail and progress on this build by clicking on the ‘down arrow’ next to the #1 and selecting ‘Console Output’
The console output will list all of the actions being carried out as part of this build and install.
This may take some time!
Once the build and install is complete you should something like this at the end of the log file
From here we can check if the application is actually running on our slave EC2 Linux Ubuntu instance.
7. On your Windows RDP session, open a new tab in Internet explorer. Enter the following address: http://localhost:3000
You should see the application running like this:
If you want to login and play with Rocket Chat you can click on the ‘Register a new account’ link (you don’t need to follow any email confirmation process – the first login can be created direct from the GUI).
If you go back to your Jenkins home page you should see the successful build reported as follows:
Accessing Rocket Chat from your Local Machine
If you want to access the Rocket Chat application from you desktop/laptop then follow these steps:
- in your EC2 management console select the Linux host
- find the ‘Public DNS’ value
- Construct the following Url: http://<public-dns-value>:3000
- Enter this Url in your browser
Note that this will only work if your security groups are configured correctly and Public DNS settings are configured. See module 1 of this course for more info.
All that remains now is to setup a few clean up actions and a few tests to make sure Jenkins reports the build/install as a success.
Part 11: Jenkins Post Build Actions and Smoke Tests
Once we have a setup that will build and install our application it’s worth configuring Jenkins to run a few checks and tests just to confirm that things completed without any issues.
We’ll use the Post-Build Actions feature to carry out two actions:
a) check the logs for successful install messages
b) use ‘wget’ to check that the Rocket Chat home page is displayed
We can set these up by following these steps:
1. go back to the Job configuration by selecting configure on the home page.
2. Scroll to the bottom of the configuration page to find the Post Build Actions section
3. Click the ‘Add post-build action’ button and select the ‘Post Build Task’ option
Check the Logs
We’ll configure a task to look for 2 log messages. Two log message that we know we should see during the install if it’s been a successful install.
4. In the Log text field enter this text (it’s a reg ex expression)
00:..:.. SCREEN -d -m node /home/ubuntu/workspace/BuildRocketChatOnNode/
Then click the ‘Add’ button and add a 2nd log message check
00:..:.. SCREEN -d -m /home/ubuntu/.meteor
These checks are specific to the Rocket Chat application install. Again when you build this for your own application you’ll have to replace these with checks specific to your build. For info though these messages are checking that the Rocket Chat processes were successfully started and running.
5. In the Script field enter this shell script text
!/bin/bash #keep checking to see when Rocket Chat is up and running for I in 1 2 3 4 5 6 7 8 9 do sleep 60 # use wget to see if we can download index page # if wget returns true we can download index page # -T 3 => wait 3 seconds for response # -t 1 => means only retry once if (exec /usr/bin/wget -T 3 -t 1 http://localhost:3000) then echo “Rocket Chat now running ” exit 0 else echo “Waited $I minute(s) - Rocket Chat not running yet ” fi done echo “ Rocket Chat didn’t start ” exit 1
Again this is specific to Rocket Chat. We’re just running this script on the Linux Ubuntu Rocket Chat server to see if Rocket Chat starts running. We’re using a Unix command called ‘wget’ which gets a web page, the Rocket Chat home page. If this command returns a 404 over 10 attempts then we fail the check. If it returns a http 200 response code then we pass the check.
6. Check the ‘Run script only if all previous steps were successful’ check box
It’s only worth running the shell script if the checks on the log file pass.
7. Check the ‘Escalate script execution status to job status’ check box
If these checks pass then mark the build in Jenkins as a pass. If any of these checks fail then mark the build as failed in Jenkins. We’ll see the reporting of these results in a minute.
Then click the ‘Save’ button
Part 12: Starting Subsequent Jenkins Builds
Now at this point, do you remember this setting in your Amazon Ec2 configuration…
Idle termination time: 30
Well your Linux slave machine has probably been idle for 30 minutes now. As a result it’s probably shut itself down. Then again maybe you went through this really quickly and it’s still running. Either way, depending on the state of this slave machine Jenkins will do one of two things when we start a new build:
a. If it has shut itself down:
When we kick off a new build it will start from the beginning and create a new EC2 instance. NOTE: if the node has been shutdown and then a new node run up for a subsequent build then the new node with have a NEW IP ADDRESS and HOSTNAME.
b. If it hasn’t shut itself down:
The slave is still running, so Jenkins just needs to start a new build and install using the existing slave machine
If you want to avoid these nodes shutting down you can always extend the Idle termination time to more than 30 minutes. Or set it to 0 so that it always stays running (mind your AWS spend if you do this though).
Start the next build
1. Return to the Jenkins home page
2. Click the Job name (just to the right of it) and select ‘Build Now’ from the drop down menu
Assuming you didn’t have a slave running the whole process will kick off again, starting with the running up of the Amazon EC2 instance. Then the build/install will take place. Then we’ll run our completing checks/tests.
Don’t forget it can take a few minutes for these Amazon EC2 instances to run up. It can also take up to 10 minutes to complete the full job including the build and install. This is because we’re using free tier instances within AWS. Please be patient.
If the build, install and tests all succeed then we’ll see this Blue lamp and the sun icon on the Jenkins home page:
If anything fails we’ll see a red lamp and the weather will start to turn cloudy:
Getting it badly wrong over a number of builds and the weather turns to Thunder and Lightning.
In short the lamp indicates the state of the last job. The weather is an indication of job stability. So an aggregation of results across the last few builds.
Building on our AWS environment we’ve configured Jenkins so that we can control a slave machine (an Ubuntu Linux machine) and build and install the application under test. We’ve also factored in a few small checks to make sure everything is up and running.
The key take away here is that Jenkins is not just for building software. It’s not even just for implementing continuous integration. Both of these are great goals. However, on it’s own it’s just a great tool for testers. It allows us to automate a lot of our repetitive tasks like running up test environments and installing the software we need to test.
Then as we add to this the capability to kick of automated tests we’re into making our test process far more efficient. And what we’ll move on to next, in module 3 of this course, is linking in automation tools like Selenium.