Setup Appium on OSX

Background

Appium is an open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS, Android, and Windows apps using the WebDriver protocol.
– http://appium.io

Prerequisites

  • Xcode updated to latest version (through the App Store)
  • Simulator started or a physical device plugged into your Mac via USB
  • Java SDK installed and JAVA_HOME path set

Run the following commands, in Terminal, to install the necessary packages:

  1. homebrew
    • /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  2. npm
    • $ brew install node
  3. appium
    • $ npm i -g appium
  4. libimobliedevice
    • $ brew install libimobiledevice
  5. ios-deploy
    • $ npm install -g ios-deploy
  6. (Optional) appium-repl
    • $ npm install appium-repl -g
  7. Xcode command line tools
    • xcode-select --install
  8. Carthage
    • $ brew install carthage

Getting and Installing Appium

  1. In your web browser, navigate to https://github.com/appium/appium-desktop/releases/latest
  2. Download appium-desktop-#.#.#.dmg
  3. Open the .dmg file
  4. Drag the Appium icon to your Applications folder
    • Once complete, close the dmg’s window and delete the .dmg file
  5. Start Appium
    • (If prompted) Select “Open”
    • (Optional) Right-click the Appium icon in the dock and then select “Options” → “Keep in Dock”
  6. Click the “Start Server v#.#.#” button
  7. Click “⏸” (Stop Server) button
  8. Close the window to exit Appium Desktop

Start Inspector Session Using Xcode’s iOS Simulator

  1. Open XCode
  2. Select “Xcode” → “Open Developer Tool” → “Simulator”
    1. (Optional) Right-click the Simulator icon in the dock and then select “Options” → “Keep in Dock”
  3. Select “Hardware” → “Device” → “Manage Devices…”
    • Keep this window up for later
  4. Open Appium Desktop
  5. Click the “Start Server v#.#.#” button
  6. Click the “🔍” (Start Inspector Session) button
  7. Enter the following Desired Capabilities
    • automationName : XCUITest
    • platformName : iOS
    • deviceName : iPhone Simulator
    • app : {path_to_app}
  8. (Optional) Save the “Capability Set” by clicking the “Save” button
  9. Click the “Start Session” button
    • Note: The Simulator will close and a new “Appium” Simulator will be launched

Start Inspector Session Using a Physical iOS Device

The process is the same as with the Simulator, with two exceptions:

  1. Desired Capability – “app”
    • Should point to an “.ipa” instead of a “.app”
  2. Desired Capability – “deviceName”
    • Should match the Name under “Xcode” → “Window” → “Devices”

Selenium Selector Cheat Sheet

Background

Selenium uses the By class to find elements on a page. There are a number of tools to help you find a selector but once you figure out how they work you should be able to figure it out using just the element’s HTML. Creating a selector that is reliable enough to find your element, no matter its position on the page, is a learned skill. It is highly recommeded that you review the documentation for CSS Selectors and XPath Syntax.

Selenium Selector Cheat Sheet

The following tables focus on FindElement (and not FindElements), so any JavaScript function that returns multiple results is directed to return only the first (at index 0).

Basic Selenium Selectors

Description Selenium Selector JavaScript Selector
By.ByClassName  By.ClassName(“gsfi”)  document.getElementsByClassName(“gsfi”)[0]
By.ByCssSelector By.CssSelector(“#hplogo”) document.querySelector(“#hplogo”)
By.ById By.Id(“hplogo”) document.getElementById(“hplogo”)
By.ByLinkText  By.LinkText(“About”)  $x(“//a[text()=’About’]”)
By.ByName  By.Name(“q”)  document.getElementsByName(“q”)[0]
By.ByTagName  By.TagName(“a”)  document.getElementsByTagName(“input”)[0]
By.ByXPath By.XPath(“//*[@id=’hplogo’]”) $x(“//*[@id=’hplogo’]”)
  • All XPath expressions can be evaluated using jQuery. jQuery commands typically start with $
  • In XPath * is a wildcard. “//*[…” says to find any element type. “//a[…” says to find only <a> element types.

Advanced Selenium Selectors

Description Selenium Selector JavaScript Selector
Parent Element  childElement.FindElement(By.XPath(“..”)) childElement.parentElement
Element Containing Text  By.XPath(“//*[text()[contains(.,’Privacy’)]]”)  $x(“//*[text()[contains(.,’Privacy’)]]”)[0]
 Element Attribute By.CssSelector(“input[value=’3.2′]”)  document.querySelector(“input[value=’3.2′]”)
 Element Attribute By.XPath(“//input[@value=’3.2′]”)  $x(“//input[@value=’3.2′]”)

Selector Tips

Long Selectors

Using Chrome (covered below), I got a selector for Google’s “Google Search” button: #tsf > div.tsf-p > div.jsb > center > input[type="submit"]:nth-child(1). This selector reads, “Get the element with ID=’tsf’, then its child div with class=’tsf-p’, then its child div with class=’jsb’, then its child ‘center’ element > then its child input with type=’button’ and is the first child”. This selector can be trimmed down to any of the following:

  1. “center > input”
    • Returns the first <center> element’s first <input>
    • Brittle warning: If the element is moved, <center> element is removed, or another <input> becomes the first child.
  2. “input[type=submit]”
    • Returns an<input> element where type=”submit”
  3. “//*[@value=’Google Search’]”
    • Returns any element where value=”Google Search”
    • Brittle warning: If another element has the same value and appears first in the document.

Dynamic Attributes

Lets say we have a selector, “#div123 > a” and the “div123″ is dynamically generated and changes every time the page is reloaded. One simple trick is to edit the element and remove the id=”” attribute. This is done by inspecting the element and then double clicking the id attribute. When you get a new selector, it will have gone up the document tree a bit and might now look like “section > div > a” where ‘div” was the dynamically ID’d element.

 

Multiple Child Elements

This can be related to dynamic classes, too. A <ul> might have 5 <li> children. When the <ul> is first presented, all the <li> have the same class. Once one is selected the class of the selected element changes. For this example, let’s say that it goes from “li.defaultState” to “li.selected”. If your selector is “ul > li.defaultState:nth-child(5)” (to select the last <li>) you would receive an error since now only 4 elements have that class. This selector can be modified to the following:

  1. “ul > li:nth-child(5)”
    • Returns the 5th <li> of the first <ul>
    • Brittle warning: If the <ul> child count changes or if another <ul> appears first in the document.
  2. “//li[text()=’Coffee’]”
    • Returns the first <li> found where the textContent=”Coffee”
    • Brittle warning: If another <ll> appears first in the document with the same textContent value.

Getting a Selector Using Chrome’s DevTools

  1. Navigate to https://www.google.com/
  2. Right-click on the logo and then select “Inspect”
  3. The element is highlighted in the “Elements” tab of Chrome’s DevTools
  4. Right-click on the highlighted element and then select “Copy → Copy selector”

Testing a Selector Using Chrome’s DevTools Element Search

  1. Open Chrome’s DevTools [F12] and then select the “Elements” tab
  2. Press [Ctrl]+[F] to open the search
  3. Paste in your selector and if the element is found, then it will be highlighted

Testing a Selector Using JavaScript

  1. Open Chrome’s DevTools [F12] and then select the “Console” tab
  2. In the command line type document.querySelector("#hplogo") and then press [Enter]
  3. Hover over the returned element and the element will be highlighted on the page

Selenium Selectors and JavaScript

As seen in the previous section, you can test CSS Selectors using plain ol’ JavaScript. The document object is the deserialized HTML. We then call a method of that object. This can be querySelector (returns the first element matching the expression) or querySelectorAll (returns all elements matching the expression). These methods match Selenium’s functionality:

document.querySelector("your-selector-here") → driver.FindElement(By.CssSelector("your-selector-here"))

document.querySelectorAll("your-selector-here") → driver.FindElements(By.CssSelector("your-selector-here"))

The State of Selenium 3 WebDrivers

Background on Selenium 3 WebDrivers

… all the major browser vendors ship their own implementations (Apple, Google, Microsoft, and Mozilla).
… there is now a W3C specification for browser automation, based on the Open Source WebDriver. This has yet to reach “recommendation” status, but the people working on it (including members of the Selenium project!) are now focusing on finishing the text and writing the implementations.
– https://seleniumhq.wordpress.com/2016/10/13/selenium-3-0-out-now/

The Issue

While official support from browser vendors and an accompanying W3C spec are a step in the right direction, we are in the midst of a transition phase. The spec is still in “working” status. So any vague specs are either implemented as the browser vendor sees fit or not at all.

Examples

Mozilla and “Actions”

So the deal is that “actions” is terribly underspecified. I have been trying to fix that, but it’s very unclear to me what the semantics are supposed to be (e.g. when you move the mouse, should events be dispatched on all the elements along the pointers’ path). Until that is clear it’s not going to be possible to make this work the same way across multiple implementations.
– https://github.com/SeleniumHQ/selenium/issues/2285#issuecomment-227742441

Track the status of the implementation:

Apple and “Wait”

I haven’t found a bug tracker for this issue but the (above) screenshot shows that “Wait” is not implemented in Safari. This test runs fine on all the other browsers you see in the screenshot.

Workarounds

Until such time as the WebDrivers are fully fleshed out and implemented you’ll have to craft your own code if you want to test in all browsers. I do what I can in JavaScript using the JavascriptExecutor.

The Method:

The Usage:

 Non-Selenium 3 Specific Issue – IE .Click()

The IE WebDriver is still packaged with the Selenium download [Source] and to my knowledge not made by Microsoft (their efforts seem focused on Edge at this time). The .Click() implementation has mixed results [Source]. So I use JavaScript’s .click() instead.

Working with Selenium 3 Grid

Selenium 3 Grid

Selenium-Grid allows you run your tests on different machines against different browsers in parallel. That is, running multiple tests at the same time against different machines running different browsers and operating systems. Essentially, Selenium-Grid support distributed test execution. It allows for running your tests in a distributed test execution environment.
http://www.seleniumhq.org/docs/07_selenium_grid.jsp#what-is-selenium-grid

Getting the Necessary Files

This post will cover a Selenium 3 Grid setup with a Windows hub and an OSX Safari node. I highly recommend having a readme.txt file to each folder so that you can record the version number and URL for each file. If you change the file names you are going to have a bad time.

Windows

For your convenience I have uploaded a .zip of the necessary files for Windows. Download Link: Selenium_Windows.zip

Download the following files and place them in C:\Selenium

Set Your Path File

  1. Right-click “This PC” and select “Properties”
  2. In the left navigation menu, select “Advanced system settings”
  3. Select the “Advanced” tab
  4. Select “Environment Variables…”
  5. Under “System variables” select “Path” and then select “Edit…”
  6. Select “New”
  7. Enter C:\Selenium\
  8. Select “OK”

OSX

Download the following files and place them in ~/Documents/Selenium

Enable Automation Playback in Safari

  1. Ensure that the Develop menu is available.
    • It can be turned on by opening Safari preferences (Safari > Preferences in the menu bar), going to the Advanced tab, and ensuring that the Show Develop menu in menu bar checkbox is checked.
  2. Enable Remote Automation in the Develop menu.
    • This is toggled via Develop > Allow Remote Automation in the menu bar.
  3. Authorize safaridriver to launch the webdriverd service which hosts the local web server.
    • To permit this, run /usr/bin/safaridriver once manually and complete the authentication prompt.

Creating the Necessary Scripts

Windows – Start Grid Hub

Create a new file called start-grid-hub.bat and paste in the following:

cd C:\Selenium
java -jar selenium-server-standalone-3.3.1.jar -role hub

Note: Be sure to replace 3.3.1 with the current version number.

Windows – Start Grid Node

Create a new file called start-grid-node.bat and paste in the following:

cd C:\Selenium
java -Dwebdriver.chrome.driver=C:\Selenium\chromedriver.exe -Dwebdriver.gecko.driver=C:\Selenium\geckodriver.exe -Dwebdriver.edge.driver=C:\Selenium\MicrosoftWebDriver.exe -jar selenium-server-standalone-3.3.1.jar -role node -hub http://localhost:4444/grid/register

Note: Be sure to replace 3.3.1 with the current version number and localhost with your hub’s machine name or IP.

OSX – Start Grid Node

Create a new file called start-grid-node.sh and paste in the following:

#!/bin/bash
cd ~/Documents/Selenium/
java -jar selenium-server-standalone-3.3.1.jar -browser browserName=safari,platform=MAC -role node -hub http://localhost:4444/grid/register

Note: Be sure to replace 3.3.1 with the current version number and localhost with your hub’s machine name or IP.

WebDriver Setup

To create a driver that uses the remote instance of Safari add the following to your code:

DesiredCapabilities capabilities = DesiredCapabilities.Safari();
capabilities.SetCapability(CapabilityType.BrowserName, "safari");
capabilities.SetCapability(CapabilityType.Platform, "MAC");
driver = new RemoteWebDriver(hubUri, capabilities);

Note: The code (above) is C# so you may have to massage it a bit to work in other languages.

Running the Test

  1. On the Windows machine, run start-grid-hub.bat
  2. On the OSX machine:
    1. Open Terminal
    2. Drag-and-drop the start-grid-node.sh file into the Terminal window and then press [Enter]
  3. On the Windows machine, run your test(s)

Fix webdriver.chrome.driver system property Error

Background – webdriver.chrome.driver system property

I was following along Selenium’s quick-start guide but kept getting an error, “The path to the driver executable must be set by the webdriver.chrome.driver system property“. Most of the answers I found were for Java and simply set the value in code. I tried a similar fix in C# to no avail. Finally, I found a solution that works that sets the values as command-line parameters when I launch the Node.

Files

All files were placed in C:\Selenium. An entry was added to my Windows PATH variable by:

  1. Right-click “This PC” and select “Properties”
  2. In the left navigation menu, select “Advanced system settings”
  3. Select the “Advanced” tab
  4. Select “Environment Variables…”
  5. Under “System variables” select “Path” and then select “Edit…”
  6. Select “New”
  7. Enter C:\Selenium\
  8. Select “OK”

Start the Hub

The following script will start the Grid Hub (on the default port of 4444).

cd C:\Selenium
java -jar selenium-server-standalone-3.0.0.jar -role hub

Start the Node

The following script will start a Grid Node (on the default port of 5555). It will also register the locations of the Chrome and Firefox WebDrivers (respectively).

cd C:\Selenium
java -Dwebdriver.chrome.driver=C:\Selenium\chromedriver.exe -Dwebdriver.gecko.driver=C:\Selenium\geckodriver.exe -jar selenium-server-standalone-3.0.0.jar -role node -hub http://localhost:4444/grid/register

Automated API Testing Using Powershell

Background

About a year ago I was worked on a project where we had to write a RESTful API from the ground up in Node. After the first few controllers, manual testing become quite tedious and labor intensive. One of the biggest rules in SQA is “automate what you can”. So, once we had a few endpoints solidified to match the RESTful pattern, I got to work. It took about 8 hours to come up with this automated testing solution and it saved countless hours of manual QA and tickets bouncing back-and-forth between QA and DEV (because they didn’t have unit tests). I put the solution on a VM that anyone working on the project could access, modify, and run. Let’s get started with API Testing Using Powershell.

Why Powershell?

I wanted to use the HttpClient from .Net 4.5 that I used in a lot of previous projects. The devs I was working with were MEAN and LAMP stack proficient, so I had to be sure to “Keep It Stupid Simple” and be sure to comment.

The Setup

As previously mentioned, I use the HttpClient (System.Net.Http.dll). I also import the JSON.Net (Newtonsoft.Json.dll) library to parse the JSON response from the RESTful API. In the project you’ll also see a .CSV file (check it out on GitHub). The CSV file is what makes this a data-driven testing solution that is easily updated and read by both machine and man.

The Code

You can view/fork the code from my GitHub project. You’ll need Powershell to run it, but I highly recommend using Powershell ISE. I am not going to copy+paste the code like in my other posts but rather I will explain it at a high-level.

What’s Going On

  1. Get the Present Working Directory (pwd) and save it for the next few steps
  2. Import the .dll files (see “The Setup” above)
  3. Set the BaseUrl, so that only the endpoints need to be specified in the .CSV file
    • This allows you to easily switch environments
  4. Create a new instance of the HttpClient
    • This will hold your session, so that you can log-in with one test and then do some call that requires a valid session afterwards
  5. Import the .CSV file
  6. Parse the .CSV file using a foreach loop
    1. Create an error array
    2. Create a description of the current test (row of the .CSV)
    3. Build a request with the given data
      • If the Method is PUT or POST, then add the body content
    4. Send the request and save the response
    5. Save the response’s status code
    6. Save the response’s content
    7. If the expected status code does not match the response’s, then we add that error message to the error array
    8. If the error array has any length, then we print out an error (with red text-color)
      • Else, print out a success message (with green-text)
    9. If there is a comment in the .CSV file’s current row, then print that out (with gray-text)

API Testing Using Powershell in Action

API Testing Using Powershell Screenshot

Load Testing Using PowerShell

Background

I was tasked with doing a load test for a client. At the start of work on the ticket I found that Telerik Load Testing wasn’t finding my dynamic target. This is because it was looking at query-string parameters but I needed to target values sent via a POST with a multi-part form as its body content. So, I took some code I used for another project where I used the HttpClient from the .Net 4.5 library in PowerShell.

Getting the Required Assembly

The code we will cover uses a .Net 4.5 dynamic link library (.dll). You can get this by installing .Net 4.5 (or higher) SDK. I have also added them to the sample project at the end of the article. Ss a convenience, I like to copy the .dll file(s) I need to the same directory to the PowerShell script (.ps1 file).

Understanding the Basics

A virtual user will make a series of URL requests, often with wait times (called “Think Time”) peppered in to simulate an actual users interaction with the server). We will use the HttpClient for the virtual user’s session as it is thread safe. It needs to be thread safe because we will use PowerShell RunSpaces and Jobs to spin up concurrent users. The calculator for finding the number of concurrent users can be found at webperfomance.com. Note that the number of concurrent users you can actually use is limited by the hardware the script is executing on as we are spinning up multiple instances of PowerShell.

Multi-Threading  PowerShell

The code for multi-threading comes from TheSurleyAdmin.com. The code works like this: Each thread is started up and given the task within the $ScriptBlock. Once the jobs are all done, the results are spit out into a GridView.

# http://thesurlyadmin.com/2013/02/11/multithreading-powershell-scripts/
cls

$Throttle = 5
$ScriptBlock = {
    Param (
        [int]$RunNumber
    )
    $RanNumber = Get-Random -Minimum 1 -Maximum 10
    Start-Sleep -Seconds $RanNumber
    $RunResult = New-Object PSObject -Property @{
        RunNumber = $RunNumber
        Sleep = $RanNumber
    }
    Return $RunResult
}

$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $Throttle)
$RunspacePool.Open()
$Jobs = @()

1..20 | % {
    $Job = ::Create().AddScript($ScriptBlock).AddArgument($_)
    $Job.RunspacePool = $RunspacePool
    $Jobs += New-Object PSObject -Property @{
        RunNum = $_
        Pipe = $Job
        Result = $Job.BeginInvoke()
    }
}

Write-Host "Waiting.." -NoNewline
Do {
    Write-Host "." -NoNewline
    Start-Sleep -Seconds 1
} While ( $Jobs.Result.IsCompleted -contains $false)
Write-Host "All jobs completed!"

$Results = @()
ForEach ($Job in $Jobs)
{   
    $Results += $Job.Pipe.EndInvoke($Job.Result)
}

$Results | Out-GridView

Adding in the HttpClient

Now that we have our base, we’ll need to modify it to suit our needs. The first thing we’ll need to add is the HttpClient. Because we are firing up multiple instances of PowerShell, the import will need to happen in the $ScriptBlock.

$ScriptBlock = {
    ...
    # Get the current working directory
    $pwd = pwd
    # Import the .Net Http Dynamic Link Library
    Add-Type -Path "$pwd\System.Net.Http.dll"
    # Instantiate a new http client, using the imported library
    $client = New-Object -TypeName System.Net.Http.Httpclient
    ...
}

Driving the Test Using a Data-Bound CSV File

Instead of having a list of URLs in the script, I like to use a .csv file as the PowerShell cmdlet is amazingly simple to use. In this CSV I have just a few columns; ID, METHOD, and URL. Method was needed so that when a POST was encountered, I could create the body dynamically. More on that in a moment.

$ScriptBlock = {
    ...
    # Define the path to the CSV file that holds all the URLs to be tested
    $pathToCsv = "$pwd\URLs.csv"
    # Import the CSV (defined above)
    $csv = Import-csv -path $pathToCsv
    ...
}

Handling POSTs

For my load test, the POSTs needed to have unique email addresses. For this I use one of my favorite tricks: the Gmail “+” trick and UnixTimeStamp. With Gmail you can add + to make a unique email address that routes back to the original. As an example, “tim+1@gmail.com” is actually “tim@gmail.com”. So after the “+” I just throw in a UnixTimeStamp. I found that this wasn’t unique enough so I also added the thread number to the string. Also, notice the use of MultipartFormDataContent. That is the bit that creates the form to be used as the POST’s body content.

$ScriptBlock = {
    ...
    # For Loop to run through each row of data in the CSV
    foreach($line in $csv) {
        $startTime = Get-Date
        $email = ''
        $statusCode = ''
        # If the request is a POST, then we need to set a unique email address
        if ($line.Method -eq "POST")  {
            $mfdc = New-Object -TypeName System.Net.Http.MultipartFormDataContent
            $pathToFormDataCsv = "$pwd\Form-Data.csv"
            $formData = Import-csv -path $pathToFormDataCsv
            foreach ($row in $formData)
            {
                $key = $row.Key
                $value = New-Object -TypeName System.Net.Http.StringContent $row.Value
                if ($key -eq "email") {
                    $email = "tim+$RunNumber{0:G}@spiredigital.com" -f [int][double]::Parse((Get-Date -UFormat %s))
                    $value = New-Object -TypeName System.Net.Http.StringContent $email
                }
                $mfdc.Add($value, $key)
            }
            $response = $client.PostAsync($line.URL, $mfdc).Result
            # Get the status code
            $statusCode = $response.StatusCode
        }
        # Otherwise, just make the request
        else {
            $request = New-Object -TypeName System.Net.Http.HttpRequestMessage
            $request.RequestUri = $line.URL
            $request.Method = $line.Method
            $response = $client.SendAsync($request).Result
            # Get the status code
            $statusCode = $response.StatusCode
        }
        $endTime = Get-Date
        ...
    }
}

Handling Results

After each request, I log the time it took and add my test data to a PowerShell Object using Add-Member to add properties to the generic object. That object is returned as the result of the job. Then after all jobs are complete, the results are aggregated and spit out in a GridView (same as the original script from TheSurleyAdmin).

$ScriptBlock = {
    ...
    # For Loop to run through each row of data in the CSV
    foreach($line in $csv) {
        ...
        $runTime = New-TimeSpan -Start $startTime -End $endTime
        # Save the test data to a PowerShell object
        $iterationResult = New-Object PSObject
        $iterationResult | Add-Member -NotePropertyName "Thread" -NotePropertyValue $RunNumber
        $iterationResult | Add-Member -NotePropertyName "StatusCode" -NotePropertyValue $statusCode
        $iterationResult | Add-Member -NotePropertyName "URL" -NotePropertyValue $line.URL
        $iterationResult | Add-Member -NotePropertyName "Time" -NotePropertyValue $runTime
        # Add the new object to the test result
        $threadResults += $iterationResult
    }
    # Return the results array for this thread
    Return $threadResults
}

Putting It All Together

Here is the final script:

# Set the number of threads we want to run concurrently
$threads = 5
# The script block is the code to be executed by each thread. Threads do not share data so if a library needs to be imported, then each thread needs to do so.
$ScriptBlock = {
    # Parameter - https://technet.microsoft.com/en-us/magazine/jj554301.aspx
    Param ( [int]$RunNumber )
    # Get the current working directory
    $pwd = pwd
    # Import the .Net Http Dynamic Link Library
    Add-Type -Path "$pwd\System.Net.Http.dll"
    # Instantiate a new http client, using the imported library
    $client = New-Object -TypeName System.Net.Http.Httpclient
    # Define the path to the CSV file that holds all the URLs to be tested
    $pathToCsv = "$pwd\URLs.csv"
    # Import the CSV (defined above)
    $csv = Import-csv -path $pathToCsv
    # Create an array to hold iteration results
    $threadResults = @()
    # For Loop to run through each row of data in the CSV
    foreach($line in $csv) {
        $startTime = Get-Date
        $email = ''
        $statusCode = ''
        # If the request is a POST, then we need to set a unique email address
        if ($line.Method -eq "POST")  {
            $mfdc = New-Object -TypeName System.Net.Http.MultipartFormDataContent
            $pathToFormDataCsv = "$pwd\Form-Data.csv"
            $formData = Import-csv -path $pathToFormDataCsv
            foreach ($row in $formData)
            {
                $key = $row.Key
                $value = New-Object -TypeName System.Net.Http.StringContent $row.Value
                if ($key -eq "email") {
                    $email = "tim+$RunNumber{0:G}@spiredigital.com" -f [int][double]::Parse((Get-Date -UFormat %s))
                    $value = New-Object -TypeName System.Net.Http.StringContent $email
                }
                $mfdc.Add($value, $key)
            }
            $response = $client.PostAsync($line.URL, $mfdc).Result
            # Get the status code
            $statusCode = $response.StatusCode
        }
        # Otherwise, just make the request
        else {
            $request = New-Object -TypeName System.Net.Http.HttpRequestMessage
            $request.RequestUri = $line.URL
            $request.Method = $line.Method
            $response = $client.SendAsync($request).Result
            # Get the status code
            $statusCode = $response.StatusCode
        }
        $endTime = Get-Date
        $runTime = New-TimeSpan -Start $startTime -End $endTime
        # Save the test data to a PowerShell object
        $iterationResult = New-Object PSObject
        $iterationResult | Add-Member -NotePropertyName "Thread" -NotePropertyValue $RunNumber
        $iterationResult | Add-Member -NotePropertyName "StatusCode" -NotePropertyValue $statusCode
        $iterationResult | Add-Member -NotePropertyName "URL" -NotePropertyValue $line.URL
        $iterationResult | Add-Member -NotePropertyName "Time" -NotePropertyValue $runTime
        # Add the new object to the test result
        $threadResults += $iterationResult
    }
    # Return the results array for this thread
    Return $threadResults
}
# Create a pool of powershell instances
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $threads)
$RunspacePool.Open()
# Create an Jobs array and invoke each job as they are added to the array
$Jobs = @()
1..$threads | % {
    $Job = ::Create().AddScript($ScriptBlock).AddArgument($_)
    $Job.RunspacePool = $RunspacePool
    $Jobs += New-Object PSObject -Property @{
        RunNum = $_
        Pipe = $Job
        Result = $Job.BeginInvoke()
    }
}
# Clear screen
cls
# Write a status message to the console, until all jobs finish
Write-Host "Working.." -NoNewline
Do {
    Write-Host "." -NoNewline
    Start-Sleep -Seconds 1
} While ( $Jobs.Result.IsCompleted -contains $false)
Write-Host "All jobs completed!"
# Package all threads results into one object
$Results = @()
ForEach ($Job in $Jobs) {   
    $Results += $Job.Pipe.EndInvoke($Job.Result)
}
# Display the results in a table
$Results | Out-GridView

gridview-example

Download the Sample

Note that I have removed the URLs as they are subject to a confidentiality agreement with the client.

Load Test (compressed via .zip).

Performance Enhancements and Load Testing Tips

Testing – What to Look For

Enhancements – Low Hanging Fruits

  1. Optimize images
    1. Images should not exceed 1MB if at all possible
    2. Should be optimized
      1. https://imageoptim.com/ – for Mac OSX
      2. https://tinypng.com/ – Online tool
    3. Large images that are resized via html/js should be that size in the first place
  2. Minify all JS/CSS
  3. Do not load videos on page load (YouTube videos like to buffer)
    1. Instead, use an image, that when clicked will load the video
  4. Enable gzip compression (server setting)
  5. Limit API calls
    1. For auto-complete search:
      1. Do not submit until at least 2 or 3 characters are entered
      2. And/or use a timer to send a request only after the user has stopped typing for 2 seconds

Enhancements – Other Possible Enhancements

  1. Set up a caching mechanism
    1. Caching will respond to a request with a previous response if the requests are the same
      1. WordPress has tons of plug-ins for this
      2. http://memcached.org/ – *nix
      3. Many more options exist
  2. Set up a CDN
    1. Put assets on another server to reduce load on the server that is handling requests
      1. AWS has a service called CloudFront
      2. Many more options exist

Setup Eclipse for Automation Testing

Update

This article is still relevant but a more comprehensive guide exists at http://timothycope.com/c-selenium-automation. This new guide also covers Page Object Design Pattern (POM) and Selenium Server (Grid).

Background

Eclipse is an IDE for many languages and in this case we will use it for Java development. Tests created using Selenium IDE can be exported to Java and ran in Eclipse using the JUnit plugin.

Getting What you Need

Eclipse will be an executable while jUnit and Selenium will be .jar files.

  1. Eclipse (for Java Developers) – http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/mars1
  2. JUnit – https://github.com/junit-team/junit/wiki/Download-and-Install 
  3. Selenium (Server and Client) – http://www.seleniumhq.org/download/
  4. Selenium IDE – http://www.seleniumhq.org/download/

Setting It All Up

  1. Install Eclipse
    1. For OSX, when the .tar.gz file opens move the extracted app to your Applications folder.
  2. Run Eclipse
    1. For OSX, open your Applications folder and then right-click Eclipse and select Open. Select Open from the prompt.
  3. For now, just use the default workspace. So, simply select Ok when the workspace prompt appears
  4. Close the welcome screen by clicking the X on its tab
    1. You should now see a blank project explorer
  5. From the menu select File -> New -> Java Project
  6. Name the project “Example” and select Finish
  7. Right-click in the Package Explorer pane and select New -> Folder
  8. Select “Example” as the parent folder
  9. Name the new folder “lib” and select Finish
    1. This will act as our “libraries” folder
  10. Copy over the .jar files to the “lib” folder (you can drag and drop)
    1. junit-#.#.jar
    2. selenium-server-standalone-#.#.#.jar
    3. selenium-java-#.#.#.jar
  11. Select Copy files on the prompt
  12. Select all files in the lib folder, right-click and select Build Path -> Add to Build Path
    1. This will copy the files into a folder called “Referenced Libraries” in your Package Explorer

Creating an Example Test Using Selenium IDE

  1. Install Selenium IDE if you haven’t already
    1. Use Firefox to download using the link on the Selenium downloads page
  2. Open Firefox
  3. Click the Selenium icon to the right of the address bar
  4. Enter https://www.google.com as your Base URL
  5. Click the record button (if not already recording)
    1. Navigate to https://www.google.com/
    2. Type Tim
    3. Click Search button
    4. Right-click on the page and select waitForTitle Tim – Google Search from the context menu
    5. Click the record button to stop recording
  6. Click the Play Current Test button
    • Ensure all steps are highlighted in green, meaning the test passed
  7. Click File -> Export Test Case As… and select Java/JUnit4/Webdriver
  8. Save as “test01.java

Adding the Test to the Eclipse Project

  1. Add/Drag-and-drop the .java file into the “src” folder of your project
    1. Select Copy file if prompted
  2. There will be errors reported during the import so let’s fix those:
    1. In the Package Explorer, drill down to the newly added test and double-click it
      1. Example -> src -> default package -> test01.java
    2.  The declared package “com.example.tests” does not match the expected package “” test01.java /Example/src line 1 Java Problem
      1. Remove the line “package com.example.tests;”
    3. The public type Test01 must be defined in its own file test01.java /Example/src line 12 Java Problem
      1. Rename the class “test01”
    4. Save the changes and refresh the package explorer (F5)

Running Your First Test

  1. Right-click the test in the package explorer and select Run As -> JUnit Test
  2. Eclipse will open the JUnit pane and execute the test in Firefox (the default WebDriver).
  3. Upon completion you should see the result(s)

Running More Tests

You can add more test to the default package. When you want to run them all you can select the default package folder and run as a JUnit test.

Running Tests Using a Headless Browser

A headless browser is an “internet browser” without a GUI. For this example, we will use HtmlUnit. It is an HtmlClient wrapper for Java applications. Get it here, http://sourceforge.net/projects/htmlunit/files/htmlunit/.

  1. Download the .jar file
  2. Add the .jar file to your “lib” folder
  3. Add the file to your build path
  4. Using the test created (above):
    1. Using //, comment out the line private WebDriver driver;
    2. Add a line under that, private HtmlUnitDriver driver;
      1. Hover over WebClient and select Import ‘HtmlUnitDriver’ (org.openqa.selenium.htmlunit)
      2. This will add the reference for WebClient from the HtmlUnit jar
    3. Using //, comment out the line driver = new FirefoxDriver();
    4. Add a line under that, driver = new HtmlUnitDriver();
    5. Add a line under that, driver.setJavascriptEnabled(true);
  5. Save your changes
  6. Execute the test
    1. Note: HtmlUnit will show a bunch of warnings, ignore them for now as it is normal behavior
      1. These warnings related to the page being loaded (akin to DevTools Console) and not the test itself
    2. Note: Google changes IDs based on browsers so this example will fail since it was recorded in Firefox and will be ran using HtmlUnit [Source].
      1. You can set the User-Agent by setting, driver = new HtmlUnitDriver(BrowserVersion.FIREFOX_38);
      2. You will also have to hover over BrowserVerison and add the missing reference, Import ‘BrowserVersion’ (com.gargoylesoftware.htmlunit)
      3. Documentation on BrowserVersion

Selenium Locators

Background

For many Selenium commands, a target is required. This target identifies an element in the content of the web application, and consists of the location strategy followed by the location in the format locatorType=location. The locator type can be omitted in many cases. The various locator types are explained below with examples for each.
http://www.seleniumhq.org/docs/02_selenium_ide.jsp#locating-elements

A Note on Locators

Most locators will find the first element that matches the criteria. So, if you have “link=Click Here” as your target and you have multiple “Click Here” links on the page, then Selenium will run with the first one it find in the DOM.

Selenium IDE

While using Selenium IDE it will chose the selector for you. Most often this is fine but in the case of dynamic IDs, you’ll need to know how to make your own. The order in which Selenium IDE picks a selector seems to be:

  1. ID
  2. Link Text (for <a> only)
  3. Name + Type (optional) + Value (optional)
  4. Xpath

There are other locators, but I have yet to see Selenium IDE use them. You can see all of them by clicking on the link in the Background quote (above).

Locating By ID

This is probably the simplest as an ID should be unique. A challenge would be if the ID is dynamic. If so, use another locator method (below).

Example (Source):

<html>
  <body>
    <form id="loginForm">
      <input name="username" type="text" />
      <input name="password" type="password" />
      <input name="continue" type="submit" value="Login" />
      <input name="continue" type="button" value="Clear" />
    </form>
  </body>
</html>

To target the <form> your target will be id=loginForm

Locating by Link Text

Generally speaking, links do not have many attributes. As such we can use the <a> tag and it’s TextContent.

Example (Source):

<html>
  <body>
    <p>Are you sure you want to do this?</p>
    <a href="continue.html">Continue</a>
    <a href="cancel.html">Cancel</a>
  </body>
<html>

To target the <a> whose TextContent is “Continue” your target will be link=Continue

Locating By Name + Type + Value

This uses the element attributes to target the element. Name is required but Type and Value are optional.

Example (Source):

<html>
  <body>
    <form id="loginForm">
      <input name="username" type="text" />
      <input name="password" type="password" />
      <input name="continue" type="submit" value="Login" />
      <input name="continue" type="button" value="Clear" />
    </form>
  </body>
<html>

To target the <input> whose name is “continue”, type is “button”, and value is “Clear” your target will be name=continue type=button value=Clear. Note that if you don’t specify a value, then Selenium will find the element whose value is “Login” as it is the first element to meet the conditions.

Locating by Xpath

Xpath was intended for XML but works with HTML as well. This uses an elements position in the DOM as a locator. An Xpath of html/div/a[1] would look in the <html> section of the DOM, drill down into the first <div>, and then find the first <a>.

Example (Source):

<html>
  <body>
    <form id="loginForm">
      <input name="username" type="text" />
      <input name="password" type="password" />
      <input name="continue" type="submit" value="Login" />
      <input name="continue" type="button" value="Clear" />
    </form>
  </body>
<html>

To target the <form> your target will be xpath=/html/body/form[1]

Locating with CSS Instead of Xpath

Xpath is great when it works. When it doesn’t it’s usually because the element changed position in the DOM. For this reason we should target elements in a way that front-end developers would. That is CSS selectors. To do so your target will be css={insert path here}

Tips on Building Xpath and CSS Locators

In Chrome you can inspect an element, right-click, and select ‘Copy CSS Path’ or ‘Copy Xpath’.

Screen Shot 2015-11-12 at 3.05.57 PM

In Firefox you can do the same thing but you will need the Firebug add-on.

Screen Shot 2015-11-12 at 3.07.57 PM