Test Automation Strategy: Developing A

Developing a
test automation strategy

why are we here?
regression testing
the greatest cost of development
regression testing
you have already paid for this at least once

why regression test?
quality ensures what was working, still works

quality ensures what was working, still works
risk
deliverables, release, schedule
money
lost revenue, market, contractual
a project without
thorough and continuous regression testing
is like a road trip
where you do not look out the window
until the end of the trip
why automate? (vs. manual)
Money Time
Money
Time
automation strategy
what to consider...
tests are living specifications of the system

automation strategy
what to consider...
ARE THE TESTS GOING TO BE USED?

automation strategy
what to consider...
business needs
return on investment
organizational structure and culture

automation strategy
business needs...
determine areas of focus of the automation

automation strategy
return on investment...
limits and prioritizes the areas of focus of the

automation
determines at which level to automate a test
determines the automation framework

automation strategy
organization and culture...
determines the success or failure
determines the speed, ability & openness to change
determines the communication
determines the ability to learn

automation strategy
organization and culture...
determines who uses and maintains the tests
determines the automation framework too

automation strategy
what else to consider...
automation strategy
development process and lifecycle
maintenance of the tests
technology stacks
automation strategy
development process and lifecycle...
legacy (system without tests) to greenfields

automation strategy
system under test development process
methodology: waterfall agile xp ad hoc
is automation development part of the same process?

automation strategy
is the automator part of the development team?
how much access to the system developers?

automation strategy
maintenance of the tests...
who and how much can they maintain?
how are the system developers involved?

automation strategy
maintenance of the tests...
how timely can/must tests be maintained
what reporting needed to maintain tests

automation strategy
finally, technology stacks...
languages and technology already used
unit and integration testing
source code control, continuous integration
devops or operations
where to automate?
who is going to use the tests?
developers, maybe...
depends
where to automate?
“Its essential point is that you should
have many more low-level unit tests than
high level end-to-end tests running
through a GUI.” Martin Fowler
https://msdn.microsoft.com
where to automate?
XP
UNIT LEVEL
INTEGRATION LEVEL test driven development
GUI INTEGRATION LEVEL
ACCEPTANCE LEVEL
BDD
where to automate?
acceptance tests
only if the business is engaged and
going to use them
ACCEPTANCE LEVEL
BDD
where to automate?
what to consider...
DRY
where to automate?
what to consider...
look for low-hanging fruit

risk-based testing
what computers do well, humans not so much
fragility of test
which automation framework to use?
Gherkin/Cucumber JUnit
Robot Framework
Jasmine
Selenium Webdriver
Spock/Geb
TestNG
what to consider...
ORGANIZATIONAL STRUCTURE AND CULTURE

what to consider...
ORGANIZATIONAL STRUCTURE AND CULTURE

who is reading the tests?
who is writing and maintaining the tests?
balance of right tool with existing technology,

knowledge, and ability to learn
natural language for high level specification
goal is to develop tests NOT framework
and...
grids...
integration...
reporting...
teh sexy
gherkin/cucumber/ruby
gherkin/cucumber
generally ruby based...
but JRuby (JVM), Java, Groovy, JavaScript,

.NET (using SpecFlow)
libraries (Selenium, Watir)
Ruby has the richest ecosystem with gems
Feature: Cash Withdrawal
Scenario: Successful withdrawal from an account in credit

Given my account has been credited with $100
When I withdraw $20
Then $20 should be dispensed
And the balance of my account should be $80
Given /^my account has been credited with (#{CAPTURE_CASH_AMOUNT})$/

do |amount|
my_account.credit(amount)
end
robot framework
python based...
also runs on Jython (JVM) & IronPython (.NET)

libraries (Selenium, Watir, SOAP, etc.)
keyword based
supports “given when then”
supports tabular/data driven
*** Settings ***
Library Selenium Library
Suite Setup Start Selenium Server
Suite Teardown Stop Selenium Server
Test Setup Setup Test
Test Teardown Close All Browsers
*** Testcases ***
Login Should Succeed When the Correct
Username and Password are Entered
Enter Username AUser
Enter Password TestPass
Click the Login Button
Login is Successful
*** Keywords ***
Setup Test
Open Browser http://zdiles.chaosnet.org/ googlechrome
Maximize Browser Window
Enter Username [Arguments] ${username}

Input Text uname ${username}
Login Is Successful
Page Should Contain Welcome
*** Testcases ***
Login Should Succeed When the Correct Username and
Password are Entered
Given I Enter Username AUser
And I Enter Password TestPass
When I Click the Login Button
Then Login is Successful
given when then

spock/geb/groovy
developing in java or grails?
spock/geb/groovy
geb...
groovy-based browser automation tool
built on Selenium WebDriver
works with JUnit, TestNG, Spock
page models and jQuery selector syntax

spock/geb/groovy
spock...
groovy-based test and specification framework
supports “given when then”
tabular/data driven
import geb.spock.GebSpec
class GebHomepageSpec extends GebSpec {
def "can access The Book of Geb via homepage"() {

given:”One is on the Geb Home Page”
to GebHomePage
when: “I click the jQuery Link”

highlights.jQueryLikeApi.click()
then: “I should be at the jQuery Section”

highlights.jQueryLikeApi.isSelected
}
}
import geb.Page
class GebHomePage extends Page {

static url = "http://gebish.org"
static at = { title == "Geb - Very Groovy Browser Automation" }
static content = {
highlights { $("#sidebar .sidemenu").module(HighlightsModule) }
sectionTitles { $("#main h1")*.text() }
}
}
KS
E WOR
RA M
F
other options...
other options...
raw Selenium (Java)

excel spreadsheet data driven scripts
script capture record/playback
flat files in source control
the simplest thing that could possibly work

tests
are specifications of the system...
how does it fit?
how does it fit...
design your approach into the organization

how does it fit...
find the gaps and tight squeezes

the specifics...
the mechanics...
so what’s left?
documentation of the strategy...
execution of the strategy...
if project management...
estimation...
resources...

Test Automation Strategy: Developing A

Uploaded by

Copyright:

Available Formats

Test Automation Strategy: Developing A

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Test Automation Strategy: Developing A

Uploaded by

Copyright:

Available Formats

Developing a

test automation strategy

you have already paid for this at least once

quality ensures what was working, still works

quality ensures what was working, still works

tests are living specifications of the system

ARE THE TESTS GOING TO BE USED?

organizational structure and culture

determine areas of focus of the automation

limits and prioritizes the areas of focus of the

determines at which level to automate a test

determines the automation framework

determines the success or failure

determines the speed, ability & openness to change

determines the communication

determines the ability to learn

determines who uses and maintains the tests

determines the automation framework too

development process and lifecycle

maintenance of the tests

legacy (system without tests) to greenfields

system under test development process

methodology: waterfall agile xp ad hoc

is automation development part of the same process?

is the automator part of the development team?

how much access to the system developers?

who and how much can they maintain?

how are the system developers involved?

how timely can/must tests be maintained

what reporting needed to maintain tests

languages and technology already used

unit and integration testing

source code control, continuous integration

GUI INTEGRATION LEVEL

look for low-hanging fruit

ORGANIZATIONAL STRUCTURE AND CULTURE

ORGANIZATIONAL STRUCTURE AND CULTURE

who is writing and maintaining the tests?

balance of right tool with existing technology,

natural language for high level specification

goal is to develop tests NOT framework

but JRuby (JVM), Java, Groovy, JavaScript,

Scenario: Successful withdrawal from an account in credit

Given /^my account has been credited with (#{CAPTURE_CASH_AMOUNT})$/

also runs on Jython (JVM) & IronPython (.NET)

Enter Username [Arguments] ${username}

given when then

groovy-based browser automation tool

built on Selenium WebDriver

works with JUnit, TestNG, Spock

page models and jQuery selector syntax

groovy-based test and specification framework

supports “given when then”

class GebHomepageSpec extends GebSpec {

def "can access The Book of Geb via homepage"() {

when: “I click the jQuery Link”

then: “I should be at the jQuery Section”

class GebHomePage extends Page {

static at = { title == "Geb - Very Groovy Browser Automation" }