Software Test Automation

With Open Source Software

Itai Agmon

Who Am I?

• CTO of Top-Q
• Main contributor of the JSystem project
• Creator of the Difido project
• My laptop runs Linux

1
 Why do I like OSS Projects?
• It’s built by passionate communities of developer
• Real solutions to real problems
• Rate of innovation is faster.
• There’s no vendor lock-in. Remember SilverLight
• You can “look under the hood”
• No back doors. _NSAKEY
• Oh, and it’s free
“Given enough eyeballs, all bugs are shallow.”
Eric S. Raymond

2
3
Automation tools and libraries

4
So, Selenium?

5
 Test Automation Frameworks

Tests Layer
Inversion of Control
Could be
Programmatically, Text Based, GUI Parameters

Reporting
Common Interface
Infrastructure Layer Setup and Teardown

Assertion and Validation

Layered Architecture

Let’s select A framework

6
 First - Functional Tests or NFT

• Non Functional • Functional

– Performance – Black box testing
– Security – Describes what the
– Usability system does
– Stability
– Feed input,
– Availability & Resilience
examine the output
– Configuration
– Data migrating

First - What is the type of the functional tests?

Experimental
manual tests

The higher we go
• More unreliability End
QA/
• Greater cost
• Slower feedback
to End Automation
/GUI Teams

Integration/
API

R&D
Unit Tests

7
 What is the programming language?

Who Are The Automation Customers?

Coding Skill
Coding Skill

Coding Skill

QA Automation Eng. R&D Developer

• Common in small organization, startups

• Automation engineer write, execute and debug
tests
• It is better to use the code based approach

8
 Code Based Approach

private String firstName = "Zhang";

private String lastName = "Wei";

@Test
public void testEditAccount(){
DashboardPage dashboardPage = web.getDashboardPage();
AccountsPage accountsPage = dashboardPage.
getTopMenuPage().
clickOnCustomersItm().
clickOnAccountsItm();

AccountDetailsPage accountDetailsPage = accountsPage.clickOnAccountByIndex(0);

EditAccountDetailsMainTab mainTab =
accountDetailsPage.clickOnEditAccountLnkAndGoToEditAccountsPage();
mainTab.enterTextToFirstName(firstName);
mainTab.enterTextToLastName(lastName);

EditAccountDetailsAdditionalInfoTab additionlInfoTab = mainTab.clickOnAdditionInfoTab();

additionlInfoTab.selectCountry(Country.Hungary);
additionlInfoTab.clickOnOkBtn();
}

Code Based Approach Frameworks

MSTest

9
 Who Are The Automation Customers?

Coding Skill
Coding Skill

Coding Skill

QA Automation Eng. R&D Developer

• Common in large organization

• Automation engineers write the infrastructure
• QA are familiar with the system more then any other person
• QA are using GUI or any other UI to write, execute and debug tests

Layered Architecture

Tests Layer

Text Based, GUI

QA
Common Interface
Infrastructure Layer

Automation
Eng.

10
 UI Based Approach Frameworks

Gauge
Gauge

Robot
Framework

11
Behavior Driven Development (BDD)

BDD Automation Frameworks

12
 Who Are The Automation Customers?

Coding Skill
Coding Skill

Coding Skill
QA Automation Eng. R&D Developer

• Developers can write tests for each new feature

• Watch out! They always have their own opinion
• They will probably prefer to write code
• Need to enforce discipline

What services do I need from my

framework?

13
 Not only must Automation be done; it must also be seen to be done

• Start test
• End test
• Fail test
• Log message Console

Excel/CSV
Automation Report
Tests Manager
DB

HTML
XML

System Under
Test

Parameterization

@Parameters({ "first-name" })
<suite name="My suite"> @Test
<parameter name="first-name" value="Cedric"/> public void testSingleString(String firstName) {
System.out.println("Invoked testString " + firstName);
<test name="Simple example"> assert "Cedric".equals(firstName);
>-- ... --< }

private File file = new File(".");

private String str = "Some string";
private int i = 5;
private Date date = new Date();
private String[] strArr;
private Account account;

**/
* Test with different parameters
/*
@Test
@TestProperties(name = "Test with file '${file}' string '${str}' integer ${i}
date ${date}", paramsInclude = {
"file", "str", "i", "date", "strArr" })
public void testWithParameters() {
report.report("File: " + file.getAbsolutePath());
report.report("Date: " + date.toString());
report.report("String: " + str);
report.report("Integer: " + i);
}

14
Additional Framework Services

Inversion of Control

Parameters

Reporting

Fixtures

Assertion and Validation

Concurrency

Environment Configuration

peripheral systems

15
 Peripheral Systems Types

• Scheduler (CI/CB)
• Test Management Systems (QC, TestLink, MTM)
• Resource Manager
• Report Server
• Reports BI provider
• Source Control
• Remote Agents Manager
• Report Generator

Peripheral Systems Example

16
Q&A

17
 Automations in a
Continuous Deployment
Environment
Matan Goren QA Automation Tech Lead

Who we are
• More than 82 million registered users
• 28 million family trees
• Website, mobile apps (iOS and Android), desktop
apps (windows and Mac)
• 2.6 billion profiles (names in family trees)
• 7 billion historical documents and records. Including
the world’s largest collection of newspapers
• 200 million photos
• 42 languages
• 275 employees

18
Where we were
• All manual QA
• Repetitive tasks
• Time consuming
• Two service packs a week
• Holding back development
• SPC size limits error handling

Where we are today

• Number of tests
• Desktop Chrome: 650 (469)
• Desktop IE: 552 (0)
• Mobile-web 90 (0)
• Initial test suites for iOS and Android apps
• Runtime of full suite: 35 min for chrome + 32 min for all
mobile-web devices (51 min)
• Grids: 4 (1)
• Tests run on prod and semi staging env
• Continuous Deployment

19
“In software, when something is painful, the way to reduce
the pain is to do it more frequently, not less.”
- David Farley

The CD mindset

• We moved to CD a year ago

• On average we distribute code to production 25 times
per day
• Comes with challenges
• Automated tests on every code release
• Only minimal tests on each release
• Dev is responsible to run appropriate tests PRE-RELEASE
in staging

20
RND Goals
• Risk reduction to production
• Small incremental changes are easier to monitor and
revert in necessary
• Has lower impact on the system
• Increasing R&D velocity
• Avoiding wasted time on merges and complex
coordination before dist
• Allow R&D and product to experiment and innovate
more frequently

QA Goals
• Provide a safety net
• Avoid the bottleneck of manual sanity testing
• Reliability
• No flaky tests
• No false failures
• QA E2E test should not be the new bottleneck

21
CD Flow (simplified)

Start CD flow Dist

Run Unit Tests

Post-dist
E2E tests

Canary Pre-dist
Server E2E tests

• Continuous Deployment Applied at MyHeritage

– SlideShare

Responsibilities
• QA are responsible for E2E tests only
• Dev write and maintain Unit tests
• E2E tests are written during development process
• All QA members monitor CD suite results
• CD E2E tests are NOT acceptance tests

22
Timing
• CD flow from commit to prod is ~25 minutes
• Current QA CD suite takes 3 minutes on average
• Runs in parallel to some of the jobs in the CD flow
• Limited by time frame of parallel jobs
• Still have room for further growth

Challenges of CD
And how we face them

• Minimal suite – No full regression on commit

• Each Dev is responsible to run the desired
Automated regression tests
• Reliability
• Monitor a new test before adding it to the CD
suite
• Velocity of tests
• Run parallel tests (currently up to 16)
• Write new CD dedicated tests if existing tests are
too long
• Add hardware if needed

23
Challenges of CD cont.

• Failure visibility
• Slack messages for individual failed tests
• Browser console logs in report
• Screenshots in report
• Post dist – production server logs in the report
(Using Elasticsearch)
• Large scale tests refactoring due to changes
• Ability to run tests with a patch instead of
committing code that doesn’t correspond to
production

Challenges of CD cont.

• CD halt on failure
• Fast response time for failure. All QA members
monitor the relevant channel in slack
• Fast adaptation to feature flags
• Implemented feature flags for tests

24
Challenges of CD cont.

• Handling A\B tests

• All AB tests can be controlled with a cookie.
• All tests start with control-group variants set in the
cookies
• If needed, set a desired variant by replacing the
cookie.

“You don't have to be a genius or a visionary or

even a college graduate to be successful. You just
need a framework and a dream.”
- Michael Dell

Gray Area for full image

25
Framework
• Ruby
• Cucumber.io
• Page-Object Gem
• Grid Grouper
• Cuke_Slicer gem
• Jenkins
• Watir-webdriver
• Selenium Grid/Grid Extras
• Appium
• Appium_lib
• Xcode simulator

Ruby
• Easy language for inexperienced developers
• Open source community is very active
• It ain’t Java!

26
 Watir-webdriver
• Native to Ruby
• Cleaner than Selenium
• Adds methods not native to Selenium
• “Being able to select an element by a explicit
identifier” –watirmelon.com

Page-Object gem
• Provides a simple interface to define and interact with
elements on a page
• Works with both Watir and Selenium
• Simple way of introducing OOP without a lot of technical
knowledge required
• Page factory module handles common page classes
action

27
Cucumber.io
• BDD tool
• Tests are written in plain English
• Makes for easy debugging
• Easy move from Manual test to automatic test
• Helps soften the introduction to programming
• “Top down” programming
• Write a step
• Define the step
• Make it work
• Pre and post test hooks

Cucumber Feature
Tags specify the suites

Directly with Watir-Webdriver

Using page-object

28
Step Definitions
Directly with Watir-Webdriver

Using page-object

Page-object example
Defining page-object

Verify page loaded with

page-object

Using page-object

29
Native Mobile Apps
• Tests run nightly on the build or on demand
• Using two Mac minis as test machines
• Device parameters are passed from Jenkins (or
configuration if running locally)
• The apps are built on the tests machines and deployed
to the simulator\emulator
• Support for iOS and Android within existing infra
• iOS simulator, Android emulator and Appium server are
opened by command from within the code

Native Mobile Apps

• Support for iOS and Android within existing infra

30
Native Mobile Apps
• iOS simulator, Android emulator and Appium server are
opened by command from within the code

Native Mobile Test Flow

• Start E2E flow
• Build App
• Start test flow
• Open simulator\emulator
• Open Appium server
• Start Appium driver
• Perform test
• Close Appium driver
• Close Appium server
• Build Report
• Send slack notifications and emails

31
Grid Grouper
• Homegrown tool
• Takes a given suite and distributes the test across several
groups
• Utilizes cuke_slicer gem
• Checks the feature flags and filters out tests that are
turned off
• Multiple reports problem is solved by a report merger
tool that was developed in house

Jenkins
• Chuck Norris!
• Was already in use by RND
• Open source (lot’s of available plugins)
• Schedule automated builds
• Kick off manual builds with a few clicks
• Control build parameters (server, suite, etc)
• Can build in any of our environments with any
combination of tags
• Holds build history for a specified amount of time
• Merged report is stored in each build
• Slack and email notifications

32
The Grids
• Comprised of Selenium Grid and Selenium-Grid-Extras
• Selenium-Grid-Extras makes the basic setup streamlined
• Gives extra features such as:
• Pulling in updated drivers for all browsers
• Also for Selenium itself

How we use the grids

• We have 4 grids
• Each grid has 4 VMs
• The VMs include 1 hub-node and 3 other nodes
• The hubs are Jenkins slaves
• Each node can run 4 parallel tests on Chrome
• One grid is dedicated to CD
• We can pull a grid “offline” for testing infra upgrades
• Ability to have multiple test runs in parallel on different
environments and tags
• Only runs through Jenkins
• Jenkins selects grids automatically based on
availability and run history

33
“A challenge only becomes an obstacle when
you bow to it.”
- Ray A. Davis

Challenges QA faces

• Maintaining production data integrity

• Reliance on pre-existing data
• Running on local staging env
• Identifying failure reasons
• Need a true BDD/TDD mindset

34
Questions?

We are hiring

Thanks!
AA Milne

35
Mobile Testing
Open(ish) Source Solutions

https://www.youtube.com/watch?v=p8A4yq
N4_9A

Mobile Testing
Open(ish) Source Solutions

We ain’t gonna talk about ringtones…

111
 What’s ahead

• Mobile evolution
• Motivation
• Define UI Testing (with mobile emphasis)
• UI Mobile testing – Than VS Now

Mobile evolution

112
Mobile evolution

Mobile evolution

113
Mobile evolution

Mobile evolution

114
Motivation

Motivation

115
Motivation

Motivation

116
Motivation

Motivation

117
 UI Testing
Experimental
manual tests
The higher we
go
End
QA/
• More unreliability to End Automation
• Greater cost
• Slower feedback /GUI Teams

Integration/
API
R&D
Unit Tests

Define UI Testing

• Functional Black box testing

• Running on actual (or as close as possible)
device
• Manual Repetitive/Time/Cost

118
 UI Testing – Then

• Image based

UI Testing – Then

119
 UI Testing – Then

UI Testing – Then

• Baseline per resolution

• Non adaptive for changes
• Restriction to single IDE
• Many devices
• Hybrid apps treated the same

120
 UI Testing – Now

• Instrumentation based
• Virtualization support, better simulators
• Android UiAutomator/IOS UIAutomation
Frameworks
• Requires signature/permission

UI Testing – Now

• UI Automator Viewer

121
 UI Testing – Now

• Appium
– Allows to locate UI elements and perform
various operations on them
– Multiple OS Support (Win, Lin, Mac)
– Supports native and hybrid applications
– Supports iOS and Android
– Supports real devices and simulators

UI Testing – Now

122
 UI Testing – Now

DesiredCapabilities capabilities = new DesiredCapabilities();

capabilities.setCapability("platformName", "Android");
capabilities.setCapability("app","src/test/resources/com.twitter.android_5.5.0.apk");
capabilities.setCapability("app-package", "com.twitter.android");
capabilities.setCapability("app-activity", ".LoginActivity");

AppiumDriver driver = new AppiumDriver(new URL("http://127.0.0.1:4723/wd/hub"),

capabilities);

//Find the element in page using a selector locating strategy

WebElement element = driver.findElement(MobileBy.className("android.widget.EditText"));

//Perform operation on the found element

element.sendKeys("jsystemdemo");

UI Testing – Now

123
 UI Testing – Experitest

• Utilize cloud Devices

• Write code and run it using SeeTest on
hosted devices
• Build your own Device lab
• Network Virtualization

https://youtu.be/fWM_DBQ95p8

UI Testing – SauceLabs

• Appium Hub
• Run your code against multiple devices
online
• No need to configure prior testing
• Manual Testing

https://youtu.be/kCJwwRXUTdo

124
 UI Testing – GenyMotion

• Local Running android devices

• Faster emulations thanks to usage of GPU
• Must have devices configured
• Demanding

https://www.youtube.com/watch?v=50PR7
ocy_IU

Page Object Design Pattern

• Within your application UI there are areas that your

tests interact with.
A Page Object simply models these as objects within
the test code.
• The Page Object’s methods represent the services
offered by a particular page.
• The page object encapsulate the UI elements. if the UI
changes, the fix need only be applied in one place.

125
 Resources

http://powerlisting.wikia.com/wiki/File:Devolution_of_Man.jpg
http://www.geek.com/mobile/evolution-of-the-cell-phone-depicted-in-papercraft-1309666/
http://www.hexapolis.com/2015/07/28/12-video-game-franchises-then-and-now-the-graphics-evolution/
App Annie, BI Inteligence
http://worthstuff.blogspot.com/2010/05/lg-ally-mobile-phone-with-slide-out.html
http://www.kinvey.com/how-long-to-build-an-app-infographic/
http://www.kinvey.com/wp-content/uploads/2013/01/how-long-does-it-take-to-build-an-app-kinvey-1500x834.png
http://www.factslides.com/s-Mobile-Phones
http://jwtyler.blogspot.co.il/2010_10_01_archive.html
http://www.androidauthority.com/nokia-qwerty-keyboar-107145/
http://www.formotus.com/14018/blog-mobility/figuring-the-costs-of-custom-mobile-business-app-development
http://decor10blog.com/design-decorate/decorating-ideas/apple-design-team-abandoning-sketchbooks-for-the-apple-
pencil-says-jonathan-ive.html
https://www.quora.com/Why-Microsoft-failed-at-mobile

126
 The Road To Automated Testing
Tal Tsach-Cohen
QA Manager Hexagon Technologies

About Me

• In QA 6 Years
• Studied .Net 2yrs at SELA while working
• Introduced automation to company when QA
was 2 members. Today I manage a team of
14.
• Previously was a chef in San Francisco for 8
years. This experience was more relevant than
you would think.

127
 Who Are We?

• Development division for leading Forex

Company.
• 70 Employees – 15 QA
• B2B, B2C, CRM, CMS, Web, Mobile
(iOS, Android) currently 2 brands for all
applications.
• All developed in house.

What Does QA Look Like?

• 15 Members.
• 5 Teams
• 7 Automation, 7 Manual including team
leaders.
• 4000 tests run nightly.

128
 Components
•.Net C#
•Selenium, Appium, Applitools, Git, NUnit
•All frameworks developed in house page object model for GUI.
•2 Solutions Server and Client systems.
•Same tests for both web applications, and same tests for iOS and
Android only element mapping is changed via configuration files.
•Jenkins for deploy to QA and running tests.
•Custom reporter including video, HTML Report and integration with
Elastic Search.
•Test analysis with Kibana and Elastic Search.
•Notifications via Slack.

Why C#?

•At time of birth of automation .Net was 95% of

existing platform.
•Existing in house knowledge and libraries in C#.
•Built in load testing framework with visual studio
ultimate edition.
•In retrospect would probably go with Java.

129
 Why Jenkins?

•Open Source, easy to use.

•QA was first to implement automated deployment in Jenkins.
•Efficient for creating automated test jobs which can be
executed by manual testers/developers as needed.
•Integration with email, slack, Jira.

Challenges Specific To Hexagon

•Off the clock POC was needed to demonstrate need for investing in
automation.
•System is extremely complex, upwards of 50 components working
together making stability an issue.
•Core components have no unit testing so E2E testing was needed to
make up for deficit.
•Due to system complexity and need for known system wide state running
parallel tests on core business component (Trading Platform) is almost
impossible.
•Multitude of technologies used in system require high competency of
automation developers. Oracle DB, redis, kafka, couchDB, mongoDB,
NodeJS, Cassandra, Linux, ZMQ and more…

130
 More Challenges Specific To Hexagon

•Server side automation team (No GUI) needs hard to find skill set of high
level developer with QA orientation.
•Automation developers are responsible for writing code as well as
defining tests. Finding testers with aptitude for both disciplines has been a
challenge.
•Inability to use external solutions due to lack of access to QA
environments from outside LAN.

Where We Are Today

•Server side testing
– ~ 2500 tests written
– 2000 part of nightly regression
– 900 run parallel in first 15 minutes
– Regression done in 6 hours
– 95% coverage
•B2B – CRM Testing
– 1300 Tests all run nightly
– 89% First run pass 22% in second pass on failed tests
– 80% coverage of component
– We are working on parallelizing the tests should be 4 hours when done.
•Web – Trading and Website
– 800 tests nightly regression
– 6 hours chrome 75% first run pass 36% pass on failed run pass
– 11 hours firefox 63% first run pass 39% pass on failed run pass
– 6 hours explorer 45% first run pass 52% pass on failed run pass

131
 Where We Are Today

•Mobile testing started this week

– 62 tests 30% regression coverage
– 88% pass rate, 50% on second pass.
– 1.5 hours

132
133
134
135
 What Is Next?

• Image Recognition for GUI design validation.

• Improve Explorer browser stability, Edge
testing.
• Jmeter load testing, and browser performance
testing. We use ZMQ protocol between
servers, Created plugin for our purposes.

136
137
 Page Object Design Pattern – De-facto
Standard for Test Automation Projects

Itai Agmon

What is software architecture?

“Software application architecture is the process of defining a

structured solution that meets all of the technical and
operational requirements, while optimizing common quality
attributes such as performance, security, and
manageability. It involves a series of decisions based on a
wide range of factors, and each of these decisions can
have considerable impact on the quality, performance,
maintainability, and overall success of the application.”
Microsoft

138
139
Image courtesy of Stuart Miles at FreeDigitalPhotos.net

First Things First

• I am not going to talk about Selenium

• The examples are in Java
• There is a lot of code involved. Can be
downloaded from here
• https://github.com/itaiag/page-objects-
training

140
 So, What is That Page Object Thingy Anyway?

• Architecture for GUI test automation

• Mostly used for web application testing
• I use it for every GUI (Desktop, Web, Mobile
and even cli menus)
• Requires OO language
• De facto industry standard.
• It is a design pattern, not recipe.

141
 How About Those Record and Playback technologies?

@Test
public void testRegisterAndLogin() throws Exception {
// Register new user
driver.findElement(By.linkText("Register")).click();
driver.findElement(By.id("firstName")).sendKeys(FIRST_NAME);
driver.findElement(By.name("lastName")).sendKeys(LAST_NAME);
String userName = FIRST_NAME + String.valueOf(System.currentTimeMillis());
driver.findElement(By.id("username")).sendKeys(userName);
driver.findElement(By.id("password")).sendKeys(PASSWORD);
driver.findElement(By.tagName("button")).click();
wait.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By
.xpath("//div[text()='Registration successful']")));

// Login
driver.findElement(By.id("username")).sendKeys(userName);
driver.findElement(By.id("password")).sendKeys(PASSWORD);
driver.findElement(By.tagName("button")).click();

wait.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By
.xpath("//h1[text()='Hi " + FIRST_NAME + "!']")));
driver.findElement(By.linkText("Logout")).click();

142
 No Biggy, Let’s Use Functions

@Test
public void testRegisterAndLogin() throws Exception {
String userName = registerNewUser(FIRST_NAME, LAST_NAME,
PASSWORD);
Login(userName, PASSWORD);
logout();
}
private void logout() {
driver.findElement(By.linkText("Logout")).click();
}

private void Login(String userName, String password) {

driver.findElement(By.id("username")).sendKeys(userName);
driver.findElement(By.id("password")).sendKeys(password);
driver.findElement(By.tagName("button")).click();
wait.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.xpath("//h1[text()='Hi " + FIRST_NAME + "!']")));
}

private String registerNewUser(String firstName, String lastName, String password) {

// Register new user
driver.findElement(By.linkText("Register")).click();
driver.findElement(By.id("firstName")).sendKeys(firstName);
driver.findElement(By.name("lastName")).sendKeys(lastName);
String userName = FIRST_NAME + String.valueOf(System.currentTimeMillis());
driver.findElement(By.id("username")).sendKeys(userName);
driver.findElement(By.id("password")).sendKeys(password);
driver.findElement(By.tagName("button")).click();
wait.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By
.xpath("//div[text()='Registration successful']")));
return userName;
}

Functions Are Words

Image courtesy of Stuart Miles at FreeDigitalPhotos.net

143
 Granularity – Let’s Talk Negative

//I can do this

loginAndSuccess(user, password);
loginAndFail(user, password);

//Or this
login(user, password, expectSuccess);

//Or maybe this

login(user, password);
assertInHomePage();
assertInLoginPage();

Image courtesy of artur84 at FreeDigitalPhotos.net

Image courtesy of Bill Longshaw at FreeDigitalPhotos.net

144
 Page Object Pattern

public class LoginPage extends AbstractPage {

public LoginPage(WebDriver driver) {

super(driver);
}

public void typeToUserNameTb(String userName) {

driver.findElement(By.id("username")).sendKeys(userName);
}

public void typeToPasswordTb(String password) {

driver.findElement(By.id("password")).sendKeys(password);
}

public void clickOnLoginBtn() {

driver.findElement(By.tagName("button")).click();
}

public void clickOnRegisterLnk() {

driver.findElement(By.linkText("Register")).click();
}

The Test

@Test
public void testRegisterAndLogin() throws Exception {
LoginPage loginPage = new LoginPage(driver);
loginPage.clickOnRegisterLnk();

RegisterPage registerPage = new RegisterPage(driver);

registerPage.typeToFirstName(FIRST_NAME);
registerPage.typeToLastName(LAST_NAME);
String userName = FIRST_NAME + String.valueOf(System.currentTimeMillis());
registerPage.typeToUserName(userName);
registerPage.typeToPasswordTb(PASSWORD);
registerPage.clickOnRegisterBtn();

// OH MY GOD... DON'T USE SLEEP!!!

Thread.sleep(1000);

loginPage = new LoginPage(driver);

loginPage.typeToUserNameTb(userName);
loginPage.typeToPasswordTb(PASSWORD);
loginPage.clickOnLoginBtn();

HomePage homePage = new HomePage(driver);

homePage.clickOnLogoutBtn();

145
 Navigation

Image courtesy http://www.webseoanalytics.com/blog/link-structure-analyzing-the-most-important-methods/

…Adding Navigations

RegisterPage
public LoginPage clickOnRegisterBtn() {
driver.findElement(By.tagName("button")).click();
return new LoginPage(driver);
}

TestRegistration
@Test
public void testRegisterAndLogin() throws Exception {
LoginPage loginPage = new LoginPage(driver);
RegisterPage registerPage = loginPage.clickOnRegisterLnk();

registerPage.typeToFirstName(FIRST_NAME);
registerPage.typeToLastName(LAST_NAME);
String userName = FIRST_NAME + String.valueOf(System.currentTimeMillis()); And what about
registerPage.typeToUserName(userName); negative tests?
registerPage.typeToPasswordTb(PASSWORD);
loginPage = registerPage.clickOnRegisterBtn();

loginPage.typeToUserNameTb(userName);
loginPage.typeToPasswordTb(PASSWORD);
HomePage homePage = loginPage.clickOnLoginBtn();

homePage.clickOnLogoutBtn();

146
 Negative Test

@Test
public void testLoginWithWrongPassword(){
LoginPage loginPage = new LoginPage(driver);
loginPage.typeToUserNameTb(FIRST_NAME);
loginPage.typeToPasswordTb("WRONG PASSWORD");
loginPage.clickOnLoginBtnAndStayInLoginPage();

@Test
public void testSuccessfullLogin(){
LoginPage loginPage = new LoginPage(driver);
loginPage.typeToUserNameTb(FIRST_NAME);
loginPage.typeToPasswordTb(PASSWORD);
HomePage homePage = loginPage.clickOnLoginBtnAndGoToHomePage();
}
What do you
actually test
here ?

Image courtesy of graur codrin at FreeDigitalPhotos.net

There Are Two Kinds of Assertions

@Test
public void testLoginWithWrongPassword() throws Exception {
LoginPage loginPage = new LoginPage(driver);
loginPage.typeToUserNameTb(FIRST_NAME);
loginPage.typeToPasswordTb("WRONG PASSWORD");
loginPage. clickOnLoginBtnAndStayInLoginPage();

loginPage.assertAlertMessage("Username or password is incorrect");

}

@Test
public void testLoginWithWrongPassword() {
LoginPage loginPage = new LoginPage(driver);
loginPage.typeToUserNameTb(FIRST_NAME);
loginPage.typeToPasswordTb("WRONG PASSWORD");
loginPage.clickOnLoginBtnAndStayInLoginPage();

String text = loginPage.waitForAlertMessage();

Assert.assertEquals("Failed login message is incorrect",
"Username or password is incorrect", text);
}

Image courtesy of Stuart Miles at FreeDigitalPhotos.net

147
 We Sure Like Things Organized
public class LoginPage extends AbstractPage {

private By userNameBy = By.id("username");

private By passwordBy = By.id("password");
private By loginBtnBy = By.tagName("button");
private By registerLnkBy = By.linkText("Register");
private By alertDivBy = By.className("alert");

public LoginPage(WebDriver driver) {

super(driver);
}

public HomePage clickOnLoginBtnAndGoToHomePage() {

driver.findElement(loginBtnBy).click();
return new HomePage(driver);
}

public void clickOnLoginBtnAndDoNotLogin() {

driver.findElement(loginBtnBy).click();
}
}

If You Don’t Have Anything to Return, Return Yourself

LoginPage
public LoginPage typeToUserNameTb(String userName) {
driver.findElement(userNameBy).sendKeys(userName);
return this;
}

public LoginPage typeToPasswordTb(String password) {

driver.findElement(passwordBy).sendKeys(password);
return this;
}

public HomePage clickOnLoginBtnAndGoToHomePage() {

driver.findElement(loginBtnBy).click();
return new HomePage(driver);
}

public LoginPage clickOnLoginBtnAndDoNotLogin() {

driver.findElement(loginBtnBy).click();
return this;
}

148
 We Now Have a Fluent Test

@Test
public void testRegisterAndLogin() throws Exception {
LoginPage loginPage = new LoginPage(driver);
RegisterPage registerPage = loginPage.
clickOnRegisterLnkAndGoToRegisterPage();

String userName = FIRST_NAME + String.valueOf(System.currentTimeMillis());

loginPage = registerPage.
typeToFirstName(FIRST_NAME).
typeToLastName(LAST_NAME).
typeToUserName(userName).
typeToPasswordTb(PASSWORD).
clickOnRegisterBtnAndGoToLoginPage();

HomePage homePage = loginPage.

typeToUserNameTb(userName).
typeToPasswordTb(PASSWORD).
clickOnLoginBtnAndGoToHomePage();

homePage.clickOnLogoutBtnAndGoToLoginPage();

Binding Functions Together

@Test
public void testRegisterAndLogin() {
LoginPage loginPage = new LoginPage(driver);
RegisterPage registerPage = loginPage.clickOnRegisterLnkAndGoToRegisterPage();

String userName = FIRST_NAME + String.valueOf(System.currentTimeMillis());

loginPage = registerPage.doRegistration(FIRST_NAME, LAST_NAME, userName,
PASSWORD);

HomePage homePage = loginPage.doLogin(userName, PASSWORD);

homePage.clickOnLogoutBtnAndGoToLoginPage();

149
Different Browsers, Different Problems

• Not always click

• Can’t delete cookies

• Doesn’t scroll to links out of view

Layered Architecture

Tests Layer

Page Objects Layer

ActionBot

150
 Meet ActionBot

public class ActionBot {

private final WebDriver driver;

public ActionBot(WebDriver driver){

this.driver = driver;
}

public void clickOnElementLocatedBy(By by){

WebElement element = driver.findElement(by);
element.click();
}

public void sendKeysToElementLocatedBy(By by,String keys){

WebElement element = driver.findElement(by);
element.clear();
element.sendKeys(keys);
}

Image courtesy of Simon Howden at FreeDigitalPhotos.net

Q&A

151
152
 Tips and Tricks Appium Via Selenium

June 2016

On The Agenda

Intro of Gett & Automation FrameWork

Intro of Selenium Page Object Design Pattern
Examples of use
Web-Locators Tricks for Object Repository
Intro for mobile Client Automation infra
Drill Down into ios Platform using Appium
Example of Use for webDriver on AppiumDriver

153
Some Numbers

$520 700 50,000+

Million Employees Taxis Running
In Founding On Gett Technology

4000+ 70 30,000,000+
Enterprise clients Cities Passengers
Around the globe On all platforms

Automation
Challenges

★ Multi platforms under same test

★ Challenge to manage small testable units

★ Challenge managing test run time

★ Location based product

154
Gett Automation FrameWork

Automation Platforms

GUI Platform:
→ Android - client and Driver app
→ Iphone DBX and client
→ Web -
● Company web
● Customer Care
● Private Web
● Mobile Web
Headles
Hybrid

155
Automation periodic runs

•Headless E2E - Main Server - content related - every pull request

•Headless Regression - Nightly run

•Api tests - micro services related - every pull request

•GUI Regression - every regression phase in Sprint

•API Regression - every regression phase in Sprint

Load Zone

●Load Infra inherit / based On Automation API infra

●Automation script with Jmeter adaptations run on Jmeter tool
Advantages:
→ Maintenance in automation zone reflect automatically on the load zone
→ Easy to create test scenarios
→ Long terms of use
→ Load is no longer belong to the load professionals …
→ Inherit Jmeters strength to load the system

156
 Gett Page Object Design pattern within interfaces

Page object is a design pattern that can be implemented as a selenium best practices.

• The Page Object pattern represents the screens of your web app as a series of objects and
encapsulates the features represented by a page.

• It allows us to model the UI in our tests.

• A page object is an object-oriented class that serves as an interface to a page of your AUT.

• Using common wrapper class for basic web functionality is recommended

• When we have page with the same functionality/ Object in two different pages u se interfase

Gett Page Object Design

pattern Tips

Using - AbstractPage

Inside - AbstractPage class

• AbstractPage Class is the place to holds all the wrappers for selenium methods
• Abstract Methods serves all webPages and contain only baseWeb Functionality regardless Buisness logic

157
 Gett Web Page Design pattern within interfaces

inherit
AbstractPage()

inherit
AbstractPage()

Gett Page Object Design pattern within interfaces

• The Use in Abstract class - implement the interface for common Functionality && common objects

The Middle abstract serves the

Common TariffAbstract{} Two Implements for
TariffService{}

inherit
AbstractPage()

158
 Gett Page Object Design pattern within interfaces
TariffService class

The Interface - for Tariff functionality that implement by two different pages

Gett Page Object Design

Pattern within interfaces - TariffAbstract class

inside TariffAbstract class

Not a common Method

A Common Method

• The method FillFixChargeForm() is not common

• The method setFixTariffrFromAddressDetailes() is common and also part of the calls in the above

159
Gett Page Object Design PhoneUserTariffServiceImpl class

Pattern - within interfaces

• The method FillFixChargeForm() contain its unique functionality and also use
setFixTariffrFromAddressDetailes()

Gett Page Object Design Pattern

Tip - isInModule

• Every Page holds Sync point to be sure that we in the correct place and the page was loaded
• Wait contain polling - until object found limit by X time !!!
• In case of Failure throws exception

160
Tip - Web Locator and Object Repository - mapping the objects

WebLocator Class
Web Locators Business Logic

Repository

Hold one focal point that capture the

locators Differentiate by logical
Zone and name for objects

PhoneUserTariffServiceImpl class
Page object Design Pattern -
Main Rules

Transition to the Next class

FixPriceDetailsPage

• Every page implement in a class

• Every Page get the selenium driver and use it to perform actions on objects
• Every Page pass it’s Driver to the new Page in the flow

161
 Mobile Client Automation FrameWork

Appium is “cross-platform”: it allows you to write tests against multiple platforms (iOS,
Android), using the same API , based on Selenium API .

Tip - One interface for Business

logic per client

• The test use one interface object that implements differents platform - Main interface obj.
• The main business logic object perform all the business logic calls in the system

162
 Trick -The Implementor for
IOS platform - Appium Init

iphoneDriver = getTaxiAppium.connect();
driver = iphoneDriver;
// AbstractPage by passing to it the webDriver ;

pageImpl= new AppiumDriverPageImpl(driver)

Appium Driver PageImpl class

163
The Implementor for IOS
platform - Appium
pageImpl.clickOnButton.

(iphoneButtons.get(getTaxiButton.name()));

Inside AbstractPage Class

• Example of using the obj “pageImpl” to click on btn using selenium Api
• The Method iphoneButtons.get() transform the the getTaxiButton into our the WebLocatore Object

The Implementor for IOS

platform - Appium

pageImpl.selectAutoCompleteOptionWithMouse

(iphoneButtons.get(GetTaxiButton.OK.name()));

• Example of using the obj “pageImpl” to click on Mouse using Events selenium Api

164
WE ARE HIRING
● QA Testers

● Automation developers

● Architect

● Fullstack Web Developer

● iOS / Android

● Product Manager

● Product Analyst

● BI

● DevOps

● Big Data

● UX/UI designers

165
 Reports
Get the most out of your tests

“Not only must automation be

done; it must also be seen to be
done”
someone, somewhere

166
Who Cares About Reports?

167
 JSystem

• My favorite test automation framework

• I’m the main project contributor
• Used in most of my company’s projects

168
 It has some drawbacks

• Ugly
• Confusing
• Slow
• Unmaintainable
• Hard to customize

169
 JUnit Example
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<testsuite errors="1" failures="1" hostName="agmon-PC" name="JSystem Reporter" tests="9" time="1022.419" timeStamp="Sun May 29
11:13:29 IDT 2016">
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="reportError" time="0.227">
<error message="Error" type="class java.lang.Exception">java.lang.Exception: Error
at jsystem.framework.report.RunnerListenersManager.addError(RunnerListenersManager.java:355)
at jsystem.runner.remote.RemoteExecutorImpl$ReaderThread.run(RemoteExecutorImpl.java:368)
</error>
</testcase>
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="controlParametersAttributes" time="0.173" />
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="testWithParameterProvider" time="0.205" />
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="reportWithLevels" time="0.224" />
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="testWithReturnParameter" time="0.2" />
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="testWithParameters" time="0.161" />
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="reportWarning" time="0.136" />
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="reportFailure" time="0.142">
<failure message="Fail report was submitted" type="class
junit.framework.AssertionFailedError">junit.framework.AssertionFailedError: Fail report was submitted
at jsystem.framework.report.RunnerListenersManager.addFailure(RunnerListenersManager.java:466)
at jsystem.runner.remote.RemoteExecutorImpl$ReaderThread.run(RemoteExecutorImpl.java:373)
</failure>
</testcase>
<testcase classname="il.co.topq.auto.tests_project.TestsExamples" name="reportSuccess" time="0.105" />
</testsuite>

Cucumber Example

170
 TestNG Example

ReportNG Example

171
 So I decided to write something on my own
It should be:
• Real time!
• Easy on the eye
• Fast
• Flexible – Add screenshot, files, tables.
• No server
• It should have a weird name

Frameworks and Languages

I also work with a few frameworks and

languages:
• C# with NUnit and MSTest
• Python with PYTest and Nose
• Java with TestNG and JSystem
• Javascript with Jasmine

172
 Difido Architecture

• At that point, I didn’t know much about

Javascript.
• But I did know that MVC is the way to go
Controller

View Model

Difido Architecture

173
 Demo of HTML report

Now I want a server

Requirements:
• An addition to the local reports
• Central web interface
• Keep history
• Join results from different stations
• Real time, real time, real time
• Damn fast
• Easy to deploy

174
Difido Server Architecture

Difido Server Demo

175
 How about BI?

• We gathered a lot of data. Can I do

something with it?

Elasticsearch

• Open source
• Java Based
• NoSQL
• Very Scalable (Peta bytes of data) Shay Banon

• Includes a text search engine (Lucene)

• Part of the ELK stack

176
Difido Server Architecture

Difido Server Architecture

177
 More Features

I needed some more features

• Send mail at the end of the run
• Delete old reports

Success Stories

178
 You Can Use It too

• Site: http://top-q.github.io/difido-reports/
• Documentation: https://github.com/Top-Q/difido-
reports/wiki
• Github: https://github.com/Top-Q/difido-reports

Q&A

179
180