Download as PDF, PPTX
![10
MochaJS with Webdriver Wrapper
Provided Mocha test wrapper with Promise Manager
Selenium-webdriver’s wrapper for mocha methods that automatically handles all calls into the
promise manager which makes the code very sync like.
var test = require('selenium-webdriver/testing');
var webdriver = require('selenium-webdriver');
var By = require('selenium-webdriver').By;
var Until = require('selenium-webdriver').until;
test.it('Login and make sure the job menu is there', function() {
driver.get(url, 5000);
driver.findElement(By.css('input#email')).sendKeys('useremail@email.com');
driver.findElement(By.css('input#password')).sendKeys(password);
driver.findElement(By.css('button[type="submit"]')).click();
driver.wait(Until.elementLocated(By.css('li.active > a.jobs')));
var job = driver.findElement(By.css('li.active a.jobs'));
job.getText().then(function (text) {
assert.equal(text, 'Jobs', 'Job link title is correct');
});
});
Mocha wrapper makes
the code very
“synchronous” like.](https://image.slidesharecdn.com/airwareseleniummeetup-end-to-endtestautomationwithnode-161020054638/75/Fullstack-End-to-end-test-automation-with-Node-js-one-year-later-10-2048.jpg)
![16
JobsPage.prototype.getJobList = function () {
this.waitForDisplayed(By.css('.job-table'));
const jobNames = [];
const defer = promise.defer();
const flow = promise.controlFlow();
const jobList = this.driver.findElement(By.css('.job-table'));
// get entries
flow.execute(() => {
jobList.findElements(By.css('.show-pointer')).then((jobs) => {
// Get text on all the elements
jobs.forEach((job) => {
let jobName;
flow.execute(function () {
job.findElement(By.css('.job-table-row-name')).then((element) => {
element.getText().then((text) => {
jobName = text;
});
// look up more table cells...
});
}).then(() => {
jobNames.push({
jobName: jobName
});
});
});
});
}).then(() => {
// fulfill results
defer.fulfill(jobNames);
});
return defer.promise;
};
...and the Really Ugly
Not planning for complexity + promise chaining =
We want our tests to be:
● Readable / Flat structure ❌
● Agnostic / Context-free ❌
● ‘De-asynchronous’ ✔
● In line with ECMA standards ❌](https://image.slidesharecdn.com/airwareseleniummeetup-end-to-endtestautomationwithnode-161020054638/75/Fullstack-End-to-end-test-automation-with-Node-js-one-year-later-16-2048.jpg)
Uploaded byMek Srunyu Stittri
Fullstack End-to-end test automation with Node.js, one year later
The document discusses the evolution of end-to-end test automation using Node.js, focusing on the advancements made since a presentation at the San Francisco Selenium Meetup in 2015. Key highlights include the transition from Selenium WebDriver's control flow to async/await for cleaner, more readable code, as well as the integration of type-safe JavaScript with Facebook's Flow type library. Future strategies involve employing robust visual testing methods and improving reporting mechanisms to enhance developer efficiency.
More Related Content
![[CLIW] Web testing](https://cdn.slidesharecdn.com/ss_thumbnails/cliw-120110000833-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Fullstack End-to-end test automation with Node.js, one year later
![Governance, Deployment & Methodologies for Agentic Automation [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day2-251112135038-a386476d-thumbnail.jpg?width=640&height=640&fit=bounds)
Fullstack End-to-end test automation with Node.js, one year later
- 1. Fullstack End-to-End Test Automationwith Node.js One year later SF Selenium Meetup @ Saucelabs Chris Clayman Mek Srunyu Stittri
- 2. 2 Agenda ● Background ○ Whatwe presented last year ● Async / Await - an alternative to Webdriver’s built-in control flow. ○ Limitations with control flow ○ Use Babel to write the latest ES6 JavaScript syntax. ES6 Pageobjects ● Extending MochaJS ○ Custom reporter with screenshots from Sauce Labs ○ Parallel tests and accurate reporting ● Type-safe JavaScript with Facebook’s Flow-type library. ● Robust visual diffs ● What’s next
- 3. 3 Background Back in 2015,we started looking at node.js for end-to-end functional test framework. ● Kept Node.js adoption in mind ○ More and more company moving to node.js ○ Share code with fullstack developers ● Team presented at San Francisco Selenium Meetup in Nov 2015 ○ Event - http://www.meetup.com/seleniumsanfrancisco/events/226089563/ ○ Recording - https://www.youtube.com/watch?v=CqeCUyoIEo8 ○ Slides - http://www.slideshare.net/MekSrunyuStittri/nodejs-and-selenium-webdriver-a-journey-from-the-java-side
- 4. 4 Why we chosenode.js QA, Automation engineers Frontend engineers Backend engineers Java Javascript Java Python Javascript Java Ruby Javascript Node.js Company A Company B Company C Node.js Javascript Node.js Go, Python Airware
- 5. 5 What we presentedlast year Input / update data Get data Input / update data Get data UI Framework : node.js ● selenium-webdriver ● mocha + wrapper ● Applitools ● co-wrap for webclient ● chai (asserts) Rest API Framework : node.js ● co-requests ● mocha ● co-mocha ● chai (asserts) ● json, jayschema WebClients Pageobjects Webclient adaptor Database Backend : Go, Python Browser Frontend : Javascript Microservice #1 Microservice #2 ... #n Rest APIs
- 6.
- 7.
- 8. 8 Heavily dependent onselenium-webdriver control flows http://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/lib/promise.html What is the promise manager / control flow ● Implicitly synchronizes asynchronous actions ● Coordinate the scheduling and execution of all commands. Maintains a queue of scheduled tasks and executing them. What we presented last year driver.get('http://www.google.com/ncr'); driver.findElement({name: 'q'}).sendKeys('webdriver'); driver.findElement({name: 'btnGn'}).click(); driver.get('http://www.google.com/ncr') .then(function() { return driver.findElement({name: 'q'}); }) .then(function(q) { return q.sendKeys('webdriver'); }) .then(function() { return driver.findElement({name: 'btnG'}); }) .then(function(btnG) { return btnG.click(); }); The core Webdriver API is built on top of the control flow, allowing users to write the below. Instead of thatThis
- 9. 9 Achieving Sync-like Code Codewritten using Webdriver Promise Manager JavaScript selenium tests using Promise Manager driver.get("http://www.google.com"); driver.findElement(webdriver.By.name('q')).sendKeys('webdriver'); driver.findElement(webdriver.By.name('btnG')).click(); driver.getTitle().then(function(title) { console.log(title); }); Equivalent Java code driver.get("http://www.google.com"); driver.findElement(By.name("q")).sendKeys("webdriver"); driver.findElement(By.name("btnG")).click(); assertEquals("webdriver - Google Search", driver.getTitle()); Hey, we look similar now!
- 10. 10 MochaJS with WebdriverWrapper Provided Mocha test wrapper with Promise Manager Selenium-webdriver’s wrapper for mocha methods that automatically handles all calls into the promise manager which makes the code very sync like. var test = require('selenium-webdriver/testing'); var webdriver = require('selenium-webdriver'); var By = require('selenium-webdriver').By; var Until = require('selenium-webdriver').until; test.it('Login and make sure the job menu is there', function() { driver.get(url, 5000); driver.findElement(By.css('input#email')).sendKeys('[email protected]'); driver.findElement(By.css('input#password')).sendKeys(password); driver.findElement(By.css('button[type="submit"]')).click(); driver.wait(Until.elementLocated(By.css('li.active > a.jobs'))); var job = driver.findElement(By.css('li.active a.jobs')); job.getText().then(function (text) { assert.equal(text, 'Jobs', 'Job link title is correct'); }); }); Mocha wrapper makes the code very “synchronous” like.
- 11. 11 test.it('Verify data fromboth frontend and backend', function() { var webClient = new WebClient(); var projectFromBackend; // API Portion of the test var flow = webdriver.promise.controlFlow(); flow.execute(function *(){ yield webClient.login(Constants.USER001_EMAIL, Constants.USER_PASSWORD); var projects = yield webClient.getProjects(); projectFromBackend = projectutil.getProjectByName(projects, Constants.QE_PROJECT); }); // UI Portion of the test var login = new LoginPage(driver); login.enterUserInfo(Constants.USER001_EMAIL, Constants.USER_PASSWORD); var topNav = new TopNav(driver); topNav.getProjects().then(function (projects){ Logger.debug('Projects from backend:', projectsFromBackend); Logger.debug('Projects from frontend:', projects); assert.equal(projectsFromBackend.size, projects.size); }); Heavily dependent on the promise manager / control flow Here we handle execution order including a generator call What We Presented Last Year Context switching to REST API calls
- 12. Alternatives to Webdriver’sBuilt-in Control Flow Async / Await
- 13. JS and Webdriver:the Good, the Bad... Good Stuff ● Functional programming ● More Collaboration with front-end development teams ● JavaScript Developers writing Selenium tests ● Fast paced open source community ● Able to build things really quick ● JavaScript is fun! Bad Stuff ● Webdriver’s Control Flow & Promise Manager ○ Not agnostic ○ Parent variable declarations ○ Iteration can be hacky ● Context switching between Webdriver and non-Webdriver asynchronous function calls
- 14. 14 ...and the Ugly constpromise = require('selenium-webdriver').promise PageObject.prototype.getMessages = function() { const els = this.driver.findElements(By.css('.classname'); const defer = promise.defer(); const flow = promise.controlFlow(); flow.execute(function* () { const textArray = yield els.map((el) => { return el.getText(); }); defer.fulfill(textArray); }); return defer.promise; } We want our tests to be: ● Readable / Flat structure ✔ ● Agnostic / Context-free ❌ ● De-asynchronous ✔ ● In line with ECMA standards ❌ Context Switch
- 15. 15 ...and the (kindof) Ugly test.it('Archives job', function () { const flow = promise.controlFlow(); let job; flow.execute(function* () { // Create a job through API job = yield webClient.createJob(); driverutil.goToJob(driver, job.id); }); const jobInfo = new ViewJob(driver); jobInfo.clickArchive(); jobInfo.isModalDisplayed().then((displayed) => { assert.isTrue(displayed, 'Modal should be displayed'); }); flow.execute(function* () { yield webClient.deleteJob(job.id); }); }); We want our tests to be: ● Readable / Flat structure ● Agnostic / Context-free ❌ ● De-asynchronous ✔ ● In line with ECMA standards ❌ Context Switch Context Switch
- 16. 16 JobsPage.prototype.getJobList = function() { this.waitForDisplayed(By.css('.job-table')); const jobNames = []; const defer = promise.defer(); const flow = promise.controlFlow(); const jobList = this.driver.findElement(By.css('.job-table')); // get entries flow.execute(() => { jobList.findElements(By.css('.show-pointer')).then((jobs) => { // Get text on all the elements jobs.forEach((job) => { let jobName; flow.execute(function () { job.findElement(By.css('.job-table-row-name')).then((element) => { element.getText().then((text) => { jobName = text; }); // look up more table cells... }); }).then(() => { jobNames.push({ jobName: jobName }); }); }); }); }).then(() => { // fulfill results defer.fulfill(jobNames); }); return defer.promise; }; ...and the Really Ugly Not planning for complexity + promise chaining = We want our tests to be: ● Readable / Flat structure ❌ ● Agnostic / Context-free ❌ ● ‘De-asynchronous’ ✔ ● In line with ECMA standards ❌
- 17. 17 As if wehad trainer wheels on... Doing complex things with selenium-webdriver promise manager ended up taking more time and being more cumbersome. What It Felt Like
- 18.
- 19. 19 Async/Await Introducing Async/Await ● ES2016language specification grabbed from C# ● Async functions awaits and returns a promise ● No callbacks, no control flow libraries, no promise chaining, nothing but simple syntax. ● ‘De-asynchronous’ done easy
- 20. 20 function notAsync (){ foo().then((bar) => { console.log(bar) }); } Async/Await async function isAsync() { const bar = await foo(); console.log(bar); } Given function foo() that returns a promise... ES5 Javascript ES2016 Latest Javascript
- 21. 21 PageObject.prototype.getMessages = function(){ const els = this.driver.findElements(By.css('.classname'); const defer = promise.defer(); const flow = promise.controlFlow(); flow.execute(function* () { const textArray = yield* els.map((el) => { return el.getText(); }); defer.fulfill(textArray); }); return defer.promise; } async getMessages() { const els = await this.driver.findElements(By.css('.classname'); return Promise.all(els.map((el) => { return el.getText(); })); } We want our tests to be ● Readable / Flat structure ✔ ● Portable / Context-free ✔ ● De-asynchronous ✔ ● In line with ECMA standards ✔ Promise Manager vs. Async/Await
- 22. 22 test.it('Archives job', function() { const flow = promise.controlFlow(); let job; flow.execute(function* () { // Create a job through API job = yield webClient.createJob(); driverutil.goToJob(driver, job.id); }); const jobInfo = new ViewJob(driver); jobInfo.clickArchive(); jobInfo.isModalDisplayed().then((displayed) => { assert.isTrue(displayed, 'Modal should be displayed'); }); flow.execute(function* () { yield webClient.deleteJob(job.id); }); }); Promise Manager vs. Async/Await it('Archives job', async function () { const job = await webClient.createJob(); await driverutil.goToJob(driver, job.id); const jobInfo = new ViewJob(driver); await jobInfo.clickArchive(); const displayed = await jobInfo.isModalDisplayed(); assert.isTrue(displayed, 'Modal should be displayed'); await webClient.deleteJob(job.id); }); We want our tests to be ● Readable / Flat structure ✔ ● Portable / Context-free ✔ ● De-asynchronous ✔ ● In line with ECMA standards ✔
- 23. 23 How to useAsync / Await How did we get Async / Await? ● Babel compiles latest JS syntax to ES5 compatible code ● Babel-register can be a pre-runtime compiler in Mocha. ● See the repo!
- 24. 24 'use strict'; var BasePage= require('./BasePage'); var By = require('selenium-webdriver').By; //Constructor for the Top Navigation Bar function TopNav(webdriver) { BasePage.call(this, webdriver); this.isLoaded(); } //BasePage and Constructor wiring TopNav.prototype = Object.create(BasePage.prototype); TopNav.prototype.constructor = TopNav; TopNav.prototype.isLoaded = function () { this.waitForDisplayed(By.css('.options')); return this; }; TopNav.prototype.openProjectDropdown = function () { this.waitForDisplayed(By.css('.options')); this.waitForEnabled(By.css('.options ul:nth-of-type(1)')); this.click(By.css('.options ul:nth-of-type(1)')); return this; }; 'use strict'; import BasePage from './BasePage'; import { By } from 'selenium-webdriver'; import ui from './../util/ui-util'; export default class TopNav extends BasePage { //Constructor for the Top Navigation Bar constructor(webdriver: WebDriverClass) { super(webdriver); } async isLoaded(): Promise<this> { await ui.waitForDisplayed(this.driver, By.css('.options')); return this; } async openProjectDropdown(): Promise<this> { await ui.waitForDisplayed(this.driver, By.css('.options')); await ui.waitForEnabled(this.driver, By.css('.options ul:nth-of-type(1)')); await ui.click(this.driver, By.css('.options ul:nth-of-type(1)')); return this; } } ES6 Pageobjects with flow annotation
- 25. 25 What we presentedlast year Input / update data Get data Input / update data Get data UI Framework : node.js ● selenium-webdriver ● mocha + wrapper ● Applitools ● co-wrap for webclient ● chai (asserts) Rest API Framework : node.js ● co-requests ● mocha ● co-mocha ● chai (asserts) ● json, jayschema WebClients Pageobjects Webclient adaptor Database Backend : Go, Python Browser Frontend : Javascript Microservice #1 Microservice #2 ... #n Rest APIs
- 26. 26 Current Database Backend : Go,Python Browser Read & Write - Click, Drag - Enter text - Get text Frontend : Javascript Microservice #1 Microservice #2 ... #n Rest APIs Read & Write API Calls Get, Post, Put, Delete UI tests ● selenium-webdriver ● requests Rest API tests ● requests ● Json, jayschema WebClients - Job Client - Project Client - File Client - etc.. UI Pageobjects - Project List - Job List - Job info - Maps - Annotations Node.js common tooling ● Mocha ● Babel (ES6) ● Bluebird ● Asserts (Chai) ● Flow type (Facebook) Visual tests ● Applitools (visual diffs)
- 27. 27 Async / AwaitCons ● Error handling / stack tracing ● Forces devs to actually understand promises ● Async or bust - difficult to incrementally refactor ● Promise Wrapper optimizations gone ● Chewier syntax than control flow - `await` everywhere
- 28. 28 ● Async /await was almost designed for browser automation ‘desync’ing. The glove fits ● Refactoring out of Promise Manager is cumbersome ● Simplifying test syntax -> less dependencies and opinion -> happy and efficient devs The Bottom Line...
- 29. Custom reporter withscreenshots from Saucelabs Extending MochaJS
- 30. 30 Why Roll YourOwn Reporter? ● Start a conversation with developers: ○ What is the most important data you need to see in order to be efficient and successful? ● 5 things important to Airware cloud team: ○ Failures First ○ Flakiness vs. Failure ○ Assertion messaging ○ Screenshots ○ Video
- 31.
- 32. 32 Parallelization Try mocha-parallel-tests.But be careful!
- 33. 33 ● Take advantageof scalability to deliver what other devs want ● Keep an eye on the OS-community for new solutions The Bottom Line...
- 34.
- 35. 35 Flowtype and Eslint Don’tlike the willy-nilly-ness of JavaScript? Lint! Type check! ● Static type analysis is available with Flow and TypeScript ● Both have awesome IDE / editor plugins ● We picked Flow in order to start annotating our code piece by piece. ● We added Webdriver, Mocha, and Chai type definitions ● ESlint has some great plugins related to test structuring. We’ve also written our own ● The bottom line: the best time to capture test errors is as you write them
- 36. And best practices Robustvisual diffs
- 37. 37 Robust automated visualtests Powered by
- 38. 38 Size of test Thetest pyramid* Cost Time Coverage Martin Fowler - Test Pyramid, Google Testing Blog 2014 Google Test Automation Conference Big E2E tests Medium Integration tests Small Unit tests As you move up the pyramid, your tests gets bigger. At the same time the number of tests (pyramid width) gets smaller. ● Cost: Test execution, setup time and maintenance is less as you come down ● Feedback: Detection cycle (time) is less as you come down ● Stability: Smaller tests are less flaky ● Coverage: E2E workflow tests have better coverage but with tradeoffs; longer time, flakiness # Number of tests
- 39. 39 Google suggests a70/20/10 ratio for test amount allocation. (Of a total 100 tests) ○ 70% unit tests, ○ 20% integration tests ○ 10% end-to-end tests The exact mix will be different for each team, but we should try to retain that pyramid shape. The ideal ratio Google Testing Blog 2014 Google Test Automation Conference 10 20 70
- 40. 40 Visual diff testpyramid Visual tests UI Selenium tests REST API tests QA test pyramid Big E2E tests Medium Integration tests Small Unit tests Engineering test pyramid Browser Screenshots Backend image diffs Visual diff test pyramid Apply the test pyramid concept to automated visual test suite ● Browser screenshot tests are big E2E tests ● Add smaller visual testing with commandline image diffs. No browsers involved.
- 41. 41 Applitools commandline imagediff Powered by Using Applitools eyes.images SDK var Eyes = require('eyes.images').Eyes; await image = getImage("store.applitools.com","/download/contact_us.png/" + version); // Visual validation point #1 await eyes.checkImage(img, 'Contact-us page'); ● Downloads or streams the images as PNG format ● Uploads images to validate against Applitools service More info - https://eyes.applitools.com/app/tutorial.html?accountId=m989aQAuq8e107L5sKPP9tWCCPU10JcYV8FtXpBk1pRrlE110 Thanks Liran Barokas !!
- 42. 42 Visual diff testpyramid Browser Screenshots Backend image diffs Powered by
- 43. Test trend analysisand etc. Whats next ?
- 44. 44 Important attributes ofCI/CD systems ● Trustworthy results ● Tests that do not add value are removed ● Tests have the privilege (not the right) to run in CI ● Metadata from build and test results ● Trend analysis ● Is the feature ready to ship ● Cutting edge technology ○ Visual diff ■ Applitools ○ Kubernetes ○ Containers ○ Unikernels Denali Lumma (Uber), testing in 2020
- 45. 45 Test trend analytics Workin Progress!
- 46. 46 Testability as aproduct requirement Collaboration between Frontend and Automation teams Surface data attributes in the UI DOM ● uuids - test can cross validate by making API calls in UI tests ● Image group, image names, photo clustering attributes ● etc.. Testability as a product requirement! :)
- 47. 47 Github links The AirwareGithub repos https://github.com/airware Our Async / Await Example https://github.com/airware/webdriver-mocha-async-await-example Flowtype interfaces for Webdriver, Chai, Mocha https://github.com/airware/forseti-flow-interfaces
- 48.
- 49.