If you were to look at the automated testing of most teams, you are almost guaranteed to find some automated UI tests. While I would like to hope there are more unit tests in the world, I reckon UI tests come a close second! Automated UI tests allow teams to directly test on the user interface their customers use, allow teams to automate customer journeys, trigger specific behaviour and test downstream integrations. However, automated UI testing is used as a synonym for end-to-end testing in a lot of teams, which means they are missing out on many other ways they could use automation to test their UIs. End-to-end tests definitely have their place, but we can do more tests that target specific risks and allow for more targeted tests.
In this article, we are going to explore five common types of automation UI testing; Visual Testing, End-to-End (E2E) Testing, UI Testing, Component Testing and Cross-Browser testing.
Visual Testing
Visual Testing is a technique focused on testing how the application looks to users. The technique is not about whether the application works, it’s purely about how the application looks and renders. Is the font correct, the spacing, the colours, the layout and much more. Visual testing utilises the gold master approach, whereby a baseline is captured, approved and used as a baseline for future comparison. All subsequent test executions will take a new screenshot and compare to the baseline and raise any differences detected.
Visual testing is the technique teams should be using instead of checking text on the screen. It’s an assertion we’ve all done, but when we check text on the screen, it doesn’t tell us where that text is. Is it the right colour? The right font? The right size? Is there an image covering some of it? Such tests are an example of a badly designed assertion, because while we do care about text on the screen, there was almost an inferred assumption we were asserting that, but without a visual test, you definitely aren’t.
Risks We Can Mitigate:
- Incorrect font, size or colour
- Inconsistent layout
- Broken images or icons
- Incorrect page colours
- In correct responsive designs
How It's Performed
There are a plethora of tools in this domain, some more sophisticated than others. As mentioned, it’s based on creating a baseline image and then the tools comparing that with the captured image during the test. The majority of tools rely on pixel-by-pixel comparison, which will never be an exact match, but the tools allow you to set a threshold such as 5% difference. However, that 5% could be your company's logo, so you have to be selective with the pages/parts of a page you are comparing to reduce that risk. More sophisticated tools rely on AI to detect change and allow you to draw areas you want the comparison to ignore. As we are comparing images it’s important to have strong data management, so you can assure the data is the same on the screenshots to reduce false positives.
When to Use Visual Testing?
Visual testing should be used sparingly as they can be slower to execute, but I’d always look to have UI tests on our most critical screens, and especially on errors shown to the user. It’s common to test errors by checking the error message on the screen, but that doesn’t confirm the error is red, in view and has a clear icon, which is actually what we want to be testing.
End-to-End (E2E) Testing
E2E tests are designed to exercise the full application by triggering a behaviour on the UI, having that navigate the full stack and produce a result on the screen. This could be a journey on a single page, or typically across multiple pages in the application. Automation complete customer journeys is another common approach to E2E tests. They provide the ability to exercise the system in its entirety.
That strength is also its disadvantage, as typically a E2E UI test would make several full journeys up and down the stack, which means a lot of moving parts, and an increased chance of test flakiness. They are also usually slower to execute as they are exercising all the layers; sending data to an API, API processing it, sending it to a database, and then back up to the UI for each page.
Risks We Can Mitigate:
- Integration issues between frontend and backend
- Navigation failures
- Customer flow issues
- Issues with elements on the page
How It's Performed
E2E testing requires a tool that can simulate user behaviour on the UI. So, in the context of a web site, we need a tool that can simulate clicks, waits, typing and navigation. Once a tool is sourced, a typical approach is to exercise the behaviour manually to understand the screens and flows, then mimic that behaviour in the tool.
When to Use E2E Testing?
In an ideal context, E2E tests should be kept to critical user journeys and business flows. Exercising such flows will give teams confidence that the whole system is configured correctly and all the parts are communicating. We want to avoid having too many E2E as these are usually large tests due to traversing the full stack multiple times, which also increases the chance of test flakiness.
UI Testing
UI testing is very similar to E2E testing, the main difference is we try to avoid using the full stack. With UI testing our focus is on the interface itself, not how it interacts with the wider system. We’d look to use API mocks when creating UI tests. This gives us two big advantages; we are targeting UI risks directly, and mocks give us more control over the data, which in turn allows us to exercise the UI. We typically do UI testing using a browser to get as close to the environment our customers will be using as possible.
Risks We Can Mitigate:
- Inconsistent page behaviour/functionality
- Rendering of the elements
- Ensuring the page is only making the API calls it needs to
- Correct page layout
- Ensuring API data is mapped to correct elements on the page
How It's Performed
UI testing requires a tool that can simulate user behaviour on the UI. So, in the context of a web site, we need a tool that can simulate clicks, waits, typing and navigation. Almost identical to E2E testing apart from one big difference, we also need a tool that allows us to mock API calls, to allow us to isolate the UI, and have full control over the data it receives.
When to Use UI Testing?
When the risk is specific to a behaviour or feature of the page itself not the whole system. Perhap there is a specific layout on the page, or some client side javascript that renders something on the page. In those contexts it’s advisable to use UI testing and mock any IP calls so we can target the specific risk, and keep our tests small.
Component Testing
A relatively new approach to UI testing space thanks to the evolution of frameworks like React and Angular. Component testing allows us to test components of the page independently, you can think of them as unit testing for the frontend. Component testing allows us to render a single component, and interact with it as if it were on a web page. We can inject specific data, trigger events and check if it's rendered correctly. A typical UI is made up of multiple components and these days those components are used on multiple pages, with component testing we can test the component once in isolation, and gain confidence that the component will work on all the pages it's used. Examples of components are buttons, avatar, menu, card, modals, everything on the page really if you are building with components.
Risks We Can Mitigate:
- Incorrect logic within the component
- Invalid state and data usage
- Not responding to events
- Incorrect rendering
How It's Performed
In order to do component testing we need a tool that allows us to render a component in isolation, usually in a virtual browser. Component testing is done in a virtual browser because we only need the bare bones of a browser, so this allows for quicker execution, usually in the milliseconds. The component is loaded along with all the state for the scenario being tested, that state will be things like data and props. We may also need to mock any external dependencies for example if the component makes an API call to get/send data, as we want to test the component in isolation.
When to Use Component Testing?
If your application is using a library that supports components then you should definitely be taking advantage of component testing. If those components are also used in multiple parts of the application, the justification is even higher. Testing components in isolation results in smaller tests, and speeds up debugging when things go wrong, as we know it’s the component and not an API or something else on the page.
Cross-Browser/Device/Platform Testing
While we’ve seen more stability in devices, platforms and browsers over recent years with the adoption of shared standards, there are still many contexts where cross-X tests are needed. Cross-X testing focuses on testing an application on different versions of the environment it’s running on, such as browser versions, different browsers, as well as different operating systems. A lot of applications rely on functionality from their environment, such as browser APIs, operating system functionality and much more. These environments are in a constant state of change and usually you have no control over their updates, so it’s important to test across these devices to avoid customers facing issues.
The number of permutations is infinite, and the cost of doing such type of testing can be costly in terms of virtual machines, real devices, cloud tools and the cost of maintaining them. So, it’s crucial that such testing is done based on real customer data, there is little value in testing a platform/version combination your customers aren’t using. Analytics is a great place to get this insight.
Risks We Can Mitigate:
- Relying on outdated platform APIs
- Functionality behaving differently on different browser/versions
- Layout/Rendering issues
- Performance issues
- Legal/contractual requirements to test on specific versions
How It's Performed
Typically such testing is done using the same tooling as the UI and E2E testing. Majority of these tools allow you to configure the test to run on different browsers, devices and versions. An alternative is to point the test execution to a device cloud, a service that provides all the popular combinations for convenience. It’s a good practice to select the combinations based on customer usage, and business needs. Testing all permutations is excessive and likely testing would never complete.
When to Use It
Cross-X testing should be used when the business is supporting multiple browsers, OS versions and devices. It could also be the case that you have legal obligations to test on real browsers. Perhaps you have to maintain support for legacy devices, due to contractual reasons or a large customer base using them, and you don’t want to lose that business. If you have a global/diverse customer base, it would also be justification for cross-X testing.
Conclusion
UI Automated Testing has come a long way in recent years with a plethora of new approaches available to us. It’s important to view your applications UI in multiple layers and identify the right layer to mitigate the risk in question. It’s also important to consider the type of assertion you wish to do, as often that dictates the technique required.
Taking a layered approach to UI testing will ensure your tests are small and targeted, which usually results in faster creation, execution and debugging. Solely relying on large E2E tests is no longer a viable strategy, the tools and techniques exist to go beyond them, and we should utilise them.
About Automated Testing series
You're at the second part of our Automated Testing series - a collaboration with Richard Bradshaw. The series will explore many facets of Testing Automation and its implementations.
In the first part, we explored the Manual and Automated Testing paradox. Check it out here.
The next part, coming out on 24 June 2025, will explore different types of API automated tests and their best use cases. Stay tuned!
