Why do you need performance tests?

Why should you care specifically about application performance? How does it impact the business chances to succeed in an application-based economy? This post talks about performance testing and its role in producing software that delivers business value.

What are performance tests?

If software applications were cat play tunnels, performance testing would be finding out whether or not the tunnel will break before too many cats use it, before they even start.*

So what would be a well-performing cat play tunnel? The one that serves our cats well and they can play in the tunnel freely 😼

What if we speak about web applications? A well-performing application is one that lets users carry out a given task without undue perceived delay and irritation.

Here we get into the crux of performance testing. It determines whether your software meets speed, scalability and stability requirements under expected workloads.

To understand performance testing we need to think of these two metrics — latency and throughput. These metrics let you figure out if your application is performing well or not.

What is Latency? ⏱️

Latency in our example is how much time it takes for a cat to get from the beginning of the tunnel to the end of it. Latency is a measure of delay and is measured in milliseconds.

What is Throughput? 🐈🐈🐈

Throughput on the other hand is the maximum amount of cats walking through the tunnel over time during a test. As more cats enter the tunnel they need to wait, creating a queue. Throughput is often expressed as requests/transactions per second, or RPS/TPS.

We can look at performance testing like at a dance between latency and throughput where we are aiming at reducing latency and finding a sweet spot called the Optimal Rate — here the throughput is at its peak and the latency starts to increase, but not drastically.

Before starting a performance test it is common to have a throughput goal that the application needs to be able to meet. What we already know from the figure above is that latency drives throughput.

Why is performance testing important?

Many applications struggle to deliver good performance when it counts the most. For example during sales season in eCommerce or when everyone goes on lockdown and starts using Netflix heavily. Such conditions of peak loading if not diagnosed, could cause serious business implications.

Often it’s just a matter of milliseconds. Google found an extra 500 milliseconds in search page generation time dropped traffic by 20%. A 100-millisecond delay in website load time can hurt conversion rates by 7 percent.

The goal of performance testing is not to find bugs (although you will be surprised that you will found them as well) but to eliminate performance bottlenecks and check whether the system can sustain continuous expected load.

If done right, and incorporated early enough in the software development lifecycle, it provides stakeholders with crucial information about their application that affect their business. By doing performance tests you avoid:

• running slow or going down with your application
• users experiencing poor usability
• missing software vulnerabilities

Since you can’t measure what you can’t see, you need to discover and deal with the symptoms of performance problems, dig out their root cause. Hence, performance testing is important and ensures business continuity and success. The figure below from “The Art of Application Performance Testing” illustrates this challenge perfectly.

What are the types of performance tests?

• Latency testing — required to measure the time taken for moving data messages from one point within a computer network to another.
• Stress testing: measures the system on its robustness and error handling capabilities under extremely high load.
• Load testing: models the expected usage of a software program by simulating multiple users accessing it concurrently.
• Targeted infrastructure testing: individual layer and/or fragment test in an end to end application configuration.
• Failover testing: validates a system’s ability to be able to allocate extra resource when it faces heavy traffic and usage to ensure that the end user’s experience is not affected.
• Capacity testing: checks whether the application and environment can handle the amount of traffic that it was designed to handle. Important for benchmarking.
• Soak testing: measures the performance of a system to validate its behavior in the production environment.

What to remember when doing performance tests?

Prepare your methodology

Developers appreciate the value of testing, but there’s always a push-and-pull when it comes to the time and costs of it. Thus it’s important to decide what level of coverage is appropriate for your project. Before you start, define a list of things that you want to accomplish, use cases that focus on key business functionalities (e.g. in eCommerce it could be a buying process, or product search). Here you’ll need to understand your application and its users. Then ask yourself what problems might occur and how would they manifest themselves. Later, write down activities and define owners for each activity to make sure any problems you find will get a fix promptly.

Involve developers from the very beginning

Test as early as possible in the development process and remember you can test individual units both separately and together. Also remember to create a team that consists of developers and testers.

Manage your expectations

Your tests will never be 100% accurate and you won’t have time to test everything for too long. Today’s systems depend on thousands of different parameters and testing everything is literally impossible. Create benchmarks, look over them with your team and suggest changes. Defining thresholds for metrics you don’t want to cross is actually the essence of testing.

Compare and use production data

Analyse each test’s results and compare the metrics to the KPIs. Also remember to take into accounts the results of previous test runs and use the data from production to ensure your tests are closer to production scenarios.

Use performance tests in your CI

Running performance tests should be part of your functional testing cycle, not just the last part of the development process.

Tools for performance testing

There are many tools available for performance testing. Here are a few we’ve explored:

Get Maria Kucharczyk’s stories in your inbox

Join Medium for free to get updates from this writer.

JMeter — a 100% pure Java application designed to load test functional behavior and measure performance. It’s free and may be used to test performance both on static and dynamic resources, web dynamic applications. It can be used to simulate a heavy load on a server, group of servers, network or object to test its strength or to analyse overall performance under different load types.

Gatling — is an open-source load testing tool for web applications, designed for DevOps and Continuous Integration. It’s a highly capable load testing tool. It is designed for ease of use, maintainability and high performance.

Here’s Andrzej Ludwikowski talking about Gatling’s pros and cons:

Liked the video? Want to use Gatling for performance testing? Join hands-on training — Gatling fundamentals for performance testing >>

Locust — is an easy-to-use, distributed, user load testing tool. It is intended for load-testing web sites (or other systems) and figuring out how many concurrent users a system can handle.

How do Netflix and Youtube run performance tests?

Business benefits of doing performance tests

Successful market entrance

When you first deploy your application, you need to take into account something called “novelty factor”. If you add to it a proper marketing and promotion, expect quite a lot of users interested in your product. There is no other moment as important as launching a new product and performance testing helps you better prepare for this big day.

Happy customers

Customers have little patience for slow or broken apps. The faster the app works, the better users can load the site or use the app and actually benefit from your services. They will stay with your business as you keep them engaged and happy, because your product delivers what is promised.

Higher revenue

When you support your market claims, keep your customers engaged and ensure your application is safe under heavy loads, also in terms of cyber attacks, your business gets traction and delivers more value, also in times of peak loading determined by business characteristics.

Good brand reputation

Application performance has a direct correlation to brand trust and brand loyalty. A consumer’s perception of a brand is directly affected by their initial impressions when engaging with the brand.

Scalability

Obviously business wants more customers. Performance tests let you avoid unrealistic expectations of concurrent end users of your application.

Better safe than sorry

With performance tests you can find vulnerabilities before they become full-blown outages and resolve the risks that might impair the application in real world scenarios.

Do you need performance tests?

Performance testing allows you to deliver a user-friendly software. Do you remember when HBO Go went down as Game of Thrones Season 7 premiered? Thousands of disappointed fans went on Twitter to vent.

While some could argue that such technical failure may actually work in your favour and translate into bigger media coverage, usually it does the opposite. Application performance issues impact business directly, harming the brand by causing problems with continuous integration od updates frequency.

Your software’s performance should be on your mind from the moment you have initiated a project to minimize risks, obtain feedback and learn about best practices over time.

Like this post and interested in learning more?
Follow us on Medium!
Need help with your Casandra, Kafka or Scala projects?
Just contact us here.