Load Testing from Multiple Clients with a Pipeline
Video size:
Abstract
Load test as code is a type of performance test checking the upper limits of your system by subjecting it to extreme load conditions. Using automated pipelines, a load test can be further improved to simulate real-world scenarios, as traffic can be forced upon multiple servers (with different IPs)
Summary
-
Using load testing tools, we can build load test codes and simulate multiple user requests sent based on a specific load and timeline. In choosing a tool, we consider the following in making a decision on the tool that we will use.
-
Gatling is commonly coded in Scala, but it's now available with Java. Simulations contain your load tests. The load test will produce an HTML report that shows various metrics. With these data, you can easily confirm whether something needs to be tweaked in your infrastructure.
-
Gatling can be used to simulate load testing from a lot more clients. The GitHub actions code of running Gatling is pretty straightforward. A solution doesn't have to be complicated. If you know different tools used in DevOps, maybe combining them can be all ready.
Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hi everyone, today I'll be sharing about the topic loadtesting for
multiple clients with a pipeline. We recently migrated
our platform to a more modern architecture and has to ensure
that the sizing of the new architecture maintains the same high availability
of the previous one. To do this, we used
load test as code, where we simulated real production
traffic in an isolated nonproduction environment.
Loadtesting in its definition is a type of performance test checking
the upper limits of your system by subjecting it to extreme
load conditions. Here we see a single client making a
request to the server. Using load testing tools,
we can build load test codes and simulate multiple
user requests sent based on a specific load
and timeline that we specify.
There are several load testing tools in the market. In choosing
a tool, we consider the following in making a decision on the
tool that we will use. Most of the tools here, like Jmeter,
locust and Gatling have the same features, so the final
decision can also be a matter of preference, like if
your developers have experience using a particular tool or
are more used to the programming language used by a certain tool.
So for us it was Katlin. And again, I won't
be showing some data on how to select one, but would like to
give an overview of how architects or engineers usually make
this decision based on DevOps considerations.
Now let's get to coding and walking through the components of the load test
code in Gatling. Here we see the configuration
file of Gatling. This is named Gatling Conf.
Inside the conf directory of the code, a few
mandatory lines that we need to update are the simulations and resources
directory. We need to instruct Gatling
about our code structure and where we are expecting it to load
the code to run the tests.
Resources are a set of values that you feed to your simulation
tests. You can set up a scenarios and use values
in any resource file for load tests where you
are simulating different parameter values for a single scenario.
You can also add a resource file for sample data or anything
that you will be using in the simulation code.
Next are simulations. Simulations contain your
load tests. It's commonly coded in Scala, but it's now
available with Java. If you are more familiar with this language in
creating simulation code, let's add this code structure to
a scala file. Diving deeper to
the simulation code, we can add variables that we can use
throughout the simulation. An example here is how we can
assign credentials with value that we feed with
a map. This map can refer back to the resource section
of our code here.
We can also create an HTTP configuration and
store it in a variable. We can also add common headers like
accept and content type.
Next, we need a scenario which is the actual test.
We can feed the credentials, execute an HTTP call,
add headers, and also state the expected result from the API
call.
Finally, we set up the scenario and inject users
based on common injection options where could
be either open or closed. If we have several scenarios,
we can add them in the same file delimited by a comma.
We run the simulation by a shell script that contains instructions to
run the Java program. This will open up a selection for
the number of simulations we have in our directory and allow selection
for which simulation to run.
During the run, the screen will refresh every few seconds
showing the current load test until it completes.
The load test will produce an HTML report that shows various metrics
like number of success versus failed requests,
individual metrics like response times of each request,
execution, success and failures.
It will also show the increase, decrease, or constancy
of the users sent by the load test tool and
its equivalent responses per second.
There's a few more graphs in the HTML report produced by Gatling
that may be helpful in what you're trying to find out. With these data,
you can easily confirm whether something needs to be tweaked in
your infrastructure based on how it responds.
Now, as a platform, we need to be able to simulate
load testing from a lot more clients, even if
one client is sending traffic with a heavy load.
In the real world scenario, we expect the load to come
from a distributed number of clients with different ips.
So the simple solution is putting this in a pipelines.
We simply put our code in GitHub and trigger the pipelines
in parallel to spin up multiple GitHub runners, each with
their own unique server ips, and each one sends
traffic to our platform, thus simulating the real
world scenario more accurately. And that's it.
That's the solution. So before
we mark this as done, I'll quickly walk you through some other integration decisions
we made when putting together the Gatling and GitHub actions architecture.
First is how to programmatically select a simulation.
This one's fairly easy, as there's a config file used by
Gatling where we can force simulation selection by
merely updating it in the config file. This way we
are not prompted with what file to run. It is automatically
selected once we trigger the run command.
The simple trick ips replacing this in the file in one of
the GitHub action steps.
Now we also want to make this reusable across environments in
our platform. We distinguish this with the domain of the API call.
Again the solution to this IPS programmatically update
the environment in the simulation code using a simple said command.
Problems three and four are pretty much similar to the previous ones.
We also do this programmatically by means of an input to the
GitHub actions workflow and updating the values in the Gatling load test
code. The GitHub actions code
of running Gatling is pretty straightforward. We install the dependency
we need, which is Java, then run the load test using the shell provided
script. In terms of output.
Since the reports are produced in the server, in this
case the GitHub runner, we can use a GitHub action to
simply create an artifact to expose this after the workflow runs.
We can add other things here, like putting it in a remote server,
or sending the file in an email or a slack message and that's
it. We ran the code, get a report,
and everything is automated. Some important
things to note the main purpose of load testing is to test the performance of
our platform or even an application. We want to
design load tests such that it can test the upper limits or
close to the breaking point of a system. We can try to adjust the
load accordingly during runs, or determine a base scenario
based on production data. We can also design load
tests to continuously run in a schedule. We can send
heavier traffic on a particular time of day and lower traffic on
off hours. And as all systems go,
this is a personal preference, but going serverless means reducing
the headache that comes with maintaining your own servers.
Use cloud as much as possible and choose the serverless option
when you can. If you want to learn more about this,
I also have a blog created where I further explained how to develop load test
codes. Even if you don't code at all. You can find
this link here, published at developers at Asurion Devcommunity
and that's it. A solution doesn't have to be complicated.
If you know different tools used in DevOps, maybe combining
them can be all ready. Be a solution to a problem that
you're wanting to solve. Just keep things simple.
I hope you enjoyed the presentation.