python系列29:压测工具locust

1. 介绍

使用pip进行安装,下面是个简单例子:
新建一个test.py

from locust import HttpUser, task
import json
payload = json.load(filename) # 替换成post请求的payload文件
class TestUser(HttpUser):
    @task
    def hello_world(self):
        self.client.get("/hello")
        self.client.post("/world",json.dumps(payload),headers = {'Content-Type':'application/json'})

执行 locust -f test.py
然后打开web页面:
在这里插入图片描述点击start,会执行脚本代码,调用hello和world接口。
ramp-up的设置,一般而言:100以内的并发用户数,ramp-up时间设置为1-2s;100-500左右,rramp-up时间设置为2-3s;500以上,ramp-up时间设置为5-10s。
压测的结果如图:
在这里插入图片描述

也可以使用命令行:

$ locust --headless --users 10 --spawn-rate 1 -H http://your-server.com
[2021-07-24 10:41:10,947] .../INFO/locust.main: No run time limit set, use CTRL+C to interrupt.
[2021-07-24 10:41:10,947] .../INFO/locust.main: Starting Locust 2.28.0
[2021-07-24 10:41:10,949] .../INFO/locust.runners: Ramping to 10 users using a 1.00 spawn rate
Name              # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
----------------------------------------------------------------------------------------------
GET /hello             1     0(0.00%)  |     115     115     115     115  |    0.00    0.00
GET /world             1     0(0.00%)  |     119     119     119     119  |    0.00    0.00
----------------------------------------------------------------------------------------------
Aggregated             2     0(0.00%)  |     117     115     119     117  |    0.00    0.00

[2021-07-24 10:44:42,484] .../INFO/locust.runners: All users spawned: {"HelloWorldUser": 10} (10 total users)

2. 测试脚本分析

import time
from locust import HttpUser, task, between

class QuickstartUser(HttpUser):
    wait_time = between(1, 5) # 或者可以使用

    @task
    def hello_world(self):
        self.client.get("/hello")
        self.client.get("/world")

    @task(3)
    def view_items(self):
        for item_id in range(10):
            self.client.get(f"/item?id={item_id}", name="/item")
            time.sleep(1)

    def on_start(self):
        self.client.post("/login", json={"username":"foo", "password":"bar"})

脚本中至少需要包含一个class。HttpUser继承自HttpSession,会给每一个用户创建一个self.client。
@task会给每一个用户创建一个greenlet(协程),然后就可以调用clinet的get和post方法来调用http服务:

response = self.client.get("/my-profile")
response = self.client.post("/login", {"username":"testuser", "password":"secret"})

@task中的数值表示权重。每个用户会以权重概率,从所有的task中选取一个执行。
view_items中的name属性,会将所有的url都归类到同一个/item下,而不是分成10个单独的url进行统计。

wait_time除了between还有如下几种:
constant(X):等待固定时间
constant_throughput: for an adaptive time that ensures the task runs (at most) X times per second.
constant_pacing:for an adaptive time that ensures the task runs (at most) once every X seconds
也可以自定义wait_time函数:

class MyUser(User):
    last_wait_time = 0

    def wait_time(self):
        self.last_wait_time += 1
        return self.last_wait_time

@tag标签可以用来指定跑哪些tasks。例如下面的例子中:

from locust import User, constant, task, tag

class MyUser(User):
    wait_time = constant(1)

    @tag('tag1')
    @task
    def task1(self):
        pass

    @tag('tag1', 'tag2')
    @task
    def task2(self):
        pass

    @tag('tag3')
    @task
    def task3(self):
        pass

    @task
    def task4(self):
        pass

If you started this test with --tags tag1, only task1 and task2 would be executed during the test. If you started it with --tags tag2 tag3, only task2 and task3 would be executed.

–exclude-tags will behave in the exact opposite way. So, if you start the test with --exclude-tags tag3, only task1, task2, and task4 will be executed. Exclusion always wins over inclusion, so if a task has a tag you’ve included and a tag you’ve excluded, it will not be executed.

3. 启动函数

如果需要在整个测试开始前或结束后执行代码,则需要下面的函数:

from locust import events

@events.test_start.add_listener
def on_test_start(environment, **kwargs):
    print("A new test is starting")

@events.test_stop.add_listener
def on_test_stop(environment, **kwargs):
    print("A new test is ending")

如果需要在每个process开始前执行一些代码,则需要下面的函数:

from locust import events
from locust.runners import MasterRunner

@events.init.add_listener
def on_locust_init(environment, **kwargs):
    if isinstance(environment.runner, MasterRunner):
        print("I'm on master node")
    else:
        print("I'm on a worker or standalone node")

如果是每次request请求前都要执行一些代码,则需要在User里执行on_start函数:

class QuickstartUser(HttpUser):
    def on_start(self):
        self.client.post("/login", json={"username":"foo", "password":"bar"})

4. FastHttpUser

FastHttpUser使用了geventhttpclient,性能比基于requests的HttpUser快了数倍。In a best case scenario (doing small requests inside a while True-loop) a single Locust process (limited to one CPU core) can do around 16000 requests per second using FastHttpUser, and 4000 using HttpUser (tested on a 2021 M1 MacBook Pro and Python 3.11)
下面是一个例子:

from locust import FastHttpUser, run_single_user, task
from locust.contrib.fasthttp import RestResponseContextManager
from locust.user.wait_time import constant

from collections.abc import Generator
from contextlib import contextmanager


class MyUser(FastHttpUser):
    host = "https://postman-echo.com"
    wait_time = constant(180)  # be nice to postman-echo.com, and dont run this at scale.

    @task
    def t(self):
        # should work
        with self.rest("GET", "/get", json={"foo": 1}) as resp:
            if resp.js["args"]["foo"] != 1:
                resp.failure(f"Unexpected value of foo in response {resp.text}")

        # should work
        with self.rest("POST", "/post", json={"foo": 1}) as resp:
            if resp.js["data"]["foo"] != 1:
                resp.failure(f"Unexpected value of foo in response {resp.text}")
            # assertions are a nice short way to express your expectations about the response. The AssertionError thrown will be caught
            # and fail the request, including the message and the payload in the failure content.
            assert resp.js["data"]["foo"] == 1, "Unexpected value of foo in response"

        # assertions are a nice short way to validate the response. The AssertionError they raise
        # will be caught by rest() and mark the request as failed

        with self.rest("POST", "/post", json={"foo": 1}) as resp:
            # mark the request as failed with the message "Assertion failed"
            assert resp.js["data"]["foo"] == 2

        with self.rest("POST", "/post", json={"foo": 1}) as resp:
            # custom failure message
            assert resp.js["data"]["foo"] == 2, "my custom error message"

        with self.rest("POST", "/post", json={"foo": 1}) as resp:
            # use a trailing comma to append the response text to the custom message
            assert resp.js["data"]["foo"] == 2, "my custom error message with response text,"

        with self.rest("", "/post", json={"foo": 1}) as resp:
            # assign and assert in one line
            assert (foo := resp.js["foo"])
            print(f"the number {foo} is awesome")

        # rest() catches most exceptions, so any programming mistakes you make automatically marks the request as a failure
        # and stores the callstack in the failure message
        with self.rest("POST", "/post", json={"foo": 1}) as resp:
            1 / 0  # pylint: disable=pointless-statement

        # response isn't even json, but RestUser will already have been marked it as a failure, so we dont have to do it again
        with self.rest("GET", "/") as resp:
            pass

        with self.rest("GET", "/") as resp:
            # If resp.js is None (which it will be when there is a connection failure, a non-json responses etc),
            # reading from resp.js will raise a TypeError (instead of an AssertionError), so lets avoid that:
            if resp.js:
                assert resp.js["foo"] == 2
            # or, as a mildly confusing oneliner:
            assert not resp.js or resp.js["foo"] == 2

        # 404
        with self.rest("GET", "http://example.com/") as resp:
            pass

        # connection closed
        with self.rest("POST", "http://example.com:42/", json={"foo": 1}) as resp:
            pass


# An example of how you might write a common base class for an API that always requires
# certain headers, or where you always want to check the response in a certain way
class RestUserThatLooksAtErrors(FastHttpUser):
    abstract = True

    @contextmanager
    def rest(self, method, url, **kwargs) -> Generator[RestResponseContextManager, None, None]:
        extra_headers = {"my_header": "my_value"}
        with super().rest(method, url, headers=extra_headers, **kwargs) as resp:
            if resp.js and "error" in resp.js and resp.js["error"] is not None:
                resp.failure(resp.js["error"])
            yield resp


class MyOtherRestUser(RestUserThatLooksAtErrors):
    host = "https://postman-echo.com"
    wait_time = constant(180)  # be nice to postman-echo.com, and dont run this at scale.

    @task
    def t(self):
        with self.rest("GET", "/") as _resp:
            pass


if __name__ == "__main__":
    run_single_user(MyUser)

5. 其他参数

locust --help
Usage: locust [options] [UserClass ...]

Common options:
  -h, --help            show this help message and exit
  -f <filename>, --locustfile <filename>
                        The Python file or module that contains your test,
                        e.g. 'my_test.py'. Accepts multiple comma-separated
                        .py files, a package name/directory or a url to a
                        remote locustfile. Defaults to 'locustfile'.
  --config <filename>   File to read additional configuration from. See https:
                        //docs.locust.io/en/stable/configuration.html#configur
                        ation-file
  -H <base url>, --host <base url>
                        Host to load test, in the following format:
                        https://www.example.com
  -u <int>, --users <int>
                        Peak number of concurrent Locust users. Primarily used
                        together with --headless or --autostart. Can be
                        changed during a test by keyboard inputs w, W (spawn
                        1, 10 users) and s, S (stop 1, 10 users)
  -r <float>, --spawn-rate <float>
                        Rate to spawn users at (users per second). Primarily
                        used together with --headless or --autostart
  -t <time string>, --run-time <time string>
                        Stop after the specified amount of time, e.g. (300s,
                        20m, 3h, 1h30m, etc.). Only used together with
                        --headless or --autostart. Defaults to run forever.
  -l, --list            Show list of possible User classes and exit
  --config-users [CONFIG_USERS ...]
                        User configuration as a JSON string or file. A list of
                        arguments or an Array of JSON configuration may be
                        provided

Web UI options:
  --web-host <ip>       Host to bind the web interface to. Defaults to '*'
                        (all interfaces)
  --web-port <port number>, -P <port number>
                        Port on which to run web host
  --headless            Disable the web interface, and start the test
                        immediately. Use -u and -t to control user count and
                        run time
  --autostart           Starts the test immediately (like --headless, but
                        without disabling the web UI)
  --autoquit <seconds>  Quits Locust entirely, X seconds after the run is
                        finished. Only used together with --autostart. The
                        default is to keep Locust running until you shut it
                        down using CTRL+C
  --web-login           Protects the web interface with a login page. See
                        https://docs.locust.io/en/stable/extending-
                        locust.html#authentication
  --tls-cert <filename>
                        Optional path to TLS certificate to use to serve over
                        HTTPS
  --tls-key <filename>  Optional path to TLS private key to use to serve over
                        HTTPS
  --class-picker        Enable select boxes in the web interface to choose
                        from all available User classes and Shape classes

Master options:
  Options for running a Locust Master node when running Locust distributed. A Master node need Worker nodes that connect to it before it can run load tests.

  --master              Launch locust as a master node, to which worker nodes
                        connect.
  --master-bind-host <ip>
                        IP address for the master to listen on, e.g
                        '192.168.1.1'. Defaults to * (all available
                        interfaces).
  --master-bind-port <port number>
                        Port for the master to listen on. Defaults to 5557.
  --expect-workers <int>
                        Delay starting the test until this number of workers
                        have connected (only used in combination with
                        --headless/--autostart).
  --expect-workers-max-wait <int>
                        How long should the master wait for workers to connect
                        before giving up. Defaults to wait forever
  --enable-rebalancing  Re-distribute users if new workers are added or
                        removed during a test run. Experimental.

Worker options:
  Options for running a Locust Worker node when running Locust distributed.
  Typically ONLY these options (and --locustfile) need to be specified on workers, since other options (-u, -r, -t, ...) are controlled by the master node.

  --worker              Set locust to run in distributed mode with this
                        process as worker. Can be combined with setting
                        --locustfile to '-' to download it from master.
  --processes <int>     Number of times to fork the locust process, to enable
                        using system. Combine with --worker flag or let it
                        automatically set --worker and --master flags for an
                        all-in-one-solution. Not available on Windows.
                        Experimental.
  --master-host <hostname>
                        Hostname of locust master node to connect to. Defaults
                        to 127.0.0.1.
  --master-port <port number>
                        Port to connect to on master node. Defaults to 5557.

Tag options:
  Locust tasks can be tagged using the @tag decorator. These options let specify which tasks to include or exclude during a test.

  -T [<tag> ...], --tags [<tag> ...]
                        List of tags to include in the test, so only tasks
                        with at least one matching tag will be executed
  -E [<tag> ...], --exclude-tags [<tag> ...]
                        List of tags to exclude from the test, so only tasks
                        with no matching tags will be executed

Request statistics options:
  --csv <filename>      Store request stats to files in CSV format. Setting
                        this option will generate three files:
                        <filename>_stats.csv, <filename>_stats_history.csv and
                        <filename>_failures.csv. Any folders part of the
                        prefix will be automatically created
  --csv-full-history    Store each stats entry in CSV format to
                        _stats_history.csv file. You must also specify the '--
                        csv' argument to enable this.
  --print-stats         Enable periodic printing of request stats in UI runs
  --only-summary        Disable periodic printing of request stats during
                        --headless run
  --reset-stats         Reset statistics once spawning has been completed.
                        Should be set on both master and workers when running
                        in distributed mode
  --html <filename>     Store HTML report to file path specified
  --json                Prints the final stats in JSON format to stdout.
                        Useful for parsing the results in other
                        programs/scripts. Use together with --headless and
                        --skip-log for an output only with the json data.

Logging options:
  --skip-log-setup      Disable Locust's logging setup. Instead, the
                        configuration is provided by the Locust test or Python
                        defaults.
  --loglevel <level>, -L <level>
                        Choose between DEBUG/INFO/WARNING/ERROR/CRITICAL.
                        Default is INFO.
  --logfile <filename>  Path to log file. If not set, log will go to stderr

Other options:
  --show-task-ratio     Print table of the User classes' task execution ratio.
                        Use this with non-zero --user option if some classes
                        define non-zero fixed_count attribute.
  --show-task-ratio-json
                        Print json data of the User classes' task execution
                        ratio. Use this with non-zero --user option if some
                        classes define non-zero fixed_count attribute.
  --version, -V         Show program's version number and exit
  --exit-code-on-error <int>
                        Sets the process exit code to use when a test result
                        contain any failure or error. Defaults to 1.
  -s <number>, --stop-timeout <number>
                        Number of seconds to wait for a simulated user to
                        complete any executing task before exiting. Default is
                        to terminate immediately. When running distributed,
                        this only needs to be specified on the master.
  --equal-weights       Use equally distributed task weights, overriding the
                        weights specified in the locustfile.

User classes:
  <UserClass1 UserClass2>
                        At the end of the command line, you can list User
                        classes to be used (available User classes can be
                        listed with --list). LOCUST_USER_CLASSES environment
                        variable can also be used to specify User classes.
                        Default is to use all available User classes

Examples:

    locust -f my_test.py -H https://www.example.com

    locust --headless -u 100 -t 20m --processes 4 MyHttpUser AnotherUser

See documentation for more details, including how to set options using a file or environment variables: https://docs.locust.io/en/stable/configuration.html
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值