Python异步框架大战：FastAPI、Sanic、Tornado VS Go 的 Gin

一、前言

异步编程在构建高性能 Web 应用中起着关键作用，而 FastAPI、Sanic、Tornado 都声称具有卓越的性能。本文将通过性能压测对这些框架与Go的Gin框架进行全面对比，揭示它们之间的差异。

二、环境准备

系统环境配置

编程语言

压测工具

这里选择 wrk 工具进行压测，mac 安装直接通过brew快速安装

brew install wrk

window安装可能要依赖它的子系统才方便安装，或者换成其他的压测工具例如JMeter。

web框架

数据库配置

三、wrk 工具 http压测

FastAPI

普通http请求压测

依赖安装

pip install fastapi==0.103.1
pip install uvicorn==0.23.2

编写测试路由

from fastapi import FastAPI


app = FastAPI(summary="fastapi性能测试")


@app.get(path="/http/fastapi/test")
async def fastapi_test():
    return {"code": 0, "message": "fastapi_http_test", "data": {}}

Uvicorn 运行，这里是起四个进程运行部署

uvicorn fastapi_test:app --log-level critical --port 8000 --workers 4

wrk压测

开20个线程，建立500个连接，持续请求30s

wrk -t20 -d30s -c500 http://127.0.0.1:8000/http/fastapi/test

压测结果

➜  ~ wrk -t20 -d30s -c500 http://127.0.0.1:8000/http/fastapi/test

Running 30s test @ http://127.0.0.1:8000/http/fastapi/test
  20 threads and 500 connections
  
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.06ms    2.89ms  36.65ms   85.34%
    Req/Sec     3.85k     3.15k   41.59k    70.05%
    
  2298746 requests in 30.11s, 383.64MB read
  Socket errors: connect 267, read 100, write 0, timeout 0
  
Requests/sec:  76357.51
Transfer/sec:     12.74MB

Thread Stats 这里是 20、30个压测线程的平均结果指标

• 平均延迟(Avg Latency):每个线程的平均响应延迟
• 标准差(Stdev Latency):每个线程延迟的标准差
• 最大延迟(Max Latency):每个线程遇到的最大延迟
• 延迟分布(+/- Stdev Latency):每个线程延迟分布情况
• 每秒请求数(Req/Sec):每个线程每秒完成的请求数
• 请求数分布(+/- Stdev Req/Sec):每个线程请求数的分布情况

Socket errors: connect 267, read 100, write 0, timeout 0，是压测过程中socket的错误统计

• connect:连接错误,表示在压测过程中,总共有 267 次连接异常
• read:读取错误,表示有 100 次读取数据异常
• write:写入错误,表示有0次写入异常
• timeout:超时错误,表示有0次超时

MySQL数据查询请求压测

这里在简单试下数据库查询时候的情况

首先先补充下项目依赖

pip install hui-tools[db-orm, db-redis]==0.2.0

hui-tools是我自己开发的一个工具库，欢迎大家一起来贡献。https://github.com/HuiDBK/py-tools

#!/usr/bin/python3
# -*- coding: utf-8 -*-
# @Author: Hui
# @Desc: { fastapi性能测试 }
# @Date: 2023/09/10 12:24
import uvicorn
from fastapi import FastAPI
from py_tools.connections.db.mysql import SQLAlchemyManager, DBManager

app = FastAPI(summary="fastapi性能测试")


async def init_orm():
    db_client = SQLAlchemyManager(
        host="127.0.0.1",
        port=3306,
        user="root",
        password="123456",
        db_name="house_rental"
    )
    db_client.init_mysql_engine()
    DBManager.init_db_client(db_client)


@app.on_event("startup")
async def startup_event():
    """项目启动时准备环境"""

    await init_orm()
    
@app.get(path="/http/fastapi/mysql/test")
async def fastapi_mysql_query_test():
    sql = "select id, username, role from user_basic where username='hui'"
    ret = await DBManager().run_sql(sql)

    column_names = [desc[0] for desc in ret.cursor.description]
    result_tuple = ret.fetchone()
    user_info = dict(zip(column_names, result_tuple))

    return {"code": 0, "message": "fastapi_http_test", "data": {**user_info}}

wrk压测

wrk -t20 -d30s -c500 http://127.0.0.1:8000/http/fastapi/mysql/test

➜  ~ wrk -t20 -d30s -c500 http://127.0.0.1:8000/http/fastapi/mysql/test

Running 30s test @ http://127.0.0.1:8000/http/fastapi/mysql/test
  20 threads and 500 connections
  
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    38.81ms   19.35ms 226.42ms   76.86%
    Req/Sec   317.65    227.19   848.00     57.21%
    
  180255 requests in 30.09s, 36.95MB read
  Socket errors: connect 267, read 239, write 0, timeout 0
  Non-2xx or 3xx responses: 140
  
Requests/sec:   5989.59
Transfer/sec:      1.23MB

可以发现就加入一个简单的数据库查询，QPS从 76357.51 降到 5989.59 足足降了有10倍多，其实是单机数据库处理不过来太多请求，并发的瓶颈是在数据库，可以尝试加个redis缓存对比MySQL来说并发提升了多少。

Redis缓存查询压测

#!/usr/bin/python3
# -*- coding: utf-8 -*-
# @Author: Hui
# @Desc: { fastapi性能测试 }
# @Date: 2023/09/10 12:24
import json
from datetime import timedelta

import uvicorn
from fastapi import FastAPI
from py_tools.connections.db.mysql import SQLAlchemyManager, DBManager
from py_tools.connections.db.redis_client import RedisManager

app = FastAPI(summary="fastapi性能测试")


async def init_orm():
    db_client = SQLAlchemyManager(
        host="127.0.0.1",
        port=3306,
        user="root",
        password="123456",
        db_name="house_rental"
    )
    db_client.init_mysql_engine()
    DBManager.init_db_client(db_client)


async def init_redis():
    RedisManager.init_redis_client(
        async_client=True,
        host="127.0.0.1",
        port=6379,
        db=0,
    )


@app.on_event("startup")
async def startup_event():
    """项目启动时准备环境"""

    await init_orm()

    await init_redis()


@app.get(path="/http/fastapi/redis/{username}")
async def fastapi_redis_query_test(username: str):
    # 先判断缓存有没有
    user_info = await RedisManager.client.get(name=username)
    if user_info:
        user_info = json.loads(user_info)
        return {"code": 0, "message": "fastapi_redis_test", "data": {**user_info}}

    sql = f"select id, username, role from user_basic where username='{username}'"
    ret = await DBManager().run_sql(sql)

    column_names = [desc[0] for desc in ret.cursor.description]
    result_tuple = ret.fetchone()
    user_info = dict(zip(column_names, result_tuple))

    # 存入redis缓存中, 3min
    await RedisManager.client.set(
        name=user_info.get("username"),
        value=json.dumps(user_info),
        ex=timedelta(minutes=3)
    )

    return {"code": 0, "message": "fastapi_redis_test", "data": {**user_info}}


if __name__ == '__main__':
    uvicorn.run(app)

运行

wrk -t20 -d30s -c500 http://127.0.0.1:8000/http/fastapi/redis/hui

结果

➜  ~ wrk -t20 -d30s -c500 http://127.0.0.1:8000