tc mqprio

已于 2024-08-02 18:37:23 修改
阅读量311 收藏
点赞数 2
CC 4.0 BY-SA版权
文章标签： java 前端 数据库
于 2024-08-02 18:34:38 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/mishuang2017/article/details/140877813

如果是24个cpu的机器，可以用下面的命令创建tc mqprio qdisc:

echo 'legacy' > /sys/class/net/enp8s0f0/compat/devlink/mode
echo 'switchdev' > /sys/class/net/enp8s0f0/compat/devlink/mode
/opt/mellanox/iproute2/sbin/ip link set dev enp8s0f0 up
/opt/mellanox/iproute2/sbin/ip addr add 193.237.149.223/16 dev enp8s0f0
/opt/mellanox/ethtool/sbin/ethtool --set-channels enp8s0f0 combined 24
/opt/mellanox/iproute2/sbin/tc qdisc del dev enp8s0f0 root mqprio
/opt/mellanox/iproute2/sbin/tc qdisc add dev enp8s0f0 root mqprio num_tc 7 map 1 5 4 1 1 1 1 1 2 0 1 1 6 1 3 1 queues 6@0 6@6 6@12 1@18 1@19 1@20 3@21 hw 1 mode channel shaper bw_rlimit max_rate 12.14Gbit 1.14Gbit 1.14Gbit 2.14Gbit 1.14Gbit 1.14Gbit 1.14Gbit

所有map对应的queue加起来要等于24，否则会报错。

输出入下：
 

[root@c-234-181-80-083 ~]# tc qdisc show dev $link
qdisc mqprio 800e: root tc 7 map 1 5 4 1 1 1 1 1 2 0 1 1 6 1 3 1
             queues:(0:5) (6:11) (12:17) (18:18) (19:19) (20:20) (21:23)
             mode:channel
             shaper:bw_rlimit   max_rate:12140Mbit 1140Mbit 1140Mbit 2140Mbit 1140Mbit 1140Mbit 1140Mbit
qdisc pfifo_fast 0: parent 800e:18 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:17 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:16 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:15 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:14 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:13 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:12 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:11 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:10 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:f bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:e bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:d bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:c bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:b bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:a bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:9 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:8 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:7 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:6 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:5 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:4 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:3 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc pfifo_fast 0: parent 800e:1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

打印一些debug：
 

[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 6, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 6, offset: 6
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 6, offset: 12
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 1, offset: 18
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 1, offset: 19
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 1, offset: 20
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 3, offset: 21
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_mqprio_build_tc_to_txq: count: 0, offset: 0
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 0
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16386
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 1
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16386
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 2
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16386
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 3
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16386
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 4
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16386
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 5
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16386
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 6
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16387
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 7
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16387
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 8
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16387
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 9
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16387
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 10
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16387
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 11
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16387
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 12
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16388
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 13
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16388
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 14
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16388
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 15
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16388
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 16
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16388
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 17
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16388
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 18
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16389
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 19
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16390
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 20
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16391
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 21
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16392
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 22
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16392
[Fri Aug  2 10:53:43 2024] mlx5e_open_channels: chs->num: 24, 23
[Fri Aug  2 10:53:43 2024] mlx5e_open_sqs: group_id: 16392
enp8s0f0
mlx5e_netdev_ops
dev.num_tc: 7
(struct netdev_tc_txq [16]){
        {
                .count = (u16)6,
                .offset = (u16)0,
        },
        {
                .count = (u16)6,
                .offset = (u16)6,
        },
        {
                .count = (u16)6,
                .offset = (u16)12,
        },
        {
                .count = (u16)1,
                .offset = (u16)18,
        },
        {
                .count = (u16)1,
                .offset = (u16)19,
        },
        {
                .count = (u16)1,
                .offset = (u16)20,
        },
        {
                .count = (u16)3,
                .offset = (u16)21,
        },
}
dev.real_num_tx_queues: 24
mlx5e_priv.channels.params.mqprio
(struct <anonymous>){
        .mode = (u16)1,
        .num_tc = (u8)7,
        .tc_to_txq = (struct netdev_tc_txq [16]){
                {
                        .count = (u16)6,
                        .offset = (u16)0,
                },
                {
                        .count = (u16)6,
                        .offset = (u16)6,
                },
                {
                        .count = (u16)6,
                        .offset = (u16)12,
                },
                {
                        .count = (u16)1,
                        .offset = (u16)18,
                },
                {
                        .count = (u16)1,
                        .offset = (u16)19,
                },
                {
                        .count = (u16)1,
                        .offset = (u16)20,
                },
                {
                        .count = (u16)3,
                        .offset = (u16)21,
                },
        },
        .channel = (struct <anonymous>){
                .max_rate = (u64 [16]){ 1517500000, 142500000, 142500000, 267500000, 142500000, 142500000, 142500000 },
                .hw_id = (u32 [16]){ 16386, 16387, 16388, 16389, 16390, 16391, 16392 },
        },
}
*(struct mlx5e_mqprio_rl *)0xffff8e038c64e880 = {
        .mdev = (struct mlx5_core_dev *)0xffff8e03d2a3c380,
        .root_id = (u32)16672,
        .leaves_id = (u32 *)0xffff8e038c64e860,
        .num_tc = (u8)7,
}
mlx5e_priv.mqprio_rl.leaves_id
(u32)16386
(u32)16387
(u32)16388
(u32)16389
(u32)16390
(u32)16391
(u32)16392

每个tcX映射一个或多个channel，或者叫netdev queues，创建channel的时候会传进去一个hw_id，这样每个channel就知道了自己对应qos的信息。

下面的命令是一个42 cpu的机器上使用的，还没研究。

/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 0 equal 16
/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 16 equal 16 context new
/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 32 equal 4 context new
/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 36 equal 1 context new
/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 37 equal 1 context new
/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 38 equal 2 context new
/opt/mellanox/ethtool/sbin/ethtool -X enp8s0f0 start 40 equal 2 context new
/opt/mellanox/ethtool/sbin/ethtool --show-ntuple enp8s0f0
/opt/mellanox/ethtool/sbin/ethtool --show-ntuple enp8s0f0
/opt/mellanox/ethtool/sbin/ethtool --show-ntuple enp8s0f0
/opt/mellanox/ethtool/sbin/ethtool --show-ntuple enp8s0f0
/opt/mellanox/ethtool/sbin/ethtool --config-ntuple enp8s0f0 flow-type  tcp4 dst-ip 193.237.149.223 dst-port 19493 context 5 loc 792
/opt/mellanox/ethtool/sbin/ethtool --config-ntuple enp8s0f0 flow-type  tcp4 dst-ip 193.237.149.223 dst-port 19490 context 2 loc 20
/opt/mellanox/ethtool/sbin/ethtool --config-ntuple enp8s0f0 flow-type  tcp4 dst-ip 193.237.149.223 dst-port 19489 context 0 loc 345
/opt/mellanox/ethtool/sbin/ethtool --config-ntuple enp8s0f0 flow-type  tcp4 dst-ip 193.237.149.223 dst-port 19494 context 6 loc 369
/opt/mellanox/ethtool/sbin/ethtool --config-ntuple enp8s0f0 flow-type  tcp4 dst-ip 193.237.149.223 dst-port 19491 context 3 loc 351
/opt/mellanox/ethtool/sbin/ethtool --config-ntuple enp8s0f0 flow-type  tcp4 dst-ip 193.237.149.223 dst-port 19492 context 4 loc 780

mishuang2017
关注
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值