RTDETR融合FMNet中的FrequencyAttention模块

​

RT-DETR使用教程： RT-DETR使用教程

RT-DETR改进汇总贴：RT-DETR更新汇总贴

《FMNet: Frequency-Assisted Mamba-Like Linear Attention Network for Camouflaged Object Detection》

一、 模块介绍

        论文链接：https://arxiv.org/abs/2503.11030

论文速览：

        伪装物体检测 （COD） 具有挑战性，因为伪装物体与其周围环境之间具有很强的相似性，这使得识别变得复杂。现有方法主要依赖于空间局部特征，无法捕获全局信息，而 Transformer 增加了计算成本。为了解决这个问题，提出了频率辅助类曼巴线性注意力网络（FMNet），它利用频域学习来有效地捕获全局特征并减轻物体与背景之间的模糊性。FMNet 引入了多尺度频率辅助曼巴样线性注意力 （MFM） 模块，通过多尺度结构集成频率和空间特征，以处理尺度变化，同时降低计算复杂性。此外，金字塔频率注意力提取 （PFAE） 模块和频率反向解码器 （FRD） 增强了语义并重建了特征。

总结：本文更新其中的FrequencyAttention模块代码及使用方法。​

⭐⭐本文二创模块仅更新于付费群中，往期免费教程可看下方链接⭐⭐

RT-DETR更新汇总贴（含免费教程）文章浏览阅读264次。RT-DETR使用教程：缝合教程： RT-DETR中的yaml文件详解：labelimg使用教程：_rt-deterhttps://xy2668825911.blog.csdn.net/article/details/143696113https://xy2668825911.blog.csdn.net/article/details/143696113

二、二创融合模块

2.1 相关代码

# https://blog.csdn.net/StopAndGoyyy?spm=1011.2124.3001.5343
# https://arxiv.org/pdf/2503.11030?
# FMNet: Frequency-Assisted Mamba-Like Linear Attention Network for Camouflaged Object Detection
from einops import rearrange, repeat
from torch.nn import Softmax

def custom_complex_normalization(input_tensor, dim=-1):
    real_part = input_tensor.real
    imag_part = input_tensor.imag
    norm_real = F.softmax(real_part, dim=dim)
    norm_imag = F.softmax(imag_part, dim=dim)

    normalized_tensor = torch.complex(norm_real, norm_imag)

    return normalized_tensor

class FrequencyAttention(nn.Module):
    def __init__(self, in_dim):
        super(FrequencyAttention, self).__init__()

        down_dim = in_dim // 2

        self.conv1 = nn.Sequential(
            nn.Conv2d(in_dim, down_dim, kernel_size=1), nn.BatchNorm2d(down_dim), nn.ReLU(True)
        )

        self.conv2 = nn.Sequential(
            nn.Conv2d(in_dim, down_dim, kernel_size=3, dilation=3, padding=3), nn.BatchNorm2d(down_dim), nn.ReLU(True)
        )
        self.query_conv2 = nn.Conv2d(in_channels=down_dim, out_channels=down_dim//8, kernel_size=1)
        self.key_conv2 = nn.Conv2d(in_channels=down_dim, out_channels=down_dim//8, kernel_size=1)
        self.value_conv2 = nn.Conv2d(in_channels=down_dim, out_channels=down_dim, kernel_size=1)
        self.gamma2 = nn.Parameter(torch.zeros(1))

        self.temperature = nn.Parameter(torch.ones(8, 1, 1))

        self.weight = nn.Sequential(
            nn.Conv2d(down_dim, down_dim // 16, 1, bias=True),
            nn.BatchNorm2d(down_dim // 16),
            nn.ReLU(True),
            nn.Conv2d(down_dim // 16, down_dim, 1, bias=True),
            nn.Sigmoid())

        self.softmax = Softmax(dim=-1)
        self.norm = nn.BatchNorm2d(down_dim)
        self.relu = nn.ReLU(True)
        self.num_heads = 8

    def forward(self, x):

        conv2 = self.conv2(x)
        b, c, h, w = conv2.shape

        q_f_2 = torch.fft.fft2(conv2.float())
        k_f_2 = torch.fft.fft2(conv2.float())
        v_f_2 = torch.fft.fft2(conv2.float())
        tepqkv = torch.fft.fft2(conv2.float())

        q_f_2 = rearrange(q_f_2, 'b (head c) h w -> b head c (h w)', head=self.num_heads)
        k_f_2 = rearrange(k_f_2, 'b (head c) h w -> b head c (h w)', head=self.num_heads)
        v_f_2 = rearrange(v_f_2, 'b (head c) h w -> b head c (h w)', head=self.num_heads)

        q_f_2 = torch.nn.functional.normalize(q_f_2, dim=-1)
        k_f_2 = torch.nn.functional.normalize(k_f_2, dim=-1)
        attn_f_2 = (q_f_2 @ k_f_2.transpose(-2, -1)) * self.temperature
        attn_f_2 = custom_complex_normalization(attn_f_2, dim=-1)
        out_f_2 = torch.abs(torch.fft.ifft2(attn_f_2 @ v_f_2))
        out_f_2 = rearrange(out_f_2, 'b head c (h w) -> b (head c) h w', head=self.num_heads, h=h, w=w)
        out_f_l_2 = torch.abs(torch.fft.ifft2(self.weight(tepqkv.real)*tepqkv))
        out_2 = torch.cat((out_f_2,out_f_l_2),1)

        F_2 = torch.add(out_2, x)

        return F_2

2.2 更改yaml文件 （以自研模型加入为例）

yam文件解读：YOLO系列 “.yaml“文件解读_yolo yaml文件-CSDN博客

       打开更改ultralytics/cfg/models/rt-detr路径下的rtdetr-l.yaml文件，替换原有模块。

​​

# Ultralytics YOLO 🚀, AGPL-3.0 license
# RT-DETR-l object detection model with P3-P5 outputs. For details see https://docs.ultralytics.com/models/rtdetr
# ⭐⭐Powered by https://blog.csdn.net/StopAndGoyyy,  技术指导QQ:2668825911⭐⭐

# Parameters
nc: 80 # number of classes
scales: # model compound scaling constants, i.e. 'model=yolov8n-cls.yaml' will call yolov8-cls.yaml with scale 'n'
  # [depth, width, max_channels]
  l: [1.00, 1.00, 512]
#  n: [ 0.33, 0.25, 1024 ]
#  s: [ 0.33, 0.50, 1024 ]
#  m: [ 0.67, 0.75, 768 ]
#  l: [ 1.00, 1.00, 512 ]
#  x: [ 1.00, 1.25, 512 ]
# ⭐⭐Powered by https://blog.csdn.net/StopAndGoyyy,  技术指导QQ:2668825911⭐⭐

backbone:
  # [from, repeats, module, args]
  - [-1, 1, Conv, [64, 3, 2]] # 0-P1/2
  - [-1, 1, Conv, [128, 3, 2]] # 1-P2/4
  - [-1, 2, CCRI, [128, 5, True, False]]
  - [-1, 1, Conv, [256, 3, 2]] # 3-P3/8
  - [-1, 4, CCRI, [256, 3, True, True]]
  - [-1, 1, Conv, [512, 3, 2]] # 5-P4/16
  - [-1, 4, CCRI, [512, 3, True, True]]
  - [-1, 1, Conv, [1024, 3, 2]] # 7-P5/32
  - [-1, 2, CCRI, [1024, 3, True, False]]

head:
  - [-1, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 9 input_proj.2
  - [-1, 1, FrequencyAttention, []]
  - [-1, 1, Conv, [256, 1, 1]] # 11, Y5, lateral_convs.0

  - [-1, 1, nn.Upsample, [None, 2, "nearest"]]
  - [6, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 13 input_proj.1
  - [[-2, -1], 1, Concat, [1]]
  - [-1, 2, RepC4, [256]] # 15, fpn_blocks.0
  - [-1, 1, Conv, [256, 1, 1]] # 16, Y4, lateral_convs.1

  - [-1, 1, nn.Upsample, [None, 2, "nearest"]]
  - [4, 1, Conv, [256, 1, 1, None, 1, 1, False]] # 18 input_proj.0
  - [[-2, -1], 1, Concat, [1]] # cat backbone P4
  - [-1, 2, RepC4, [256]] # X3 (20), fpn_blocks.1

  - [-1, 1, Conv, [256, 3, 2]] # 22, downsample_convs.0
  - [[-1, 16], 1, Concat, [1]] # cat Y4
  - [-1, 2, RepC4, [256]] # F4 (23), pan_blocks.0

  - [-1, 1, Conv, [256, 3, 2]] # 24, downsample_convs.1
  - [[-1, 11], 1, Concat, [1]] # cat Y5
  - [-1, 2, RepC4, [256]] # F5 (26), pan_blocks.1

  - [[20, 23, 26], 1, RTDETRDecoder, [nc]] # Detect(P3, P4, P5)
# ⭐⭐Powered by https://blog.csdn.net/StopAndGoyyy,  技术指导QQ:2668825911⭐⭐

 2.2 修改train.py文件

       创建Train_RT脚本用于训练。

from ultralytics.models import RTDETR
import os
os.environ['KMP_DUPLICATE_LIB_OK'] = 'True'

if __name__ == '__main__':
    model = RTDETR(model='ultralytics/cfg/models/rt-detr/rtdetr-l.yaml')
    # model.load('yolov8n.pt')
    model.train(data='./data.yaml', epochs=2, batch=1, device='0', imgsz=640, workers=2, cache=False,
                amp=True, mosaic=False, project='runs/train', name='exp')

​​

         在train.py脚本中填入修改好的yaml路径，运行即可训。