Attention-based destruction and construction learning for infrared object fine-grained categorization

Hao Cui; Ran Li; Yan Li; Liang Zhang; Bingyang Li

doi:10.3934/mfc.2022046

Article Contents

2023, Volume 6, Issue 4: 742-752. Doi: 10.3934/mfc.2022046

This issue Previous Article Almost sure convergence for the maxima and minima of strongly dependent nonstationary multivariate Gaussian sequences Next Article Federated learning for minimizing nonsmooth convex loss functions

Attention-based destruction and construction learning for infrared object fine-grained categorization

1.
China Airborne Missile Academy, Luoyang, China
2.
School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
3.
School of Cyber Science and Engineering, University of International Relations, Beijing, China

^*Corresponding author: Yan Li
^*Corresponding author: Yan Li

†Equal contributions

Received: April 2022

Revised: October 2022

Early access: October 28, 2022

Published online: October 28, 2022

The first author is supported by National Natural Science Foundation of China (Grant number: 61976066), Beijing Natural Science Foundation (Grant number: 4212031), and Research Funds for NSD Construction, University of International Relations (Grant numbers:2019GA43; 2021GA07).

Abstract / Introduction Full Text(HTML) Figure(9) / Table(4) Related Papers Cited by

Abstract

The infrared ship fine-grained categorization technology has wide applications in both civil and military fields. In order to improve the fine-grained categorization accuracy of infrared ships, this paper integrates the channel attention mechanism to design a SE-Resnet module. We use it as the backbone fed into the destruction and construction learning(DCL) framework, which has been proved as one of the most effective fine-grained categorization methods for infrared small object recognition. The experimental results show that our module helps DCL locate key sub-regions, and can further improve the accuracy of fine-grained categorization for infrared ship targets.

Keywords:

Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.

Citation:

Full Text(HTML)

Figure 1. A model for fine-grained picture categorization

Download: Full-size image PowerPoint slide

Figure 2. SE-ResNet Module

Download: Full-size image PowerPoint slide

Figure 3. Regional alignment network

Download: Full-size image PowerPoint slide

Figure 4. Display of Infrared Ship Datasets

Download: Full-size image PowerPoint slide

Figure 5. Burke

Download: Full-size image PowerPoint slide

Figure 6. Ford

Download: Full-size image PowerPoint slide

Figure 7. Nimitz

Download: Full-size image PowerPoint slide

Figure 8. Tikon

Download: Full-size image PowerPoint slide

Figure 9. Others

Download: Full-size image PowerPoint slide

Table 1. Training equipment and training parameters

Inter(R) Core(TM) i7-7700HQ CPU @ 2.80GHz 2.80GHz GPU:NVIDIA GeForce GTX 1050, Windows64-bit operating system Python3.6, pytorch0.4.0, CUDA10.0

Method	Optimizer	Learning rate	Epochs
DCL	SGD	0.0008	180

| Show Table

DownLoad: CSV

Table 2. Comparison of different fine-grained image classification methods

Method	Backbone	Train Accuracy	Test Accuracy

RA-CNN [9]	VGG19	88.67%	87.55%
OPAM [22]	VGG16	89.31%	88.15%
Kernel-Pooling [4]	ResNet50	89.43%	88.87%
DCL [4]	ResNet50	90.54%	90.17%
DCL-SE (ours)	ResNet50	91.73%	93.61%

| Show Table

DownLoad: CSV

Table 3. Results of fine-grained categorization experiments

Epochs = 180 train:val=7:3 SE-ResNet50	1Burke	2Ford	3Nimitz	4Ticon	5Others (Including Sun, Kiev, King Kong, Cruise, and Sacramento).	Training set results	Test set results

Training set	998	1008	1005	1001	483	90.54%	—
Test set	427	431	430	428	207	—	90.17%
Number of misidentifications	40	27	35	59	28
Top-1 Accuracy	90.63%	93.74%	91.86%	86.21%	86.47%

| Show Table

DownLoad: CSV

Table 4. Results of fine-grained categorization experiments

Epochs = 180 train:val=7:3 SE-ResNet50	1Burke	2Ford	3Nimitz	4Ticon	5Others (Including Sun, Kiev, King Kong, Cruise, and Sacramento).	Training set results	Test set results

Training set	998	1008	1005	1001	483	91.73%	—
Test set	427	431	430	428	207	—	93.61%
Number of misidentifications	27	16	21	54	22
Top-1 Accuracy	93.68%	96.29%	95.12%	91.36%	89.37%

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	S. Cai, W. Zuo and L. Zhang, Higher-Order Integration of Hierarchical Convolutional Activations for Fine-Grained Visual Categorization, Proceedings of the IEEE international conference on computer vision, 2017.
[2]	H. Chen, et al., Binarized neural architecture search for efficient object recognition, International Journal of Computer Vision, 129 (2021), 501-516.
[3]	R.-M. Chen, et al., Infrared aircraft few-shot classification method based on meta learning, Journal of Infrared And Millimeter Waves, 40 (2021), 554-560.
[4]	Y. Chen, et al., Destruction and Construction Learning for Fine-Grained Image Recognition, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019.
[5]	J. Choi, et al., Visual Tracking using Attention-Modulated Disintegration and Integration, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[6]	X. Chu, et al., Multi-Context Attention for Human Pose Estimation, Proceedings of the IEEE Conference on computer vision and pattern recognition, 2017.
[7]	R. Du, et al., Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches, European Conference on Computer Vision. Springer, Cham, 2020.
[8]	H. Feng, et al., CNN models for readability of Chinese texts, Mathematical Foundations of Computing, 5 (2022), 351.
[9]	J. Fu, H. Zheng and M. Tao, Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
[10]	Y. Gao, et al., Compact Bilinear Pooling, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[11]	W. Ge, X. Lin and Y. Yu, Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Up, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[12]	J. Hu, S. Li and G. Sun, Squeeze-and-Excitation Networks, Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
[13]	S. Huang, et al., Part-Stacked CNN for Fine-Grained Visual Categorization, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[14]	D. Lin, et al., Deep Lac: Deep Localization, Alignment and Classification for Fine-Grained Recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
[15]	T.-Y. Lin, R. C. Aruni and M. Subhransu, Bilinear CNN Models for Fine-Grained Visual Recognition, Proceedings of the IEEE international conference on computer vision, 2015.
[16]	X. Liu, et al., Fully convolutional attention networks for fine-grained recognition, arXiv preprint, (2016). arXiv: 1603.06765.
[17]	W. Luo, et al., Cross-X Learning for Fine-Grained Visual Categorization, Proceedings of the IEEE/CVF international conference on computer vision, 2019.
[18]	M. Mao, et al., IffDetector: Inference-Aware Feature Filtering for Object Detection, IEEE Transactions on Neural Networks and Learning Systems, 2021.
[19]	M. Mao, et al., SEMANTIC-RTAB-MAP (SRM): A semantic SLAM system with CNNs on depth images, Mathematical Foundations of Computing, 2 (2019), 29.
[20]	T. Mao, Z. Shi and D. Zhou, Approximating functions with multi-features by deep convolutional neural networks, Analysis and Applications, 21 (2023), in press.
[21]	Z. Miao, X. Zhao and J. Wang, et al., Using fine-grained recognition technology in the military domain, National Defense Technology, 42 (2021), 57-63.
[22]	Y. Peng, X. He and J. Zhao, Object-part attention model for fine-grained image classification, IEEE Transactions on Image Processing, 27 (2018), 1487-1500. doi: 10.1109/TIP.2017.2774041.
[23]	S. Ren, et al., Faster R-CNN: Towards real-time object detection with region proposal networks, Advances in Neural Information Processing Systems, 28 (2015).
[24]	S. Sharma, K. Ryan and S. Ruslan, Action recognition using visual attention, arXiv preprint, (2015). arXiv: 1511.04119.
[25]	M. Sun, et al., Multi-Attention Multi-Class Constraint for Fine-Grained Image Recognition, Proceedings of the European Conference on Computer Vision (ECCV), 2018.
[26]	J. B. Tenenbaum and W. T. Freeman, Separating style and content with bilinear models, Neural Computation, 12 (2000), 1247-1283.
[27]	A. Vaswani, et al., Attention is all you need, Advances in neural information processing systems, 30 (2017).
[28]	F. Wang, et al., Residual Attention Network for Image Classification, Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
[29]	Q. Wang, P. Li and L. Zhang, G2DeNet: Global Gaussian Distribution Embedding Network and its Application to Visual Recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
[30]	Y. Wang, V. I. Morariu and L. S. Davis, Learning a Discriminative Filter Bank within a CNN for Fine-Grained Recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
[31]	Y. Wang, Y. Wu and C. Xu, Infrared ship target detection algorithm based on deep transfer learning, Air & Space Defense, 4 (2021), 61-66.
[32]	C. Yu, et al., Hierarchical Bilinear Pooling for Fine-Grained Visual Recognition, Proceedings of the European conference on computer vision (ECCV), 2018.
[33]	H. Zhang, et al., Spda-CNN: Unifying Semantic Part Detection and Abstraction for Fine-Grained Recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[34]	H. Zheng, et al., Learning deep bilinear transformation for fine-grained image representation, Advances in Neural Information Processing Systems, 32 (2019). doi: 10.1109/TNNLS.2017.2665555.
[35]	N. Zhang, et al., Part-Based R-CNNs for Fine-Grained Category Detection, European conference on computer vision. Springer, Cham, 2014.
[36]	H.-H. Zhou, J. Jiang and P. Zhang, et al., Research on fine-grained recognition of ship infrared wake image based on multiple morphological spectrum, Techniques of Automation & Application, 40 (2021), 108-112.
[37]	P. Zhuang, Y. Wang and Q. Yu, Learning attentive pairwise interaction for fine-grained classification, Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. No. 07. 2020.