A Systematic Benchmark of Attention Mechanisms on the NEU-DET Strip Surface Defect Dataset Using YOLOv11n

Abstract

Strip surface defect detection is an important part of the steel industry. Attention mechanisms have been widely used to enhance the feature representation capability of detection networks; however, their effectiveness on small industrial defect datasets lacks systematic evaluation. In this paper, we take the lightweight detector YOLOv11n as the baseline and integrate 15 attention mechanisms into a unified framework, covering channel attentions (SE and ECA), spatial attention (SimAM), hybrid attentions (CBAM and GAM), coordinate attention (CA), efficient multi-scale attention (EMA), and others. Strictly controlled experiments are conducted on the NEU-DET strip surface defect dataset. All models are trained under identical conditions: 50 epochs, 200×200 input resolution, batch size 16, 70/30 train/validation split, and random seed 42. Experimental results show that the original YOLOv11n baseline achieves the highest mAP@0.5 and mAP@0.5:0.95, outperforming all attention variants. The three best-performing attentions, namely ELA, TripletAttention, and SimAM, all achieve a mAP@0.5 of 0.735, only 0.003 lower than the baseline. In contrast, CBAM and SE, which have more parameters, drop by 0.066 and 0.076, respectively, and training curve analysis confirms that the degradation stems from overfitting. Per-class analysis reveals that crazing and rolled-in scale are the most challenging defect categories, with no attention mechanism improving detection accuracy on these classes over the baseline. The parameter-free SimAM shows the smallest drop, supporting the view that lightweight designs are safer on small datasets. Supplementary experiments at 640×640 resolution further validate the superiority of the baseline (baseline 0.772 vs. SimAM 0.761, ELA 0.741, SE 0.684). All models meet the real-time detection requirements. This baseline provides the real-world results of model selection for small industrial defect samples: a good baseline is often enough, attention should be emphasized, and parameter free or low-parameter designs should be chosen.