Segmentation method of the abnormal area of coal infrared thermal image
-
摘要: 红外辐射可反映煤岩受载破坏情况，用于监测和预防煤岩动力灾害，但红外热像仪生成的红外热像图像素分辨率低、噪声较大，导致检测结果受主观因素影响较大，无法准确识别煤体损伤区域。将深度学习和红外热像结合进行无损检测已成为趋势，但目前结合深度学习和红外热像对煤体受载破坏进行识别检测的研究相对较少。针对上述问题，提出一种基于多尺度通道注意力模块(MS−CAM)U−Net模型的煤体红外热像异常区域分割方法。在传统U−Net模型的编码器中引入MS−CAM，设计了基于MS−CAM的U−Net模型结构，使模型在关注煤体红外热像异常区域显著特征的同时，还关注异常区域小目标特征，以提高异常区域分割精度。为降低煤体红外热像数据集匮乏对模型准确率和适用性的影响，对创建的煤体红外热像数据集进行数据增强操作，并采用MS COCO数据集对基于MS−CAM的U−Net模型进行预训练，再采用煤体红外热像数据集训练，得出最终网络权重。实验结果表明，该方法可有效分割煤体红外热像异常区域，精确率、F1分数、Dice系数和平均交并比分别为94.75%，94.94%，94.65%，90.03%，均优于Deeplab模型、U−Net模型和基于SENet注意力机制的U−Net模型。Abstract: Infrared radiation can reflect the damage of coal and rock under load, and can be used to monitor and prevent the dynamic disaster of coal and rock. But the infrared thermal image generated by the infrared thermal imager has low pixel resolution and large noise, which leads to the detection result being greatly affected by subjective factors. Therefore, the damaged area of the coal body cannot be accurately identified. It has become a trend to combine deep learning with infrared thermal imaging for nondestructive testing. But the research on the identification and detection of coal damage under load by combining deep learning and infrared thermal imaging is relatively few. In order to solve the above problems, a segmentation method of the abnormal area of coal infrared thermal image based on multi-scale channel attention module (MS-CAM) U-Net model is proposed. The MS-CAM is introduced into the encoder of the traditional U-Net model, and the U-Net model structure based on MS-CAM is designed. The model not only pays attention to the major characteristics of the coal infrared thermal image abnormal area, but also pays attention to the small target characteristics of the abnormal area, so as to improve the segmentation accuracy of the abnormal area. In order to reduce the influence of the lack of coal infrared thermal image data set on the accuracy and applicability of the model, the data enhancement operation is carried out on the created coal infrared thermal image data set. The MS-CAM-based U-Net model is pre-trained by using the MS COCO data set. Then the coal infrared thermal image data set is used for training to obtain the final network weight. The experimental result shows that the method can effectively segment the abnormal areas of the infrared thermal image of the coal body. The accuracy rate, the F1 score, the Dice coefficient and the average cross-combination ratio are 94.75%, 94.94%, 94.65%, and 90. 03% respectively. The results are superior to the Deeplab model, the U-Net model and the U-Net model based on the attention mechanism of the SENet.
-
-
![]()
图 1 基于MS−CAM的U−Net模型结构
Figure 1. U-Net model structure based on multi-scale channel attention module(MS-CAM)
![]()
图 4 煤样3种加载破坏时期红外热像图
Figure 4. Three infrared thermal images of coal samples during loading pressure period
![]()
图 5 LabelMe工具中煤样红外热像异常区域标注
Figure 5. Abnormal area tagging in infrared thermal images of coal samples in LabelMe tool
![]()
图 9 不同模型对煤体红外热像异常区域的分割结果
Figure 9. Segmentation results of infrared thermal images of coal samples by different models
![]()
图 10 U−Net（SENet）模型和U−Net（MS−CAM）模型分割结果的类激活热力图
Figure 10. Class activation heat map of segmentation results by U-Net（SENet） model and U-Net（MS-CAM）model
表 1 基于MS−CAM的U−Net模型网络结构及对应特征图
Table 1 Network structures of U-Net model based on MS-CAM and corresponding characteristic images
网络结构 特征图尺寸 卷积核参数 Conv_1 256×256×64
256×256×643×3×64
3×3×64MS−CAM_1 256×256×64 − DownSampling_1 128×128×64 2×2 Conv_2 128×128×128
128×128×1283×3×128
3×3×128MS−CAM_2 128×128×128 − DownSampling_2 64×64×128 2×2 Conv_3 64×64×256
64×64×2563×3×256
3×3×256MS−CAM_3 64×64×256 − DownSampling_3 32×32×256 2×2 Conv_4 32×32×512
32×32×5123×3×512
3×3×512MS−CAM_4 32×32×512 − DownSampling_4 16×16×512 2×2 Conv 16×16×512
16×16×5123×3×512
3×3×512UpSampling_1 32×32×512 2×2 Concatenate_1 32×32×1024 UpSampling_1+ MS−CAM_4 conv_1 32×32×256
32×32×2563×3×256
3×3×256UpSampling_2 64×64×256 2×2 Concatenate_2 64×64×512 UpSampling_2+ MS−CAM_3 conv_2 64×64×128
64×64×1283×3×128
3×3×128UpSampling_3 128×128×128 2×2 Concatenate_3 128×128×256 UpSampling_3+ MS−CAM_2 conv_3 128×128×64
128×128×643×3×64
3×3×64UpSampling_4 256×256×64 2×2 Concatenate_4 256×256×128 UpSampling_4+ MS−CAM_1 conv_4 256×256×64
256×256×643×3×64
3×3×64Conv1×1 256×256×1 1×1×1 
下载: 导出CSV
表 2 不同模型分割结果评价指标对比
Table 2 Comparison of evaluation indexes for segmentation results of different models
% 模型 精确率 F1分数 Dice系数 MIoU Deeplab 88.65 90.31 87.92 83.78 U−Net 92.28 91.67 91.58 84.85 U−Net(SENet) 93.46 93.56 92.81 87.28 U−Net（MS−CAM） 94.75 94.94 94.65 90.03
下载: 导出CSV
-
[1] 程富起,李忠辉,魏洋,等. 基于单轴压缩红外辐射的煤岩损伤演化特征[J]. 工矿自动化,2018,44(5):64-70. DOI: 10.13272/j.issn.1671-251x.2017110064 CHENG Fuqi,LI Zhonghui,WEI Yang,et al. Coal-rock damage evolution characteristics based on infrared radiation under uniaxial compression[J]. Industry and Mine Automation,2018,44(5):64-70. DOI: 10.13272/j.issn.1671-251x.2017110064
[2] 娄全,李忠辉,李爱国,等. 混凝土变形破坏的红外辐射特征研究[J]. 工矿自动化,2015,41(7):44-48. LOU Quan,LI Zhonghui,LI Aiguo,et al. Research of infrared radiation characteristics of concrete deformation and failure[J]. Industry and Mine Automation,2015,41(7):44-48.
[3] TIAN He,LI Zhonghui,SHEN Xiaofan,et al. Identification method of infrared radiation precursor information of coal sample failure and instability under uniaxial compression[J]. Infrared Physics & Technology,2021,119:103957.
[4] SUN Hai,MA Liqiang,LIU Wei,et al. The response mechanism of acoustic and thermal effect when stress causes rock damage[J]. Applied Acoustics,2021,180:108093. DOI: 10.1016/j.apacoust.2021.108093
[5] LIU Wei,MA Liqiang,SUN Hai,et al. Using the characteristics of infrared radiation b-value during the rock fracture process to offer a precursor for serious failure[J]. Infrared Physics & Technology,2021,114:103644.
[6] 宋晶晶,李忠辉,张昕,等. 岩样损伤红外热像的归一化直方图表征研究[J]. 红外技术,2021,43(8):777-783. SONG Jingjing,LI Zhonghui,ZHANG Xin,et al. Research on normalized histogram characterization of infrared thermal image of rock sample damage[J]. Infrared Technology,2021,43(8):777-783.
[7] CAO K,YUAN Q,XIE G,et al. Infrared radiation characteristics during crack development in water-bearing sandstone[J]. Soil Mechanics and Foundation Engineering,2021,58(3):209-214. DOI: 10.1007/s11204-021-09730-2
[8] LI Zhonghui,YIN Shan,NIU Yue,et al. Experimental study on the infrared thermal imaging of a coal fracture under the coupled effects of stress and gas[J]. Journal of Natural Gas Science and Engineering,2018,55:444-451. DOI: 10.1016/j.jngse.2018.05.019
[9] ZHANG Xueliang,WANG Deliang. Deep learning based binaural speech separation in reverberant environments[J]. IEEE/ACM Transactions on Audio,Speech,and Language Processing,2017,25(5):1075-1084. DOI: 10.1109/TASLP.2017.2687104
[10] WANG Bin,DONG Ming,REN Ming,et al. Automatic fault diagnosis of infrared insulator images based on image instance segmentation and temperature analysis[J]. IEEE Transactions on Instrumentation and Measurement,2020,69(8):5345-5355. DOI: 10.1109/TIM.2020.2965635
[11] AKRAM M W,LI Guiqiang,JIN Yi,et al. Automatic detection of photovoltaic module defects in infrared images with isolated and develop-model transfer deep learning[J]. Solar Energy,2020,198:175-186. DOI: 10.1016/j.solener.2020.01.055
[12] BANG H T,PARK S,JEON H. Defect identification in composite materials via thermography and deep learning techniques[J]. Composite Structures,2020,246:112405. DOI: 10.1016/j.compstruct.2020.112405
[13] DAI Yimian, GIESEKE F, OEHMCKE S, et al. Attentional feature fusion[C]. The IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, 2021: 3560-3569.
[14] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]. International Conference on Machine Learning, Lille, 2015: 448-456.
[15] SHARMA G, LIU D, MAJI S, et al. Parsenet: a parametric surface fitting network for 3D point clouds[C]. European Conference on Computer Vision, Glasgow, 2020: 261-276.
[16] WU Yuxin,HE Kaiming. Group normalization[J]. International Journal of Computer Vision,2018,128(3):742-755.
[17] TAN Chuanqi, SUN Fuchuan, KONG Tao, et al. A survey on deep transfer learning[C]. International Conference on Artificial Neural Networks, Rhodes, 2018: 270-279.
[18] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]. European Conference on Computer Vision, Zurich, 2014: 740-755.
[19] MILLETARI F, NAVAB N, AHMADI S A. V-net: fully convolutional neural networks for volumetric medical image segmentation[C]. The 4th International Conference on 3D Vision, Stanford, 2016: 565-571.
[20] SHELHAMER E,LONG J,DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,39(4):640-651.
-
期刊类型引用(26)
1. 程继杰，刘毅. 基于深度特征的煤矿冲击地压和煤与瓦斯突出图像感知报警方法. 煤炭科学技术. 2024(03): 245-257 . 
百度学术
2. 梁运培，郑梦浩，李全贵，毛树人，栗小雨，李建波，周俊江. 我国煤与瓦斯突出预测与预警研究现状. 煤炭学报. 2023(08): 2976-2994 . 百度学术
3. 赵旭生，马国龙，周密. 煤与瓦斯突出智能预警方法及系统. 矿业安全与环保. 2022(04): 150-156+162 . 百度学术
4. 朱墨然，王麒翔，张庆华. 多源数据驱动的防突预警指标自适应技术研究. 煤炭科学技术. 2022(08): 75-81 . 百度学术
5. 曾明圣，施式亮，鲁义，李贺，吴宽，凌紫城. 基于免疫机理的瓦斯异常涌出预警. 湖南科技大学学报(自然科学版). 2022(03): 14-19 . 百度学术
6. 谷雷，朱权洁，李莹莹，张尔辉，张震. 基于CiteSpace的瓦斯灾害智能预警文献可视化分析. 华北科技学院学报. 2021(02): 20-27 . 百度学术
7. 程晓阳，蒲阳，宋志强. 基于瓦斯监控数据源的突出预警系统在下峪口煤矿的应用. 矿业研究与开发. 2021(07): 161-165 . 百度学术
8. 付京斌. 防治煤与瓦斯突出信息系统设计研究. 煤炭与化工. 2021(05): 61-63+66 . 百度学术
9. 陈凯，戴英健，张丽娟，李小龙. 基于煤与瓦斯突出声电实时监测预警系统的应用研究. 科技视界. 2021(34): 159-161 . 百度学术
10. 谈国文. 复杂矿区煤与瓦斯突出灾害多参量预警系统建设与应用. 煤炭工程. 2020(03): 17-20 . 百度学术
11. 张天宇，王红娟. 基于正态检验的瓦斯涌出异常预警方法. 软件导刊. 2020(03): 99-103 . 百度学术
12. 高建成，宁小亮，覃木广. 深部动力灾害煤层防突预警技术在平煤十三矿的应用. 煤矿安全. 2020(05): 150-153 . 百度学术
13. 刘纯. 基于大数据的煤与瓦斯突出灾害预测方法与技术研究. 江苏科技信息. 2020(08): 50-52 . 百度学术
14. 胡业林，占晨阳，郑晓亮，宋晓. 基于改进PSO算法优化GRNN的煤与瓦斯突出预测. 长江大学学报(自然科学版). 2020(02): 92-97 . 百度学术
15. 温廷新，靳露露. 基于预处理的IFOA-ELM煤与瓦斯突出预测模型. 中国安全科学学报. 2020(01): 35-41 . 百度学术
16. 袁亮. 煤矿典型动力灾害风险判识及监控预警技术研究进展. 煤炭学报. 2020(05): 1557-1566 . 百度学术
17. 宁小亮. 基于多源信息融合的煤与瓦斯突出动态预警模型. 矿业安全与环保. 2020(03): 1-5+16 . 百度学术
18. 马国龙. 基于防突预测特征的地质异常智能判识方法. 工矿自动化. 2020(07): 49-56 . 本站查看
19. 张庆华，宁小亮，宋志强，和树栋. 瓦斯灾害区域安全态势预警技术. 工矿自动化. 2020(07): 42-48 . 本站查看
20. 张尔辉，朱权洁，刘衍，高林生，张震，谷雷. 煤层突出危险性综合预测及验证. 华北科技学院学报. 2020(04): 19-26+42 . 百度学术
21. 卢新明，阚淑婷. 煤矿动力灾害本源预警方法关键技术与展望. 煤炭学报. 2020(S1): 128-139 . 百度学术
22. 邓敢博. 瓦斯涌出特征动态分析系统在杨河煤业的应用. 内蒙古煤炭经济. 2020(19): 54-55 . 百度学术
23. 张庆华，马国龙. 我国煤矿重大灾害预警技术现状及智能化发展展望. 智能矿山. 2020(01): 52-62 . 百度学术
24. 代晨昱. 管段式矿用钻孔超声波流量计研究. 工业仪表与自动化装置. 2019(04): 108-112 . 百度学术
25. 宁小亮. 煤与瓦斯突出预警技术研究现状及发展趋势. 工矿自动化. 2019(08): 25-31+37 . 本站查看
26. 覃木广，赵旭生，张庆华，岳俊，邹云龙. 基于瓦斯地质特征的突出预警模型研究. 煤炭科学技术. 2018(12): 138-144 . 百度学术
其他类型引用(3)
计量
- 文章访问数: 276
- HTML全文浏览量: 55
- PDF下载量: 38
- 被引次数: 29
