煤矿副井矿车装载物智能识别方法

靳舒凯; 魏冠楠; 王春明; 王统海; 吴忠伦; 杨克虎

doi:10.13272/j.issn.1671-251x.2021110055

煤矿副井矿车装载物智能识别方法

靳舒凯^1,,
魏冠楠¹,
王春明¹,
王统海²,
吴忠伦³,
杨克虎^{1, 4, ,}

1.
中国矿业大学(北京) 机电与信息工程学院, 北京　100083
2.
临沂矿业集团菏泽煤电有限公司, 山东菏泽　274700
3.
临沂矿业集团菏泽煤电有限公司郭屯煤矿, 山东菏泽　274700
4.
中国矿业大学(北京) 煤矿智能化与机器人创新应用应急管理部重点实验室, 北京　100083

基金项目: 国家自然科学基金资助项目（61936008,61973307）。

详细信息

作者简介:
靳舒凯（1994－），男，山西神池人，硕士研究生，研究方向为煤矿智能开采技术，E-mail：569082993@qq.com

通讯作者:
杨克虎（1982－），男，湖北仙桃人，教授，博士，主要从事煤矿智能开采、电力电子变换器建模与控制等教学及研究工作，E-mail：ykh@cumtb.edu.cn

中图分类号: TD54/67
计量
- 文章访问数: 338
- HTML全文浏览量: 207
- PDF下载量: 52
出版历程
- 收稿日期: 2021-11-21
- 修回日期: 2022-03-28
- 网络出版日期: 2022-03-07
- 刊出日期: 2022-04-24

Intelligent identification method for mine car load in coal mine auxiliary shaft

1.
School of Mechanical Electronic and Information Engineering, China University of Mining and Technology(Beijing), Beijing 100083, China
2.
Linyi Mining Group Heze Coal Power Co., Ltd., Heze 274700, China
3.
Guotun Coal Mine, Linyi Mining Group Heze Coal Power Co., Ltd., Heze 274700, China
4.
Key Laboratory of Coal Mine Intelligence and Robot Innovative Application, Ministry of Emergency Management, China University of Mining and Technology(Beijing), Beijing 100083, China

摘要

摘要: 基于卷积神经网络实现煤矿副井矿车装载物自动分类在实际应用中因触发条件简易导致误判与漏判情况，且非矿车物体经过检测区域时会引起司控道岔误动作。针对该问题，提出了基于目标检测模型的煤矿副井矿车装载物智能识别方法。在煤矿副井井口安装工业相机采集矿车装载物图像并进行人工标注，构建矿车识别数据集，对Faster R−CNN，YOLOv4，SSD 3种目标检测模型的识别准确率与实时性进行评估，根据评估结果，得出YOLOv4模型更适用于矿车装载物识别任务的结论；为降低模型大小，提高识别速度，对YOLOv4模型进行改进，采用轻量级网络MobileNet替换原有主干特征提取网络CSPDarknet53，构建MobileNetv3−YOLOv4模型，测试结果表明MobileNetv3−YOLOv4模型的平均精度均值（mAP）为95.03%，识别速度为44 帧/s，较YOLOv4模型分别提高了0.77%，27 帧/s；为方便现场应用和部署，提高矿车装载物识别模型在嵌入式平台上的性能，提出了基于层间融合和模型量化的模型加速方法，并将加速前后的MobileNetv3−YOLOv4模型移植到Jetson TX2进行矿车装载物识别现场试验，结果表明识别速度由MobileNetv3−YOLOv4模型加速前的18.3 帧/s提升至35.42 帧/s，mAP为94.68%，满足现场实时、精确检测需求，且仅在矿车经过检测区域时启动检测任务，避免了因其他物体引起的司控道岔误动作现象。
- 煤矿智能化 /
- 副井轨道分运 /
- 矿车装载物识别 /
- 目标检测模型 /
- YOLOv4 /
- 模型加速
Abstract: The automatic classification of mine car load based on convolution neural network in coal mine auxiliary shaft is realized in practical application. The misdetection and false alarm are caused by simple trigger conditions. Non-mine car objects passing through the detection area can cause the misoperation of driver controlled switch. In order to solve this problem, an intelligent identification method for mine car load in coal mine auxiliary shaft based on target detection model is proposed. An industrial camera is installed at the wellhead of a coal mine auxiliary shaft to collect the images of mine car load and the images are manually labeled so as to construct a mine car identification data set. And the identification accuracy and the real-time performance of three target detection models, namely Faster R-CNN, YOLOv4 and SSD are evaluated. According to the evaluation results, it is concluded that the YOLOv4 model is more suitable for the identification task of mine car load. In order to reduce the model size and improve the identification speed, the YOLOv4 model is improved. The lightweight network MobileNet is used to replace the original backbone characteristic extraction network CSPDarknet53. So the MobileNetv3-YOLOv4 model is constructed. The test results show that the mean average precision(mAP) of the MobileNetv3-YOLOv4 model is 95.03%, and the identification speed is 44 frames/s, which is 0.77% and 27 frames/s higher than that of the YOLOv4 model respectively. In order to facilitate field application and deployment and improve the performance of the mine car load identification model on the embedded platform, a model acceleration method based on inter-layer fusion and model quantization is proposed. The MobileNetv3-YOLOv4 model before and after the acceleration is transplanted to Jetson TX2 for field test of mine car load identification. The results show that the identification speed is increased from 18.3 frames/s before the acceleration of the MobileNetv3-YOLOv4 model to 35.42 frames/s, and the mAP is 94.68%, which meets the real-time and precise detection requirements in the field. And the detection task is only started when the mine car passes the detection area, which avoids the misoperation of driver controlled switch caused by other objects.
- intelligent coal mine /
- auxiliary shaft track transportation /
- mine car load identification /
- object detection model /
- YOLOv4 /
- model acceleration

HTML全文

图 1 部分矿车装载物采集图像

Figure 1. Part of collected mine car load images

下载: 全尺寸图片幻灯片

图 2 YOLOv4模型优化

Figure 2. Optimization of YOLOv4 model

下载: 全尺寸图片幻灯片

图 3 神经网络模型层间横向融合过程

Figure 3. Horizontal interlamination fusion process of neural network model

下载: 全尺寸图片幻灯片

图 4 CSPDarknet53第1种残差块层间融合过程

Figure 4. Interlamination fusion process of the first residual block in CSPDarknet53

下载: 全尺寸图片幻灯片

图 5 矿车装载物智能识别现场试验平台

Figure 5. Field test platform for intelligent identification of mine car load

下载: 全尺寸图片幻灯片

图 6 矿车装载物智能识别现场试验结果

Figure 6. Field test results of intelligent identification of mine car load

下载: 全尺寸图片幻灯片

表 1 3种目标检测模型训练结果对比

Table 1 Comparison of training results of three target detection models

指标	Faster R−CNN	YOLOv4	SSD
epoch	100	97	49
训练集损失函数值	0.092 1	0.724	0.925
验证集损失函数值	0.144 8	0.807	0.989

下载: 导出CSV

表 2 3种目标检测模型检测性能对比

Table 2 Comparison of detection performance of three target detection models

指标	Faster R−CNN	YOLOv4	SSD
mAP/%	96.61	94.26	81.92
识别速度/（帧·s⁻¹）	6	17	24

下载: 导出CSV

表 3 YOLOv4模型优化前后检测性能对比

Table 3 Comparison of detection performance of YOLOv4 model before and after optimization

指标	YOLOv4	MobileNetv1− YOLOv4	MobileNetv2− YOLOv4	MobileNetv3− YOLOv4
mAP/%	94.26	93.26	92.19	95.03
识别速度/ （帧·s⁻¹）	17	39	47	44

下载: 导出CSV

表 4 目标检测模型加速前后对比

Table 4 Comparison of target detection models before and after acceleration

指标		Faster R−CNN	YOLOv4	SSD	MobileNetv3− YOLOv4
mAP/%	加速前	96.61	94.26	81.92	95.03
mAP/%	加速后	95.83	93.52	79.23	94.68
识别速度/ （帧·s⁻¹）	加速前	6	17	24	44
识别速度/ （帧·s⁻¹）	加速后	14	43	51	76

下载: 导出CSV

表 5 Jetson TX2上MobileNetv3−YOLOv4模型检测性能

Table 5 Detection performance of MobileNetv3-YOLOv4 model on Jetson TX2

指标	MobileNetv3−YOLOv4	加速后的 MobileNetv3−YOLOv4
mAP/%	95.03	94.68
识别速度/（帧·s⁻¹）	18.3	35.42

下载: 导出CSV

参考文献(16)

[1]	韩安,陈晓晶,贺耀宜,等. 智能矿山综合管控平台建设构思[J]. 工矿自动化,2021,47(8):7-14. HAN An,CHEN Xiaojing,HE Yaoyi,et al. Construction conception of intelligent mine integrated management and control platform[J]. Industry and Mine Automation,2021,47(8):7-14.
[2]	王国法,任怀伟,赵国瑞,等. 煤矿智能化十大“痛点”解析及对策[J]. 工矿自动化,2021,47(6):1-11. WANG Guofa,REN Huaiwei,ZHAO Guorui,et al. Analysis and countermeasures of ten 'pain points' of intelligent coal mine[J]. Industry and Mine Automation,2021,47(6):1-11.
[3]	李慧. 副井井口自动化运输系统研究[J]. 工矿自动化,2021,47(6):124-127,132. LI Hui. Research on automatic transport system in auxilary shaft pithead[J]. Industry and Mine Automation,2021,47(6):124-127,132.
[4]	宋庆军, 姜海燕, 王忠勇, 等. 一种矿车运输智能煤矸识别及分运装置: CN201610922689.2[P]. 2019-08-13. SONG Qingjun, JIANG Haiyan, WANG Zhongyong, et al. An intelligent coal gangue identification and separation device for mine car transportation: CN201610922689.2[P]. 2019-08-13.
[5]	袁源,汪嘉文,朱德昇,等. 顶煤放落过程煤矸声信号特征提取与分类方法[J]. 矿业科学学报,2021,6(6):711-720. YUAN Yuan,WANG Jiawen,ZHU Desheng,et al. Feature extraction and classification method of coal gangue acoustic signal during top coal caving[J]. Journal of Mining Science and Technology,2021,6(6):711-720.
[6]	饶中钰,吴景涛,李明. 煤矸石图像分类方法[J]. 工矿自动化,2020,46(3):69-73. RAO Zhongyu,WU Jingtao,LI Ming. Coal-gangue image classification method[J]. Industry and Mine Automation,2020,46(3):69-73.
[7]	KIDO S, HIRANO Y, HASHIMOTO N. Detection and classification of lung abnormalities by use of convolutional neural network（ CNN） and regions with CNN features （R-CNN）[C]//International Workshop on Advanced Image Technology, Chiang Mai, 2018: 1-4.
[8]	刘彪, 郭翔, 张帆, 等. 基于卷积神经网络的煤矿副井矿车装载物自动分类系统[C]//第32届中国控制与决策会议论文集, 合肥, 2020: 580-585. LIU Biao, GUO Xiang, ZHANG Fan, et al. Automatic classification system of auxiliary mine car load based on convolutional neural network[C]//Proceedings of the 32nd Chinese Control and Decision Conference, Hefei, 2020: 580-585.
[9]	ROH M C, LEE J Y. Refining faster-RCNN for accurate object detection[C]//The 15th International Conference on Machine Vision Applications, Nagoya, 2017: 514-517.
[10]	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[Z/OL]. arvPrint, arXiv: 2004.10934. https://arxiv.org/abs/2004.10934.
[11]	张帆,栾佳星,崔东林,等. 基于SSD−LeNet的矿井移动目标检测与识别方法[J]. 矿业科学学报,2021,6(1):100-108. ZHANG Fan,LUAN Jiaxing,CUI Donglin,et al. SSD-LeNet based method of mine moving target detection and recognition[J]. Journal of Mining Science and Technology,2021,6(1):100-108.
[12]	PANG Shanchen,WANG Shuo,RODRIGUEZ-PATON A,et al. An artificial intelligent diagnostic system on mobile Android terminals for cholelithiasis by lightweight convolutional neural network[J]. PLoS One,2019,14(9):e0221720. DOI: 10.1371/journal.pone.0221720
[13]	陈智超,焦海宁,杨杰,等. 基于改进MobileNet v2的垃圾图像分类算法[J]. 浙江大学学报(工学版),2021,55(8):1490-1499. CHEN Zhichao,JIAO Haining,YANG Jie,et al. Garbage image classification algorithm based on improved MobileNet v2[J]. Journal of Zhejiang University(Engineering Science),2021,55(8):1490-1499.
[14]	邵伟平,王兴,曹昭睿,等. 基于MobileNet与YOLOv3的轻量化卷积神经网络设计[J]. 计算机应用,2020,40(增刊1):8-13. SHAO Weiping,WANG Xing,CAO Zhaorui,et al. Design of lightweight convolutional neural network based on MobileNet and YOLOv3[J]. Journal of Computer Applications,2020,40(S1):8-13.
[15]	MA C,MU X,SHA D. Multi-Layers feature fusion of convolutional neural network for scene classification of remote sensing[J]. IEEE Access,2019,7:121685-121694. DOI: 10.1109/ACCESS.2019.2936215
[16]	BAO Jian, ZHOU Bin. Optimization of neural network with fixed-point weights and touch-screen calibration[C]//The 4th IEEE Conference on Industrial Electronics and Applications, Xi'an, 2009: 3704-3708.

施引文献(7)

期刊类型引用(7)

1.	薛生，郑晓亮，袁亮，来文豪，张玉婷. 基于机器学习的煤与瓦斯突出预测研究进展及展望. 煤炭学报. 2024(02): 664-694 . 百度学术
2.	林海飞，周捷，金洪伟，李树刚，赵鹏翔，刘时豪. 基于特征选择与机器学习的煤与瓦斯突出危险等级协同预测方法. 采矿与安全工程学报. 2023(02): 361-370 . 百度学术
3.	李燕，南新元，蔺万科. 煤与瓦斯突出危险性预测. 工矿自动化. 2022(03): 99-106 . 本站查看
4.	于世雷，佘九华，张羽，陈小林，周立林，王健. 煤层煤与瓦斯突出多指标量化评价方法探讨. 煤炭技术. 2022(05): 119-124 . 百度学术
5.	郭亚玲，江泽标，扶祥祥，吴少康. 基于模糊Bow-tie模型对煤与瓦斯突出危险性分析. 矿业工程研究. 2022(02): 42-48 . 百度学术
6.	刘万仓. 智能化高产高效综采工作面的研究与应用. 煤炭科学技术. 2021(S1): 81-83 . 百度学术
7.	于丽雅，赵永芳，张凌云，陈光波. 基于云模型和D—S理论的煤与瓦斯突出危险性评价. 工矿自动化. 2020(11): 106-112 . 本站查看