Research status and prospects of perception technology for unmanned mining vehicle driving environment
-
摘要: 矿井辅助运输系统是煤矿企业运输人员和重要物料、装备的必备系统，实现矿井无人驾驶是提高运输效率、保障运输安全的必然要求，也是落实国家煤矿智能化建设部署的必由之路。矿井无人驾驶依赖于准确实时的环境感知，即利用激光雷达、毫米波雷达等车载感知器件和车联网支持下的协同感知，实现车辆局部甚至矿井全局的精确详尽感知。对矿井无人驾驶环境感知技术的研究现状进行了系统梳理，指出巷道特殊环境使得矿井车载感知设备的性能都将出现不同程度的下降，并对各种车载感知设备的优劣进行了总结归纳；详细阐述了矿井无人驾驶环境感知的关键技术，包括基于可见光图像或激光点云的单传感器障碍物识别方法，多传感器融合感知的分类及可见光图像+激光点云、可见光图像+毫米波点云、可见光图像+激光点云+毫米波点云、4D毫米波雷达+其他感知器件等多传感器融合方式，智能网联协同感知的实现方式、数据处理方法及其对无人驾驶的促进作用，井下巷道交通标志检测与识别方法，井下无轨胶轮车和有轨机车的巷道可行驶区域分割方法等；对矿井无人驾驶环境感知技术的发展方向进行了展望，建议提高矿井多传感器融合性能、研究矿井自适应感知算法并突破矿井智能网联协同感知技术。Abstract: The auxiliary transportation system for coal mine is an essential system for transporting personnel, important materials, and equipment in coal mine enterprises. Realizing unmanned driving in coal mine is an inevitable requirement for improving transportation efficiency and ensuring transportation safety, and is also the only way to implement the national coal mine intelligent construction deployment. The mine unmanned driving relies on accurate and real-time environmental perception. By using onboard perception devices such as LiDAR and millimeter wave radar, as well as collaborative perception supported by the Internet of vehicles, the precise and detailed perception of local vehicles and even the entire mine is achieved. A systematic review is conducted on the research status of unmanned driving environment perception technology in mines. It is pointed out that the special environment of coal mine will lead to varying degrees of degradation in the performance of mine onboard perception devices. The advantages and disadvantages of various onboard perception devices are summarized. The key technologies of mine unmanned driving environment perception are elaborated in detail. The technologies include single-sensor obstacle recognition methods based on visible light images or laser point clouds, the classification of multi-sensor fusion perception, and multi-sensor fusion methods such as visible light images+laser point clouds, visible light images+millimeter wave point clouds, visible light images+laser point clouds+millimeter wave point clouds, 4D millimeter wave radar+other perception devices. The technologies include the implementation, data processing methods of intelligent networked collaborative perception, and their promoting effects on unmanned driving. The technologies also include methods for detecting and recognizing traffic signs in underground roadways, and methods for segmenting the driving area of underground trackless rubber wheeled vehicles and tracked locomotives in roadways. The development direction of unmanned driving environment perception technology in mines is pointed out. It is recommended to improve the fusion performance of multiple sensors in mines, study adaptive perception algorithms in mines, and break through the intelligent networked collaborative perception technology in mines.
-
图 1 矿井无人驾驶环境感知关键技术
Figure 1. Key perception technologies of unmanned mining vehicle driving environment
图 4 融合可见光图像与激光点云的3D目标检测模型
Figure 4. 3D target detection model based on the fusion of visible light mage and laser point cloud
图 5 图像与激光点云融合感知路况
Figure 5. Perception of road conditions based on fusion of image and laser point cloud
图 6 毫米波雷达与可见光摄像机两级融合策略
Figure 6. Two level fusion strategy for millimeter wave radar and visible light camera
图 7 基于多传感器融合的井下车辆路况识别
Figure 7. Underground vehicle road condition recognition based on multi-sensor fusion
图 8 基于传统图像处理的轨道检测算法一般流程
Figure 8. General process of track inspection algorithm based on traditional image processing
表 1 常用的矿井车载感知器件作用及其优劣
Table 1. Functions of common mine vehicle perception devices and their advantages and disadvantages
感知器件名称 作用 优点 缺点 激光雷达 生成实时点云地图，障碍物识别与测距 测距精度高，分辨率高 受粉尘影响大，价格高，体积大 毫米波雷达 障碍物测距 不受光照影响，受粉尘影响小，测距测速精度高，体积小，成本低 分辨率低，检测距离短，难以区分物体形状 4D毫米波
雷达障碍物识别与测距 不受光照影响，受粉尘影响小，测距测速精度高，角分辨率高，可测高成像，探测距离远，价格适中 点云稀疏 超声波雷达 测量矿车与巷道壁的距离，短距离防碰撞 近距离测量优势大，成本低 方向性差，回波信号弱，检测距离短 可见光
摄像机环境图像信息采集，障碍物识别与测距 分辨率高，有颜色信息，体积小，成本低 难以测距，受光照、粉尘影响大，可靠性低 红外摄像机 障碍物识别 受光照、粉尘影响小 分辨率低，探测距离近 深度相机 获取前方物体深度，障碍物识别与测距 有配套算法进行解析，使用方便 获取信息少，信息易偏移，受光照、粉尘影响大 双目视觉相机 获取边缘信息，进行场景重构，障碍物识别与测距 可测距，分辨率高，有颜色信息，体积小，成本低 受光照、粉尘影响大，可靠性低 
下载: 导出CSV
-
[1] 王国法. 加快煤矿智能化建设 推进煤炭行业高质量发展[J]. 中国煤炭,2021,47(1):2-10.WANG Guofa. Speeding up intelligent construction of coal mine and promoting high-quality development of coal industry[J]. China Coal,2021,47(1):2-10. [2] 王国法. 煤矿智能化最新技术进展与问题探讨[J]. 煤炭科学技术,2022,50(1):1-27. doi: 10.3969/j.issn.0253-2336.2022.1.mtkxjs202201001WANG Guofa. New technological progress of coal mine intelligence and its problems[J]. Coal Science and Technology,2022,50(1):1-27. doi: 10.3969/j.issn.0253-2336.2022.1.mtkxjs202201001 [3] 孙继平,江嬴. 矿井车辆无人驾驶关键技术研究[J]. 工矿自动化,2022,48(5):1-5,31.SUN Jiping,JIANG Ying. Research on key technologies of mine unmanned vehicle[J]. Journal of Mine Automation,2022,48(5):1-5,31. [4] 刘琴. 无人驾驶无轨胶轮车路径规划与路径跟踪技术研究[D]. 徐州: 中国矿业大学, 2021.LIU Qin. Research on path planning and path tracking technology of driverless trackless rubber-tyred vehicle[D]. Xuzhou: China University of Mining and Technology, 2021. [5] ZHU Hao,YUEN K-V,MIHAYLOVA L,et al. Overview of environment perception for intelligent vehicles[J]. IEEE Transactions on Intelligent Transportation Systems,2017,18(10):2584-2601. doi: 10.1109/TITS.2017.2658662 [6] 杨春雨,张鑫. 煤矿机器人环境感知与路径规划关键技术[J]. 煤炭学报,2022,47(7):2844-2872.YANG Chunyu,ZHANG Xin. Key technologies of coal mine robots for environment perception and path planning[J]. Journal of China Coal Society,2022,47(7):2844-2872. [7] 任柯燕,谷美颖,袁正谦,等. 自动驾驶3D目标检测研究综述[J]. 控制与决策,2023,38(4):865-889. doi: 10.13195/j.kzyjc.2022.0618REN Keyan,GU Meiying,YUAN Zhengqian,et al. 3D object detection algorithms in autonomous driving:a review[J]. Control and Decision,2023,38(4):865-889. doi: 10.13195/j.kzyjc.2022.0618 [8] 李庆玲,张慧祥,赵旭阳,等. 露天矿无人驾驶自卸卡车发展综述[J]. 煤炭工程,2021,53(2):29-34.LI Qingling,ZHANG Huixiang,ZHAO Xuyang,et al. Overview of unmanned mining trucks in open-pit mine[J]. Coal Engineering,2021,53(2):29-34. [9] CHENG Yuwei, SU Jingran, CHEN Hongyu, et al. A new automotive radar 4D point clouds detector by using deep learning[C]. IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, 2021: 8398-8402. [10] THRUN S. Toward robotic cars[J]. Communications of the ACM,2010,53(4):99-106. doi: 10.1145/1721654.1721679 [11] 李爱娟,巩春鹏,黄欣,等. 自动驾驶汽车目标检测方法综述[J]. 山东交通学院学报,2022,30(3):20-29.LI Aijuan,GONG Chunpeng,HUANG Xin,et al. Overview of target detection methods for autonomous vehicles[J]. Journal of Shandong Jiaotong University,2022,30(3):20-29. [12] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 779-788. [13] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]. European Conference on Computer Vision, Amsterdam, 2016: 21-37. [14] LIN T-Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(2):318-327. doi: 10.1109/TPAMI.2018.2858826 [15] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 2014: 580-587. [16] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916. doi: 10.1109/TPAMI.2015.2389824 [17] GIRSHICK R. Fast R-CNN[C]. IEEE International Conference on Computer Vision, Santiago, 2015: 1440-1448. [18] REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031 [19] 安果维,王耀南,周显恩,等. 基于显著性检测的双目测距系统[J]. 智能系统学报,2018,13(6):913-920.AN Guowei,WANG Yaonan,ZHOU Xian'en,et al. Binocular distance measurement system based on saliency detection[J]. CAAI Transactions on Intelligent Systems,2018,13(6):913-920. [20] LIU Fayao,SHEN Chunhua,LIN Guosheng,et al. Learning depth from single monocular images using deep convolutional neural fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(10):2024-2039. [21] WANG Wenshan,WANG Shuang,ZHAO Yanqiu,et al. Real-time obstacle detection method in the driving process of driverless rail locomotives based on DeblurGANv2 and improved YOLOv4[J]. Applied Sciences-Basel,2023,13(6):3861-3879. doi: 10.3390/app13063861 [22] 袁稼轩. 基于深度学习的井下巷道行人检测与距离估计[D]. 合肥: 合肥工业大学, 2019.YUAN Jiaxuan. Pedestrian detection and distance estimation of underground roadway based on deep learning[D]. Hefei: Hefei University of Technology, 2019. [23] WEI Xing,ZHANG Haitao,LIU Shaofan,et al. Pedestrian detection in underground mines via parallel feature transfer network[J]. Pattern Recognition,2020,103:107195-107207. doi: 10.1016/j.patcog.2020.107195 [24] 郭永存, 杨豚, 王爽. 基于改进YOLOv4−Tiny的矿井电机车多目标实时检测研究[J/OL]. 工程科学与技术: 1-11[2023-05-04]. https://doi.org/10.15961/j.jsuese.202200019.GUO Yongcun, YANG Tun, WANG Shuang. Research on multi-object real-time detection of mine electric locomotive based on improved YOLOv4-Tiny[J/OL]. Advanced Engineering Sciences: 1-11 [2023-05-04]. https://doi.org/10.15961/j.jsuese.202200019. [25] 霍威乐, 荆涛, 任爽. 面向自动驾驶的三维目标检测综述[J/OL]. 计算机科学: 1-20[2023-05-04]. http://kns.cnki.net/kcms/detail/50.1075.TP.20230407.1928.030.html.HUO Weile, JING Tao, REN Shuang. Review of 3D object detection for autonomous driving[J/OL]. Computer Science: 1-20[2023-05-04]. http://kns.cnki.net/kcms/detail/50.1075.TP.20230407.1928.030.html. [26] CHEN Xiaozhi, MA Huimin, WAN Ji, et al. Multi-view 3D object detection network for autonomous driving[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 6526-6534. [27] ZHOU Yin, TUZEL O. VoxelNet: end-to-end learning for point cloud based 3D object detection[C]. Computer Vision and Pattern Recognition, Salt Lake City, 2018: 4490-4499. [28] CHARLES R Q, SU Hao, KAICHUN M, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 77-85. [29] SHI Shaoshuai, WANG Xiaogang, LI Hongsheng. PointRCNN: 3D object proposal generation and detection from point cloud[C]. Computer Vision and Pattern Recognition, Long Beach, 2019: 770-779. [30] YANG Zetong, SUN Yanan, LIU Shu, et al. STD: sparse-to-dense 3D object detector for point cloud[C]. IEEE/CVF International Conference on Computer Vision, Seoul, 2019: 1951-1960. [31] CHEN Yilun, LIU Shu, SHEN Xiaoyong, et al. Fast point R-CNN[C]. IEEE/CVF International Conference on Computer Vision, Seoul, 2019: 9774-9783. [32] LI Jiale,SUN Yu,LUO Shujie,et al. P2V-RCNN:point to voxel feature learning for 3D object detection from point clouds[J]. IEEE Access,2021,9:98249-98260. doi: 10.1109/ACCESS.2021.3094562 [33] SHI Shaoshuai,JIANG Li,DENG Jiajun,et al. PV-RCNN plus plus:point-voxel feature set abstraction with local vector representation for 3D object detection[J]. International Journal of Computer Vision,2023,131(2):531-551. doi: 10.1007/s11263-022-01710-9 [34] 祖国强,何俏君,张志德,等. 无人驾驶环境感知中多源数据融合应用综述[J]. 汽车文摘,2022(8):8-13.ZU Guoqiang,HE Qiaojun,ZHANG Zhide,et al. Application overview on multi-source data fusion in unmanned environment perception[J]. Automotive Digest,2022(8):8-13. [35] WANG Zhangjing,WU Yu,NIU Qingqing. Multi-sensor fusion in automated driving:a survey[J]. IEEE Access,2020,8:2847-2868. doi: 10.1109/ACCESS.2019.2962554 [36] 秦沛霖,张传伟,周李兵,等. 煤矿井下无人驾驶无轨胶轮车目标3D检测研究[J]. 工矿自动化,2022,48(2):35-41.QIN Peilin,ZHANG Chuanwei,ZHOU Libing,et al. Research on 3D target detection of unmanned trackless rubber-tyred vehicle in coal mine[J]. Industry and Mine Automation,2022,48(2):35-41. [37] 骆彬. 井下蓄电池无轨胶轮车无人驾驶系统设计研究[D]. 徐州: 中国矿业大学, 2019.LUO Bin. Design of driverless system of underground battery trackless rubber tire vehicle[D]. Xuzhou: China University of Mining and Technology, 2019 . [38] 陈相蒙,王恩标,王刚. 煤矿电机车无人驾驶技术研究[J]. 煤炭科学技术,2020,48(增刊2):159-164.CHEN Xiangmeng,WANG Enbiao,WANG Gang. Research on electric locomotive self-driving technology in coal mine[J]. Coal Science and Technology,2020,48(S2):159-164. [39] 谢厚抗. 无人驾驶无轨胶轮车多传感信息融合与智能感知技术研究[D]. 徐州: 中国矿业大学, 2021.XIE Houkang. Research on multi-sensor information fusion and intelligent sensing technology for driverless trackless rubber-tyred vehicles[D]. Xuzhou: China University of Mining and Technology, 2021. [40] 周李兵. 煤矿井下无轨胶轮车无人驾驶系统研究[J]. 工矿自动化,2022,48(6):36-48.ZHOU Libing. Research on unmanned driving system of underground trackless rubber-tyred vehicle in coal mine[J]. Journal of Mine Automation,2022,48(6):36-48. [41] 侯刚,王国法,薛忠新,等. 煤矿辅助运输自动驾驶关键技术与装备[J]. 采矿与岩层控制工程学报,2022,4(3):5-17.HOU Gang,WANG Guofa,XUE Zhongxin,et al. Key technologies and equipment for automatic driving of coal mine auxiliary transportation[J]. Journal of Mining and Strata Control Engineering,2022,4(3):5-17. [42] 贺海涛,廖志伟,郭卫. 煤矿井下无轨胶轮车无人驾驶技术研究与探索[J]. 煤炭科学技术,2022,50(增刊1):212-217.HE Haitao,LIAO Zhiwei,GUO Wei. Research and exploration on driverless technology of trackless rubbertyred vehicle in coal mine[J]. Coal Science and Technology,2022,50(S1):212-217. [43] Dr. Wang. 4D成像毫米波雷达系统的发展与趋势[J]. 汽车与配件,2022(21):50-53.Dr. Wang. Development and trends of 4D imaging millimeter wave radar systems[J]. Automobile & Parts,2022(21):50-53. [44] 李晨鑫,张立亚. 煤矿井下网联式自动驾驶技术研究[J]. 工矿自动化,2022,48(6):49-55.LI Chenxin,ZHANG Liya. Research on the network connected automatic driving technology in underground coal mine[J]. Journal of Mine Automation,2022,48(6):49-55. [45] 邢亚男. 车路协同感知融合研究[D]. 长春: 吉林大学, 2022.XING Yanan. Research on vehicle-road collaborative perception fusion[D]. Changchun: Jilin University, 2022. [46] 陈晓晶. 井工煤矿运输系统智能化技术现状及发展趋势[J]. 工矿自动化,2022,48(6):6-14,35.CHEN Xiaojing. Current status and development trend of intelligent technology of underground coal mine transportation system[J]. Journal of Mine Automation,2022,48(6):6-14,35. [47] 贺海涛,张立辉,王旭峰,等. 面向矿用辅助运输车辆的智能无人运输系统研究[J]. 工矿自动化,2022,48(增刊1):72-75.HE Haitao,ZHANG Lihui,WANG Xufeng,et al. Research on intelligent unmanned transportation system for mine-used auxiliary transportation vehicles[J]. Journal of Mine Automation,2022,48(S1):72-75. [48] 阮顺领,焦鑫,景莹,等. 一种露天矿区非结构化道路分割检测方法[J]. 测绘科学,2022,47(6):204-212.RUAN Shunling,JIAO Xin,JING Ying,et al. Road detection in mining area based on bilateral segmentation optimization network[J]. Science of Surveying and Mapping,2022,47(6):204-212. [49] 张莹,黄影平,郭志阳,等. 基于点云与图像交叉融合的道路分割方法[J]. 光电工程,2021,48(12):32-43.ZHANG Ying,HUANG Yingping,GUO Zhiyang,et al. Point cloud-image data fusion for road segmentation[J]. Opto-Electronic Engineering,2021,48(12):32-43. [50] 张云佐, 郑宇鑫, 武存宇, 等. 基于双特征提取网络的复杂环境车道线精准检测[J/OL]. 吉林大学学报（工学版）: 1-9[2023-05-04]. https://doi.org/10.13229/j.cnki.jdxbgxb20221129.ZHANG Yunzuo, ZHENG Yuxin, WU Cunyu, et al. Accurate lane detection of complex environment based on double feature extraction network[J/OL]. Journal of Jilin University（Engineering and Technology Edition）: 1-9 [2023-05-04]. https://doi.org/10.13229/j.cnki.jdxbgxb20221129. [51] 方遒, 李伟林, 梁卓凡, 等. 基于多尺度复合卷积和图像分割融合的车道线检测算法[J/OL]. 北京理工大学学报: 1-11[2023-05-04]. https://doi.org/10.15918/j.tbit1001-0645.2022.238.FANG Qiu, LI Weilin, LIANG Zhuofan, et al. Lane line detection algorithm based on multi-scale composite convolution and image segmentation fusion[J/OL]. Transactions of Beijing Institute of Technology: 1-11[2023-05-04]. https://doi.org/10.15918/j.tbit1001-0645.2022.238. [52] 乔晓敏. 基于深度学习的端到端井下轨道检测方法研究[D]. 合肥: 合肥工业大学, 2019.QIAO Xiaomin. The study on end-to-end downhole track detectionbased on deep learning[D]. Hefei: Hefei University of Technology, 2019. [53] 卫星,杨国强,李佳,等. 结合多尺度条件生成对抗网络的井下轨道检测[J]. 中国图象图形学报,2020,25(2):282-293.WEI Xing,YANG Guoqiang,LI Jia,et al. Multiscale information-based conditional generative adversarial nets for downhole track detection[J]. Journal of Image and Graphics,2020,25(2):282-293. [54] 于骞翔,张元生. 井下电机车轨道障碍物图像处理方法的智能识别技术[J]. 金属矿山,2021(8):150-157.YU Qianxiang,ZHANG Yuansheng. Track obstacle intelligent recognition technology of mine electric locomotive based on image processing[J]. Metal Mine,2021(8):150-157. [55] 王京华,王李管,毕林. 基于计算机视觉技术的矿井电机车无人驾驶障碍物检测技术[J]. 黄金科学技术,2021,29(1):136-146.WANG Jinghua,WANG Liguan,BI Lin. Obstacle detection technology of mine electric locomotive driverless based on computer vision technology[J]. Gold Science and Technology,2021,29(1):136-146. [56] WANG Wenshan,WANG Shuang,GUO Yongcun,et al. Detection method of obstacles in the dangerous area of electric locomotive driving based on MSE-YOLOv4-Tiny[J]. Measurement Science and Technology,2022,33(11):115403-115415. doi: 10.1088/1361-6501/ac82db [57] 韩江洪,卫星,陆阳,等. 煤矿井下机车无人驾驶系统关键技术[J]. 煤炭学报,2020,45(6):2104-2115.HAN Jianghong,WEI Xing,LU Yang,et al. Driverless technology of underground locomotive in coal mine[J]. Journal of China Coal Society,2020,45(6):2104-2115. [58] 周华平,郑锐. 基于改进BiSeNet的煤矿井下轨道检测算法[J]. 湖北民族大学学报(自然科学版),2021,39(4):398-403.ZHOU Huaping,ZHENG Rui. Underground rail detection algorithm based on improved BiSeNet[J]. Journal of Hubei Minzu University(Natural Science Edition),2021,39(4):398-403. [59] 郭永存,童佳乐,王爽. 井下无人驾驶电机车行驶场景中多目标检测研究[J]. 工矿自动化,2022,48(6):56-63.GUO Yongcun,TONG Jiale,WANG Shuang. Research on multi-object detection in driving scene of underground unmanned electric locomotive[J]. Journal of Mine Automation,2022,48(6):56-63. [60] 胡青松,梁天河,李世银,等. 基于粒子群与模拟退火算法的改进矿井克里金定位方法[J]. 金属矿山,2023(5):221-227.HU Qingsong,LIANG Tianhe,LI Shiyin,et al. An improved Kriging location method based on particle swarm optimization and simulated annealing algorithms[J]. Metal Mine,2023(5):221-227. -
点击查看大图
图(9) / 表(1)
计量
- 文章访问数: 856
- HTML全文浏览量: 40
- PDF下载量: 123
- 被引次数: 0
