Volume 49 Issue 6
Jun.  2023
Turn off MathJax
Article Contents
Article Navigation >  Journal of Mine Automation  > 2023  >  49(6): 128-140
HU Qingsong, MENG Chunlei, LI Shiyin, et al. Research status and prospects of perception technology for unmanned mining vehicle driving environment[J]. Journal of Mine Automation，2023，49(6)：128-140.  doi: 10.13272/j.issn.1671-251x.18115
Citation: HU Qingsong, MENG Chunlei, LI Shiyin, et al. Research status and prospects of perception technology for unmanned mining vehicle driving environment[J]. Journal of Mine Automation，2023，49(6)：128-140.  doi: 10.13272/j.issn.1671-251x.18115
shu

Research status and prospects of perception technology for unmanned mining vehicle driving environment

doi: 10.13272/j.issn.1671-251x.18115
  • 1.

    Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, China University of Mining and Technology, Xuzhou 221116, China

  • 2.

    School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China

  • 3.

    Xuzhou Engineering Research Center of Intelligent Industry Safety and Emergency Collaboration, China University of Mining and Technology, Xuzhou 221116, China

  • Received Date: 2023-05-07
  • Rev Recd Date: 2023-06-01
    • Available Online: 2023-06-30
    Abstract
  • The auxiliary transportation system for coal mine is an essential system for transporting personnel, important materials, and equipment in coal mine enterprises. Realizing unmanned driving in coal mine is an inevitable requirement for improving transportation efficiency and ensuring transportation safety, and is also the only way to implement the national coal mine intelligent construction deployment. The mine unmanned driving relies on accurate and real-time environmental perception. By using onboard perception devices such as LiDAR and millimeter wave radar, as well as collaborative perception supported by the Internet of vehicles, the precise and detailed perception of local vehicles and even the entire mine is achieved. A systematic review is conducted on the research status of unmanned driving environment perception technology in mines. It is pointed out that the special environment of coal mine will lead to varying degrees of degradation in the performance of mine onboard perception devices. The advantages and disadvantages of various onboard perception devices are summarized. The key technologies of mine unmanned driving environment perception are elaborated in detail. The technologies include single-sensor obstacle recognition methods based on visible light images or laser point clouds, the classification of multi-sensor fusion perception, and multi-sensor fusion methods such as visible light images+laser point clouds, visible light images+millimeter wave point clouds, visible light images+laser point clouds+millimeter wave point clouds, 4D millimeter wave radar+other perception devices. The technologies include the implementation, data processing methods of intelligent networked collaborative perception, and their promoting effects on unmanned driving. The technologies also include methods for detecting and recognizing traffic signs in underground roadways, and methods for segmenting the driving area of underground trackless rubber wheeled vehicles and tracked locomotives in roadways. The development direction of unmanned driving environment perception technology in mines is pointed out. It is recommended to improve the fusion performance of multiple sensors in mines, study adaptive perception algorithms in mines, and break through the intelligent networked collaborative perception technology in mines.

     

    • FullText(HTML)
  • loading
    • References(60)
  • [1]
    王国法. 加快煤矿智能化建设 推进煤炭行业高质量发展[J]. 中国煤炭,2021,47(1):2-10.

    WANG Guofa. Speeding up intelligent construction of coal mine and promoting high-quality development of coal industry[J]. China Coal,2021,47(1):2-10.
    [2]
    王国法. 煤矿智能化最新技术进展与问题探讨[J]. 煤炭科学技术,2022,50(1):1-27. doi: 10.3969/j.issn.0253-2336.2022.1.mtkxjs202201001

    WANG Guofa. New technological progress of coal mine intelligence and its problems[J]. Coal Science and Technology,2022,50(1):1-27. doi: 10.3969/j.issn.0253-2336.2022.1.mtkxjs202201001
    [3]
    孙继平,江嬴. 矿井车辆无人驾驶关键技术研究[J]. 工矿自动化,2022,48(5):1-5,31.

    SUN Jiping,JIANG Ying. Research on key technologies of mine unmanned vehicle[J]. Journal of Mine Automation,2022,48(5):1-5,31.
    [4]
    刘琴. 无人驾驶无轨胶轮车路径规划与路径跟踪技术研究[D]. 徐州: 中国矿业大学, 2021.

    LIU Qin. Research on path planning and path tracking technology of driverless trackless rubber-tyred vehicle[D]. Xuzhou: China University of Mining and Technology, 2021.
    [5]
    ZHU Hao,YUEN K-V,MIHAYLOVA L,et al. Overview of environment perception for intelligent vehicles[J]. IEEE Transactions on Intelligent Transportation Systems,2017,18(10):2584-2601. doi: 10.1109/TITS.2017.2658662
    [6]
    杨春雨,张鑫. 煤矿机器人环境感知与路径规划关键技术[J]. 煤炭学报,2022,47(7):2844-2872.

    YANG Chunyu,ZHANG Xin. Key technologies of coal mine robots for environment perception and path planning[J]. Journal of China Coal Society,2022,47(7):2844-2872.
    [7]
    任柯燕,谷美颖,袁正谦,等. 自动驾驶3D目标检测研究综述[J]. 控制与决策,2023,38(4):865-889. doi: 10.13195/j.kzyjc.2022.0618

    REN Keyan,GU Meiying,YUAN Zhengqian,et al. 3D object detection algorithms in autonomous driving:a review[J]. Control and Decision,2023,38(4):865-889. doi: 10.13195/j.kzyjc.2022.0618
    [8]
    李庆玲,张慧祥,赵旭阳,等. 露天矿无人驾驶自卸卡车发展综述[J]. 煤炭工程,2021,53(2):29-34.

    LI Qingling,ZHANG Huixiang,ZHAO Xuyang,et al. Overview of unmanned mining trucks in open-pit mine[J]. Coal Engineering,2021,53(2):29-34.
    [9]
    CHENG Yuwei, SU Jingran, CHEN Hongyu, et al. A new automotive radar 4D point clouds detector by using deep learning[C]. IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, 2021: 8398-8402.
    [10]
    THRUN S. Toward robotic cars[J]. Communications of the ACM,2010,53(4):99-106. doi: 10.1145/1721654.1721679
    [11]
    李爱娟,巩春鹏,黄欣,等. 自动驾驶汽车目标检测方法综述[J]. 山东交通学院学报,2022,30(3):20-29.

    LI Aijuan,GONG Chunpeng,HUANG Xin,et al. Overview of target detection methods for autonomous vehicles[J]. Journal of Shandong Jiaotong University,2022,30(3):20-29.
    [12]
    REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]. IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 779-788.
    [13]
    LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]. European Conference on Computer Vision, Amsterdam, 2016: 21-37.
    [14]
    LIN T-Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(2):318-327. doi: 10.1109/TPAMI.2018.2858826
    [15]
    GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 2014: 580-587.
    [16]
    HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916. doi: 10.1109/TPAMI.2015.2389824
    [17]
    GIRSHICK R. Fast R-CNN[C]. IEEE International Conference on Computer Vision, Santiago, 2015: 1440-1448.
    [18]
    REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031
    [19]
    安果维,王耀南,周显恩,等. 基于显著性检测的双目测距系统[J]. 智能系统学报,2018,13(6):913-920.

    AN Guowei,WANG Yaonan,ZHOU Xian'en,et al. Binocular distance measurement system based on saliency detection[J]. CAAI Transactions on Intelligent Systems,2018,13(6):913-920.
    [20]
    LIU Fayao,SHEN Chunhua,LIN Guosheng,et al. Learning depth from single monocular images using deep convolutional neural fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,38(10):2024-2039.
    [21]
    WANG Wenshan,WANG Shuang,ZHAO Yanqiu,et al. Real-time obstacle detection method in the driving process of driverless rail locomotives based on DeblurGANv2 and improved YOLOv4[J]. Applied Sciences-Basel,2023,13(6):3861-3879. doi: 10.3390/app13063861
    [22]
    袁稼轩. 基于深度学习的井下巷道行人检测与距离估计[D]. 合肥: 合肥工业大学, 2019.

    YUAN Jiaxuan. Pedestrian detection and distance estimation of underground roadway based on deep learning[D]. Hefei: Hefei University of Technology, 2019.
    [23]
    WEI Xing,ZHANG Haitao,LIU Shaofan,et al. Pedestrian detection in underground mines via parallel feature transfer network[J]. Pattern Recognition,2020,103:107195-107207. doi: 10.1016/j.patcog.2020.107195
    [24]
    郭永存, 杨豚, 王爽. 基于改进YOLOv4−Tiny的矿井电机车多目标实时检测研究[J/OL]. 工程科学与技术: 1-11[2023-05-04]. https://doi.org/10.15961/j.jsuese.202200019.

    GUO Yongcun, YANG Tun, WANG Shuang. Research on multi-object real-time detection of mine electric locomotive based on improved YOLOv4-Tiny[J/OL]. Advanced Engineering Sciences: 1-11 [2023-05-04]. https://doi.org/10.15961/j.jsuese.202200019.
    [25]
    霍威乐, 荆涛, 任爽. 面向自动驾驶的三维目标检测综述[J/OL]. 计算机科学: 1-20[2023-05-04]. http://kns.cnki.net/kcms/detail/50.1075.TP.20230407.1928.030.html.

    HUO Weile, JING Tao, REN Shuang. Review of 3D object detection for autonomous driving[J/OL]. Computer Science: 1-20[2023-05-04]. http://kns.cnki.net/kcms/detail/50.1075.TP.20230407.1928.030.html.
    [26]
    CHEN Xiaozhi, MA Huimin, WAN Ji, et al. Multi-view 3D object detection network for autonomous driving[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 6526-6534.
    [27]
    ZHOU Yin, TUZEL O. VoxelNet: end-to-end learning for point cloud based 3D object detection[C]. Computer Vision and Pattern Recognition, Salt Lake City, 2018: 4490-4499.
    [28]
    CHARLES R Q, SU Hao, KAICHUN M, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017: 77-85.
    [29]
    SHI Shaoshuai, WANG Xiaogang, LI Hongsheng. PointRCNN: 3D object proposal generation and detection from point cloud[C]. Computer Vision and Pattern Recognition, Long Beach, 2019: 770-779.
    [30]
    YANG Zetong, SUN Yanan, LIU Shu, et al. STD: sparse-to-dense 3D object detector for point cloud[C]. IEEE/CVF International Conference on Computer Vision, Seoul, 2019: 1951-1960.
    [31]
    CHEN Yilun, LIU Shu, SHEN Xiaoyong, et al. Fast point R-CNN[C]. IEEE/CVF International Conference on Computer Vision, Seoul, 2019: 9774-9783.
    [32]
    LI Jiale,SUN Yu,LUO Shujie,et al. P2V-RCNN:point to voxel feature learning for 3D object detection from point clouds[J]. IEEE Access,2021,9:98249-98260. doi: 10.1109/ACCESS.2021.3094562
    [33]
    SHI Shaoshuai,JIANG Li,DENG Jiajun,et al. PV-RCNN plus plus:point-voxel feature set abstraction with local vector representation for 3D object detection[J]. International Journal of Computer Vision,2023,131(2):531-551. doi: 10.1007/s11263-022-01710-9
    [34]
    祖国强,何俏君,张志德,等. 无人驾驶环境感知中多源数据融合应用综述[J]. 汽车文摘,2022(8):8-13.

    ZU Guoqiang,HE Qiaojun,ZHANG Zhide,et al. Application overview on multi-source data fusion in unmanned environment perception[J]. Automotive Digest,2022(8):8-13.
    [35]
    WANG Zhangjing,WU Yu,NIU Qingqing. Multi-sensor fusion in automated driving:a survey[J]. IEEE Access,2020,8:2847-2868. doi: 10.1109/ACCESS.2019.2962554
    [36]
    秦沛霖,张传伟,周李兵,等. 煤矿井下无人驾驶无轨胶轮车目标3D检测研究[J]. 工矿自动化,2022,48(2):35-41.

    QIN Peilin,ZHANG Chuanwei,ZHOU Libing,et al. Research on 3D target detection of unmanned trackless rubber-tyred vehicle in coal mine[J]. Industry and Mine Automation,2022,48(2):35-41.
    [37]
    骆彬. 井下蓄电池无轨胶轮车无人驾驶系统设计研究[D]. 徐州: 中国矿业大学, 2019.

    LUO Bin. Design of driverless system of underground battery trackless rubber tire vehicle[D]. Xuzhou: China University of Mining and Technology, 2019 .
    [38]
    陈相蒙,王恩标,王刚. 煤矿电机车无人驾驶技术研究[J]. 煤炭科学技术,2020,48(增刊2):159-164.

    CHEN Xiangmeng,WANG Enbiao,WANG Gang. Research on electric locomotive self-driving technology in coal mine[J]. Coal Science and Technology,2020,48(S2):159-164.
    [39]
    谢厚抗. 无人驾驶无轨胶轮车多传感信息融合与智能感知技术研究[D]. 徐州: 中国矿业大学, 2021.

    XIE Houkang. Research on multi-sensor information fusion and intelligent sensing technology for driverless trackless rubber-tyred vehicles[D]. Xuzhou: China University of Mining and Technology, 2021.
    [40]
    周李兵. 煤矿井下无轨胶轮车无人驾驶系统研究[J]. 工矿自动化,2022,48(6):36-48.

    ZHOU Libing. Research on unmanned driving system of underground trackless rubber-tyred vehicle in coal mine[J]. Journal of Mine Automation,2022,48(6):36-48.
    [41]
    侯刚,王国法,薛忠新,等. 煤矿辅助运输自动驾驶关键技术与装备[J]. 采矿与岩层控制工程学报,2022,4(3):5-17.

    HOU Gang,WANG Guofa,XUE Zhongxin,et al. Key technologies and equipment for automatic driving of coal mine auxiliary transportation[J]. Journal of Mining and Strata Control Engineering,2022,4(3):5-17.
    [42]
    贺海涛,廖志伟,郭卫. 煤矿井下无轨胶轮车无人驾驶技术研究与探索[J]. 煤炭科学技术,2022,50(增刊1):212-217.

    HE Haitao,LIAO Zhiwei,GUO Wei. Research and exploration on driverless technology of trackless rubbertyred vehicle in coal mine[J]. Coal Science and Technology,2022,50(S1):212-217.
    [43]
    Dr. Wang. 4D成像毫米波雷达系统的发展与趋势[J]. 汽车与配件,2022(21):50-53.

    Dr. Wang. Development and trends of 4D imaging millimeter wave radar systems[J]. Automobile & Parts,2022(21):50-53.
    [44]
    李晨鑫,张立亚. 煤矿井下网联式自动驾驶技术研究[J]. 工矿自动化,2022,48(6):49-55.

    LI Chenxin,ZHANG Liya. Research on the network connected automatic driving technology in underground coal mine[J]. Journal of Mine Automation,2022,48(6):49-55.
    [45]
    邢亚男. 车路协同感知融合研究[D]. 长春: 吉林大学, 2022.

    XING Yanan. Research on vehicle-road collaborative perception fusion[D]. Changchun: Jilin University, 2022.
    [46]
    陈晓晶. 井工煤矿运输系统智能化技术现状及发展趋势[J]. 工矿自动化,2022,48(6):6-14,35.

    CHEN Xiaojing. Current status and development trend of intelligent technology of underground coal mine transportation system[J]. Journal of Mine Automation,2022,48(6):6-14,35.
    [47]
    贺海涛,张立辉,王旭峰,等. 面向矿用辅助运输车辆的智能无人运输系统研究[J]. 工矿自动化,2022,48(增刊1):72-75.

    HE Haitao,ZHANG Lihui,WANG Xufeng,et al. Research on intelligent unmanned transportation system for mine-used auxiliary transportation vehicles[J]. Journal of Mine Automation,2022,48(S1):72-75.
    [48]
    阮顺领,焦鑫,景莹,等. 一种露天矿区非结构化道路分割检测方法[J]. 测绘科学,2022,47(6):204-212.

    RUAN Shunling,JIAO Xin,JING Ying,et al. Road detection in mining area based on bilateral segmentation optimization network[J]. Science of Surveying and Mapping,2022,47(6):204-212.
    [49]
    张莹,黄影平,郭志阳,等. 基于点云与图像交叉融合的道路分割方法[J]. 光电工程,2021,48(12):32-43.

    ZHANG Ying,HUANG Yingping,GUO Zhiyang,et al. Point cloud-image data fusion for road segmentation[J]. Opto-Electronic Engineering,2021,48(12):32-43.
    [50]
    张云佐, 郑宇鑫, 武存宇, 等. 基于双特征提取网络的复杂环境车道线精准检测[J/OL]. 吉林大学学报（工学版）: 1-9[2023-05-04]. https://doi.org/10.13229/j.cnki.jdxbgxb20221129.

    ZHANG Yunzuo, ZHENG Yuxin, WU Cunyu, et al. Accurate lane detection of complex environment based on double feature extraction network[J/OL]. Journal of Jilin University（Engineering and Technology Edition）: 1-9 [2023-05-04]. https://doi.org/10.13229/j.cnki.jdxbgxb20221129.
    [51]
    方遒, 李伟林, 梁卓凡, 等. 基于多尺度复合卷积和图像分割融合的车道线检测算法[J/OL]. 北京理工大学学报: 1-11[2023-05-04]. https://doi.org/10.15918/j.tbit1001-0645.2022.238.

    FANG Qiu, LI Weilin, LIANG Zhuofan, et al. Lane line detection algorithm based on multi-scale composite convolution and image segmentation fusion[J/OL]. Transactions of Beijing Institute of Technology: 1-11[2023-05-04]. https://doi.org/10.15918/j.tbit1001-0645.2022.238.
    [52]
    乔晓敏. 基于深度学习的端到端井下轨道检测方法研究[D]. 合肥: 合肥工业大学, 2019.

    QIAO Xiaomin. The study on end-to-end downhole track detectionbased on deep learning[D]. Hefei: Hefei University of Technology, 2019.
    [53]
    卫星,杨国强,李佳,等. 结合多尺度条件生成对抗网络的井下轨道检测[J]. 中国图象图形学报,2020,25(2):282-293.

    WEI Xing,YANG Guoqiang,LI Jia,et al. Multiscale information-based conditional generative adversarial nets for downhole track detection[J]. Journal of Image and Graphics,2020,25(2):282-293.
    [54]
    于骞翔,张元生. 井下电机车轨道障碍物图像处理方法的智能识别技术[J]. 金属矿山,2021(8):150-157.

    YU Qianxiang,ZHANG Yuansheng. Track obstacle intelligent recognition technology of mine electric locomotive based on image processing[J]. Metal Mine,2021(8):150-157.
    [55]
    王京华,王李管,毕林. 基于计算机视觉技术的矿井电机车无人驾驶障碍物检测技术[J]. 黄金科学技术,2021,29(1):136-146.

    WANG Jinghua,WANG Liguan,BI Lin. Obstacle detection technology of mine electric locomotive driverless based on computer vision technology[J]. Gold Science and Technology,2021,29(1):136-146.
    [56]
    WANG Wenshan,WANG Shuang,GUO Yongcun,et al. Detection method of obstacles in the dangerous area of electric locomotive driving based on MSE-YOLOv4-Tiny[J]. Measurement Science and Technology,2022,33(11):115403-115415. doi: 10.1088/1361-6501/ac82db
    [57]
    韩江洪,卫星,陆阳,等. 煤矿井下机车无人驾驶系统关键技术[J]. 煤炭学报,2020,45(6):2104-2115.

    HAN Jianghong,WEI Xing,LU Yang,et al. Driverless technology of underground locomotive in coal mine[J]. Journal of China Coal Society,2020,45(6):2104-2115.
    [58]
    周华平,郑锐. 基于改进BiSeNet的煤矿井下轨道检测算法[J]. 湖北民族大学学报(自然科学版),2021,39(4):398-403.

    ZHOU Huaping,ZHENG Rui. Underground rail detection algorithm based on improved BiSeNet[J]. Journal of Hubei Minzu University(Natural Science Edition),2021,39(4):398-403.
    [59]
    郭永存,童佳乐,王爽. 井下无人驾驶电机车行驶场景中多目标检测研究[J]. 工矿自动化,2022,48(6):56-63.

    GUO Yongcun,TONG Jiale,WANG Shuang. Research on multi-object detection in driving scene of underground unmanned electric locomotive[J]. Journal of Mine Automation,2022,48(6):56-63.
    [60]
    胡青松,梁天河,李世银,等. 基于粒子群与模拟退火算法的改进矿井克里金定位方法[J]. 金属矿山,2023(5):221-227.

    HU Qingsong,LIANG Tianhe,LI Shiyin,et al. An improved Kriging location method based on particle swarm optimization and simulated annealing algorithms[J]. Metal Mine,2023(5):221-227.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(1)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (851) PDF downloads(123) Cited by()
    Proportional views
    Related

    苏ICP备 05016349号-2

    Supported by: Beijing Renhe Information Technology Co. Ltd.  Visits：    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return