Volume 49 Issue 5
May  2023
Turn off MathJax
Article Contents
Abstract
References
Related Articles
REN Wei. A fully mechanized working face inspection system based on SLAM and virtual reality[J]. Journal of Mine Automation，2023，49(5)：59-65. DOI: 10.13272/j.issn.1671-251x.18076
Citation: REN Wei. A fully mechanized working face inspection system based on SLAM and virtual reality[J]. Journal of Mine Automation，2023，49(5)：59-65. DOI: 10.13272/j.issn.1671-251x.18076
shu

A fully mechanized working face inspection system based on SLAM and virtual reality

  • Beijing Tianma Intelligent Control Technology Co., Ltd., Beijing 101399, China

More Information
  • Received Date: February 06, 2023
  • Revised Date: May 14, 2023
  • Available Online: May 23, 2023
    Graphical Abstract
    • Abstract
  • The reliability of the inspection robot in the fully mechanized working face is low due to the lack of scale information. In order to solve the above problem, virtual reality (VR) technology is introduced into the inspection of fully mechanized working face. A fully mechanized working face inspection system based on simultaneous localization and mapping (SLAM) and VR is designed. The system includes two parts: an inspection robot subsystem located underground and a VR real-time rendering subsystem located on the ground. The inspection robot subsystem utilizes laser SLAM technology to achieve real-time 3D scanning and establish a 3D map. At the same time, a panoramic camera is used to capture the scene of the fully mechanized working face in real-time. The real-time obtained laser point cloud and panoramic video are transmitted to the VR real-time rendering subsystem. The VR real-time rendering subsystem uses GPU acceleration technology to color laser point clouds. By customizing the rendering part of the Unreal 3D engine, real-time rendering of the laser point cloud is achieved, and the laser point cloud is projected onto the VR glasses. Remote operators obtain real-time 3D scenes through VR glasses, and remotely control the movements of the inspection robot through the operating handle. The fully mechanized working face inspection based on the first perspective is achieved. The underground industrial test results show that the system can achieve free switching of perspectives and zoom in on the scene. It enables better observation of details, with higher accuracy and reliability. By using GPU acceleration technology for point cloud coloring, the processing time is significantly shorter than CPU processing time. GPU has higher real-time performance, and the entire system's latency can meet the requirements of inspection tasks.
    • FullText(HTML)
    • References (22)
  • [1]
    王国法,范京道,徐亚军,等. 煤炭智能化开采关键技术创新进展与展望[J]. 工矿自动化,2018,44(2):5-12.

    WANG Guofa,FAN Jingdao,XU Yajun,et al. Innovation progress and prospect on key technologies of intelligent coal mining[J]. Industry and Mine Automation,2018,44(2):5-12.
    [2]
    张守祥,张学亮,张磊,等. 综采巡检机器人关键技术研究[J]. 煤炭科学技术,2022,50(1):247-255.

    ZHANG Shouxiang,ZHANG Xueliang,ZHANG Lei,et al. Research on key technology of patrol robot in fully-mechanized mining face[J]. Coal Science and Technology,2022,50(1):247-255.
    [3]
    梁占泽. 矿用带式输送机巡检机器人驱动系统设计[J]. 工矿自动化,2021,47(4):108-112. DOI: 10.13272/j.issn.1671-251x.2021010003

    LIANG Zhanze. Driving system design of inspection robot for mine belt conveyor[J]. Industry and Mine Automation,2021,47(4):108-112. DOI: 10.13272/j.issn.1671-251x.2021010003
    [4]
    毛浩,薛忠新,范生军,等. 张家峁煤矿回风巷道智能巡检机器人系统[J]. 煤矿安全,2021,52(7):107-111. DOI: 10.13347/j.cnki.mkaq.2021.07.019

    MAO Hao,XUE Zhongxin,FAN Shengjun,et al. Intelligent inspection robot system for return air roadway in Zhangjiamao Coal Mine[J]. Safety in Coal Mines,2021,52(7):107-111. DOI: 10.13347/j.cnki.mkaq.2021.07.019
    [5]
    商德勇,崔栓伟,周丹,等. 薄煤层巡检机器人行走机构跨沟性能分析及试验研究[J]. 煤炭工程,2017,49(4):136-138.

    SHANG Deyong,CUI Shuanwei,ZHOU Dan,et al. Ditch-crossing performance analysis and experimental study on inspection robot walking mechanism in thin coal seam[J]. Coal Engineering,2017,49(4):136-138.
    [6]
    张树生,马静雅,岑强,等. 煤矿综采工作面巡检机器人系统研究[J]. 煤炭科学技术,2019,47(10):136-140.

    ZHANG Shusheng,MA Jingya,CEN Qiang,et al. Research on inspection robot system for fully-mechanized mining face in coal mine[J]. Coal Science and Technology,2019,47(10):136-140.
    [7]
    李森,王峰,刘帅,等. 综采工作面巡检机器人关键技术研究[J]. 煤炭科学技术,2020,48(7):218-225.

    LI Sen,WANG Feng,LIU Shuai,et al. Study on key technology of patrol robots for fully-mechanized mining face[J]. Coal Science and Technology,2020,48(7):218-225.
    [8]
    郝勇,袁智. 综采工作面自动巡检机器人系统设计[J]. 煤炭科学技术,2020,48(8):145-149. DOI: 10.13199/j.cnki.cst.2020.08.018

    HAO Yong,YUAN Zhi. Design of automatic inspection robot system for fully-mechanized coal mining face[J]. Coal Science and Technology,2020,48(8):145-149. DOI: 10.13199/j.cnki.cst.2020.08.018
    [9]
    YASHIN G A, TRINITATOVA D, AGISHEV R T, et al. Aerovr: virtual reality-based teleoperation with tactile feedback for aerial manipulation[C]. The 19th International Conference on Advanced Robotics, Belo Horizonte, 2019: 767-772.
    [10]
    BARENTINE C, MCNAY A, PFAFFENBICHLER R, et al. A VR teleoperation suite with manipulation assist[C]. The 16th ACM/IEEE International Conference on Human-Robot Interaction, Boulder, 2021: 442-446.
    [11]
    GUZZI J, ABBATE G, PAOLILLO A, et al. Interacting with a conveyor belt in virtual reality using pointing gestures[C]. The 17th ACM/IEEE International Conference on Human-Robot Interaction, Sapporo, 2022: 1194-1195.
    [12]
    KENNEL-MAUSHART F, PORANNE R, COROS S. Multi-arm payload manipulation via mixed reality[C]. International Conference on Robotics and Automation, Philadelphia, 2022. DOI: 10.1109/ICRA46639.2022.9811580.
    [13]
    KUO C-Y, HUANG C-C, TSAI C-H, et al. Development of an immersive SLAM-based VR system for teleoperation of a mobile manipulator in an unknown environment[J]. Computers in Industry, 2021, 132. DOI: 10.1016/j.compind.2021.103502.
    [14]
    WONSICK M, KELEȘTEMUR T, ALT S, et al. Telemanipulation via virtual reality interfaces with enhanced environment models[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, Prague, 2021: 2999-3004.
    [15]
    迟焕磊,袁智,曹琰,等. 基于数字孪生的智能化工作面三维监测技术研究[J]. 煤炭科学技术,2021,49(10):153-161.

    CHI Huanlei,YUAN Zhi,CAO Yan,et al. Study on digital twin-based smart fully-mechanized coal mining workface monitoring technology[J]. Coal Science and Technology,2021,49(10):153-161.
    [16]
    李首滨,李森,张守祥,等. 综采工作面智能感知与智能控制关键技术与应用[J]. 煤炭科学技术,2021,49(4):28-39. DOI: 10.13199/j.cnki.cst.2021.04.004

    LI Shoubin,LI Sen,ZHANG Shouxiang,et al. Key technology and application of intelligent perception and intelligent control in fully mechanized mining face[J]. Coal Science and Technology,2021,49(4):28-39. DOI: 10.13199/j.cnki.cst.2021.04.004
    [17]
    杨生华,周永昌,芮丰,等. 薄煤层开采与成套装备技术的发展趋势[J]. 煤炭科学技术,2020,48(3):49-58. DOI: 10.13199/j.cnki.cst.2020.03.003

    YANG Shenghua,ZHOU Yongchang,RUI Feng,et al. Development trend of thin coal seam mining and complete equipment technology[J]. Coal Science and Technology,2020,48(3):49-58. DOI: 10.13199/j.cnki.cst.2020.03.003
    [18]
    亓玉浩,关士远. 基于激光SLAM的综采工作面实时三维建图方法[J]. 工矿自动化,2022,48(11):139-144. DOI: 10.13272/j.issn.1671-251x.2022060047

    QI Yuhao,GUAN Shiyuan. Real-time 3D mapping method of fully mechanized working face based on laser SLAM[J]. Journal of Mine Automation,2022,48(11):139-144. DOI: 10.13272/j.issn.1671-251x.2022060047
    [19]
    任伟. 综采工作面多目全景摄像仪的研制与应用[J]. 煤炭工程,2022,54(12):102-108.

    REN Wei. Development and application of multi view panoramic camera in fully mechanized mining face[J]. Coal Engineering,2022,54(12):102-108.
    [20]
    CHENG Xianwei, ZHAO Hui, KANDEMIR M, et al. AMOEBA: a coarse grained reconfigurable architecture for dynamic GPU scaling[C]. The 34th ACM International Conference on Supercomputing, 2020: 1-13. DOI: 10.1145/3392717.3392738.
    [21]
    CHENG Xianwei, ZHAO Hui, KANDEMIR M, et al. Alleviating bottlenecks for DNN execution on GPUs via opportunistic computing[C]. The 21st International Symposium on Quality Electronic Design, Santa Clara, 2020. DOI: 10.1109/ISQED48828.2020.9136967.
    [22]
    KIRK D B, HWU W-M W. Programming massively parallel processors[M]. 3rd ed. San Francisco: Morgan Kaufmann, 2016.
    • Related Articles

  • Related Articles

    [1]ZHANG Qilong, ZHOU Bing, WANG Guoqiang, TANG Wenxuan, WANG Qianzi, LIU Xin. Intelligent perception method for real-time airflow parameters in metal mines and its application[J]. Journal of Mine Automation, 2025, 51(2): 121-130. DOI: 10.13272/j.issn.1671-251x.2024090057
    [2]QI Yuhao, GUAN Shiyuan. Real-time 3D mapping method of fully mechanized working face based on laser SLAM[J]. Journal of Mine Automation, 2022, 48(11): 139-144. DOI: 10.13272/j.issn.1671-251x.2022060047
    [3]QIN Peilin, ZHANG Chuanwei, ZHOU Libing, WANG Jianlong. Research on 3D target detection of unmanned trackless rubber-tyred vehicle in coal mine[J]. Journal of Mine Automation, 2022, 48(2): 35-41. DOI: 10.13272/j.issn.1671-251x.2021110068
    [4]YANG Fan, LI Chong-gui. 3D Modeling Method of Mine Laneway Based on Multipath[J]. Journal of Mine Automation, 2012, 38(12): 58-62.
    [5]WANG Li-li. 3D Implementation of SVG[J]. Journal of Mine Automation, 2012, 38(12): 33-36.
    [6]SHU Li-chu. 3D Visualization Platform of Mine Based on 3D GIS[J]. Journal of Mine Automation, 2011, 37(6): 7-11.
    [7]CAO Yan-jie, CHEN Bi. Design of 3D Simulation Platform of Concrete Spraying Robot Based on MFC and OpenGL[J]. Journal of Mine Automation, 2010, 36(12): 17-20.
    [8]YANG Ping~, YANG Liang-yu~, HUANG Xiang-dong~. Design of 3D Imaging System for Multi-arm Caliper Logging Tool Based on OpenGL[J]. Journal of Mine Automation, 2009, 35(2): 42-45.
    [9]GUO Ju. Design of 3D Virtual Mine Based on 3D GIS Technology[J]. Journal of Mine Automation, 2007, 33(5): 1-4.
    [10]YU Xiao-ya. Programming Technnique of Interactive 3-Dimensional Animation Based on Java 3D[J]. Journal of Mine Automation, 2003, 29(6): 57-59.

  • Cited by

    Periodical cited type(6)

    1. 李黎黎，李合菊. 基于大数据的采煤机运行状态监测与评估. 煤矿机械. 2025(03): 208-211 .
    2. 侯京平. 薄煤层综采工作面智能化控制系统的设计及应用分析. 当代化工研究. 2024(22): 129-131 .
    3. 邵明育，赵兖锋，梁汝光. 基于RBF预测的采煤机电气运行数据获取系统设计. 电气技术与经济. 2024(12): 384-386+390 .
    4. 李翠平，陈永松. 大数据分析技术下给煤机运行中的异常分析与处理. 科学技术创新. 2023(13): 9-12 .
    5. 薛军. 基于循环神经网络的采煤机状态预测方法. 煤炭技术. 2023(09): 252-254 .
    6. 赵晓璐. 采煤机运行状态监控系统设计及应用分析. 西部探矿工程. 2023(12): 162-164 .

    Other cited types(0)

Catalog

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (246) PDF downloads (55) Cited by(6)
    Related

    苏ICP备 05016349号-2

    Supported by: Beijing Renhe Information Technology Co., Ltd. Visits：1253586    Baidu Analytics

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return