Research on the cantilever roadheader positioning based on near-infrared binocular stereo vision
-
摘要: 针对现有掘进机无法实时定位、定位不准确、视觉定位中相机视野被遮挡导致定位失败等问题，提出了一种将基于近红外双目立体视觉的悬臂式掘进机定位方案。在悬臂式掘进机机身与机臂安装近红外LED标靶，以LED作为近红外标靶构建掘进机特征信息，通过图像处理、位姿解算实现掘进机机身与截割部的三维空间定位。双目立体视觉相机安装在巷道顶部，随着掘进机不断推进，掘进机与双目立体视觉相机距离逐渐增加，造成双目图像获取失败，进而导致视觉解算截割部位姿失败。针对该问题，引入基于一维卷积神经网络（1D−CNN）的掘进机截割部磁场辅助定位方法。在掘进机机身两侧安装三轴数字磁场计，并在机臂处安装永磁体，以磁场的强度分量和双目立体视觉相机获取的位姿数据作为训练数据，构建1D−CNN模型，输出在视觉测量失效情况下掘进机截割部位姿。从深度信息和掘进机机身及截割部位姿2个方面对基于近红外双目立体视觉的悬臂式掘进机定位方案进行实验，结果表明：机身测量误差在±11 mm以内，相对误差在0.4%以内。截割部测量误差在±50 mm以内，相对误差在1%以内；掘进机机身与截割部间的相对位姿误差在±2.5°以内，俯仰角的均方根误差为根0.930 1°，偏航角的均方根误差为0.922 0°。上述误差在巷道作业允许范围内，验证了该方案的有效性和可靠性。对基于1D−CNN的掘进机截割部磁场辅助定位方法进行了有效性验证，为模拟井下复杂磁场环境，在掘进机附近随机添加干扰磁源，结果表明：该方法对掘进机截割部俯仰角、偏航角、翻滚角的预测值与测量的真实值基本吻合，预测的俯仰角、偏航角、翻滚角决定系数分别为0.992 4，0.995 9，0.917 4，说明基于1D−CNN的掘进机截割部磁场辅助定位方法能够较好地满足在视觉定位失效下的掘进机定位需求。Abstract: The existing roadheader has problems, such as unable real-time positioning, inaccurate positioning, and positioning failure caused by camera view occlusion in visual positioning. In order to solve the above problems, a positioning scheme of the cantilever roadheader based on near-infrared binocular stereo vision is proposed. A near-infrared LED target is arranged on the fuselage and arm of cantilever roadheader. Taking LED as the near-infrared target, the characteristic information of the roadheader is constructed. The three-dimensional spatial positioning of the roadheader fuselage and the cutting part is realized through image processing and pose calculation. The binocular stereo vision camera is arranged at the top of a roadway. The distance between the roadheader and the binocular stereo vision camera gradually increases as the roadheader continues to advance. It leads to the failure of binocular image acquisition, which leads to the failure of the visual solution of the pose of the cutting part. In order to solve this problem, a magnetic field assisted positioning method of the cutting part based on one-dimensional convolution neural network (1D-CNN) is introduced. Three-axis digital magnetometers are arranged on two sides of the fuselage of the roadheader. The permanent magnet is arranged on the machine arm. The strength component of a magnetic field and pose data obtained by binocular stereo vision camera are used as training data to construct the 1D-CNN model, and the pose of a cutting part of the roadheader is output under the condition that vision measurement fails. The cantilever roadheader positioning based on near-infraared binocular stereo vision scheme is tested from the aspects of depth information and its fuselage of the roadheader and the cutting position are verified. The results showed that the measurement error of the fuselage is within ±11 mm, and the relative error is within 0.4%. The measurement error of the cutting part is within ±50 mm, and the relative error is within 1%.The relative pose error between the roadheader fuselage and the cutting part is within ±2.5°, the root-mean-square error of the pitch angle is 0.930 1°, and the root-mean-square error of the yaw angle is 0.922 0°. The errors are within the allowable range of roadway operation. The results show that the cantilever roadheader positioning scheme based on near-infrared binocular stereo vision is effective and reliable. The effectiveness of the magnetic field assisted positioning method based on 1D-CNN is verified. In order to simulate the complex magnetic field environment in coal mine underground, the interference magnetic source is randomly added near the roadheader. The results show that the predicted values of the pitch angle, yaw angle and rolling angle of the cutting part of the roadheader by this method are basically consistent with the measured real values. The determination coefficients of the predicted pitch angle, yaw angle and rolling angle are 0.992 4, 0.995 9 and 0.917 4 respectively. It shows that the magnetic field assisted positioning method of the cutting part of the roadheader based on 1D-CNN can better meet the positioning requirements of the roadheader in the case of visual positioning failure.
-
图 1 悬臂式掘进机机身及截割部位姿测量系统结构
Figure 1. Structure of cantilever roadheader fuselage and cutting part pose measurement system
图 2 悬臂式掘进机机身及截割部位姿测量方案
Figure 2. Pose measurement scheme of cantilever roadheader fuselage and cutting part
图 3 悬臂式掘进机机身及截割部位姿空间坐标系
Figure 3. Pose space coordinate systems of cantilever roadheader fuselage and cutting part
图 5 双目视觉测量失效下磁场辅助定位时相机前移
Figure 5. Camera moves forward in magnetic field assisted positioning under the failure of binocular vision measurement
图 6 掘进机机身与截割部空间坐标变化
Figure 6. Space coordinate change of roadheader fuselage and cutting part
图 7 巷道坐标系下的机身与截割部位姿变化
Figure 7. The relation between the cutting part and pose of the fuselage in the tunnel coordinate system
图 8 掘进机机身与截割部相对位姿误差
Figure 8. The relative pose error of roadheader fuselage and cutting part
图 9 掘进机截割部俯仰角预测结果
Figure 9. Prediction results of pitch angle of roadheader cutting part
图 10 掘进机截割部偏航角预测结果
Figure 10. Prediction results of yaw angle of roadheader cutting part
图 11 掘进机截割部翻滚角预测结果
Figure 11. Prediction results of rolling angle of roadheader cutting part
表 1 掘进机机身与截割部空间深度及误差
Table 1. Space depth and error of roadheader fuselage and cutting part
数据采集点
序号机身 截割部 测量值/mm 真实值/mm 误差/mm 相对误差/% 测量值/mm 真实值/mm 误差/mm 相对误差/% 1 2 953.23 2 948.49 4.74 0.16 4 953.81 4 951.44 2.37 0.05 2 2 953.23 2 949.27 3.97 0.13 4 885.71 4 934.01 48.30 0.98 3 2 953.23 2 949.06 4.17 0.14 4 953.81 4 940.48 13.33 0.27 4 2 953.36 2 943.01 10.35 0.35 4 875.18 4 910.01 34.83 0.71 5 2 953.23 2 943.47 9.76 0.33 4 799.32 4 826.18 26.87 0.56 
下载: 导出CSV
-
[1] 刘峰,曹文君,张建明,等. 我国煤炭工业科技创新进展及“十四五”发展方向[J]. 煤炭学报,2021,46(1):1-15.LIU Feng,CAO Wenjun,ZHANG Jianming,et al. Current technological innovation and development direction of the 14th Five-Year Plan period in China coal industry[J]. Journal of China Coal Society,2021,46(1):1-15. [2] 杜雨馨,刘停,童敏明,等. 基于机器视觉的悬臂式掘进机机身位姿检测系统[J]. 煤炭学报,2016,41(11):2897-2906.DU Yuxin,LIU Ting,TONG Minming,et al. Pose measurement system of boom-type roadheader based on machine vision[J]. Journal of China Coal Society,2016,41(11):2897-2906. [3] 雷孟宇, 张旭辉, 杨文娟, 等. 煤矿掘进装备视觉位姿检测与控制研究现状与趋势[J/OL]. 煤炭学报: 1-14 [2022-01-06]. DOI: 10.13225/j. cnki. jccs. JJ21.0534.LEI Mengyu, ZHANG Xuhui, YANG Wenjuan, et al. Current status and trend of research on visual pose detection and control of tunneling equipment in coal mines[J/OL]. Journal of China Coal Society: 1-14[202201-06]. DOI: 10.13225/j.cnki.jccs.JJ21.0534. [4] 朱信平,李睿,高娟,等. 基于全站仪的掘进机机身位姿参数测量方法[J]. 煤炭工程,2011(6):113-115. doi: 10.3969/j.issn.1671-0959.2011.06.044ZHU Xinping,LI Rui,GAO Juan,et al. Measurement method of body pose parameters of roadheader based on total station[J]. Coal Engineering,2011(6):113-115. doi: 10.3969/j.issn.1671-0959.2011.06.044 [5] 田原. 基于零速修正的掘进机惯性导航定位方法[J]. 工矿自动化,2019,45(8):70-73.TIAN Yuan. Inertial navigation positioning method of roadheader based on zero-velocity update[J]. Industry and Mine Automation,2019,45(8):70-73. [6] 陶云飞,宗凯,张敏骏,等. 基于iGPS的掘进机单站多点分时机身位姿测量方法[J]. 煤炭学报,2015,40(11):2611-2616.TAO Yunfei,ZONG Kai,ZHANG Minjun,et al. Aposition and orientation measurement method of single-station,multipoint and time-sharing for roadheader body based on iGPS[J]. Journal of China Coal Society,2015,40(11):2611-2616. [7] 陶云飞,李瑞,李嘉赓,等. iGPS的单站多点分时测量系统对掘进机偏向位移精度研究[J]. 煤炭技术,2017,36(2):246-247.TAO Yunfei,LI Rui,LI Jiageng,et al. Research on positioning accuracy of roadheader based on singlestation,multipoint and time-shared of iGPS measurement system[J]. Coal Technology,2017,36(2):246-247. [8] FU Shishen,LI Yiming,ZONG Kai,et al. Ultra-wideband pose detection method based on TDOA positioning model for boom-type roadheader[J]. AEU-International Journal of Electronics and Communications,2018,99:70-80. [9] 符世琛,李一鸣,杨健健,等. 基于超宽带技术的掘进机自主定位定向方法研究[J]. 煤炭学报,2015,40(11):2603-2610.FU Shichen,LI Yiming,YANG Jianjian,et al. Research on autonomous positioning and orientation method of roadheader based on ultra wide-band technology[J]. Journal of China Coal Society,2015,40(11):2603-2610. [10] 杨文娟,张旭辉,马宏伟,等. 悬臂式掘进机机身及截割头位姿视觉测量系统研究[J]. 煤炭科学技术,2019,47(6):50-57.YANG Wenjuan,ZHANG Xuhui,MA Hongwei,et al. Research on position and posturel measurement system of body and cutting head for boom-type roadheader based on machine vision[J]. Coal Science and Technology,2019,47(6):50-57. [11] LIANG Jiefeng,JING Tianjun,NIU Huanna,et al. Two-terminal fault location method of distribution network based on adaptive convolution neural network[J]. IEEE Access,2020,8:54035-54043. doi: 10.1109/ACCESS.2020.2980573 [12] 周飞燕,金林鹏,董军. 卷积神经网络研究综述[J]. 计算机学报,2017,40(6):1229-1251. doi: 10.11897/SP.J.1016.2017.01229ZHOU Feiyan,JIN Linpeng,DONG Jun. Review of convolutional neural network[J]. Chinese Journal of Computers,2017,40(6):1229-1251. doi: 10.11897/SP.J.1016.2017.01229 [13] 牛乃平,张小刚,丁华. 基于1D-CNN的采煤机摇臂齿轮故障诊断[J]. 煤矿机械,2020,41(11):148-150.NIU Naiping,ZHANG Xiaogang,DIGN Hua. Fault diagnosis of shearer rocker arm gear based on 1D-CNN[J]. Coal Mine Machinery,2020,41(11):148-150. [14] 马乔雨,张欣,张春雷,等. 基于一维卷积神经网络的横波速度预测[J]. 岩性油气藏,2021,33(4):111-120. doi: 10.12108/yxyqc.20210412MA Qiaoyu,ZHANG Xin,ZHANG Chunlei,et al. Shear wave velocity prediction based on one-dimensional convolutional neural network[J]. Lithologic Reservoirs,2021,33(4):111-120. doi: 10.12108/yxyqc.20210412 [15] 王志同,牛志刚,郭晨星,等. 可自动回撤煤矿探测机器人设计[J]. 工矿自动化,2018,44(5):6-12.WANG Zhitong,NIU Zhigang,GUO Chenxing,et al. Design of coal mine detection robot with automatic returning[J]. Industry and Mine Automation,2018,44(5):6-12. [16] 司垒. 采煤机智能控制关键技术研究[D]. 徐州: 中国矿业大学, 2015.SI Lei. Research on key technologies of intelligent control for shearer[D]. Xuzhou: China University of Mining and Technology, 2015. -
点击查看大图
计量
- 文章访问数: 183
- HTML全文浏览量: 26
- PDF下载量: 34
- 被引次数: 0
