基于CUDA加速动态规划优化全景拼接的刮板输送机直线状态监测

李博; 侍守伊; 张建军; 夏蕊; 王学文; 崔卫秀; 倪强

doi:10.13272/j.issn.1671-251x.18223

基于CUDA加速动态规划优化全景拼接的刮板输送机直线状态监测

李博^{1, 2, 3, 4,},
侍守伊^{2, 3},
张建军^{1, 4},
夏蕊^{2, 3},
王学文^{2, 3},
崔卫秀^{1, 4},
倪强⁵

1.
中煤张家口煤矿机械有限责任公司，河北张家口　076250
2.
太原理工大学机械工程学院，山西太原　030024
3.
煤矿综采装备山西省重点实验室，山西太原　030024
4.
河北省高端智能矿山装备技术创新中心，河北张家口　076250
5.
山西康伟集团有限公司，山西长治　046503

基金项目: 国家自然科学基金青年基金项目（52204149）；山西省基础研究计划项目（202103021223080，202203021221051）。

详细信息

作者简介:
李博（1988—），男，山西太原人，副教授，博士，研究方向为煤机装备结构与性能优化，E-mail：libo@tyut.edu.cn

中图分类号: TD634.2
计量
- 文章访问数: 72
- HTML全文浏览量: 18
- PDF下载量: 17
出版历程
- 收稿日期: 2024-10-22
- 修回日期: 2025-01-21
- 网络出版日期: 2025-02-08
- 刊出日期: 2025-01-24

Straightness monitoring of scraper conveyor based on CUDA-accelerated dynamic programming and optimized panoramic stitching

LI Bo^{1, 2, 3, 4,},
SHI Shouyi^{2, 3},
ZHANG Jianjun^{1, 4},
XIA Rui^{2, 3},
WANG Xuewen^{2, 3},
CUI Weixiu^{1, 4},
NI Qiang⁵

1.
China Coal Zhangjiakou Coal Mine Machinery Co., Ltd., Zhangjiakou 076250, China
2.
College of Mechanical Engineering, Taiyuan University of Technology, Taiyuan 030024, China
3.
Shanxi Province Key Laboratory of Fully Mechanized Coal Mining Equipment, Taiyuan 030024, China
4.
Hebei Province High-end Intelligent Mine Equipment Technology Innovation Center, Zhangjiakou 076250, China
5.
Shanxi Kangwei Group Co., Ltd., Changzhi 046503, China

摘要

摘要:
为提高井下复杂恶劣环境下刮板输送机直线状态监测精度和实时性，提出了一种基于统一计算设备架构（CUDA）加速动态规划优化全景拼接的刮板输送机直线状态监测方法。首先，同步2路摄像头获取的刮板输送机图像的帧数、分辨率参数，对输入的视频流进行暗通道清晰化处理，以消除井下煤尘、水雾等的干扰。其次，使用ORB算法检测和计算2路视频帧的特征点和描述子，通过K最近邻（KNN）匹配计算特征点间的匹配对，利用设置阈值比例的方法过滤错误的匹配点，使用随机抽样一致（RANSAC）算法计算出用于图像透视变换的单应性矩阵。然后，基于CUDA将读取Sobel算子、计算梯度、计算总能量差异、循环初始化权重与路径、寻找最佳接缝线分配到不同的线程中，并定义计算能量图和寻找最佳接缝线的核函数，完成2路图像沿接缝线融合的全景拼接。最后，使用霍夫变换方法对全景拼接的刮板输送机图像中部槽挡煤板进行直线拟合，并将拟合的直线绘制在全景拼接图像上，以反映刮板输送机的直线状态。实验及测试结果表明，CUDA加速动态规划优化全景拼接痕迹不明显，且处理速度快；通过霍夫变换对中部槽挡煤板拟合的直线与刮板输送机直线具有较好的一致性，可用于刮板输送机直线状态监测。
- 刮板输送机直线状态监测 /
- CUDA加速 /
- 动态规划 /
- 全景拼接 /
- 霍夫变换 /
- 统一计算设备架构
Abstract:
To enhance the accuracy and real-time performance of straightness monitoring for scraper conveyors in complex and harsh underground environments, a straightness monitoring method for scraper conveyors based on compute unified device architecture (CUDA)-accelerated dynamic programming and optimized panoramic stitching was proposed. First, the frame rates and resolution parameters of images captured by two synchronized cameras were aligned, and the input video stream underwent dark channel enhancement to eliminate the interference of coal dust, water mist, and other factors underground. Then, the oriented FAST and rotated BRIEF (ORB) algorithm was used to detect and calculate feature points and descriptors from the two video frames. Feature point matches were calculated using K-nearest neighbors (KNN), with incorrect matches filtered using a threshold ratio. A homography matrix for image perspective transformation was calculated using the random sample consensus (RANSAC) algorithm. CUDA was employed to accelerate the processing by assigning tasks such as Sobel operator reading, gradient computation, total energy difference calculation, loop initialization of weights and paths, and optimal seam finding to different threads. Kernel functions for energy map computation and seamline optimization were defined, enabling the seamless fusion of the two images along the seamline to complete panoramic stitching. Finally, the Hough transform was applied to perform linear fitting of the coal blocking plate in the middle trough of the stitched panoramic image. The fitted line was superimposed on the panoramic image to reflect the straightness status of the scraper conveyor. The experiment and test results showed that the CUDA-accelerated dynamic programming significantly reduced visible stitching artifacts while ensuring high processing speed. The straight line fitted by the Hough transform closely matched the actual straightness of the scraper conveyor, demonstrating its effectiveness for straightness monitoring of scraper conveyors.
- scraper conveyor straightness monitoring /
- CUDA acceleration /
- dynamic programming /
- panoramic stitching /
- Hough transform /
- compute unified device architecture

HTML全文

图 1 CUDA加速动态规划优化全景拼接流程

Figure 1. Process of CUDA-accelerated dynamic programming optimization of panoramic stitching

下载: 全尺寸图片幻灯片

图 2 图像重叠区域

Figure 2. Image overlapping area

下载: 全尺寸图片幻灯片

图 3 CUDA加速动态规划原理

Figure 3. Principle of CUDA-accelerated dynamic programming

下载: 全尺寸图片幻灯片

图 4 待拼接视频帧

Figure 4. Video frames to be stitched

下载: 全尺寸图片幻灯片

图 5 不同算法下视频帧拼接效果对比

Figure 5. Comparison of video frame stitching effects under different algorithms

下载: 全尺寸图片幻灯片

图 6 全景拼接图像边缘处理

Figure 6. Panoramic stitching image edge processing

下载: 全尺寸图片幻灯片

图 7 二值化图像的ROI

Figure 7. ROI of binary image

下载: 全尺寸图片幻灯片

图 8 全景拼接图像ROI的直线拟合

Figure 8. Linear fitting of ROI for panoramic stitching image

下载: 全尺寸图片幻灯片

图 9 全景拼接图像直线拟合

Figure 9. Panoramic stitching image line fitting

下载: 全尺寸图片幻灯片

图 10 CUDA加速动态规划与传统动态规划优化全景拼接结果对比

Figure 10. Comparison of optimized panoramic stitching results between CUDA-accelerated dynamic programming and traditional dynamic programming

下载: 全尺寸图片幻灯片

图 11 刮板输送机挡煤板边缘直线拟合结果

Figure 11. Straight line fitting result of coal blocking plate edge of scraper conveyor

下载: 全尺寸图片幻灯片

图 12 不同分辨率图像和计算平台下CUDA加速动态规划和传统动态规划算法的处理时间对比

Figure 12. Comparison of processing time between CUDA-accelerated dynamic programming algorithm and traditional dynamic programming algorithm under different resolution images and computing platforms

下载: 全尺寸图片幻灯片

表 1 不同算法下视频拼接效果评价

Table 1 Evaluation of video stitching effects under different algorithms

算法亮度系数对比度清晰度信息熵

传统动态规划 0.520 7 66.85 1 133.27 6.746 5
CUDA加速动态规划 0.621 3 68.24 1 225.19 6.749 4

下载: 导出CSV

参考文献(21)

[1]	李首滨,李森,张守祥,等. 综采工作面智能感知与智能控制关键技术与应用[J]. 煤炭科学技术,2021,49(4):28-39. LI Shoubin,LI Sen,ZHANG Shouxiang,et al. Key technology and application of intelligent perception and intelligent control in fully mechanized mining face[J]. Coal Science and Technology,2021,49(4):28-39.
[2]	张智喆,王世博,张博渊,等. 基于采煤机运动轨迹的刮板输送机布置形态检测研究[J]. 煤炭学报,2015,40(11):2514-2521. ZHANG Zhizhe,WANG Shibo,ZHANG Boyuan,et al. Shape detection of scraper conveyor based on shearer trajectory[J]. Journal of China Coal Society,2015,40(11):2514-2521.
[3]	王学文,李素华,谢嘉成,等. 机器人运动学与时序预测融合驱动的刮板输送机调直方法[J]. 煤炭学报,2021,46(2):652-666. WANG Xuewen,LI Suhua,XIE Jiacheng,et al. Straightening method of scraper conveyor driven by robot kinematics and time series prediction[J]. Journal of China Coal Society,2021,46(2):652-666.
[4]	LYU Jiakun,SHI Peng,WAN Zhijun,et al. Research on a real-time monitoring method for the three-dimensional straightness of a scraper conveyor based on binocular vision[J]. Mathematics,2022,10(19). DOI: 10.3390/math10193545.
[5]	张帆,李闯,李昊. 智能综采工作面刮板输送机直线度监测方法研究[J]. 煤炭科学技术,2022,50(4):246-255. ZHANG Fan,LI Chuang,LI Hao. Study on straightness monitoring method of scraper conveyor in intelligent fully-mechanized mining face[J]. Coal Science and Technology,2022,50(4):246-255.
[6]	王雪松,王世博,王世佳,等. 刮板输送机直线度误差预测模型[J]. 中国矿业大学学报,2023,52(1):168-177. WANG Xuesong,WANG Shibo,WANG Shijia,et al. Prediction model of straightness error of scraper conveyor[J]. Journal of China University of Mining & Technology,2023,52(1):168-177.
[7]	CHEN Ningning,FANG Xinqiu,FENG Haotian,et al. Scraper conveyor shape sensing technology based on orthogonal optical fiber strain[J]. Measurement Science and Technology,2023,34(12). DOI: 10.1088/1361-6501/aceb0e.
[8]	蓝先迪. 全景视频拼接关键技术研究[D]. 成都:电子科技大学,2020. LAN Xiandi. Research on key technologies of panoramic video stitching[D]. Chengdu:University of Electronic Science and Technology of China,2020.
[9]	杨丹,陈君,朱小勇. 一种大视差场景下的实时视频拼接系统[J]. 计算机与现代化,2020(4):19-23,29. DOI: 10.3969/j.issn.1006-2475.2020.04.004 YANG Dan,CHEN Jun,ZHU Xiaoyong. A real-time video stitching system in large parallax scene[J]. Computer and Modernization,2020(4):19-23,29. DOI: 10.3969/j.issn.1006-2475.2020.04.004
[10]	杨丹,陈君,朱小勇. 基于时序特征预测的实时视频拼接算法[J]. 网络新媒体技术,2021,10(4):32-38,59. DOI: 10.3969/j.issn.2095-347X.2021.04.005 YANG Dan,CHEN Jun,ZHU Xiaoyong. Real-time video stitching algorithm based on time-series features prediction[J]. Network New Media Technology,2021,10(4):32-38,59. DOI: 10.3969/j.issn.2095-347X.2021.04.005
[11]	LIU Suming,CHAI Lin,JIN Lizuo. Panoramic image stabilization algorithm base on distance transformation and image pyramid[C]. The 1st International Conference on New Materials,Machinery and Vehicle Engineering,Zhangjiajie,2022:369-378.
[12]	侯明利. 机坪塔台全景视频拼接系统设计分析[J]. 科技创新导报,2022,19(3):99-101. HOU Mingli. Analysis and design of a panoramic video stitching system for apron control tower[J]. Science and Technology Innovation Herald,2022,19(3):99-101.
[13]	武忆涵. 基于自适应接缝线和快速融合的全景视频拼接技术研究[D]. 西安:长安大学,2023. WU Yihan. Research on panoramic video stitching technology based on adaptive seam line and fast fusion[D]. Xi'an:Changan University,2023.
[14]	雍玖,王阳萍,雷晓妹,等. 基于改进ORB特征检测的全景视频拼接[J]. 计算机应用与软件,2017,34(5):182-188. DOI: 10.3969/j.issn.1000-386x.2017.05.032 YONG Jiu,WANG Yangping,LEI Xiaomei,et al. Panoramic video stitching method based on improved ORB feature detection[J]. Computer Applications and Software,2017,34(5):182-188. DOI: 10.3969/j.issn.1000-386x.2017.05.032
[15]	刘台,胡斌,田进龙,等. 全景视频实时拼接技术研究与实现[J]. 移动通信,2023,47(2):69-76. LIU Tai,HU Bin,TIAN Jinlong,et al. Research and implementation of real-time stitching technology of panoramic video[J]. Mobile Communications,2023,47(2):69-76.
[16]	程德强,厉航,黄晓丽,等. 基于改进随机抽样一致算法的视频拼接算法[J]. 工矿自动化,2017,43(8):50-55. CHENG Deqiang,LI Hang,HUANG Xiaoli,et al. Video mosaic algorithm based on improved random sample consensus algorithm[J]. Industry and Mine Automation,2017,43(8):50-55.
[17]	ZHANG Yun,LAI Yukun,ZHANG Fanglüe. Content-preserving image stitching with piecewise rectangular boundary constraints[J]. IEEE Transactions on Visualization and Computer Graphics,2021,27(7):3198-3212. DOI: 10.1109/TVCG.2020.2965097
[18]	管增伦,顾军,赵广源. 基于改进加速鲁棒特征的井下视频拼接算法[J]. 工矿自动化,2018,44(11):69-74. GUAN Zenglun,GU Jun,ZHAO Guangyuan. Underground video stitching algorithm based on improved speeded up robust features[J]. Industry and Mine Automation,2018,44(11):69-74.
[19]	罗永涛,王艳,张红民. 结合最佳缝合线和改进渐入渐出法的图像拼接算法[J]. 红外技术,2018,40(4):382. LUO Yongtao,WANG Yan,ZHANG Hongmin. Image-stitching algorithm by combining the optimal seam and an improved gradual fusion method[J]. Infrared Technology,2018,40(4):382.
[20]	张翔,王伟,肖迪. 基于改进最佳缝合线的图像拼接方法[J]. 计算机工程与设计,2018,39(7):1964-1970. ZHANG Xiang,WANG Wei,XIAO Di. Image mosaic method based on improved best seam-line[J]. Computer Engineering and Design,2018,39(7):1964-1970.
[21]	聂子博,曹建军,翁年凤,等. 基于直线生成的卷积霍夫线段检测[J]. 计算机技术与发展,2024,34(5):30-36. NIE Zibo,CAO Jianjun,WENG Nianfeng,et al. Convolutional Hough line segment detection based on line generation[J]. Computer Technology and Development,2024,34(5):30-36.