Volume 49 Issue 5
May  2023
Turn off MathJax
Article Contents
LI Chengcheng, MA Lisen, TIAN Yuan, et al. An onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter[J]. Journal of Mine Automation,2023,49(5):66-73.  doi: 10.13272/j.issn.1671-251x.2022100002
Citation: LI Chengcheng, MA Lisen, TIAN Yuan, et al. An onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter[J]. Journal of Mine Automation,2023,49(5):66-73.  doi: 10.13272/j.issn.1671-251x.2022100002

An onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter

doi: 10.13272/j.issn.1671-251x.2022100002
  • Received Date: 2022-10-05
  • Rev Recd Date: 2023-05-15
  • Available Online: 2022-12-05
  • During the movement or operation of coal mining equipment such as a roadheader, the vibration of the vehicle body can easily cause blurring of the onboard camera video. This leads to a decrease in the precision and reliability of machine vision detection based on the onboard video. In order to solve the above problem, an onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter is proposed. This algorithm consists of three parts: motion estimation, trajectory smoothing, and motion compensation. In the motion estimation stage, the contrast limited adaptive histogram equalization (CLAHE) algorithm is used to enhance the image of the underground roadway. The Shi-Tomasi algorithm is used to obtain the feature points of each image frame. The obtained feature points are tracked and matched by optical flow, and then the motion trajectory of the camera is calculated. In the trajectory smoothing stage, Kalman filtering is used to predict the current time value based on the optimal value of the previous frame of the video. It avoids the problem of pre-storing sampling data of mean filtering and improves the real-time performance of image stabilization. In the motion compensation stage, the jitter video is compensated frame by frame based on the relationship between the original motion path and the smooth path, generating a stable video sequence. The experimental results show the following points: ① After CLAHE enhancement processing, the success rate of feature point matching is increased by 58% compared to the non-enhancement processing and 43% compared to the HE enhancement processing. It indicates that the CLAHE algorithm can effectively improve the matching number of image feature points. ② Through pixel offset analysis, differential image analysis, and peak signal-to-noise ratio (PSNR) analysis, it is verified that the onboard video stabilization algorithm for roadheader based on CLAHE and Kalman filter has a good image stabilization effect. ③ Compared with the traditional HE+mean filtering algorithm, the algorithm based on CLAHE and Kalman filter reduces the overall time consumption of processing 100 frames of video images by 0.379 seconds, effectively improving the real-time performance of the video stabilization while removing jitter.

     

  • loading
  • [1]
    张旭辉,杨文娟,薛旭升,等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报,2022,47(1):579-597.

    ZHANG Xuhui,YANG Wenjuan,XUE Xusheng,et al. Challenges and developing of the intelligent remote controlon roadheaders in coal mine[J]. Journal of China Coal Society,2022,47(1):579-597.
    [2]
    WANG Guofa,REN Huaiwei,ZHAO Guorui,et al. Research and practice of intelligent coal mine technology systems in China[J]. International Journal of Coal Science & Technology,2022,9(2):19-35.
    [3]
    魏闪闪,谢巍,贺志强. 数字视频稳像技术综述[J]. 计算机研究与发展,2017,54(9):2044-2058. doi: 10.7544/issn1000-1239.2017.20160078

    WEI Shanshan,XIE Wei,HE Zhiqiang. Digital video stabilization techniques:a survey[J]. Journal of Computer Research and Development,2017,54(9):2044-2058. doi: 10.7544/issn1000-1239.2017.20160078
    [4]
    熊炜,王传胜,李利荣,等. 结合光流法和卡尔曼滤波的视频稳像算法[J]. 计算机工程与科学,2020,42(3):493-499. doi: 10.3969/j.issn.1007-130X.2020.03.015

    XIONG Wei,WANG Chuansheng,LI Lirong,et al. Video stabilization algorithm based on optical flow method and Kalman filtering[J]. Computer Engineering & Science,2020,42(3):493-499. doi: 10.3969/j.issn.1007-130X.2020.03.015
    [5]
    熊炜,王传胜,管来福,等. 基于特征跟踪和网格路径运动的视频稳像算法[J]. 计算机工程与科学,2020,42(5):843-850.

    XIONG Wei,WANG Chuansheng,GUAN Laifu,et al. A video stabilization algorithm based on feature tracking and mesh path motion[J]. Computer Engineering & Science,2020,42(5):843-850.
    [6]
    朱娟娟,范静,郭宝龙. 抗前景干扰的自适应电子稳像算法[J]. 光子学报,2015,44(6):45-52.

    ZHU Juanjuan,FAN Jing,GUO Baolong. Adaptive electronic image stabilization algorithm resistant to foreground moving object[J]. Acta Photonica Sinica,2015,44(6):45-52.
    [7]
    范叶平,郭政,张锐. 基于下采样灰度投影的电子稳像算法研究[J]. 工矿自动化,2017,43(4):22-27.

    FAN Yeping,GUO Zheng,ZHANG Rui. Research on electronic image stabilization algorithm based on subsample gray-scale projection[J]. Industry and Mine Automation,2017,43(4):22-27.
    [8]
    王施鳗,许文海,董丽丽,等. 基于改进的Harris角点的机载红外图像电子稳像[J]. 红外技术,2020,42(6):573-579. doi: 10.3724/SP.J.7102068595

    WANG Shiman,XU Wenhai,DONG Lili,et al. Electronic image stabilization of airborne infrared images based on improved Harris corner detection[J]. Infrared Technology,2020,42(6):573-579. doi: 10.3724/SP.J.7102068595
    [9]
    程德强,郭政,刘洁,等. 一种基于改进光流法的电子稳像算法[J]. 煤炭学报,2015,40(3):707-712.

    CHENG Deqiang,GUO Zheng,LIU Jie,et al. An electronic image stabilization algorithm based on improved optical flow method[J]. Journal of China Coal Society,2015,40(3):707-712.
    [10]
    孙继平,田子建. 矿井图像监视系统与关键技术[J]. 煤炭科学技术,2014,42(1):65-68.

    SUN Jiping,TIAN Zijian. Image monitoring system and key technology in underground mine[J]. Coal Science and Technology,2014,42(1):65-68.
    [11]
    王诚聪,刘亚静. 矿井复杂环境视频监控图像增强算法研究[J]. 煤炭工程,2021,53(4):147-151.

    WANG Chengcong,LIU Yajing. Image enhancement algorithm for video surveillance in complex environment of underground mine[J]. Coal Engineering,2021,53(4):147-151.
    [12]
    王殿伟,王晶,许志杰,等. 一种光照不均匀图像的自适应校正算法[J]. 系统工程与电子技术,2017,39(6):1383-1390. doi: 10.3969/j.issn.1001-506X.2017.06.29

    WANG Dianwei,WANG Jing,XU Zhijie,et al. Adaptive correction algorithm for non-uniform illumination images[J]. Systems Engineering and Electronics,2017,39(6):1383-1390. doi: 10.3969/j.issn.1001-506X.2017.06.29
    [13]
    余成波,孔庆达,田桐. 基于CLAHE与同态滤波的细胞图像增强新方法[J]. 微型机与应用,2017,36(4):51-52,62.

    YU Chengbo,KONG Qingda,TIAN Tong. New approach to the cell image enhancement based on adaptive histogram equalization and homomorphic filtering[J]. Microcomputer & Its Applications,2017,36(4):51-52,62.
    [14]
    YADAV G, MAHESHWARI S, AGARWAL A. Contrastlimited adaptive histogram equalization based enhancement for real time video system[C]. International Conference on Advances in Computing, Communications and Informatics, Delhi, 2014: 2392-2397.
    [15]
    BANSAL M,KUMAR M,KUMAR K,et al. An efficient technique for object recognition using Shi-Tomasi corner detection algorithm[J]. Soft Computing,2021,25(6):4423-4432. doi: 10.1007/s00500-020-05453-y
    [16]
    WANG Zhen, YANG Xiaojun. Moving target detection and tracking based on pyramid Lucas-Kanade optical flow[C]. The 3rd IEEE International Conference on Image, Vision and Computing, Chongqing, 2018: 66-69.
    [17]
    RODRÍGUEZ-PADILLA I,CASTELLE B,MARIEU V,et al. A simple and efficient image stabilization method for coastal monitoring video systems[J]. Remote Sensing,2019,12(1):1-21. doi: 10.3390/rs12010001
    [18]
    ZHAI Bo,ZHENG Jin,LI Bo. Digital image stabilization based on adaptive motion filtering with feedback correction[J]. Multimedia Tools and Applications,2016,75:12173-12200. doi: 10.1007/s11042-015-3183-3
    [19]
    LI Lengyi, MA Xiaohong, ZHAO Zheng. Real-time video stabilization based on fast block matching and improved Kalman filter[C]. The Fifth International Conference on Intelligent Control and Information Processing, Dalian, 2014: 394-397.
    [20]
    程德强,黄晓丽,厉航,等. 矿井车载视频图像稳像算法研究[J]. 工矿自动化,2017,43(11):21-26.

    CHENG Deqiang,HUANG Xiaoli,LI Hang,et al. Research on vehicle video image stabilization algorithm for mine[J]. Industry and Mine Automation,2017,43(11):21-26.
    [21]
    SOUZA M R E,MAIA H D A,PEDRINI H. Survey on digital video stabilization:concepts,methods,and challenges[J]. ACM Computing Surveys,2022,55(3):1-37.
    [22]
    陈滨,赵建军,杨利斌. 视频稳像评价方法研究[J]. 科学技术与工程,2016,16(22):219-224,230. doi: 10.3969/j.issn.1671-1815.2016.22.039

    CHEN Bin,ZHAO Jianjun,YANG Libin. Research on evaluation method of video stabilization[J]. Science Technology and Engineering,2016,16(22):219-224,230. doi: 10.3969/j.issn.1671-1815.2016.22.039
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(10)  / Tables(2)

    Article Metrics

    Article views (267) PDF downloads(13) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return