A line feature matching algorithm for mine images based on line segment detection and LT descriptors
-
摘要: 图像匹配是同步定位与地图构建（SLAM）技术中极为重要的一环，用于根据图像之间的变换关系确定相机位姿。基于线特征的图像匹配方法具有较强的鲁棒性和抗噪能力，更加适用于井下图像匹配，基于深度学习的线描述符对线段遮挡等场景具有较高的鲁棒性，性能优于传统描述符，但卷积神经网络架构的描述符将可变长度线段抽象为固定维进行描述，不利于线段长度及视差变化较大图像的匹配。针对上述问题，提出一种基于直线段检测和线描述符的矿井图像线特征匹配算法。在频域利用单参数同态滤波降低图像的照射分量，并增强反射分量，提升亮度及对比度；在YUV空间利用对比度受限的自适应直方图均衡化（CLAHE）算法对亮度分量进行均衡，使亮度分布更加均匀；变换至RGB空间提取直线段检测（LSD）线，引入一种基于Transformer架构的LT描述符构建LSD线的特征向量，最后完成线特征匹配。实验结果表明：该算法结合了同态滤波和CLAHE算法的优点，增强后图像的亮度适中，对比度良好，灰度分布均匀，增强效果优于单参数同态滤波算法、EnlightenGAN算法；该算法提取的线特征数较原图平均提升了32.92%，在不同相似纹理占比、不同程度旋转与平移变化的井下图像匹配中鲁棒性好，平均正确匹配数为61.75对，平均精度为86.83%，优于线二进制描述符（LBD）算法、LBD_NNDR算法、LT算法，能够满足矿井图像稳健匹配的需求。Abstract: Image matching is an extremely important part of simultaneous localization and mapping (SLAM) technology. It is used to determine camera position and posture based on the transformation relationship between images. The image matching method based on line features has strong robustness and noise resistance, making it more suitable for underground image matching. The line descriptors based on deep learning have high robustness to scenes such as line segment occlusion, and their performance is better than traditional descriptors. However, the descriptors of convolutional neural network architecture abstract variable length line segments into fixed dimensions for description, which is not conducive to matching images with large changes in line segment length and parallax. In order to solve the above problems, a line feature matching algorithm for mine images based on line segment detection and line transformers (LT) is proposed. The algorithm uses single parameter homomorphic filtering in the frequency domain to reduce the lighting component of the image, enhance the reflection component, and improve brightness and contrast. The algorithm uses contrast limited adaptive histogram equalization (CLAHE) algorithm in YUV space to balance brightness components and make brightness distribution more even. The algorithm transforms to RGB space to extract line segment detection (LSD) lines. A LT descriptor based on Transformer architecture is introduced to construct the feature vector of LSD lines, and finally complete line feature matching. The experimental results show that the algorithm combines the advantages of homomorphic filtering and CLAHE algorithm. After image enhancement, the brightness of the image is moderate, the contrast is good, the grayscale distribution is even. The enhancement effect is better than the single parameter homomorphic filtering algorithm and EnlightenGAN algorithm. The number of line features extracted by this algorithm has increased by an average of 32.92% compared to the original image. It has good robustness in matching underground images with different proportions of similar textures, varying degrees of rotation and translation changes. The average correct matching number is 61.75 pairs, with an average precision of 86.83%. It is superior to the line binary descriptor (LBD) algorithm, LBD_NNDR algorithm, and LT algorithm. It can meet the requirements of robust matching of mine images.
-
图 1 基于LSD和LT描述符的矿井图像线特征匹配算法流程
Figure 1. Process of mine image line feature matching algorithm based on line segment detector(LSD) and line transformers(LT) descriptor
图 4 CLAHE原理
Figure 4. Principle of contrast limited adaptive histogram equalization(CLAHE)
图 5 基于LSD和LT描述符的矿井图像线特征匹配算法模型
Figure 5. Model of mine image line feature matching algorithm based on LSD and LT descriptor
表 1 LT描述符的训练参数
Table 1. Training parameters of the LT descriptor
参数 值 学习率 0.001 训练轮次 1 000 图像大小 640×480 线长度最小阈值 16 最大Token数 21 Token间距 8 描述符维度 256 注意力头数量 4 编码器特征维数 [32 64 128 256] 线段描述层数量 12 前馈层内部维度 1 024 签名网络层数 7 Transformer编码器层数 12 
下载: 导出CSV
表 2 图像增强结果统计
Table 2. Statistics of image enhancement results
算法 标准差 均值 信息熵 PSNR 改进同态滤波算法 63.583 123.820 7.712 10.014 EnlightenGAN算法 63.312 153.775 7.683 7.598 本文算法 64.108 129.174 7.798 10.178
下载: 导出CSV
表 3 LSD线段提取数量
Table 3. LSD line segment extraction quantity
采集点 帧数 LSD线平均数/条 增长率/% 原图 本文算法增强图像 巷道 480 94.8 125.6 32.49 水房 218 120.5 166.4 38.09 工作面 324 138.8 181.5 30.85 避难硐室 780 184.4 244.8 32.75 平均值 144.6 192.2 32.92
下载: 导出CSV
表 4 图像属性
Table 4. Image attributes
图像序号 采集位置 旋转与平移程度 相似纹理占比 1 巷道 较小 较小 2 避难硐室 较小 较大 3 巷道 较大 较小 4 避难硐室 较大 较大
下载: 导出CSV
表 5 线特征匹配实验数据统计
Table 5. Statistics of experimental data of line feature matching
图像
序号变化
程度相似纹
理占比算法 线特征数量/条 同名直
线数/对正确匹
配数/对匹配
精度/%左 右 1 较小 较小 LBD 125 105 62 53 85.48 LBD_NNDR 125 105 58 52 89.65 LT 125 105 51 46 90.19 本文算法 149 130 63 58 92.06 2 较小 较大 LBD 180 182 50 28 56.00 LBD_NNDR 180 182 33 25 75.76 LT 180 182 62 52 83.87 本文算法 327 335 123 100 85.47 3 较大 较小 LBD 100 181 20 12 60.00 LBD_NNDR 100 181 15 11 73.33 LT 100 181 37 34 91.89 本文算法 137 258 42 39 92.86 4 较大 较大 LBD 287 252 60 9 15.00 LBD_NNDR 287 252 18 8 44.44 LT 287 252 55 38 69.09 本文算法 350 385 65 50 76.92
下载: 导出CSV
-
[1] 王国法,任世华,庞义辉,等. 煤炭工业“十三五”发展成效与“双碳”目标实施路径[J]. 煤炭科学技术,2021,49(9):1-8.WANG Guofa,REN Shihua,PANG Yihui,et al. Development achievements of China' s coal industry during the 13th Five-Year Plan period and implementation path of "dual carbon" target[J]. Coal Science and Technology,2021,49(9):1-8. [2] 程德强,钱建生,郭星歌,等. 煤矿安全生产视频AI识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349-365.CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349-365. [3] 苗升,刘小雄,黄剑雄,等. 无人机视觉SLAM环境感知发展研究[J]. 计算机测量与控制,2021,29(8):1-6,41.MIAO Sheng,LIU Xiaoxiong,HUANG Jianxiong,et al. Research on development of UAV visual SLAM environment perception[J]. Computer Measurement & Control,2021,29(8):1-6,41. [4] 孔二伟,张亚邦,李佳悦,等. 面向煤矿井下低光照图像的增强方法[J]. 工矿自动化,2023,49(4):62-69,85.KONG Erwei,ZHANG Yabang,LI Jiayue,et al. An enhancement method for low light images in coal mines[J]. Journal of Mine Automation,2023,49(4):62-69,85. [5] 赵良玉,金瑞,朱叶青,等. 基于点线特征融合的双目惯性SLAM算法[J]. 航空学报,2022,43(3):363-377.ZHAO Liangyu,JIN Rui,ZHU Yeqing,et al. Stereo visual-inertial SLAM algorithm based on merge of point and line features[J]. Acta Aeronautica et Astronautica Sinica,2022,43(3):363-377. [6] ZHU Daixian,JI Kangkang,WU Dong,et al. A coupled visual and inertial measurement units method for locating and mapping in coal mine tunnel[J]. Sensors,2022,22(19). DOI: 10.3390/s22197437. [7] 谢晓佳. 基于点线综合特征的双目视觉SLAM方法[D]. 杭州:浙江大学,2017.XIE Xiaojia. Stereo visual SLAM using point and line features[D]. Hangzhou:Zhejiang University,2017. [8] WANG Wei,GAO Wei,CUI Hainan,et al. Reconstruction of lines and planes of urban buildings with angle regularization[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2020,165:54-66. doi: 10.1016/j.isprsjprs.2020.04.013 [9] 宋佳璇,范大昭,董杨,等. 神经网络学习与灰度信息结合的跨视角影像线特征匹配算法[J]. 测绘学报,2023,52(6):990-999. doi: 10.11947/j.AGCS.2023.20220468SONG Jiaxuan,FAN Dazhao,DONG Yang,et al. Line matching algorithm for cross-view images combining neural network learning with grayscale information[J]. Acta Geodaetica et Cartographica Sinica,2023,52(6):990-999. doi: 10.11947/j.AGCS.2023.20220468 [10] CHEN Min,YAN Shaohua,QIN Rongjun,et al. Hierarchical line segment matching for wide-baseline images via exploiting viewpoint robust local structure and geometric constraints[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2021,181:48-66. doi: 10.1016/j.isprsjprs.2021.09.002 [11] CHEN Min,LI Wen,FANG Tong,et al. An adaptive feature region-based line segment matching method for viewpoint-changed images with discontinuous parallax and poor textures[J]. International Journal of Applied Earth Observation and Geoinformation,2023,117. DOI: 10.1016/j.jag.2023.103209. [12] LANGE M,SCHWEINFURTH F,SCHILLING A. DLD:a deep learning based line descriptor for line feature matching[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems,Macau,2019:5910-5915. [13] 王竞雪,刘肃艳,王伟玺. 联合共线约束与匹配冗余的组直线匹配结果检核算法[J]. 测绘学报,2020,49(6):746-756.WANG Jingxue,LIU Suyan,WANG Weixi. A checking algorithm for pair-wise line matching based on collinearity constraint and matching redundancy[J]. Acta Geodaetica et Cartographica Sinica,2020,49(6):746-756. [14] LI Gang,ZENG Yawen,HUANG Huilan,et al. A multi-feature fusion slam system attaching semantic invariant to points and lines[J]. Sensors,2021,21(4). DOI: 10.3390/s21041196. [15] ZHENG Xianwei,YUAN Zhuang,DONG Zhen,et al. Smoothly varying projective transformation for line segment matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2022,183:129-146. doi: 10.1016/j.isprsjprs.2021.10.017 [16] WANG Qiang,ZHANG Wei,LIU Xiaolong,et al. Line matching of wide baseline images in an affine projection space[J]. International Journal of Remote Sensing,2020,41(2):632-654. doi: 10.1080/01431161.2019.1646937 [17] SHEN Liang,ZHU Jiahua,XIN Qin,et al. Robust line segment mismatch removal using point-pair representation and Gaussian-uniform mixture formulation[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2023,203:314-327. doi: 10.1016/j.isprsjprs.2023.08.003 [18] 刘肃艳,王竞雪,沈昭宇,等. 结合线对几何特征及单线描述符约束的直线匹配算法[J]. 武汉大学学报(信息科学版),2023,48(6):936-949.LIU Suyan,WANG Jingxue,SHEN Zhaoyu,et al. Line matching algorithm based on pair-wise geometric features and individual line descriptor constraints[J]. Geomatics and Information Science of Wuhan University,2023,48(6):936-949. [19] 张珊,张卡,赵立科,等. 结合网状描述符和单应约束的近景影像直线匹配[J]. 地球信息科学学报,2022,24(11):2186-2197.ZHANG Shan,ZHANG Ka,ZHAO Like,et al. Close-range image line matching based on mesh descriptor and homography constraint[J]. Journal of Geo-information Science,2022,24(11):2186-2197. [20] VAKHITOV A,LEMPITSKY A. Learnable line segment descriptor for visual SLAM[J]. IEEE Access,2019,7:39923-39934. doi: 10.1109/ACCESS.2019.2901584 [21] LANGE M,RAISCH C,SCHILLING A. WLD:a wavelet and learning based line descriptor for line feature matching[M]//KRÜGER J,NIESSNER M,STÜCKLER J. Vision,modeling,and visualization. Eindhoven:The Eurographics Association,2020:39-46. [22] PAUTRAT R,LIN J T,LARSSON V,et al. SOLD2:self-supervised occlusion-aware line description and detection[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Nashville,2021:11368-11378. [23] YOON S,KIM A. Line as a visual sentence:context-aware line descriptor for visual localization[J]. IEEE Robotics and Automation Letters,2021,6(4):8726-8733. doi: 10.1109/LRA.2021.3111760 [24] GUO Chunle,LI Chongyi,GUO Jichang,et al. Zero-reference deep curve estimation for low-light image enhancement[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,2020:1777-1786. [25] 郭永坤,朱彦陈,刘莉萍,等. 空频域图像增强方法研究综述[J]. 计算机工程与应用,2022,58(11):23-32.GUO Yongkun,ZHU Yanchen,LIU Liping,et al. Research review of space-frequency domain image enhancement methods[J]. Computer Engineering and Applications,2022,58(11):23-32. [26] 王智奇,李荣冰,刘建业,等. 基于同态滤波和直方图均衡化的图像增强算法[J]. 电子测量技术,2020,43(24):75-80.WANG Zhiqi,LI Rongbing,LIU Jianye,et al. Image enhancement algorithm based on homomorphic filtering and histogram equalization[J]. Electronic Measurement Technology,2020,43(24):75-80. [27] HANA F M,MAULIDA I D. Analysis of contrast limited adaptive histogram equalization (CLAHE) parameters on finger knuckle print identification[J]. Journal of Physics:Conference Series,2021,1764. DOI: 10.1088/1742-6596/1764/1/012049. [28] JIANG Yifan,GONG Xinyu,LIU Ding,et al. EnlightenGAN:deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing,2021,30:2340-2349. doi: 10.1109/TIP.2021.3051462 [29] 高宇彤. 基于透视变换与LBD描述子约束的特征线匹配算法[D]. 阜新:辽宁工程技术大学,2022.GAO Yutong. Feature line matching algorithm based on perspective transformation and LBD descriptor constraint[D]. Fuxin:Liaoning Technical University,2022. -
点击查看大图
图(15) / 表(5)
计量
- 文章访问数: 59
- HTML全文浏览量: 19
- PDF下载量: 8
- 被引次数: 0
