Unsupervised learning-based panoramic unfolded image stitching method for rock mass borehole wall
-
摘要:
岩体孔壁全景展开图像拼接时采用传统方法时存在相邻图像间建立特征对应关系的鲁棒性不足，提取的图像特征点质量差、数量少等问题，采用有监督学习方法无法获取足够精确标注的匹配点对。针对上述问题，提出了一种基于无监督学习的岩体孔壁全景展开图像拼接方法。通过基于分组卷积改进的ResNet网络对相邻2幅待拼接的岩体孔壁全景展开图像进行多尺度特征提取；引入匹配度交叉相关计算模块识别和对齐特征图中的特征，进而确定对应特征图间的空间关系；通过全局和局部变形偏移量计算网络模块对图像进行精确的空间特征对齐；通过单应性变形模块与图像网格变形模块，有效消除相邻图像间的特征变形，实现图像间的整体对齐和精细的局部调整，精准配准局部特征和变形。实验结果表明：该方法能有效克服图像特征偏移、内容错位、细节特征丢失和拼接失败等问题，拼接处几乎没有明显的拼接痕迹，提高了拼接图像的整体质量和视觉效果；在均方根误差（RMSE）、重叠区域的峰值信噪比（PSNR）和结构相似性（SSIM）指标上均优于其他主流的图像拼接方法，显著提升了岩体孔壁全景展开图像拼接精度。
-
关键词:
- 岩体孔壁全景展开图像 /
- 图像拼接 /
- 无监督学习 /
- 全局和局部变形偏移量 /
- 单应性变形
Abstract:Traditional methods for panoramic unfolded image stitching of rock mass borehole walls suffer from insufficient robustness in establishing feature correspondences between adjacent images, as well as poor quality, limited quantity of extracted image feature points and otherc problems. Meanwhile, supervised learning methods cannot obtain sufficiently precise labeled matching point pairs. To address these issues, an unsupervised learning-based panoramic unfolded image stitching method for rock mass borehole wall is proposed. Multi-scale feature extraction was performed on two adjacent panoramic unfolded images of the rock mass borehole wall to be stitched, using a ResNet network improved with grouped convolutions. A matching degree cross-correlation calculation module was introduced to identify and align features within the feature maps, thereby determining the spatial relationships between corresponding feature maps. A global and local deformation offset calculation network module precisely aligned spatial features of the images. Furthermore, homography deformation and image grid deformation modules effectively eliminated feature distortions between adjacent images, achieving overall alignment and fine local adjustments, enabling accurate registration of local features and deformations. Experimental results showed that this method effectively overcame issues such as image feature displacement, content misalignment, loss of detailed features, and stitching failure. The stitching seams exhibited almost no visible artifacts, improving the overall quality and visual effect of the stitched images. The method outperforms other mainstream image stitching approaches in terms of Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR) in overlapping regions, and Structural Similarity (SSIM) index, significantly enhancing the stitching accuracy of panoramic unfolded images of rock mass borehole walls.
-
-
![]()
图 1 基于无监督学习的岩体孔壁全景展开图像拼接方法原理
Figure 1. Principle of unsupervised learning-based panoramic unfolded image stitching method for rock mass borehole wall
![]()
图 4 匹配度交叉相关计算模块结构
Figure 4. Structure of matching degree cross-correlation calculation module
![]()
图 6 全局变形偏移量计算网络模块结构
Figure 6. Structure of global deformation offset calculation network module
![]()
图 7 局部变形偏移量计算网络模块结构
Figure 7. Structure of local deformation offset calculation network module
![]()
图 12 不同方法下图像拼接结果对比
Figure 12. Comparison of image stitching result using different methods
表 1 不同特征提取网络性能对比
Table 1 Performance comparison of different feature extraction networks
网络 RMSE PSNR SSIM ResNet−18 2.022 7 22.667 9 0.741 6 ResNet−34 1.692 7 24.157 9 0.775 1 ResNet−50 0.152 7 25.337 9 0.835 7 
下载: 导出CSV
表 2 不同方法下图像拼接性能指标对比
Table 2 Comparison of image stitching performance metrics using different methods
方法 RMSE PSNR SSIM SIFT+RANSAC 9.432 8 21.252 6 0.718 5 APAP 6.453 1 13.471 4 0.208 9 UDHN 4.298 7 15.745 5 0.339 5 UDIS 0.170 3 24.271 8 0.776 4 EUIS 0.169 6 24.392 8 0.780 1 本文方法 0.152 7 25.337 9 0.835 7
下载: 导出CSV
-
[1] 朱合华,潘柄屹,武威,等. 岩体结构面信息采集及识别方法研究进展[J]. 应用基础与工程科学学报,2023,31(6):1339-1360. ZHU Hehua,PAN Bingyi,WU Wei,et al. Review on collection and extraction methods of rock mass discontinuity information[J]. Journal of Basic Science and Engineering,2023,31(6):1339-1360.
[2] 夏丁,葛云峰,唐辉明,等. 数字钻孔图像兴趣区域分割与岩体结构面特征识别[J]. 地球科学,2020,45(11):4207-4217. XIA Ding,GE Yunfeng,TANG Huiming,et al. Segmentation of region of interest and identification of rock discontinuities in digital borehole images[J]. Earth Science,2020,45(11):4207-4217.
[3] 邹先坚,王川婴,宋欢. 全景钻孔图像自动识别技术在工程实践中的应用研究[J]. 应用基础与工程科学学报,2022,30(1):246-256. ZOU Xianjian,WANG Chuanying,SONG Huan. Application of automatic recognition technique for the panoramic borehole images obtained in engineering practice[J]. Journal of Basic Science and Engineering,2022,30(1):246-256.
[4] 苑朝,黄诺飞,蒋阳,等. 基于改进旋转不变性二进制描述算法的电力场景图像拼接[J]. 电力科学与工程,2024,40(1):31-38. YUAN Chao,HUANG Nuofei,JIANG Yang,et al. Power scene images mosaic based on improved oriented fast and rotated brief algorithm[J]. Electric Power Science and Engineering,2024,40(1):31-38.
[5] 郑阳阳,王慧琴,王可,等. 改进网格单应性投影变换的文物多镜头光谱图像拼接方法[J]. 计算机工程与应用,2024,60(21):225-235. DOI: 10.3778/j.issn.1002-8331.2310-0386 ZHENG Yangyang,WANG Huiqin,WANG Ke,et al. Multi-lens spectral image mosaic method of cultural relics with improved grid projection transformation[J]. Computer Engineering and Applications,2024,60(21):225-235. DOI: 10.3778/j.issn.1002-8331.2310-0386
[6] 邹先坚,王川婴,宋欢. 岩体孔内摄像视频高精度快速成图方法研究[J]. 工程科学与技术,2021,53(4):158-167. ZOU Xianjian,WANG Chuanying,SONG Huan. A high-precision and fast image forming method for borehole camera video of rock mass[J]. Advanced Engineering Sciences,2021,53(4):158-167.
[7] GAO Junhong,KIM S J,BROWN M S. Constructing image panoramas using dual-homography warping[C]. Conference on Computer Vision and Pattern Recognition,Colorado Springs,2011:49-56.
[8] ZARAGOZA J,CHIN T J,BROWN M S,et al. As-projective-As-possible image stitching with moving DLT[C]. IEEE Conference on Computer Vision and Pattern Recognition,Portland,2013:2339-2346.
[9] RANI V,NABI S T,KUMAR M,et al. Self-supervised learning:a succinct review[J]. Archives of Computational Methods in Engineering,2023,30(4):2761-2775. DOI: 10.1007/s11831-023-09884-2
[10] AN Jianping,YANG Kai,GAO Xiaozheng,et al. Sustainable ultra-dense heterogeneous networks[M]//Sustainable Wireless Communications. Singapore:Springer Nature Singapore,2022:7-19.
[11] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas,2016:770-778.
[12] 李雪浩,肖秦琨,杨梦薇. 基于分组卷积的轻量级图像超分辨率重建[J]. 激光杂志,2023,44(3):126-132. LI Xuehao,XIAO Qinkun,YANG Mengwei. Lightweight image super-resolution algorithm based on grouping convolution[J]. Laser Journal,2023,44(3):126-132.
[13] COHEN T S,WELLING M. Group equivariant convolutional networks[C]. The 33rd International Conference on Machine Learning,New York,2016:2990-2999.
[14] VAN LAARHOVEN T. L2 regularization versus batch and weight normalization[EB/OL]. [2024-09-27]. https://arxiv.org/abs/1706.05350v1.
[15] GU Xiaodong,FAN Zhiwen,ZHU Siyu,et al. Cascade cost volume for high-resolution multi-view stereo and stereo matching[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,2020:2492-2501.
[16] WANG Meiqi,LU Siyuan,ZHU Danyang,et al. A high-speed and low-complexity architecture for softmax function in deep learning[C]. IEEE Asia Pacific Conference on Circuits and Systems,Chengdu,2018:223-226.
[17] 杜港,侯凌燕,佟强,等. 基于BRISK和改进RANSAC算法的图像拼接[J]. 液晶与显示,2022,37(6):758-767. DOI: 10.37188/CJLCD.2021-0292 DU Gang,HOU Lingyan,TONG Qiang,et al. Image mosaicing based on BRISK and improved RANSAC algorithm[J]. Chinese Journal of Liquid Crystals and Displays,2022,37(6):758-767. DOI: 10.37188/CJLCD.2021-0292
[18] ABDEL-AZIZ Y I,KARARA H M,HAUCK M. Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry[J]. Photogrammetric Engineering & Remote Sensing,2015,81(2):103-107.
[19] 张省,朱伟. 基于SIFT匹配和RANSAC算法的超分辨率重建[J]. 测绘通报,2019(10):119-122. ZHANG Sheng,ZHU Wei. Super-resolution reconstruction based on SIFT matching and RANSAC algorithm[J]. Bulletin of Surveying and Mapping,2019(10):119-122.
[20] NIE Lang,LIN Chunyu,LIAO Kang,et al. Unsupervised deep image stitching:reconstructing stitched features to images[J]. IEEE Transactions on Image Processing,2021,30:6184-6197. DOI: 10.1109/TIP.2021.3092828
[21] QIN Chunbin,RAN Xiaotian. Efficient unsupervised image stitching using attention mechanism with deep homography estimation[J]. Computers,Materials & Continua,2024,79(1):1319-1334.
计量
- 文章访问数: 19
- HTML全文浏览量: 8
- PDF下载量: 2
