留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于云边协同的煤矿井下尺度自适应目标跟踪方法

牟琦 韩嘉嘉 张寒 李占利

牟琦,韩嘉嘉,张寒,等. 基于云边协同的煤矿井下尺度自适应目标跟踪方法[J]. 工矿自动化,2023,49(4):50-61.  doi: 10.13272/j.issn.1671-251x.2022100093
引用本文: 牟琦,韩嘉嘉,张寒,等. 基于云边协同的煤矿井下尺度自适应目标跟踪方法[J]. 工矿自动化,2023,49(4):50-61.  doi: 10.13272/j.issn.1671-251x.2022100093
MU Qi, HAN Jiajia, ZHANG Han, et al. A scale-adaptive target tracking method for coal mine underground based on cloud-edge collaboration[J]. Journal of Mine Automation,2023,49(4):50-61.  doi: 10.13272/j.issn.1671-251x.2022100093
Citation: MU Qi, HAN Jiajia, ZHANG Han, et al. A scale-adaptive target tracking method for coal mine underground based on cloud-edge collaboration[J]. Journal of Mine Automation,2023,49(4):50-61.  doi: 10.13272/j.issn.1671-251x.2022100093

基于云边协同的煤矿井下尺度自适应目标跟踪方法

doi: 10.13272/j.issn.1671-251x.2022100093
基金项目: 国家重点研发计划资助项目(2019YFB1405000)。
详细信息
    作者简介:

    牟琦(1974—),女,陕西西安人,副教授,研究方向为计算机视觉和图像处理、人工智能,E-mail:muqi@xust.edu.cn

    通讯作者:

    李占利(1964—),男,陕西西安人,教授,研究方向为图像处理、视觉计算与可视化,E-mail:lizl@xust.edu.cn

  • 中图分类号: TD76

A scale-adaptive target tracking method for coal mine underground based on cloud-edge collaboration

  • 摘要: 煤矿井下监控视频中的运动目标通常存在较大的尺度变化和形变,导致基于计算机视觉的目标跟踪算法准确率不高,且海量的视频数据导致基于云端的集中式数据处理方式难以满足目标跟踪的实时性要求。针对上述问题,提出了一种基于云边协同的煤矿井下尺度自适应目标跟踪方法。设计了基于深度估计的尺度自适应目标跟踪算法,通过构建深度−尺度估计模型,利用目标深度值估计尺度值,实现尺度自适应目标跟踪,解决了目标尺度变化和形变导致跟踪准确率不高的问题;设计了一种基于云边协同的智能监控系统架构,将尺度自适应目标跟踪算法细粒度划分后的子模块按所需计算资源分别部署在系统的边缘端和云端,通过边缘端和云端的分布式并行处理提高算法运行效率,解决了集中式数据处理方式实时性差的问题。将基于云边协同的煤矿井下尺度自适应目标跟踪方法应用于煤矿井下视频序列,对其跟踪性能和实时性能进行实验验证,结果表明:与核相关滤波(KCF)、判别型尺度空间跟踪(DSST)算法、基于多特征融合的尺度自适应(SAMF)算法3种经典目标跟踪算法相比,基于深度估计的尺度自适应目标跟踪算法在煤矿井下目标出现较大尺度变化和形变时,具有更高的跟踪精度和成功率;与传统的云计算处理方式相比,基于云边协同的尺度自适应目标跟踪算法部署方式使算法总时延降低了32.55%,有效提升了煤矿井下智能监控系统目标跟踪的实时性能。

     

  • 图  1  深度−尺度估计模型构建过程

    Figure  1.  Construction process of depth-scale estimation model

    图  2  基于深度估计的尺度自适应目标跟踪算法

    Figure  2.  Scale-adaptive target tracking algorithm based on depth estimation

    图  3  目标跟踪算法5个子模块的运行时间

    Figure  3.  Running time of five sub-modules of the target tracking algorithm

    图  4  基于云边协同的智能监控系统架构

    Figure  4.  Architecture of cloud-edge collaborative intelligent monitoring system

    图  5  云边协同任务卸载架构

    Figure  5.  Architecture of cloud-side collaborative task unloading

    图  6  煤矿井下视频序列部分帧的RGB图像和深度图像

    Figure  6.  Part of RGB images and depth images of a coal mine underground video sequence

    图  7  深度−尺度估计模型实验结果

    Figure  7.  Experimental results of depth-scale estimation model

    图  8  4种算法对煤矿井下数据集的目标跟踪结果

    Figure  8.  Target tracking results of four algorithms for coal mine underground data set

    图  9  4种算法对OTB−100数据集的目标跟踪结果

    Figure  9.  Target tracking results of four algorithms for OTB-100 standard data set

    图  10  目标跟踪算法在煤矿井下数据集上的精度和成功率

    Figure  10.  Tracking accuracy and success rate of four target tracking algorithms for coal mine underground data set

    图  11  目标跟踪算法在OTB−100数据集上的精度和成功率

    Figure  11.  Tracking accuracy and success rate of four target tracking algorithms for OTB-100 data set

    图  12  目标跟踪算法在2种部署方式下的局部时延曲线

    Figure  12.  Local delay curves of the proposed target tracking algorithm under two deployment modes

    图  13  目标跟踪算法在2种部署方式下的总时延曲线

    Figure  13.  Overall delay curves of the proposed target tracking algorithm under two deployment mode

    图  14  不同边缘服务器数量下目标跟踪算法总时延曲线

    Figure  14.  Overall delay curves of the proposed target tracking algorithm under different groups of edge servers

    表  1  煤矿井下尺度自适应目标跟踪实验环境配置

    Table  1.   Experimental environment configuration of coal mine underground scale-adaptive target tracking method

    实验环境配置
    边缘服务器CPUAMD Ryzen 5 5600H with Radeon Graphics 3.30 GHz
    GPUGeForce RTX 3050
    系统Windows10
    云端服务器CPUIntel Xeon E5−2680 v2
    GPUNVIDIA GeForce GTX TITAN Z
    系统centos7
    编程语言Matlab
    下载: 导出CSV

    表  2  任务卸载相关参数配置

    Table  2.   Parameters configuration related to task unloading

    设备参数
    终端设备$ {B}_{i,j} $/MHz100
    $ {P}_{i,j} $/W0.1
    ${D}_{i,j} $[0.1,1.0]
    边缘服务器$ {B}_{j,{\rm{c}}} $/MHz200
    $ {P}_{j,{\rm{c}}} $/W0.5
    $ {y}_{j} $/GHz10
    $ {C}_{j} $/($ \mathrm{c}\mathrm{y}\mathrm{c}\mathrm{l}\mathrm{e}\mathrm{s}\cdot {\mathrm{b}\mathrm{i}\mathrm{t}}^{-1} $)200
    $ D_{j,{\rm{c}}} $[0.1,1.0]
    云端服务器$ {w}_{{\rm{c}}} $/GHz100
    $ {C}_{{\rm{c}}} $/($ \mathrm{c}\mathrm{y}\mathrm{c}\mathrm{l}\mathrm{e}\mathrm{s}\cdot {\mathrm{b}\mathrm{i}\mathrm{t}}^{-1} $)50
    $ {\sigma }^{2} $/dBm100
    其他$ {b}_{{\rm{v}}} $/($ \mathrm{M}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{t}\cdot {\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{k}}^{-1} $)[1,100]
    $ {b}_{{\rm{f}}} $/($ \mathrm{M}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{t}\cdot {\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{k}}^{-1} $)[1,5]
    $ {b}_{{\rm{m}}} $/($ \mathrm{M}\mathrm{i}\mathrm{b}\mathrm{i}\mathrm{t}\cdot {\mathrm{t}\mathrm{a}\mathrm{s}\mathrm{k}}^{-1} $)[1,5]
    下载: 导出CSV

    表  3  煤矿井下视频序列数据信息

    Table  3.   Video sequence data information of coal mine underground

    视频序列帧数主要影响因素
    Video1355尺度变化
    Video2170目标形变、背景杂乱
    Video3181光照不均、尺度变化
    Video4297光照极度不足、目标形变
    Video5215目标形变
    Video6286尺度变化
    下载: 导出CSV

    表  4  目标跟踪算法在2种部署方式下的传输时延和计算时延计算结果

    Table  4.   Calculated transmission delay and calculation delay of the proposed target tracking algorithm under two deployment modes ms

    视频序列云边协同方式云计算方式
    传输时延计算时延传输时延计算时延
    Video122.4910.012 78209.559 60.004 61
    Video223.3610.013 41190.456 90.004 73
    Video321.4360.012 13205.454 90.004 47
    Video422.2670.011 98193.576 80.004 34
    Video521.8790.015 76200.177 40.004 91
    Video621.2530.013 05193.115 10.004 39
    Total132.6870.079 11 1 192.340 70.027 45
     注:黑体数据为最优结果。
    下载: 导出CSV

    表  5  目标跟踪算法在2种部署方式下的总时延计算结果

    Table  5.   Calculated overall delay of the proposed target tracking algorithm under two deployment modes ms

    视频序列总时延
    云计算方式云边协同方式
    Video1209.567 6139.916 7
    Video2190.465 3140.214 6
    Video3205.462 5130.169 8
    Video4193.584 6134.247 2
    Video5200.186 1130.744 2
    Video6193.123 8128.929 4
    Total1 192.389 9804.221 9
     注:黑体数据为最优结果。
    下载: 导出CSV

    表  6  不同边缘服务器数量下目标跟踪算法时延

    Table  6.   Delay of the proposed target tracking algorithm under different groups of edge servers

    边缘服务器
    数量/台
    视频序列
    数量/个
    传统云计算
    总时延/ms
    M=12时云边协同时延/msM=18时云边协同时延/ms
    边缘端云端总时延边缘端云端总时延
    传输时延计算时延传输时延计算时延传输时延计算时延传输时延计算时延
    161192.3899132.6870.0791671.4290.0263804.2219132.6870.0791671.4290.0263804.2219
    2122384.7798132.6870.0791671.4290.0526804.2481132.6870.0791671.4290.0526804.2481
    3183577.1697265.3740.07911342.8590.07891608.391132.6870.0791671.4290.0789804.2744
    4244769.5596265.3710.07911342.8590.10521608.417265.3740.07911342.8590.10521608.417
    5305961.9495398.0610.07912014.2880.13152412.560265.3740.07911342.8590.13151608.443
    6367154.3394398.0610.07912014.2880.15782412.586265.3740.07911342.8590.15781608.470
     注:黑体数据为最优结果。
    下载: 导出CSV
  • [1] 王国法,赵国瑞,胡亚辉. 5G技术在煤矿智能化中的应用展望[J]. 煤炭学报,2020,45(1):16-23.

    WANG Guofa,ZHAO Guorui,HU Yahui. Application prospect of 5G technology in coal mine intelligence[J]. Journal of China Coal Society,2020,45(1):16-23.
    [2] 丁恩杰,俞啸,夏冰,等. 矿山信息化发展及以数字孪生为核心的智慧矿山关键技术[J]. 煤炭学报,2022,47(1):564-578.

    DING Enjie,YU Xiao,XIA Bing,et al. Development of mine informatization and key technologies of intelligent mines[J]. Journal of China Coal Society,2022,47(1):564-578.
    [3] 陈龙,王晓,杨健健,等. 平行矿山:从数字孪生到矿山智能[J]. 自动化学报,2021,47(7):1633-1645.

    CHEN Long,WANG Xiao,YANG Jianjian,et al. Parallel mining operating systems:from digital twins to mining intelligence[J]. Acta Automatica Sinica,2021,47(7):1633-1645.
    [4] 孙彦景,霍羽,陈岩,等. 矿山动态协同作业场景无线通信关键技术[J]. 煤炭学报,2021,46(1):321-332.

    SUN Yanjing,HUO Yu,CHEN Yan,et al. Key technology of service attribute driven wireless communication for dynamic cooperative operations in mining scenarios[J]. Journal of China Coal Society,2021,46(1):321-332.
    [5] 王学文,刘曙光,王雪松,等. AR/VR融合驱动的综采工作面智能监控关键技术研究与试验[J]. 煤炭学报,2022,47(2):969-985.

    WANG Xuewen,LIU Shuguang,WANG Xuesong,et al. Research and test on key technologies of intelligent monitoring and control driven by AR/VR for fully mechanized coal-mining face[J]. Journal of China Coal Society,2022,47(2):969-985.
    [6] LIU S,LIU D,SRIVASTAVA G,et al. Overview and methods of correlation filter algorithms in object tracking[J]. Complex & Intelligent Systems,2021,7(4):1895-1917.
    [7] 张旭辉,杨文娟,薛旭升,等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报,2022,47(1):579-597.

    ZHANG Xuhui,YANG Wenjuan,XUE Xusheng,et al. Challenges and developing of the intelligent remote controlon roadheaders in coal mine[J]. Journal of China Coal Society,2022,47(1):579-597.
    [8] 张帆,孙晓辉,崔东林. 基于ORB特征的矿井移动目标双目视觉跟踪与定位[J]. 煤炭学报,2018,43(增刊2):654-662.

    ZHANG Fan,SUN Xiaohui,CUI Donglin. Method of tracking and positioning for mobile target based on ORB features and binocular vision in mine[J]. Journal of China Coal Society,2018,43(S2):654-662.
    [9] 刘浩,刘海滨,孙宇,等. 煤矿井下员工不安全行为智能识别系统[J]. 煤炭学报,2021,46(增刊2):1159-1169.

    LIU Hao,LIU Haibin,SUN Yu,et al. Intelligent recognition system of unsafe behavior of underground coal miners[J]. Journal of China Coal Society,2021,46(S2):1159-1169.
    [10] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, 2010: 2544-2550.
    [11] HENRIQUES J F, CASEIRO R, MARTINES P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]. European Conference on Computer Vision, Berlin, 2012: 702-715.
    [12] HENRIQUES J F,CASEIRO R,MARTINS P,et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,37(3):583-596.
    [13] 覃剑,石昌伟,张媛,等. 边缘视频处理的细粒度划分与重组部署算法[J]. 电子学报,2021,49(11):2152-2159.

    QIN Jian,SHI Changwei,ZHANG Yuan,et al. Fine-grained partitioning and reorganization deployment strategy of edge video processing[J]. Acta Electronica Sinica,2021,49(11):2152-2159.
    [14] 卢新明,阚淑婷. 煤炭精准开采地质保障与透明地质云计算技术[J]. 煤炭学报,2019,44(8):2296-2305.

    LU Xinming,KAN Shuting. Geological guarantee and transparent geological cloud computing technology of precision coal mining[J]. Journal of China Coal Society,2019,44(8):2296-2305.
    [15] 施巍松,张星洲,王一帆,等. 边缘计算:现状与展望[J]. 计算机研究与发展,2019,56(1):69-89. doi: 10.7544/issn1000-1239.2019.20180760

    SHI Weisong,ZHANG Xingzhou,WANG Yifan,et al. Edge computing:state-of-the-art and future directions[J]. Journal of Computer Research and Development,2019,56(1):69-89. doi: 10.7544/issn1000-1239.2019.20180760
    [16] 王国法,杜毅博,任怀伟,等. 智能化煤矿顶层设计研究与实践[J]. 煤炭学报,2020,45(6):1909-1924.

    WANG Guofa,DU Yibo,REN Huaiwei,et al. Top level design and practice of smart coal mines[J]. Journal of China Coal Society,2020,45(6):1909-1924.
    [17] 陈思光,陈佳民,赵传信. 基于深度强化学习的云边协同计算迁移研究[J]. 电子学报,2021,49(1):157-166.

    CHEN Siguang,CHEN Jiamin,ZHAO Chuanxin. Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism[J]. Acta Electronica Sinica,2021,49(1):157-166.
    [18] DANELLJAN M, HAGER G, KHAN F, et al. Accurate scale estimation for robust visual tracking[C]. British Machine Vision Conference, Nottingham, 2014: 1-11.
    [19] LI Yang, ZHU Jianke. A scale adaptive kernel correlation filter tracker with feature integration[C]. European Conference on Computer Vision Workshops, Zurich, 2014: 254-265.
    [20] 孙继平,贾倪. 矿井视频图像中人员目标匹配与跟踪方法[J]. 中国矿业大学学报,2015,44(3):540-548.

    SUN Jiping,JIA Ni. Human target matching and tracking method in coal mine video[J]. Journal of China University of Mining & Technology,2015,44(3):540-548.
    [21] 程健,陈亮,王凯,等. 一种多特征融合的复杂场景动态目标跟踪算法[J]. 中国矿业大学学报,2021,50(5):1002-1010.

    CHENG Jian,CHEN Liang,WANG Kai,et al. Multi-feature fusion dynamic target tracking algorithm for complex scenes[J]. Journal of China University of Mining & Technology,2021,50(5):1002-1010.
    [22] 牟琦,张寒,何志强,等. 基于深度估计和特征融合的尺度自适应目标跟踪算法[J]. 图学学报,2021,42(4):563-571.

    MU Qi,ZHANG Han,HE Zhiqiang,et al. Scale adaptive target tracking algorithm based on depth estimation and feature fusion[J]. Journal of Graphics,2021,42(4):563-571.
    [23] GOARD C, AODHA O M, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]. Computer Vision and Pattern Recognition, Hawaii, 2017: 6602-6611.
    [24] HABER E E,NGUYEN T M,ASSI C. Joint optimization of computational cost and devices energy for task offloading in multi-tier edge-clouds[J]. IEEE Transactions on Communications,2019,67(5):3407-3421. doi: 10.1109/TCOMM.2019.2895040
    [25] CHEN Siguang,ZHENG Yimin,LU Weifeng,et al. Energy optimal dynamic computation offloading for industrial IoT in fog computing[J]. IEEE Transactions on Green Communications and Networking,2019,4(2):566-576.
    [26] 张展,张宪琦,左德承,等. 面向边缘计算的目标追踪应用部署策略研究[J]. 软件学报,2020,31(9):2691-2708.

    ZHANG Zhan,ZHANG Xianqi,ZUO Decheng,et al. Research on target tracking application deployment strategy for edge computing[J]. Journal of Software,2020,31(9):2691-2708.
    [27] 姜德义,魏立科,王翀,等. 智慧矿山边缘云协同计算技术架构与基础保障关键技术探讨[J]. 煤炭学报,2020,45(1):484-492.

    JIANG Deyi,WEI Like,WANG Chong,et al. Discussion on the technology architecture and key basic support technology for intelligent mine edge-cloud collaborative computing[J]. Journal of China Coal Society,2020,45(1):484-492.
    [28] 张文柱,余静华. 移动边缘计算中基于云边端协同的任务卸载策略[J]. 计算机研究与发展,2023,60(2):371-385.

    ZHANG Wenzhu,YU Jinghua. Task offloading strategy in mobile edge computing based on cloud-edge-end cooperation[J]. Journal of Computer Research and Development,2023,60(2):371-385.
  • 加载中
图(14) / 表(6)
计量
  • 文章访问数:  163
  • HTML全文浏览量:  50
  • PDF下载量:  22
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-10-31
  • 修回日期:  2023-04-08
  • 网络出版日期:  2023-04-27

目录

    /

    返回文章
    返回