矿用AI视频边缘计算技术研究与应用

张立亚; 郝博南; 马征; 杨志方

doi:10.13272/j.issn.1671-251x.18215

矿用AI视频边缘计算技术研究与应用

张立亚^{1, 2, 3, 4,},
郝博南^{1, 2, 3, 4},
马征^{1, 2, 3, 4},
杨志方^{1, 2, 3, 4}

1.
煤炭科学技术研究院有限公司，北京　100013
2.
煤炭智能开采与岩层控制全国重点实验室，北京　100013
3.
煤矿应急避险技术装备工程研究中心，北京　100013
4.
北京市煤矿安全工程技术研究中心，北京　100013

基金项目: 工业和信息化部2024年工业互联网创新发展工程−工业5G轻量化融合终端项目（TC240AAKM-132）；天地科技股份有限公司科技创新创业资金专项项目（2023-TD-ZD005-005，2024-TD-ZD015-01）。

详细信息

作者简介:
张立亚（1985—），男，河北定州人，研究员，博士，主要从事矿井通信与智能矿山技术研究工作，E-mail：zhangliya@ccrise.cn

中图分类号: TD67
计量
- 文章访问数: 73
- HTML全文浏览量: 12
- PDF下载量: 23
出版历程
- 收稿日期: 2024-10-24
- 修回日期: 2024-12-25
- 网络出版日期: 2025-01-07
- 刊出日期: 2024-12-24

Research and application of mining AI video edge computing technology

ZHANG Liya^{1, 2, 3, 4,},
HAO Bonan^{1, 2, 3, 4},
MA Zheng^{1, 2, 3, 4},
YANG Zhifang^{1, 2, 3, 4}

1.
CCTEG China Coal Research Institute, Beijing 100013, China
2.
National Key Laboratory of Intelligent Mining and Strata Control, Beijing 100013, China
3.
Coal Mine Emergency Evasion Technology and Equipment Engineering Research Center, Beijing 100013, China
4.
Beijing Municipal Engineering Research Center for Coal Mine Safety Technology, Beijing 100013, China

摘要

摘要:
目前矿山AI视频系统主要采用地面服务器进行分析处理，存在视频分析整体响应时延较高、多系统联动延迟、网络带宽资源占用大等问题。针对该问题，设计了一种轻量化、可边缘部署的矿用AI视频边缘计算系统。提出了基于注册机的轻量化软件开发工具包（SDK）框架，对算子进行解耦，以提升算法并行运算能力，降低SDK对存储空间的需求；对YOLOv7的卷积运算进行分组设计，利用恒等映射对Focus主干网络进行优化，以减少运算量并轻量化网络结构，同时引入Transformer中的注意力机制提高检测性能。集成国产化智能芯片和5G通信模块，研制了矿用AI视频服务器，实现了矿山边缘节点部署与计算。实验结果表明：矿用AI视频边缘计算系统响应出色，部署注册机SDK与改进YOLOv7模型后，平均推理时延为28 ms，较React Native+YOLOv7和MobileNet分别减少52%和44%；在各种负载情况下，矿用AI视频服务器的响应时间远低于矿山操作的最低要求。现场工业性测试结果表明：矿用AI视频服务器接入8路摄像仪时，响应时延为51 ms，带宽维持在45 Mbit/s，比使用地面服务器时的时延降低了59%，带宽提高了15%,实现了对井下视频数据的实时、就地分析处理，有效降低了数据传输时延，提高了视频分析的响应速率和处理效率。
- 智能矿山 /
- AI视频分析 /
- 边缘计算 /
- 注册机 /
- YOLOv7 /
- 轻量化设计
Abstract:
Currently, mining AI video systems mainly rely on ground servers for analysis and processing, which leads to issues such as high overall response latency, multi-system linkage delays, and high network bandwidth utilization. To address these issues, a lightweight and edge-deployable mining AI video edge computing system was designed. A lightweight software development kit (SDK) framework based on the register machine was proposed to decouple operators, improving the parallel computing ability of the algorithms and reducing the storage requirements of the SDK. Grouping design of YOLOv7 convolution operation was conducted, and the Focus backbone network was optimized using identity mapping to reduce computation amount and streamline the network structure. Additionally, the attention mechanism from Transformer was incorporated to improve detection performance. The mining AI video server was developed by integrating the domestic intelligent chips and 5G communication modules, enabling the deployment and calculation of mine edge nodes. The experimental results showed that the mining AI video edge computing system had excellent response. After deploying the registration machine SDK and optimizing the YOLOv7 model, the average inference latency was 28 ms, 52% and 44% lower than that of React Native+YOLOv7 and MobileNet, respectively. Under various load conditions, the response latency of the mining AI video server was significantly lower than the minimum requirements for mine operation. The field industrial test results showed that when the mining AI video server was connected to an 8-channel camera, the response latency was 51 ms, and the bandwidth was maintained at 45 Mbit/s. Compared to using a ground server, the response latency was reduced by 59%, and the bandwidth increased by 15%. This enabled real-time and on-site analysis and processing of underground video data, effectively reducing the data transmission delay and improving the response rate and processing efficiency of video analysis.
- intelligent mine /
- AI video analysis /
- edge computing /
- register machine /
- YOLOv7 /
- lightweight design

HTML全文

图 1 矿用AI视频边缘计算系统架构

Figure 1. Architecture of mining AI video edge computing system

下载: 全尺寸图片幻灯片

图 2 基于注册机的SDK框架

Figure 2. Software development kit（SDK） framework based on register machine

下载: 全尺寸图片幻灯片

图 3 标准卷积和分组卷积结构

Figure 3. Standard convolution and group convolution structure

下载: 全尺寸图片幻灯片

图 4 改进Focus网络结构

Figure 4. Improved Focus network structure

下载: 全尺寸图片幻灯片

图 5 特征融合阶段网络结构

Figure 5. Feature fusion stage network structure

下载: 全尺寸图片幻灯片

图 6 矿用AI视频服务器核心模块

Figure 6. Core modules of mining AI video server

下载: 全尺寸图片幻灯片

图 7 5G通信模块网络框架

Figure 7. 5G communication module network framwork

下载: 全尺寸图片幻灯片

图 8 AI模组架构

Figure 8. AI module architecture

下载: 全尺寸图片幻灯片

图 9 12 V过流保护电路

Figure 9. 12 V overcurrent protection circuit

下载: 全尺寸图片幻灯片

图 10 5 V电压过流保护电路

Figure 10. 5 V overcurrent protection circuit

下载: 全尺寸图片幻灯片

图 11 服务器上下行平均带宽对比

Figure 11. Comparison of server's average upstream and downstream bandwidth

下载: 全尺寸图片幻灯片

表 1 不同框架与模型组合推理性能

Table 1 Inference performance of different framework and model combinations

框架及模型	时延/ms	准确率	召回率
React Native+YOLOv7	58	0.903	0.899
React Native+MobileNet	50	0.912	0.901
React Native+本文模型	39	0.926	0.919
注册机SDK+YOLOv7	42	0.903	0.899
注册机SDK+MobileNet	36	0.912	0.901
注册机SDK+本文模型	28	0.926	0.919

下载: 导出CSV

表 2 不同网络负载的响应时间

Table 2 Response latency under different network loads

网络负载/%	平均响应时间/ms
网络负载/%	矿用AI视频服务器	地面管理服务器
0	10	60
20	14	68
40	20	72
60	27	77
80	36	98
100	50	121

下载: 导出CSV

参考文献(21)

[1]	王国法,庞义辉. 智能化示范煤矿建设成效与发展方向[J]. 智能矿山,2024,5(1):2-11. WANG Guofa,PANG Yihui. Construction effect and development direction of intelligent demonstration coal mine[J]. Journal of Intelligent Mine,2024,5(1):2-11.
[2]	程德强,寇旗旗,江鹤,等. 全矿井智能视频分析关键技术综述[J]. 工矿自动化,2023,49(11):1-21. CHENG Deqiang,KOU Qiqi,JIANG He,et al. Overview of key technologies for mine-wide intelligent video analysis[J]. Journal of Mine Automation,2023,49(11):1-21.
[3]	罗明华,朱兴林,黄春. 煤矿AI视频监控系统[J]. 智能矿山,2023,4(4):62-65. LUO Minghua,ZHU Xinglin,HUANG Chun. AI video monitoring system in coal mine[J]. Journal of Intelligent Mine,2023,4(4):62-65.
[4]	程健,李昊,马昆,等. 矿井视觉计算体系架构与关键技术[J]. 煤炭科学技术,2023,51(9):202-218. CHENG Jian,LI Hao,MA Kun,et al. Architecture and key technologies of coalmine underground vision computing[J]. Coal Science and Technology,2023,51(9):202-218.
[5]	KHAN S M,NABI A U,BHANBHRO T H. Comparative analysis between flutter and react native[J]. International Journal of Artificial Intelligence & Mathematical Sciences,2022,1(1):15-28.
[6]	MARUTHAPPA M,LEE Yunli,WONG S F. Deep learning placenta abnormalities with customized architecture using deeplearning4j[J]. Indian Journal of Computer Science and Engineering,2022,13(4):1346-1355. DOI: 10.21817/indjcse/2022/v13i4/221304180
[7]	王琳毅,白静,李文静,等. YOLO系列目标检测算法研究进展[J]. 计算机工程与应用,2023,59(14):15-29. DOI: 10.3778/j.issn.1002-8331.2301-0081 WANG Linyi,BAI Jing,LI Wenjing,et al. Research progress of YOLO series target detection algorithms[J]. Computer Engineering and Applications,2023,59(14):15-29. DOI: 10.3778/j.issn.1002-8331.2301-0081
[8]	邵延华,张铎,楚红雨,等. 基于深度学习的YOLO目标检测综述[J]. 电子与信息学报,2022,44(10):3697-3708. DOI: 10.11999/JEIT210790 SHAO Yanhua,ZHANG Duo,CHU Hongyu,et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology,2022,44(10):3697-3708. DOI: 10.11999/JEIT210790
[9]	张立亚. 基于图像识别的煤矿井下安全管控技术[J]. 煤矿安全,2021,52(2):165-168. ZHANG Liya. Safety control technology of coal mine based on image recognition[J]. Safety in Coal Mines,2021,52(2):165-168.
[10]	张立亚. 基于5G通信的矿山可视化智能监控技术[J]. 煤炭技术,2022,41(1):191-194. ZHANG Liya. Mine visual intelligent monitoring technology based on 5G communication[J]. Coal Technology,2022,41(1):191-194.
[11]	王国法,庞义辉,李爽,等. 基于煤矿时空多源信息感知的智能安控闭环体系[J]. 矿业安全与环保,2022,49(4):1-11. WANG Guofa,PANG Yihui,LI Shuang,et al. Intelligent safety closed-loop management and control system based on multi-source information perception in coal mine[J]. Mining Safety & Environmental Protection,2022,49(4):1-11.
[12]	刘彦清. 基于YOLO系列的目标检测改进算法[D]. 长春:吉林大学,2021. LIU Yanqing. Improved object detection algorithm based on YOLO series[D]. Changchun:Jilin University,2021.
[13]	ZHANG Liya,YANG Wei,HAO Bonan,et al. Edge computing resource allocation method for mining 5G communication system[J]. IEEE Access,2023,11:49730-49737. DOI: 10.1109/ACCESS.2023.3244242
[14]	ZHANG Xue,SHENG Zehua,SHEN Huiliang. FocusNet:classifying better by focusing on confusing classes[J]. Pattern Recognition,2022,129. DOI: 10.1016/j.patcog.2022.108709.
[15]	LI Rui,ZHANG Baopeng,LIU Wei,et al. PANet:an end-to-end network based on relative motion for online multi-object tracking[J]. ACM Transactions on Multimedia Computing,Communications,and Applications,2023,19(6):1-21.
[16]	李建,杜建强,朱彦陈,等. 基于Transformer的目标检测算法综述[J]. 计算机工程与应用,2023,59(10):48-64. DOI: 10.3778/j.issn.1002-8331.2211-0133 LI Jian,DU Jianqiang,ZHU Yanchen,et al. Survey of Transformer-based object detection algorithms[J]. Computer Engineering and Applications,2023,59(10):48-64. DOI: 10.3778/j.issn.1002-8331.2211-0133
[17]	付苗苗,邓淼磊,张德贤. 基于深度学习和Transformer的目标检测算法[J]. 计算机工程与应用,2023,59(1):37-48. FU Miaomiao,DENG Miaolei,ZHANG Dexian. Object detection algorithms based on deep learning and Transformer[J]. Computer Engineering and Applications,2023,59(1):37-48.
[18]	蔡腾,陈慈发,董方敏. 结合Transformer和动态特征融合的低照度目标检测[J]. 计算机工程与应用,2024,60(9):135-141. DOI: 10.3778/j.issn.1002-8331.2310-0131 CAI Teng,CHEN Cifa,DONG Fangmin. Low-light object detection combining Transformer and dynamic feature fusion[J]. Computer Engineering and Applications,2024,60(9):135-141. DOI: 10.3778/j.issn.1002-8331.2310-0131
[19]	TERVEN J,CORDOVA-ESPARZA D. A comprehensive review of YOLO architectures in computer vision:from YOLOv1 to YOLOv8 and YOLO-NAS[EB/OL]. [2024-10-15]. https://arxiv.org/ abs/2304.00501v7.
[20]	孙伟峰,李小彤,纪永刚,等. 基于检测−跟踪联动的紧凑型高频地波雷达弱目标自适应检测方法[J]. 电子与信息学报,2023,45(8):2955-2964. SUN Weifeng,LI Xiaotong,JI Yonggang,et al. An adaptive weak target detection method using joint detection and tracking for compact high frequency surface ware radar[J]. Journal of Electronics & Information Technology,2023,45(8):2955-2964.
[21]	RUKHOVICH D,SOFIIUK K,GALEEV D,et al. IterDet:iterative scheme for object detection in crowded environments[EB/OL]. [2024-10-15]. https://arxiv.org/abs/2005.05708.

施引文献(6)

期刊类型引用(5)

1.	张晓垒，权军军，杨港，段守德，郭明功，朱坤飞，邱黎明. 平煤八矿煤柱应力演化规律研究. 中国煤炭地质. 2025(03): 56-63 . 百度学术
2.	刘永冠. 四老沟矿坚硬顶板初采定向爆破弱化技术研究与应用. 陕西煤炭. 2024(12): 38-42+47 . 百度学术
3.	徐维正，许斌，唐鹏，史明伟，董文卓，张沿. 空间煤柱影响下冲击地压防治技术研究. 煤炭工程. 2023(10): 85-91 . 百度学术
4.	苗伟杰. 木瓜矿近距离采空区下无煤柱沿空掘巷围岩稳定控制研究. 煤矿现代化. 2021(04): 39-41+38 . 百度学术
5.	赵泽潭. 特厚煤层综放工作面小煤柱留设技术的应用. 煤炭与化工. 2020(12): 34-35+92 . 百度学术