基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护方法

张帆; 邵光耀; 李昱翰; 李玉雪

doi:10.13272/j.issn.1671-251x.2023090004

基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护方法

张帆^{1, 2, 3,},
邵光耀¹,
李昱翰¹,
李玉雪¹

1.
中国矿业大学（北京）人工智能学院，北京　100083
2.
煤矿智能化与机器人创新应用应急管理部重点实验室，北京　100083
3.
中国矿业大学（北京）智慧矿山与机器人研究院，北京　100083

基金项目: 国家重点研发计划项目（2022Z03010004）；国家自然科学基金面上项目（52374165）；国家自然科学基金创新研究群体项目（52121003）。

详细信息

作者简介:
张帆（1972—），男，甘肃会宁人，教授，博士研究生导师，博士，研究方向为矿山数字孪生、智能监控与通信，E-mail：zf@cumtb.edu.cn

中图分类号: TD67/355
计量
- 文章访问数: 523
- HTML全文浏览量: 42
- PDF下载量: 80
出版历程
- 收稿日期: 2023-08-31
- 修回日期: 2024-06-26
- 网络出版日期: 2024-07-03
- 刊出日期: 2024-06-29

Adaptive impact resistance support method for advanced hydraulic supports in mines based on digital twins and deep reinforcement learning

ZHANG Fan^{1, 2, 3,},
SHAO Guangyao¹,
LI Yuhan¹,
LI Yuxue¹

1.
School of Artificial Intelligence, China University of Mining and Technology-Beijing, Beijing 100083, China
2.
Key Laboratory of Intelligent Mining and Robotics of Emergency Management Department, Beijing 100083, China
3.
Institute of Intelligent Mining and Robotics, China University of Mining and Technology-Beijing, Beijing 100083, China

摘要

摘要: 受深部开采冲击地压等地质灾害扰动的影响，存在矿井超前支护系统自感知能力差、智能抗冲自适应能力弱、缺乏决策控制能力等问题。针对上述问题，提出了一种基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护方法。通过多源传感器感知巷道环境和超前液压支架支护状态，在虚拟世界中创建物理实体的数字孪生模型，其中物理模型精确展现超前液压支架的结构特征和细节，控制模型实现超前液压支架的自适应控制，机理模型实现对超前液压支架自适应支护的逻辑描述和机理解释，数据模型存储超前液压支架实体运行数据和孪生数据，仿真模型完成超前液压支架立柱仿真以实现超前液压支架与数字孪生模型虚实交互。根据基于深度Q网络（DQN）的超前液压支架自适应抗冲决策算法，对仿真环境中巷道抗冲支护进行智能决策，并依据决策结果对物理实体和数字孪生模型下达调控指令，实现超前液压支架智能控制。实验结果表明：立柱位移与压力变化一致，说明超前液压支架立柱仿真模型设计合理，从而验证了数字孪生模型的准确性；基于DQN的矿井超前液压支架自适应抗冲决策算法可通过调节液压支架控制器PID参数，自适应调控立柱压力，提升巷道安全等级，实现超前液压支架自适应抗冲支护。
- 矿井智能抗冲 /
- 超前液压支架 /
- 自适应支护 /
- 数字孪生 /
- 深度强化学习 /
- 深度Q网络 /
- DQN
Abstract: Due to the disturbance of geological disasters such as deep mining and rock burst, there are problems such as poor self perception capability, weak intelligent anti impact adaptive capability, and lack of decision-making and control capability in the advanced support system of the mine. In order to solve the above problems, a adaptive impact resistance support method for advanced hydraulic supports in mines based on digital twins and deep reinforcement learning is proposed. By sensing the roadway environment and advanced hydraulic support status through multiple sensors, a digital twin model of a physical entity is created in a virtual world. The physical model accurately displays the structural features and details of the advanced hydraulic support. The control model realizes adaptive control of the advanced hydraulic support. The mechanism model realizes logical description and mechanism explanation of the adaptive support of the advanced hydraulic support. The data model stores the physical operation data and twin data of the advanced hydraulic support. The simulation model completes the simulation of the advanced hydraulic support column to achieve virtual real interaction between the advanced hydraulic support and the digital twin model. According to the adaptive impact resistance decision-making algorithm based on deep Q-network (DQN) for advanced hydraulic support, intelligent decision-making is made for roadway impact resistance support in the simulation environment. Based on the decision results, control instructions are issued to physical entities and digital twin models to achieve intelligent control of advanced hydraulic support. The experimental results show that the displacement and pressure changes of the column are consistent, indicating that the simulation model design of the advanced hydraulic support column is reasonable, thereby verifying the accuracy of the digital twin model. The adaptive impact resistance decision-making algorithm for advanced hydraulic supports in mines based on DQN can adjust the PID parameters of the hydraulic support controller, adaptively regulate the column pressure, improve the safety level of roadways, and achieve adaptive impact resistance support for advanced hydraulic supports.
- mine intelligent impact resistance /
- advanced hydraulic support /
- adaptive support /
- digital twin /
- deep reinforcement learning /
- deep Q-network /
- DQN

HTML全文

图 1 基于数字孪生和深度强化学习的矿井超前液压支架自适应抗冲支护技术架构

Figure 1. Adaptive impact resistance support technology architecture for mine advanced hydraulic support based on digital twin and deep reinforcement learning

下载: 全尺寸图片幻灯片

图 2 GWO−BP控制原理

Figure 2. Grey wolf optimization(GWO)-BP control principle

下载: 全尺寸图片幻灯片

图 3 机理模型结构

Figure 3. Mechanism model structure

下载: 全尺寸图片幻灯片

图 4 超前液压支架自适应抗冲决策流程

Figure 4. Adaptive impact resistance decision-making flow of advanced hydraulic support

下载: 全尺寸图片幻灯片

图 5 基于DQN的超前液压支架自适应抗冲决策算法原理

Figure 5. Principle of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)

下载: 全尺寸图片幻灯片

图 6 立柱位移与压力变化曲线

Figure 6. Change curves of column displacement and pressure

下载: 全尺寸图片幻灯片

图 7 基于DQN的超前液压支架自适应抗冲决策算法奖励收敛曲线

Figure 7. Reward convergence curve of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)

下载: 全尺寸图片幻灯片

表 1 深度强化学习算法对比

Table 1 Comparison of deep reinforcement learning algorithms

算法	优点	缺点
DQN	通过经验池进行随机小批量的采样训练，打破数据之间的相关性；采用每隔N次迭代更新目标Q网络的方法，提高Q网络稳定性	对目标Q网络采取最大化操作，对Q值的估计过高；训练数据规模较小
DDPG	同时建立Q值函数和策略函数；适用于连续动作的任务，稳定性好	对参数敏感；处理规模较大、时间较长，需要很强的GPU
PPO	对智能体的策略函数进行优化，减少数据收集难度；适用于复杂场景和大规模训练	序列数据收集困难；对超参数选择敏感

下载: 导出CSV

表 2 超前液压支架立柱仿真模型参数

Table 2 Simulation model parameters of advanced hydraulic support column

参数	值	参数	值
阀芯质量/kg	0.2	活塞杆直径/mm	10
活塞杆质量/kg	15	间隙直径/mm	0.02
反馈信号比例系数	5	比例阀阀杆直径/mm	10
阀芯控制比例系数	2	比例阀阀芯直径/mm	15
液压油密度/（kg·m⁻³）	920	顶板刚度系数/（N·mm⁻¹）	15000
液压油压力/MPa	41.50	顶板阻尼系数/（N·s·m⁻¹ ）	20
活塞直径/mm	30

下载: 导出CSV

表 3 巷道应力参数及安全系数

Table 3 Stress parameters and safety factors of roadway

巷道区间/m	围岩应力/MPa	支护应力/MPa	临界应力/MPa	应力安全系数	能量安全系数	安全等级
0～33	17.50	0.28	33.67	1.9	1.09	A
33～53	26.25	0.28	33.67	1.3	1.09	C
53～65	17.50	0.28	33.67	1.9	1.09	A
65～238	24.50	0.28	33.67	1.4	1.09	C
238～338	29.75	0.28	33.67	0.9	1.09	C
338～613	24.50	0.28	33.67	1.4	1.09	C
613～713	29.75	0.28	33.67	0.9	1.09	C
713～1186	24.50	0.28	33.67	1.4	1.09	C
1186～1732	17.50	0.28	33.67	1.9	1.09	A
1732～1763	21.00	0.28	33.67	1.6	1.09	A
1763～2660	17.50	0.28	33.67	1.9	1.09	A
2660～2680	21.00	0.28	33.67	1.6	1.09	A

下载: 导出CSV

表 4 基于DQN的超前液压支架自适应抗冲决策算法训练参数

Table 4 Training parameters of adaptive impact resistance decision-making algorithm of advanced hydraulic support based on deep Q-network(DQN)

参数	值	参数	值
经验池大小	2 000	最终概率	0.1
批量大小	64	目标值Q网络更新频率/步	50
折扣因子	0.95	训练轮次	480
初始概率	0.9	最大步数	100
概率的衰减系数	0.99

下载: 导出CSV

表 5 智能决策调控前后巷道状态对比

Table 5 Comparison of roadway state before and after intelligent decision control

巷道区间/m	初始巷道状态				调控后PID参数			调控后巷道状态
巷道区间/m	立柱压力/MPa	临界应力/MPa	应力安全系数	安全等级	K_P	K_I	K_D	立柱压力/MPa	临界应力/MPa	应力安全系数	安全等级
65～238	17.21	33.67	1.4	C	0.6193	0.5841	0.6736	40.28	37.76	1.6	A
65～238	22.90	33.67	1.4	C	0.6187	0.5843	0.6738	40.31	37.76	1.6	A
338～613	21.25	33.67	1.4	C	0.6197	0.5861	0.6766	40.26	37.76	1.6	A
338～613	23.80	33.67	1.4	C	0.6193	0.5841	0.6736	40.32	37.76	1.6	A
713～1 186	17.21	33.67	1.4	C	0.6198	0.5861	0.6766	40.28	37.76	1.6	A
713～1 186	22.55	33.67	1.4	C	0.6187	0.5851	0.6746	40.41	37.76	1.6	A

下载: 导出CSV

参考文献(21)

[1]	吴拥政,付玉凯,何杰,等. 深部冲击地压巷道“卸压−支护−防护”协同防控原理与技术[J]. 煤炭学报,2021,46(1):132-144. WU Yongzheng,FU Yukai,HE Jie,et al. Principle and technology of "pressure relief-support-protection" collaborative prevention and control in deep rock burst roadway[J]. Journal of China Coal Society,2021,46(1):132-144.
[2]	王国法,牛艳奇. 超前液压支架与围岩耦合支护系统及其适应性研究[J]. 煤炭科学技术,2016,44(9):19-25. WANG Guofa,NIU Yanqi. Study on advance hydraulic powered support and surrounding rock coupling support system and suitability[J]. Coal Science and Technology,2016,44(9):19-25.
[3]	唐治,潘一山,朱小景,等. 自移式吸能防冲巷道超前支架设计与研究[J]. 煤炭学报,2016,41(4):1032-1037. TANG Zhi,PAN Yishan,ZHU Xiaojing,et al. Design and study of self-moving energy absorption and antiimpact roadway advanced support[J]. Journal of China Coal Society,2016,41(4):1032-1037.
[4]	潘一山,齐庆新,王爱文,等. 煤矿冲击地压巷道三级支护理论与技术[J]. 煤炭学报,2020,45(5):1585-1594. PAN Yishan,QI Qingxin,WANG Aiwen,et al. Theory and technology of three levels support in bump-prone roadway[J]. Journal of China Coal Society,2020,45(5):1585-1594.
[5]	杜明超,张坤,黄梁松,等. 新型抗冲击吸能装置手性结构吸能特性对比分析[J/OL]. 煤炭科学技术:1-9[2023-08-17]. https://doi.org/10.13199/j.cnki.cst.2022-1138. DU Mingchao,ZHANG Kun,HUANG Liangsong,et al. Comparative analysis of energy absorption characteristics of chiral structures of new impact absorbing devices[J/OL]. Coal Science and Technology:1-9[2023-08-17]. https://doi.org/10.13199/j.cnki.cst.2022-1138.
[6]	宿吉奎,廉自生. 乳化液泵站的模糊PID控制系统仿真[J]. 煤矿机械,2012,33(7):54-56. DOI: 10.3969/j.issn.1003-0794.2012.07.026 SU Jikui,LIAN Zisheng. Simulation of fuzzy-PID control system in emulsion pump station[J]. Coal Mine Machinery,2012,33(7):54-56. DOI: 10.3969/j.issn.1003-0794.2012.07.026
[7]	ESSA M E S M,ABOELELA M A,HASSAN M A M. Position control of hydraulic servo system using proportional-integral-derivative controller tuned by some evolutionary techniques[J]. Journal of Vibration and Control,2016,22(12):2946-2957. DOI: 10.1177/1077546314551445
[8]	KIM S W,CHO B,SHIN S,et al. Force control of a hydraulic actuator with a neural network inverse model[J]. IEEE Robotics and Automation Letters,2021,6(2):2814-2821. DOI: 10.1109/LRA.2021.3062353
[9]	FENG Hao,SONG Qianyu,MA Shoulei,et al. A new adaptive sliding mode controller based on the RBF neural network for an electro-hydraulic servo system[J]. ISA Transactions,2022,129:472-484. DOI: 10.1016/j.isatra.2021.12.044
[10]	LIU Huaiqin,YU Qinghe,WU Qu. PID control model based on back propagation neural network optimized by adversarial learning-based grey wolf optimization[J]. Applied Sciences,2023,13(8). DOI: 10.3390/app13084767.
[11]	薛光辉,管健,柴敬轩,等. 基于神经网络PID综掘巷道超前支架支撑力自适应控制[J]. 煤炭学报,2019,44(11):3596-3603. XUE Guanghui,GUAN Jian,CHAI Jingxuan,et al. Adaptive control of advance bracket support force in fully mechanized roadway based on neural network PID[J]. Journal of China Coal Society,2019,44(11):3596-3603.
[12]	胡相捧,刘新华,庞义辉,等. 基于BP神经网络PID的液压支架初撑力自适应控制[J]. 矿业科学学报,2020,5(6):662-671. HU Xiangpeng,LIU Xinhua,PANG Yihui,et al. Adaptive control of setting load of hydraulic support based on BP neural network PID[J]. Journal of Mining Science and Technology,2020,5(6):662-671.
[13]	徐亚军,张坤,李丁一,等. 超前支架自适应支护理论与应用[J]. 煤炭学报,2020,45(10):3615-3624. XU Yajun,ZHANG Kun,LI Dingyi,et al. Theory and application of self-adaptive support for advanced powered support[J]. Journal of China Coal Society,2020,45(10):3615-3624.
[14]	张帆,葛世荣,李闯. 智慧矿山数字孪生技术研究综述[J]. 煤炭科学技术,2020,48(7):168-176. ZHANG Fan,GE Shirong,LI Chuang. Research summary on digital twin technology for smart mines[J]. Coal Science and Technology,2020,48(7):168-176.
[15]	EL BAZI N,MABROUKI M,LAAYATI O,et al. Generic multi-layered digital-twin-framework-enabled asset lifecycle management for the sustainable mining industry[J]. Sustainability,2023,15(4). DOI: 10.3390/su15043470.
[16]	葛世荣,张帆,王世博,等. 数字孪生智采工作面技术架构研究[J]. 煤炭学报,2020,45(6):1925-1936. GE Shirong,ZHANG Fan,WANG Shibo,et al. Digital twin for smart coal mining workface:technological frame and construction[J]. Journal of China Coal Society,2020,45(6):1925-1936.
[17]	张帆,葛世荣. 矿山数字孪生构建方法与演化机理[J]. 煤炭学报,2023,48(1):510-522. ZHANG Fan,GE Shirong. Construction method and evolution mechanism of mine digital twins[J]. Journal of China Coal Society,2023,48(1):510-522.
[18]	陈博,朱斌,王蒙飞,等. 基于DDPG的矿井智能通风决策算法研究[J]. 煤矿机械,2023,44(10):201-204. CHEN Bo,ZHU Bin,WANG Mengfei,et al. Research on intelligent ventilation decision-making algorithm for mine based on DDPG[J]. Coal Mine Machinery,2023,44(10):201-204.
[19]	CAO Jiaqi,WANG Xiaolan,WANG Yansong,et al. An improved dueling deep Q-network with optimizing reward functions for driving decision method[J]. Proceedings of the Institution of Mechanical Engineers,Part D:Journal of Automobile Engineering,2023,237(9):2295-2309. DOI: 10.1177/09544070221106037
[20]	刘清,张龙,李天越,等. 综采工作面三机数字孪生及协同建模方法[J]. 工矿自动化,2023,49(2):47-55. LIU Qing,ZHANG Long,LI Tianyue,et al. A three machine digital twin and collaborative modeling method for fully mechanized working face[J]. Journal of Mine Automation,2023,49(2):47-55.
[21]	潘一山,肖永惠,罗浩,等. 冲击地压矿井安全性研究[J]. 煤炭学报,2023,48(5):1846-1860. PAN Yishan,XIAO Yonghui,LUO Hao,et al. Study on safety of rockburst mine[J]. Journal of China Coal Society,2023,48(5):1846-1860.

施引文献(21)

期刊类型引用(16)

1.	康岩龙. 基于轨迹盲推的人员普适定位方法. 电子设计工程. 2025(02): 68-71+76 . 百度学术
2.	何晓晗. UWB技术人员定位系统在煤矿中的应用. 江西煤炭科技. 2025(01): 171-173+178 . 百度学术
3.	陈贤. 基于UWB的TOF与TDOA井下联合定位方法. 煤矿安全. 2025(02): 220-225 . 百度学术
4.	孙继平，彭铭. 室内电磁波传播衰减统计模型用于矿井的适用性研究. 工矿自动化. 2025(02): 1-8 . 本站查看
5.	贾佳，秦冬冬，王霞. 基于BP极大似然估计井下人员定位方法研究. 煤炭技术. 2025(05): 224-228 . 百度学术
6.	彭铭. 通用无线传输路径损耗统计模型用于矿井的适用性研究. 工矿自动化. 2025(04): 57-63+85 . 本站查看
7.	王端，刘世平，王利军. 基于机器视觉的煤矿巷道人员定位研究. 矿山机械. 2024(01): 56-60 . 百度学术
8.	孙继平，彭铭. 煤矿信息综合承载网标准研究制定. 工矿自动化. 2024(04): 1-8 . 本站查看
9.	吴文臻. 基于改进时间同步的矿井UWB优化定位方法. 工矿自动化. 2024(S1): 34-38 . 本站查看
10.	张雪军，黎卓芳. 煤矿多参数复合风险智能分级决策预警系统. 工矿自动化. 2024(S1): 88-91 . 本站查看
11.	貟婧. 基于超带宽测距算法的舞台灯光控制系统与自动追踪模型研究. 自动化与仪器仪表. 2024(06): 85-88+93 . 百度学术
12.	陈代伟，胡峰平，钱正峰，唐银. 基于3D人脸识别的煤矿人员出入井唯一性识别装置设计. 煤炭科技. 2024(04): 116-120 . 百度学术
13.	陈贤，周澍，张蓉. 一种井下人员乘车识别与定位方法. 煤矿安全. 2024(11): 217-221 . 百度学术
14.	孙继平，彭铭，刘斌. 矿井无线传输测试分析与矿用5G优选工作频段研究. 工矿自动化. 2024(10): 1-11+20 . 本站查看
15.	李烨，金业勇. 小型化双向波束矿用定位终端天线设计. 工矿自动化. 2024(11): 127-131+178 . 本站查看
16.	孙继平. 煤矿用5G通信系统标准研究制定. 工矿自动化. 2023(08): 1-8 . 本站查看