智能化煤矿大数据治理关键技术研究、实践与应用

方乾; 张晓霞; 王霖; 石磊; 王雅琨

doi:10.13272/j.issn.1671-251x.18099

智能化煤矿大数据治理关键技术研究、实践与应用

方乾^{1, 2,},
张晓霞^{1, 2},
王霖^{1, 2},
石磊^{1, 2},
王雅琨^{1, 2}

1.
煤炭科学研究总院有限公司矿山大数据研究院, 北京　100013
2.
煤炭资源高效开采与洁净利用国家重点实验室, 北京　100013

基金项目: 中国煤炭科工集团重点项目（2021-TD-ZD001）。

详细信息

作者简介:
方乾（1994—），男，湖北黄冈人，助理研究员，硕士，从事煤矿大数据、机器学习等技术研究工作，E-mail：fangqian@ccteg-bigdata.com

中图分类号: TD67
计量
- 文章访问数: 1532
- HTML全文浏览量: 313
- PDF下载量: 160
出版历程
- 收稿日期: 2023-04-02
- 修回日期: 2023-05-17
- 网络出版日期: 2023-05-28
- 刊出日期: 2023-05-24

Research, practice and application of key technologies of intelligent coal mine big data governance

FANG Qian^{1, 2,},
ZHANG Xiaoxia^{1, 2},
WANG Lin^{1, 2},
SHI Lei^{1, 2},
WANG Yakun^{1, 2}

1.
Research Institute of Mine Big Data, Chinese Institute of Coal Science, Beijing 100013, China
2.
State Key Laboratory of Coal Mining and Clean Utilization, Beijing 100013, China

摘要

摘要: 针对智能化煤矿建设过程中存在的“数据孤岛”现象严重、数据质量低、数据治理体系缺乏、数据赋能不够充分等问题，分析了智能化煤矿大数据治理的基本需求，研究了智能化煤矿的数据采集和存储、数据清洗与标准化处理、数据资产规划、数据共享交换等关键技术。结合小保当煤矿数据治理现场实践情况，提出了基于工业互联网体系的智能化煤矿大数据治理总体技术架构，该架构功能与智能化煤矿大数据治理的基本需求相对应，向下实现多源异构感知数据的接入、集成和融合，向上为各种煤矿智能化应用开发提供数据服务，中间沉淀煤矿各类业务指标、模型算法，形成煤矿重要的数据资产。基于不同的数据接入协议，通过数据接入存储服务统一接入煤矿各系统的数据；通过数据清洗与标准化服务实现数据加工，提升数据质量；采用分层治理架构将数据转换为体系化的分层数据资产；通过数据共享服务将数据资产以标准接口的方式提供给其他系统使用，实现数据价值落地。从煤矿单系统应用、矿井级应用和公司级系统应用全方面展示智能化煤矿数据治理成果在不同业务场景下的实践应用情况，智能化煤矿大数据经过统一的数据治理之后，能够实现数据的融合应用，打破数据孤岛，提升数据质量，形成煤矿独有的数据资产，为煤矿生产运营提供重要价值。
- 智能化煤矿 /
- 煤矿大数据 /
- 数据治理 /
- 数据清洗与标准化 /
- 数据资产规划 /
- 数据共享服务
Abstract: In the process of intelligent coal mine construction, there are problems such as the "data island" phenomenon, low data quality, lack of data governance system, and insufficient data empowerment. In order to solve the above problems, this paper analyzes the basic requirements of intelligent coal mine big data governance. This paper studies the key technologies of intelligent coal mines such as data acquisition and storage, data cleaning and standardization, data asset planning, data sharing and exchange. Combined with the field practice of data governance in Xiaobaodang Coal Mine, the overall technical architecture of intelligent coal mine big data governance based on the Industrial Internet system is proposed. The architecture functions correspond to the basic requirements of intelligent coal mine big data governance. It realizes the access, integration and fusion of multi-source heterogeneous perception data downward, provides data services for the development of various coal mine intelligent applications upward, and sediment various business indicators and model algorithms of coal mines in the middle, forming important data assets for coal mines. Unified access to to data form various coal mine systems is achieved through data access storage services based on different data access protocols. The necessary protocol conversion and data preprocessing are realized during the access process. The data processing is achieved through data cleaning and standardization services to improve data quality. The data is transformed into systematic data assets by adopting a hierarchical governance architecture. Finally, data assets are provided to other systems through standard interfaces through data sharing services, achieving the implementation of data value. The practical application of intelligent coal mine data governance achievements in different business scenarios is demonstrated from the perspective of coal mine single system application, mine-level application and company-level system application. After unified data governance, intelligent coal mine big data can achieve data fusion applications. It can break data islands, improve data quality, form coal mine unique data assets, and provide important value for coal mine production and operation.
- intelligent coal mine /
- coal mine big data /
- data governance /
- data cleaning and standardization /
- data asset planning /
- data sharing service

HTML全文

图 1 智能化煤矿大数据治理总体技术架构

Figure 1. Overall technical architecture of intelligent coal mine big data governance

下载: 全尺寸图片幻灯片

图 2 数据采集与存储

Figure 2. Data acquisiton and storage

下载: 全尺寸图片幻灯片

图 3 数据资产总体规划

Figure 3. Overall planning of data assets

下载: 全尺寸图片幻灯片

图 4 区域风险等级评估应用

Figure 4. Application of regional risk level assessment

下载: 全尺寸图片幻灯片

表 1 数据缺失

Table 1 Data missing

时间	标签	值/A
2023−03−11T00:00:00	主泵.电流	13.9
2023−03−11T00:00:03	主泵.电流	14.5
2023−03−11T00:00:10	主泵.电流	14.2
2023−03−11T00:02:00	主泵.电流	14.1
2023−03−11 T00:02:10	主泵.电流	13.8

下载: 导出CSV

表 2 数据异常

Table 2 Data exception

时间	标签	值
2023−03−11T00:00:00	主泵.状态	1
2023−03−11T03:39:19	主泵.状态	0
2023−03−11T04:00:00	主泵.状态	1
2023−03−11T07:32:49	主泵.状态	0
2023−03−11T08:02:10	主泵.状态	3

下载: 导出CSV

表 3 小保当煤矿数据表命名

Table 3 Naming of data table of Xiaobaodang Coal Mine

公司名称	系统名称	数据表名
小保当	安全监控	ods_xbd01_safety_monitor_hi
小保当	人力资源	ods_xbd02_human_resources

下载: 导出CSV

表 4 数据资产规范结构与编码

Table 4 Data asset specification structure and coding

序号	中文名称	英文名称	说明	字段类型	数据格式	是否空值	备注
1	状态采样时间	datatime	测点最近变化的时间点	字符	d23	否	在快照表中作为主键
2	设备名称	device_name	工作面设备名称	字符	c..20	否
3	设备状态	device_status	设备状态	数值	b	否	0−运行；1−停止
4	工作面编号	workface_id	测点区域编码	字符	an..20	否
5	矿井编码	mine_id	矿井编码	字符	an6	否
6	矿井名称	mine_name	矿井名称	字符	c7	否
7	写入时间	create_time	测点请求时间	字符	d23	否

下载: 导出CSV

参考文献(21)

[1]	王国法,王虹,任怀伟,等. 智慧煤矿2025情景目标和发展路径[J]. 煤炭学报,2018,43(2):295-305. WANG Guofa,WANG Hong,REN Huaiwei,et al. 2025 scenarios and development path of intelligent coal mine[J]. Journal of China Coal Society,2018,43(2):295-305.
[2]	吴群英,蒋林,王国法,等. 智慧矿山顶层架构设计及其关键技术[J]. 煤炭科学技术,2020,48(7):80-91. WU Qunying,JIANG Lin,WANG Guofa,et al. Top-level architecture design and key technologies of smart mine[J]. Coal Science and Technology,2020,48(7):80-91.
[3]	李首滨. 煤炭工业互联网及其关键技术[J]. 煤炭科学技术,2020,48(7):98-108. LI Shoubin. Coal industry internet and its key technologies[J]. Coal Science and Technology,2020,48(7):98-108.
[4]	杜毅博,赵国瑞,巩师鑫. 智能化煤矿大数据平台架构及数据处理关键技术研究[J]. 煤炭科学技术,2020,48(7):177-185. DU Yibo,ZHAO Guorui,GONG Shixin. Study on big data platform architecture of intelligent coal mine and key technologies of data processing[J]. Coal Science and Technology,2020,48(7):177-185.
[5]	毛善君,杨乃时,高彦清,等. 煤矿分布式协同“一张图”系统的设计和关键技术[J]. 煤炭学报,2018,43(1):280-286. MAO Shanjun,YANG Naishi,GAO Yanqing,et al. Design and key technology research of coal mine distributed cooperative "one map" system[J]. Journal of China Coal Society,2018,43(1):280-286.
[6]	王国法,刘峰,孟祥军,等. 煤矿智能化(初级阶段)研究与实践[J]. 煤炭科学技术,2019,47(8):1-36. DOI: 10.13199/j.cnki.cst.2019.08.001 WANG Guofa,LIU Feng,MENG Xiangjun,et al. Research and practice on intelligent coal mine construction(primary stage)[J]. Coal Science and Technology,2019,47(8):1-36. DOI: 10.13199/j.cnki.cst.2019.08.001
[7]	高士岗,高登彦,欧阳一博,等. 煤矿智能一体化辅助生产系统及关键技术[J]. 煤炭科学技术,2020,48(7):150-160. DOI: 10.13199/j.cnki.cst.2020.07.015 GAO Shigang,GAO Dengyan,OUYANG Yibo,et al. Mine intelligent integrated auxiliary production system and key technologies[J]. Coal Science and Technology,2020,48(7):150-160. DOI: 10.13199/j.cnki.cst.2020.07.015
[8]	何敏. 智能煤矿数据治理框架与发展路径[J]. 工矿自动化,2020,46(11):23-27. HE Min. Framework and development path of data governance in intelligent coal mine[J]. Industry and Mine Automation,2020,46(11):23-27.
[9]	温亮,李丹宁. 基于EtherNet/IP的井工煤矿数据治理研究[J]. 煤炭科学技术,2022,50(增刊1):227-232. WEN Liang,LI Danning. Research on data management of coal mine based on EtherNet/IP[J]. Coal Science and Technology,2022,50(S1):227-232.
[10]	李绍俊,杨海军,黄耀欢,等. 基于NoSQL数据库的空间大数据分布式存储策略[J]. 武汉大学学报(信息科学版),2017,42(2):163-169. LI Shaojun,YANG Haijun,HUANG Yaohuan,et al. Geo-spatial big data storage based on NoSQL database[J]. Geomatics and Information Science of Wuhan University,2017,42(2):163-169.
[11]	谭霜,贾焰,韩伟红. 云存储中的数据完整性证明研究及进展[J]. 计算机学报,2015,38(1):164-177. TAN Shuang,JIA Yan,HAN Weihong. Research and development of provable data integrity in cloud storage[J]. Chinese Journal of Computers,2015,38(1):164-177.
[12]	雷德龙,郭殿升,陈崇成,等. 基于MongoDB的矢量空间数据云存储与处理系统[J]. 地球信息科学学报,2014,16(4):507-516. LEI Delong,GUO Diansheng,CHEN Chongcheng,et al. Vector spatial data cloud storage and processing based on MongoDB[J]. Journal of Geo-Information Science,2014,16(4):507-516.
[13]	林文辉. 基于Hadoop的海量网络数据处理平台的关键技术研究[D]. 北京: 北京邮电大学, 2014. LIN Wenhui. Research on key technologies of massive networkdata processing platform based on Hadoop [D]. Beijing: Beijing University of Posts and Telecommunications, 2014.
[14]	沈姝. NoSQL数据库技术及其应用研究[D]. 南京: 南京信息工程大学, 2012. SHEN Shu. Research on NoSQL database technology and application[D]. Nanjing: Nanjing University of Information Science and Technology, 2012.
[15]	高金标,何利力,邹云阳. 基于分布式存储系统的Hive与Hbase的研究[J]. 工业控制计算机,2015,28(12):44-45,47. DOI: 10.3969/j.issn.1001-182X.2015.12.021 GAO Jinbiao,HE Lili,ZOU Yunyang. Hive and Hbase based on research on Hadoop distributed file system[J]. Industrial Control Computer,2015,28(12):44-45,47. DOI: 10.3969/j.issn.1001-182X.2015.12.021
[16]	温国锋,陈立文. 煤矿安全管理数据仓库的建立与应用研究[J]. 中国矿业,2009,18(1):95-97. WEN Guofeng,CHEN Liwen. On building and applacation of coal mine security management data warehouse[J]. China Mining Magazine,2009,18(1):95-97.
[17]	赵随海. 铁路列车调度指挥系统数据仓库体系结构的研究[J]. 铁道运输与经济,2018,40(12):55-59. ZHAO Suihai. A study on the architecture of data warehouse for the railway train dispatching command system[J]. Railway Transport and Economy,2018,40(12):55-59.
[18]	王霖,方乾,张晓霞,等. 智能化煤矿数据仓库建模方法[J]. 工矿自动化,2022,48(4):5-13. DOI: 10.13272/j.issn.1671-251x.2021120007 WANG Lin,FANG Qian,ZHANG Xiaoxia,et al. Intelligent coal mine data warehouse modeling method[J]. Journal of Mine Automation,2022,48(4):5-13. DOI: 10.13272/j.issn.1671-251x.2021120007
[19]	曾志浩,姚贝,张琼林,等. 基于Hadoop平台的用户行为挖掘[J]. 计算技术与自动化,2015,34(2):100-103. ZENG Zhihao,YAO Bei,ZHANG Qionglin,et al. User behavior mining based on Hadoop platform[J]. Computing Technology and Automation,2015,34(2):100-103.
[20]	马宏伟,吴少杰,曹现刚,等. 煤矿综采设备运行状态大数据清洗建模[J]. 工矿自动化,2018,44(11):80-83. MA Hongwei,WU Shaojie,CAO Xiangang,et al. Big data cleaning modeling of operation status of coal mine fully-mechanized coal mining equipment[J]. Industry and Mine Automation,2018,44(11):80-83.
[21]	张晓霞,陈思宇,苏上海,等. 矿井智能一体化管控平台设计及应用[J]. 煤炭科学技术,2022,50(9):168-178. ZHANG Xiaoxia,CHEN Siyu,SU Shanghai,et al. Design and application of mine intelligent integrated management and control platform[J]. Coal Science and Technology,2022,50(9):168-178.