留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

综采工作面海量数据挖掘分析平台设计

王宏伟 杨焜 付翔 李进 贾思锋

王宏伟,杨焜,付翔,等. 综采工作面海量数据挖掘分析平台设计[J]. 工矿自动化,2023,49(5):30-36, 126.  doi: 10.13272/j.issn.1671-251x.18088
引用本文: 王宏伟,杨焜,付翔,等. 综采工作面海量数据挖掘分析平台设计[J]. 工矿自动化,2023,49(5):30-36, 126.  doi: 10.13272/j.issn.1671-251x.18088
WANG Hongwei, YANG Kun, FU Xiang, et al. Massive data mining and analysis platform design for fully mechanized working face[J]. Journal of Mine Automation,2023,49(5):30-36, 126.  doi: 10.13272/j.issn.1671-251x.18088
Citation: WANG Hongwei, YANG Kun, FU Xiang, et al. Massive data mining and analysis platform design for fully mechanized working face[J]. Journal of Mine Automation,2023,49(5):30-36, 126.  doi: 10.13272/j.issn.1671-251x.18088

综采工作面海量数据挖掘分析平台设计

doi: 10.13272/j.issn.1671-251x.18088
基金项目: 国家自然科学基金资助项目(52274157);山西省揭榜招标项目(20201101005);“科技兴蒙”行动重点专项项目(2022EEDSKJXM010)。
详细信息
    作者简介:

    王宏伟(1977—),女,黑龙江勃利人,教授,博士,博士研究生导师,主要研究方向为煤机装备智能化、人工智能与5G+智慧矿山等,E-mail:lntuwhw@126.com

    通讯作者:

    杨焜(1998—),男,山西长治人,硕士研究生,主要研究方向为工业互联网与大数据开发,E-mail:941077751@qq.com

  • 中图分类号: TD67

Massive data mining and analysis platform design for fully mechanized working face

  • 摘要: 当前综采工作面海量数据采集的实时性和完整性差、异常数据清洗耗时大、数据挖掘时延大,导致综采数据利用率低,无法辅助管理层实时下发决策指令。针对上述问题,设计了一种综采工作面海量数据挖掘分析平台。该平台由数据源层、数据采集存储层、数据挖掘层和前端应用层组成。数据源层由工作面各类硬件设备提供原始数据;数据采集存储层使用OPC UA网关实时采集井下传感器监测信息,再通过MQTT协议和RESTful接口将数据存入InfluxDB存储引擎;数据挖掘层利用Hive数据引擎和Yarn资源管理器筛选数据采集过程中受工作现场干扰形成的异常数据,解决因网络延时导致的数据局部采集顺序紊乱问题,并利用Spark分布式挖掘引擎挖掘工作面设备群海量工况数据的潜在价值,提高数据挖掘模型的运行速度;前端应用层利用可视化组件与后端数据库关联,再通过AJAX技术与后端数据实时交互,实现模型挖掘结果和各类监测数据的可视化展示。测试结果表明,该平台能够充分保证数据采集的实时性与完整性,清洗效率较单机MySQL查询引擎提升5倍,挖掘效率较单机Python挖掘引擎提升4倍。

     

  • 图  1  综采工作面海量数据挖掘分析平台总体架构

    Figure  1.  Overall architecture of massive data mining and analysis platform for fully mechanized working face

    图  2  海量数据采集存储技术实现流程

    Figure  2.  Massive data acquisition and storage technology implementation process

    图  3  海量数据挖掘技术实现流程

    Figure  3.  Massive data mining technology implementation process

    图  4  数据查询界面

    Figure  4.  Data query interface

    图  5  数据清洗测试流程

    Figure  5.  Data cleaning test process

    图  6  数据清洗速度对比

    Figure  6.  Data cleaning speed comparison

    图  7  数据挖掘测试流程

    Figure  7.  Data mining test process

    图  8  数据挖掘速度对比

    Figure  8.  Data mining speed comparison

  • [1] 王国法,赵国瑞,任怀伟. 智慧煤矿与智能化开采关键核心技术分析[J]. 煤炭学报,2019,44(1):34-41.

    WANG Guofa,ZHAO Guorui,REN Huaiwei. Analysis on key technologies of intelligent coal mine and intelligent mining[J]. Journal of China Coal Society,2019,44(1):34-41.
    [2] LIU Xianglan. Digital construction of coal mine big data for different platforms based on life cycle[C]. IEEE 2nd International Conference on Big Data Analysis, Beijing, 2017: 456-459.
    [3] 李福兴,李璐爔. 面向煤炭开采的大数据处理平台构建关键技术[J]. 煤炭学报,2019,44(增刊1):362-369.

    LI Fuxing,LI Luxi. Key technologies of big data processing platform construction for coal mining[J]. Journal of China Coal Society,2019,44(S1):362-369.
    [4] 杜毅博,赵国瑞,巩师鑫. 大数据关键技术在智能化煤矿中的应用与发展[J]. 煤炭科学技术,2020,48(7):177-185.

    DU Yibo,ZHAO Guorui,GONG Shixin. Study on big data platform architecture of intelligent coal mine and key technologies of data processing[J]. Coal Science and Technology,2020,48(7):177-185.
    [5] 王国法,王虹,任怀伟,等. 智慧煤矿2025情景目标和发展路径[J]. 煤炭学报,2018,43(2):295-305. doi: 10.13225/j.cnki.jccs.2018.0152

    WANG Guofa,WANG Hong,REN Huaiwei,et al. 2025 scenarios and development path of intelligent coal mine[J]. Journal of China Coal Society,2018,43(2):295-305. doi: 10.13225/j.cnki.jccs.2018.0152
    [6] 崔卫锋,田野,李旭,等. 煤矿综采工作面智能服务大数据决策平台[J]. 煤矿机械,2022,43(10):177-195.

    CUI Weifeng,TIAN Ye,LI Xu,et al. Intelligent service big data decision-making platform for fully mechanized mining face in coal mine[J]. Coal Mine Machinery,2022,43(10):177-195.
    [7] 谭章禄,王美君. 智慧矿山数据治理概念内涵、发展目标与关键技术[J]. 工矿自动化,2022,48(5):6-14. doi: 10.13272/j.issn.1671-251x.2021120090

    TAN Zhanglu,WANG Meijun. Research on the concept connotation,development goal and key technologies of data governance for smart mine[J]. Journal of Mine Automation,2022,48(5):6-14. doi: 10.13272/j.issn.1671-251x.2021120090
    [8] 葛世荣. 煤矿智采工作面概念及系统架构研究[J]. 工矿自动化,2020,46(4):1-9.

    GE Shirong. Research on concept and system architecture of smart mining workface in coal mine[J]. Industry and Mine Automation,2020,46(4):1-9.
    [9] 李首滨,李森,张守祥,等. 综采工作面智能感知与智能控制关键技术与应用[J]. 煤炭科学技术,2021,49(4):28-39.

    LI Shoubin,LI Sen,ZHANG Shouxiang,et al. Key technology and application of intelligent perception and intelligent control in fully mechanized mining face[J]. Coal Science and Technology,2021,49(4):28-39.
    [10] 苏杰,王新坤. 寸草塔煤矿综采工作面智能化建设关键技术研究与应用[J]. 煤炭科学技术,2022,50(增刊1):250-256.

    SU Jie,WANG Xinkun. Research and application of key technologies for intelligent construction of fully mechanized mining face in Cuncaota Coal Mine[J]. Coal Science and Technology,2022,50(S1):250-256.
    [11] 贺海涛. 综采工作面智能化开采系统关键技术[J]. 煤炭科学技术,2021,49(增刊1):8-15.

    HE Haitao. Key technology of intelligent mining system in fully-mechanized mining face[J]. Coal Science and Technology,2021,49(S1):8-15.
    [12] 徐亚军,张坤,李丁一,等. 超前支架自适应支护理论与应用[J]. 煤炭学报,2020,45(10):3615-3624.

    XU Yajun,ZHANG Kun,LI Dingyi,et al. Theory and application of self-adaptive support for advanced powered support[J]. Journal of China Coal Society,2020,45(10):3615-3624.
    [13] 付翔,王然风. 工作面供液系统与液压支架协同自适应控制模型设计[J]. 采矿与岩层控制工程学报,2020,2(3):90-98.

    FU Xiang,WANG Ranfeng. Cooperative self-adaptive control model of fluid feeding system and hydraulic supports in working face[J]. Journal of Mining and Strata Control Engineering,2020,2(3):90-98.
    [14] 宫文峰, 张美玲, 陈辉. 基于深度学习的旋转机械大数据智能故障诊断方法[J/OL]. 计算机集成制造系统: 1-21[2023-04-21]. http://kns.cnki.net/kcms/detail/11.5946.TP.20220612.0910.004.html.

    GONG Wenfeng, ZHANG Meiling, CHEN Hui. Intelligent fault diagnosis method based on deep learning of rotating machinery under big data[J/OL]. Computer Integrated Manufacturing Systems: 1-21[2023-04-21]. http://kns.cnki.net/kcms/detail/11.5946.TP.20220612.0910.004.html.
    [15] 丁恩杰,俞啸,廖玉波,等. 基于物联网的矿山机械设备状态智能感知与诊断[J]. 煤炭学报,2020,45(6):2308-2319. doi: 10.13225/j.cnki.jccs.zn20.0340

    DING Enjie,YU Xiao,LIAO Yubo,et al. Key technology of mine equipment state perception and online diagnosis under Internet of things[J]. Journal of China Coal Society,2020,45(6):2308-2319. doi: 10.13225/j.cnki.jccs.zn20.0340
    [16] 雷亚国,杨彬,杜兆钧,等. 大数据下机械装备故障的深度迁移诊断方法[J]. 机械工程学报,2019,55(7):1-8. doi: 10.3901/JME.2019.07.001

    LEI Yaguo,YANG Bin,DU Zhaojun,et al. Deep transfer diagnosis method for machinery in big data era[J]. Journal of Mechanical Engineering,2019,55(7):1-8. doi: 10.3901/JME.2019.07.001
    [17] 谢嘉成,王学文,杨兆建. 基于数字孪生的综采工作面生产系统设计与运行模式[J]. 计算机集成制造系统,2019,25(6):1381-1391.

    XIE Jiacheng,WANG Xuewen,YANG Zhaojian. Design and operation mode of production system of fully mechanized coal mining face based on digital twin theory[J]. Computer Integrated Manufacturing Systems,2019,25(6):1381-1391.
    [18] 张科学,徐兰欣,李旭,等. 透明工作面智能化开采大数据分析决策方法及系统研究[J]. 煤炭科学技术,2022,50(2):252-262.

    ZHANG Kexue,XU Lanxin,LI Xu,et al. Research on big data analysis and decision system of intelligent mining in transparent working face[J]. Coal Science and Technology,2022,50(2):252-262.
    [19] 李重重,刘清,刘军锋,等. 面向综采工作面的自动化软件设计与应用[J]. 工矿自动化,2023,49(3):124-130.

    LI Zhongzhong,LIU Qing,LIU Junfeng,et al. Automation software design and application for fully mechanized working face[J]. Journal of Mine Automation,2023,49(3):124-130.
    [20] 李佳,徐胜超. 基于云计算的智能电网大数据处理平台[J]. 计算机工程与设计,2018,39(10):3073-3079.

    LI Jia,XU Shengchao. Smart power system big data processing platform in cloud environments[J]. Computer Engineering and Design,2018,39(10):3073-3079.
    [21] 王万良,张兆娟,高楠,等. 基于人工智能技术的大数据分析方法研究进展[J]. 计算机集成制造系统,2019,25(3):529-547.

    WANG Wanliang,ZHANG Zhaojuan,GAO Nan,et al. Progress of big data analytics methods based on artificial intelligence technology[J]. Computer Integrated Manufacturing Systems,2019,25(3):529-547.
    [22] JIANG Dingde,WANG Yuqing,LYU Zhihan,et al. Big data analysis based network behavior insight of cellular networks for industry 4.0 applications[J]. IEEE Transactions on Industrial Informatics,2019,16(2):1310-1320.
    [23] ZAHARIA M,XIN R S,WENDELL P,et al. Apache spark:a unified engine for big data processing[J]. Communications of the Acm,2016,59(11):56-65. doi: 10.1145/2934664
    [24] 罗广恒. 基于Django和MySQL的网络化测试数据查询系统研究[J]. 智能物联技术,2019,51(2):15-21,31.

    LUO Guangheng. Research on measure data networking query system based on Django and MySQL[J]. Technology of IoT & AI,2019,51(2):15-21,31.
    [25] 张锦涛,付翔,王然风,等. 智采工作面中部液压支架集群自动化后人工调控决策模型[J]. 工矿自动化,2022,48(10):20-25.

    ZHANG Jintao,FU Xiang,WANG Ranfeng,et al. Manual regulation and control decision model of middle hydraulic support cluster automation in the intelligent working face[J]. Journal of Mine Automation,2022,48(10):20-25.
  • 加载中
图(8)
计量
  • 文章访问数:  256
  • HTML全文浏览量:  32
  • PDF下载量:  64
  • 被引次数: 0
出版历程
  • 收稿日期:  2023-03-20
  • 修回日期:  2023-05-21
  • 网络出版日期:  2023-05-29

目录

    /

    返回文章
    返回