English            | 设为主页 | 加入收藏 | 网站导航 | 读者留言 | 联系我们 | 返回首页 |
 
基于Hadoop的煤矿数据中心架构设计
Architecture design of coal mine data center based on Hadoop
【索引】韩安.基于Hadoop的煤矿数据中心架构设计[J].工矿自动化,2019,45(8):60-64.
【Reference】HAN An. Architecture design of coal mine data center based on Hadoop[J].Industry and Mine Automation,2019,45(8):60-64.
【DOI】10.13272/j.issn.1671-251x.17436
【作者】 韩安1,2
【Author】 HAN An1,2
【作者机构】1.中煤科工集团常州研究院有限公司, 江苏 常州213015;2.天地(常州)自动化股份有限公司, 江苏 常州213015
【Unit】1.CCTEG Changzhou Research Institute, Changzhou 213015, China; 2.Tiandi (Changzhou) Automation Co., Ltd., Changzhou 213015, China
【摘要】根据煤矿安全生产业务需求及智慧矿山发展要求,新型的煤矿数据中心需满足对同一时空坐标体系下煤矿海量、多元数据的高效处理、缓存、计算、存储与发布。针对传统煤矿数据中心各类数据离散存储,数据集成、业务应用及数据分析难度大等问题,设计了一种基于Hadoop的煤矿数据中心架构。采用Storm实时数据流引擎进行数据实时计算,并应用MapReduce、Spark实现批处理计算和内存计算,解决高频时序数据存储与海量数据计算问题;采用Hadoop分布式文件系统(HDFS)实现文件的可靠存储,并采用HBase分布式存储数据库实现历史数据的持久化存储,利用其无模式稀疏设计满足同一时空坐标体系下的数据分析需求;采用Redis作为实时数据库存储数据快照进行最新数据检索。基于Hadoop的煤矿数据中心充分利用大数据技术的高速数据存取和压缩性能,通过统一的云计算数据中心,有效降低了煤炭企业数据应用的复杂度及数据中心日常运维难度,可为智慧矿山建设奠定数据基础。
【Abstract】According to business requirements of coal mine safety production and development requirements of smart mine, the new coal mine data center needs to meet efficient processing, caching, calculation, storage and release of massive and multivariate data of coal mine under the same space-time coordinate system. In view of problems of discrete data storage and difficulties in data integration, business application and data analysis of traditional coal mine data center, an architecture of coal mine data center based on Hadoop was designed. Real-time data stream engine storm is adopted for real-time data calculation, and MapReduce and Spark are applied to implement batch processing and memory calculation to solve high-frequency time-series data storage and massive data calculation problems. Hadoop distributed file system (HDFS) is used for files reliable storage, and distributed storage database HBase is used for persistent storage of historical data, which can meet data analysis needs of the same space-time coordinate system by its modeless sparse design. Redis is used as a real-time database storage data snapshot for the latest data retrieval. Coal mine data center based on Hadoop makes full use of high-speed data access and compression performance of big data technology, complexity of data application in coal enterprises and daily operation and maintenance difficulty of data center are effectively reduced through unified cloud computing data center, which can lay a data foundation for the construction of smart mine.
【关键词】 智慧矿山; 煤矿数据中心; Hadoop; 大数据; 云计算
【Keywords】smart mine; coal mine data center; Hadoop; big data; cloud computing
【文献出处】工矿自动化,2019年8期
【基金】天地科技股份有限公司科技创新创业资金专项项目(2018-TD-QN019);天地(常州)自动化股份有限公司科研项目(2018GY117)
【分类号】TD67
HTML PDF下载
本网站仅提供本刊2009年之后的全文下载,其它年份的全文下载将自动转到中国知网。中国知网不支持迅雷等加速下载工具,请取消加速工具后下载。
【关 闭】