Abnormal data recognition plays an important role in mine safety monitoring system, but abnormal data generally only accounts for about 1% of the total data of the safety monitoring system, data imbalance is an intrinsic characteristics of real-time data. At present, most of machine learning algorithms have relatively poor classification accuracy and sensitivity while dealing with classification on imbalanced data sets. In order to accurately identify abnormal data, the data collected by the distributed fiber shaft deformation monitoring system of coal mine is taken as research object, RDU-SMOTE-RF abnormal data recognition method of coal mine monitoring system based on imbalanced data set was proposed. The method uses RDU algorithm for under-sampling of majority data to remove duplicate samples,uses SMOTE algorithm for oversampling of minority abnormal data to improve the imbalance of the data set by synthesizing new abnormal data, and uses the optimized data set to train random forest (RF) classification algorithm to get abnormal data recognition model. The comparison experimental results on 6 real data sets show that the method has an average recognition accuracy rate of 99.3% for abnormal data, which has good generalization and strong robustness.