Drill pipe counting method based on improved spatial-temporal graph convolution neural network

There are some problems in the existing drill pipe counting methods, such as repeated labor, large counting error, and failure to consider the timing information of actions. In order to solve the above problems, a drill pipe counting method based on an improved multi spatial-temporal graph convolution neural network (MST-GCN) model is proposed. Firstly, the video data of underground drilling is obtained through the mine monitoring camera. The Alphabose algorithm is used to extract the key points of the human body from the image sequence. The human skeleton on a single frame image and the skeleton sequence data on a continuous image sequence are obtained. The skeleton sequence representing human actions is built. Secondly, the MST-GCN model is designed based on the spatial-temporal graph convolution neural network (ST-GCN) model. The far space partition strategy is used to focus on the motion information of the key points that are far away from the skeleton. The squeeze and excitation network (SENet) is used to fuse the original space features and the far space features, so as to effectively identify the action categories on the skeleton sequence. Finally, support vector machine is used to identify the drilling pose on the drilling video to decide whether to save the skeleton sequence. If the sequence length is saved to 150 frames, the MST-GCN model is used to identify the action category. The identification interval of adjacent actions is set according to the actual drilling time, so as to record the number of actions and realize the drill pipe counting. The experimental results show that the recognition accuracy of the MST-GCN model is 91.1% on the self-built data set, which is 6.2%, 19.0% and 4.8% higher than that of ST-GCN, Alphapose-LSTM and NST-GCN, respectively. The loss value of the MST-GCN model converges below 0.2, and the learning capability is stronger. On the drilling videos under the same conditions, the average error values of the MST-GCN model, the artificial method and the improved ResNet method are 0.25, 0.75 and 21 respectively, which shows that the counting effect of the MST-GCN model is better. The average error of MST-GCN model is 9 and the miscount is low in the field application of drilling 1 300 pieces, which can meet the actual requirements.