Occlusion Solution and its Evaluation for Ball Trajectory Estimation Acquired from Broadcast Video of Table Tennis

Abstract

Recently, we have been developing a system that automatically acquires tactics and strategies of the table tennis match from broadcast video. From the broadcast video, two-dimensional coordinates of the input and output data are constructed by automatically extracting the ball trajectory with the white blog extraction process and RGB extraction process by CenterNet of deep learning (DNN). In addition, the player’s skeleton position is also estimated by CenterNet of deep learning. However, the occlusion problem occurs. An Occlusion problem is that the ball trajectory is hidden behind the player and the ball trajectory cannot be tracked. In this paper, we propose a method to solve the occlusion problem and evaluate its recognition level. If the occlusion occurs, ball positions in occlusion are estimated from the ball coordinate and trajectory before and after the occlusion area using orthogonal regression and interpolation or extrapolation. In order to show the usefulness of the occlusion solution, the accuracy of the difference between the teacher data and the estimated output of the ball coordinates in all rallies is evaluated.