In this paper, we propose a method where we represent the 3D shape of mackerels and horse mackerels flowing on a conveyor as point clouds and recognize fish species using PointNet, a point cloud data classification model. Specifically, when fish flowing on a conveyor are photographed from above(Z-axis), the rotation of the fish is assumed to be only around the Z-axis, and we propose a PointNet that is limited to rotation normalization around the Z-axis only. We conducted experiments using point cloud data which are photographed at a fish market, and the following were revealed. (1) We investigated the relations between the number of points used for recognition, accuracy rate and the recognition speed, and our findings showed that using 128 points resulted in the best recognition performance. (2) Comparing the accuracy rate of the proposed method with that of the original PointNet, the proposed method gave a higher accuracy rate for both 100 stationary fish and 308 fish moved by a conveyor. (3)Comparison of the proposed method with the method that recognizes fish species from color and depth images acquired simultaneously with the point cloud using the ResNet-50 image classification model showed that the accuracy rate of the proposed method using the point cloud is the same or higher than that of the method using color and depth images, respectively.
View full abstract