Fruit location and number estimation using photogrammetry in fruit cultivation

Abstract

In fruit growing, there are many operations based on growers’ experience, and one of them is fruit thinning. Since this task is based on the grower’s experience, it is difficult for unskilled workers to determine how much fruit should be thinned. In this study, we propose a method for estimating the position and number of fruits using a three-dimensional model of an orchard based on photogrammetry. The proposed method uses COLMAP, a high-precision photogrammetric method, and YOLOv7, a fast object detection algorithm. The 3D model and camera position information were generated by applying COLMAP to the captured video images, and the fruit recognition images were extracted by applying YOLOv7. A fruit arrangement model was generated by connecting the 3D model, camera position information, and fruit recognition images. To verify the effectiveness of the proposed method, we conducted indoor experiments in which two fruits were placed in an indoor space. As a result of the verification, the number of fruits could be identified in the indoor experiment, but the coordinates of the fruits differed from those in the real world.