Precise Positioning of Picking Grapes

Abstract

Aiming at the problem that it is difficult to determine the picking location for the picking robot, precise positioning of picking grapes based on Mask-RCNN network and Cannylines detector is proposed in this paper. Firstly, the grape images under shading, direct sunlight and backlight conditions are collected, and grape images are marked with masks to make grape data sets. Then, based on the Mask-RCNN network training data set, the trained model is used to identify the grapes, and the box of framed grape cluster and the mask approximately equal to the size of grape cluster are obtained. Then, the AOI of main stem and centroid of grape cluster are calculated according to the obtained box and mask, the obtained AOI is equalized. Finally, the Cannylines detector based on Bilateral filtering is used to detect the line segment of main stem in the AOI, and the picking location is obtained according to the minimum distance between the centroid and the line segment. 150 images from the test set were used for the experiment, with 50 images selected from each type of light, including shading, direct sunlight and backlight. Experiments show that the accuracies of picking positioning under the three types of light conditions, are 0.900, 0.920, and 0.860, respectively. This method can provide precise picking location information for the grape picking robot.