Deep learning and computer vision for assessing the number of actual berries in commercial vineyards

The number of berries is one of the most relevant yield components that drives grape production in viticulture. The goal of this work was to estimate the number of actual berries per grapevine using computer vision and deep learning in commercial vineyards. Images from the visible range (RGB) were acquired from a set of 96 grapevines (Vitis vinifera L.) at pea-size berry stage using a red, green and blue camera (RGB). At harvest, the number of berries and per vine was manually assessed as the ground-truth values. The algorithm involved computer vision to detect berries in the images and to extract canopy features, in order to gain information about canopy occlusion. These were used by the machine learning regression models built to estimate the number of actual berries per vine. A SegNet architecture was used to segment individual berries and several canopy related features. Four datasets were created combining the number of estimated visible berries and different canopy features. Three different regression models were tested on the four datasets. The best results were achieved with support vector regression (SVR) on a dataset including six canopy features. This method yielded a root mean squared error (RMSE) of 205 berries, a normalised root mean squared error (NRMSE) of 24.99% and a coefficient of determination (R) of 0.83 between the number of estimated and the number of actual berries per vine. The results show that the number of actual berries in grapevines can be assessed with high accuracy up to 60 days prior to grape harvest, using the developed algorithm based on computer vision and deep learning.Fernando Palacios would like to acknowledge the research founding FPI grant 286/2017 by Universidad de La Rioja, Gobierno de La Rioja. This work is supported by National Funds by FCT – Portuguese Foundation for Science and Technology, under the project UIDB/04033/2020