Computers and Electronics in Agriculture 178 (2020) 105796

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

journal homepage: www.elsevier.com/locate/compag

Automated grapevine flower detection and quantification method based on T

computer vision and deep learning from on-the-go imaging using a mobile
sensing platform under field conditions
Fernando Palaciosa,b, Gloria Buenoc, Jesús Salidoc, Maria P. Diagoa,b, Inés Hernándeza,b,
Javier Tardaguilaa,b,
⁎

a
Televitis Research Group, University of La Rioja, 26006 Logroño, Spain
b
Instituto de Ciencias de la Vid y del Vino (University of La Rioja, CSIC, Gobierno de La Rioja), 26007 Logroño, Spain
c
VISILAB group, University of Castilla-La Mancha, 13071 Ciudad Real, Spain

ARTICLE INFO ABSTRACT

Keywords: Grape yield forecasting is a valuable economic and quality issue for the grape and wine industry. The number of
Yield estimation flowers at bloom could be used as an early indicator towards crop forecast in viticulture. The purpose of this
Non-invasive sensing technologies work was to develop a non-invasive method for grapevine flower counting by on-the-go image acquisition, using
Precision agriculture a combination of deep learning and computer vision technology. A mobile sensing platform was used at 5 km/h
Vitis vinifera L.
to automatically capture Red Green Blue (RGB) images of vineyard canopy at night using artificial illumination
under field conditions. For the image data set, 96 vines from six grapevine varieties were selected. For ground-
truthing, the number of flowers per inflorescence was counted on a set of clusters before flowering, while yield
per vine was weighted at harvest. The developed algorithm comprised two general steps: inflorescences’ seg-
mentation, and individual flower detection. In both steps, the best results were obtained using the deep fully
convolutional neural network SegNet architecture with a VGG19 network as the encoder, with F1 score values of
0.93 and 0.73 in the inflorescences segmentation and the individual flower detection steps, respectively. These
values showed the high accuracy of the network. A determination coefficient (R2) of 0.91 between the detected
number of flowers and the actual number of flowers per vine was obtained. In addition, a linear regression model
was trained to estimate the actual number of flowers from the number of detected flowers. A root mean squared
error (RMSE) of 590 flowers per vine and a normalized root mean squared error (NRMSE) of 23.7% was ob-
tained. An R2 above 0.70 was achieved between the estimated number of actual flowers and the final yield, per
vine. These results show that the number of flowers per vine can be estimated using machine vision and deep
learning. The developed imaging platform can be used by the wine industry in commercial vineyards for a
satisfactory early crop yield forecasting.

1. Introduction Traditional methods for estimating vineyard yield rely on manually

Vineyard crop forecasting is highly relevant for the grape and wine
industry worldwide. An accurate yield prediction can lead to the de-
sired fruit quality and quantity, and in some countries, meet yield
regulations as well. While the axiom “the lower the yield the higher the
fruit and wine quality” does not hold true at all instances, it is certain
that a balance between vine vigour and yield to meet vine capacity is
desirable (Kliewer and Casteel, 2003). Therefore, abide other logistics
and economical reasons, proper yield estimation early in the season
may help to drive appropriate agronomical and canopy management
operations to achieve vine balance and fruit quality (Howell, 2001).
sampling, collecting and weighting a limited number of clusters of
certain vines within the vineyard in order to assess the final yield
(Martin and Dunn, 2003). These methods are tedious, inaccurate and
time-consuming. Therefore, new, non-invasive technologies are re-
quired for efficiently collecting a considerable amount of data, in order
to make precise and reliable estimations and predictions some months
before harvest. Nowadays, yield estimation is a major challenge for the
smart viticulture and wine industry.
Computer vision has been widely used to estimate relevant para-
meters in viticulture, such as vine pruning weight (Kicherer et al.,
2017), canopy assessment (Diago et al., 2019), cluster compactness

⁎
Corresponding author at: Televitis Research Group, University of La Rioja, 26006 Logroño, Spain.
E-mail address: javier.tardaguila@unirioja.es (J. Tardaguila).

https://doi.org/10.1016/j.compag.2020.105796
Received 2 June 2020; Received in revised form 10 September 2020; Accepted 14 September 2020
Available online 23 September 2020
0168-1699/ © 2020 Elsevier B.V. All rights reserved.
F. Palacios, et al.
(Palacios et al., 2019) as well as to assess crop yield at different phe-
nological stages (Nuske et al., 2014). In this latter work, the authors
developed a method for vineyard yield estimation based on berry de-
tection in RGB images acquired from a moving vehicle equipped with
artificial illumination. With the aim of predicting yield from image
acquisition, several trials have been conducted in different countries at
several phenological stages, ranging from pea-size to harvest. Herrero-
Huerta et al. (2015) proposed a method for yield estimation at cluster
level based on 3D cluster modelling from RGB images acquired during
day-time at harvest time. Font et al. (2015) used pixel-based image
segmentation methods for detecting red berries in images taken at
night-time using a moving vehicle with embedded artificial illumina-
tion during the harvesting period. On the other hand, Liu and Whitty
(2015) developed a method for cluster detection and counting on RGB
images manually acquired from red grapevine plants on field, during
day-time, days before harvest. Liu et al. (2017) proposed a grapevine
yield estimation method based on the automatic detection and counting
of vine shoots as early as two weeks after budburst. Aquino et al. (2018)
presented an algorithm for berry detection and quantification on RGB
images taken at night-time using artificial illumination at phenological
stages around bunch closure. Later in the season, Millan et al. (2018)
developed a new methodology based on segmenting grapevine clusters
on RGB images acquired one day before harvest, and estimating the
number of berries using a Boolean model. While several conditions and
phenological stages have been tested in these works, none of them has
addressed the grapevine flower counting for crop forecasting in viti-
culture, even though flower number per inflorescence is one of the main
determinants of grapevine yield (May 2005). Likewise, after budburst,
the processes of flowering, fertilization and fruit-set (a flower trans-
forms into a berry) are determinant of cluster components and final
yield. Fruit-set determines the final number of berries per cluster, and
together with berry weight, will determine cluster weight at harvest.
Then, this cluster weight and the number of clusters will define the final
yield in a given season. Approximately 60–70% of annual yield varia-
tion is explained by the number of clusters, while the remaining
30–40% is accounted by variations in cluster weight (Clingeleffer
2001). Of these, the number of berries per cluster is reported to be more
determinant in cluster weight than berry weight (Dunn, 2010). Since
the number of clusters is mostly fixed at winter pruning, assessing the
number of berries per cluster becomes then, essential for a proper and
early yield estimation, and this, in turn, is dependent on the number of
flowers. Both the number of flowers per inflorescence, also known as
fruitfulness, and the fruit-set rate are highly dependent on the en-
vironmental conditions of both the previous and current seasons (May
2005), such as light, temperature and water availability, with other
factors, like the variety, and some agronomical practices being also
important (Krstic et al., 2005). However, the knowledge of the number
of flowers may be highly relevant for early yield estimation should
average fruit set rate and average berry weight for a given vineyard
plot, or cultivar in a given grapegrowing area over a series of seasons
are known. Provided these values are available (or may be started to be
recorded yearly) then, a suitable yield estimation three to four months
prior to harvest can be attained, unless hail or other adverse meteor-
ological events take place during the growing season.
Until some years ago, flower counting per inflorescence had been
manually and destructively performed (Dunn and Martin, 2003). The
first works reporting non-destructive vine flower counting using com-
puter vision date to 2014, when Diago et al. (2014) presented an al-
gorithm based on image analysis for visible flower counting and the
estimation of the actual number of flowers in inflorescences’ images
manually taken. These authors used a black background to ease the
segmentation process of the inflorescences by applying the extended-
maxima transform for counting the number of brighter areas in the
segmented inflorescences, which usually corresponded to the flowers.
Following this approach, Aquino et al. (2015a) developed an applica-
tion for Android smartphones based on the methodology and analysis
2
Computers and Electronics in Agriculture 178 (2020) 105796
presented by Diago et al. (2014). Next steps addressed flower counting
on inflorescences’ images manually taken under field conditions by the
application of morphological image processing and pyramidal decom-
position (Aquino et al., 2015b). Following these two approaches, sev-
eral authors presented works for an automated flower detection and
counting in inflorescences’ images taken manually using a dark back-
ground. Among them, Millan et al. (2017) calibrated and tested several
models for the actual number of flowers’ estimation from the visible
flowers, that were automatically detected using an updated version of
the algorithm presented by Diago et al. (2014). Alternatively,
Radhouane et al. (2016) applied the watershed algorithm for the seg-
mentation of flowers, while Liu et al. (2018) proposed an alternative
image analysis method based on traditional image feature descriptors
for flower detection and counting, and Tello et al. (2019) used image
morphology analysis and the watershed algorithm for flower counting
considering several grapevine genotypes.
All previous works have addressed the visible flower counting and
actual number of flowers’ estimation using traditional image analysis
methods. Other works have handled this problem by applying more
recent techniques of deep learning and semantic segmentation.
Rudolph et al. (2019) developed an algorithm based on segmenting
inflorescences using a Fully Convolutional Network (FCN, Long et al.,
2015) and counting flowers using the Circular Hough Transform (CHT)
on manually acquired vine images under natural conditions. Grimm
et al. (2019) presented a semantic segmentation framework for detec-
tion and quantification of plant organs (inflorescences and flowers
among them) in vine images acquired manually under field conditions.
Nowadays, no previous work has attempted to count flowers in
commercial vineyards, using images acquired on-the-go from a mobile
platform. This is a relevant task for grapegrowers in order to develop a
new, and rapid approach to early crop forecast, particularly valuable in
the context of precision viticulture.
Likewise, the main goal of this work was to develop a new, non-
destructive, automated method to detect and quantify the visible flower
number per vine under field conditions, and to estimate the actual
number of flowers, with the aim of providing an early crop yield esti-
mation. Computer vision and deep learning was used for processing
RGB images acquired in commercial vineyards using a mobile imaging
platform. The amount of flowers per vine was related to yield in order
to explore a new approach for early crop forecast, several months be-
fore harvest, in commercial vineyards.
2. Materials and methods
2.1. Experimental layout
The trials were carried out during season 2018 in a commercial
vineyard located in Vergalijo (lat. 42°27′46.0″ N; long. 1°48′13.1″ W;
Navarra, Spain). Six grapevine (Vitis vinifera L.) varieties were selected,
three red (Syrah, Tempranillo, Cabernet Sauvignon) and three white
(Verdejo, Malvasia and Muscat of Alexandria). The vines were trained
onto a vertical shoot positioned (VSP) trellis system with 2 m row
spacing and 1 m vine spacing. Vineyard was partially defoliated prior to
image acquisition at flowering stage. An overall set of 96 vines (16
vines per variety) were labelled and delimited vertically using warning
tape (see Fig. 1).
2.2. Image acquisition
The RGB images were acquired on 21st May 2018, 9 days before full
flowering (109 days before harvest). A mobile platform developed at
the University of La Rioja was employed for on-the-go image acquisi-
tion in the vineyard (Fig. 2) at a speed of 5 Km/h. The platform con-
sisted of a modified all-terrain-vehicle, ATV (Trail Boss 330, Polaris
Industries, Minnesota, USA), in which a structure with artificial illu-
mination and a system for automated RGB image acquisition was
F. Palacios, et al.
Fig. 1. Vine labelled and delimited vertically using warning tape. The region of
interest (ROI) considered for the analysis of each vine is between two vertical
plastic strings.
Fig. 2. Mobile imaging platform: An all-terrain-vehicle (ATV) modified to in-
corporate a RGB camera for on-the-go image acquisition.
mounted. The artificial illumination system enabled the acquisition of
images at night-time with homogeneity on the illumination of the vines,
and the “isolation and differentiation” of the vine under evaluation
from the vines of the opposite row, which ease the image processing by
the algorithm. The components of this platform were thoroughly de-
tailed in previous works (Palacios et al., 2019; Diago et al., 2019). The
RGB camera used was a Canon EOS 5D Mark IV RGB camera (Canon
Inc. Tokyo, Japan) mounting a full-frame CMOS sensor (35 mm and
30.4 MP) equipped with a Canon EF 35 mm F/2 IS USM lens. The
camera was positioned at a distance of 0.80 m from the canopy, at a
height of 1 m from the ground, and its parameters were manually set at
the beginning of the experiment. These parameters remained fixed
during the image acquisition process: exposure time of 1/1600 s, ISO
speed of 5000 and f-number of f/8.
2.3. Ground truth measurements
In order to test the new image-based method developed in this
work, the actual number of flowers per vine was counted. The 21st and
22nd May 2018, all inflorescences per vine were manually and in-
dividually photographed using the digital camera used for on-the-go
image acquisition and a black cardboard as background. Images were
taken at daytime under natural illumination conditions. These images
were then processed by the algorithm published by Millan et al. (2017).
This algorithm processed images taken from individual inflorescences
with a dark background and automatically detect and count the visible
flowers of the inflorescence. Millan et al. (2017) also presented a
3
Computers and Electronics in Agriculture 178 (2020) 105796
nonlinear model (a multilayer feed-forward backpropagation neural
network) that used the number of detected flowers and several de-
scriptors obtained from the image to estimate the total number of
flowers (visible and non-visible) of the inflorescence photographed. For
each inflorescence, the algorithm obtained the number of visible
flowers and the total number of flowers. The number of inflorescences
per vine was also counted. Then, the sum of the estimations for all the
inflorescences of a given vine was used as the reference value of the
actual number of flowers of that specific vine. The algorithm of Millan
et al. (2017) was designed to only process images with a single in-
florescence and using a dark background, therefore, not working on
more complex conditions (like vine images or images with multiple
inflorescences), but under those conditions (images with a single in-
florescence and a dark background), it proved to obtain accurate re-
sults. This motivated the use of its results as the ground truth.
At harvest, middle of September, the yield of each labelled vine was
manually harvested and weighted.
2.4. Flower counting algorithm
The image processing algorithm for the automated flower counting
can be summarized in two general steps: inflorescences’ extraction
(Fig. 3b), and individual flower detection (Fig. 3c). For the first step a
semantic segmentation approach was followed in order to obtain a
“region of interest” (ROI) containing only aggregations of flowers em-
ploying a SegNet architecture (Badrinarayanan et al., 2017). SegNet is a
deep learning encoder-decoder architecture that performs pixel-wise
classification on RGB images. This architecture has proved to be suc-
cessful on the task of performing semantic segmentation on images
from different fields, as autonomous driving (Siam et al., 2017), med-
icine (Daimary et al., 2020) or biology (Ruiz-Santaquitaria et al., 2019).
The expertise of the authors of this work with SegNet and the good
accuracy of this architecture at performing the semantic segmentation
task on images from different fields makes its use adequate for ad-
dressing the problem presented in this work. Three different approaches
were tested for the second step. Of these, two were based on the seg-
mentation and counting the individual flowers (one performed by
SegNet and the other one by the watershed algorithm), and the third
one consisted of a prediction model based on a linear regression that
estimated the number of flowers from the area corresponding to the
aggregations of flowers previously extracted in the first step. All steps
rely on analysing small regions of the whole image. The complete image
processing is achieved following a sliding-window approach, where the
full image is reconstructed after applying the inflorescence extraction
step on each image patch, and then the individual flower detection step
is applied on the inflorescences segmented on the reconstruction of the
full image.
2.4.1. Inflorescences extraction
For a proper flower detection, the ROI in the image that contains
only groups of flowers has to be previously segmented. For this, a se-
mantic segmentation approach was followed. This consisted of a deep
learning technique that allows a pixel-wise classification on multi-
channel images given as input.
In this work, a SegNet architecture was employed (Badrinarayanan
et al., 2017). SegNet is formed by two deep neural networks to perform
semantic segmentation on RGB images given as an input. The first one
is the encoder, which is composed by several layers (convolution, batch
normalization, ReLU and pooling) that correspond to the layers of a pre-
trained Convolutional Neural Network (CNN). In this work, VGG16 and
VGG19 were considered (Simonyan and Zisserman, 2015). This encoder
performs several down-sampling operations to extract feature maps.
The decoder has a layer for each encoder layer and reverses the op-
erations performed by the encoder, up-sampling the feature map and
performing the final pixel-wise classification. Using an RGB image as
input, the result obtained by this architecture is another image where
F. Palacios, et al.
Fig. 3. Image processing stages: a) Initial full resolution image, b) inflorescences’ segmentation, and c) flowers’ detection.
each pixel is labelled with its predicted class.
The SegNet was trained with a set of 187 image patches of which 61
of them contained inflorescences, while the remaining 126 included
other elements that were present in the images (e.g. leaves, trunk or
ground). In this set, image patches were pixel-wise manually labelled
(Fig. 4), and then a data augmentation process was applied in order to
increase the variability and the amount of data until a final set of 2440
images (1220 with inflorescences and 1220 without them) was ob-
tained and used during training. The first transformations applied to the
original images in the data augmentation process were image rotations
(90° and 180°) and image flipping (vertical and horizontal). Then, the
original images and the new generated ones after the transformations
were convoluted with a gaussian kernel considering two sigma values
(2 and 4). In addition, a contrast enhancement was applied using the
Contrast-limited adaptive histogram equalization (CLAHE) algorithm
(Fig. 5) (Zuiderveld, 1994). The mentioned data augmentation adds
robustness in segmenting inflorescences photographed at different
distances, where far-located inflorescences appear unfocused on the
images and those closer to the camera exhibit higher contrast in the
separation between the flowers.
2.4.2. Individual flower detection
Three different methods were tested for flower quantification. Two
of them relied on the identification and counting of each individual
flower of the previously segmented inflorescences. For both, a seg-
mentation approach was followed. One of them used the SegNet
4
Computers and Electronics in Agriculture 178 (2020) 105796
architecture while the other one utilized the watershed algorithm. The
third method relied on estimating the number of visible flowers from
the area (in pixels) of the segmented inflorescences using a linear re-
gression model.
2.4.2.1. SegNet flower detection approach. For the SegNet approach a
VGG19 architecture was employed. A set of 35 images containing only a
group of flowers per image were manually labelled into three classes:
“contour”, “center” and “background”, which corresponded to the
contour of the flowers, the center, and the rest of the image,
respectively (Fig. 6). The data augmentation from the previous step
was also applied to obtain a set of 700 images, and this set was used to
train the VGG19 model.
After segmentation, a false positive filtering of the flower segmen-
tation was employed. It was based on considering as true positive those
flowers whose center and contour were segmented, and their contour
was surrounding the center above a certain threshold. It was observed
that, in the raw segmentation produced by the model, the structures of
pixels that did not had pixels of the “center” class or a small group of
the “contour” class, tended to be wrong artifacts that belonged to the
“background” class (Fig. 7). Several threshold values of pixels from the
“contour” class surrounding pixels from the “center” class were tested
for removing these artifacts, ranging from no filtering (no removal of
any pixel groups) to removing groups with less than 80% of the peri-
meter of their center surrounded by “contour” class pixels (see Table 2)
F. Palacios, et al.
Fig. 4. Manual labelling of inflorescences: a) Original image patch and b) the
corresponding manually labelled mask, where pixels containing inflorescences
are represented with white colour.
2.4.2.2. Watershed flower segmentation approach. The watershed
algorithm (Meyer, 1994) segments an image into several pixel blocks
of different sizes and shapes. This segmentation is performed by
“flooding” the local minima in the image until a maximum is
reached. Previously, the local minima corresponding to flower centers
had to be detected (Fig. 8). The extended-minima transform, which is
described as the regional minima of the h-minima transform (Soille,
1999), was applied in order to extent the local minima corresponding to
flower centers and suppress the rest of them, so as to reduce the
“oversegmentation” that produces the watershed segmentation
(excessive number of segmented blocks). Several h values for the the
h-minima transform were considered and their performance was tested.
2.4.2.3. Linear model estimation approach. A linear regression model
was tuned using the number of pixels of the segmented groups of
flowers from the set of 35 images used in the SegNet flower detection
approach to estimate the number of visible flowers. Mathematically this
is represented as y = ax + b, where y represents the number of visible
flowers and x the area in pixels. This method is a simpler approach than
the SegNet and watershed detection methods. While this method had
the advantage of not needing distinguishable flowers in the image (it
only needs properly segmented group of flowers), it also had the issue
of not being scale invariant, this means that the model will estimate
different numbers of visible flowers for a given group of flowers
depending on their distance to the camera. The estimation will be
5
Computers and Electronics in Agriculture 178 (2020) 105796
higher if the group is closer to the camera (as it would look larger on the
image) and it will be lower if the group is farther to the camera (as it
would look smaller on the image).
2.4.2.4. Full image processing. As it was mentioned, the two previous
steps aimed at analysing image regions. Full image processing was
achieved by a sliding window approach, where on each iteration a
480 × 480 image patch was extracted and processed by the first SegNet
for the inflorescence extraction. A 50% percentage overlap ratio
between each extracted patch and the one of the previous iteration
was considered (horizontally and vertically) in order to add robustness
to the inflorescence segmentation (Fig. 9).
The SegNet model obtained for each pixel of the extracted patches a
confidence score in the range [0,1] for the inflorescence class. A full
confidence score image was generated by joining the confidence score
patches and averaging the values of the overlapped regions between
patches. Then, an automatic confidence threshold was calculated in
each image using the Otsu method (Otsu,1979). Those pixels with a
confidence score above the threshold were considered inflorescence
pixels (Fig. 10). After obtaining a mask of the full resolution image
containing only the segmented inflorescences, each inflorescence was
processed individually in the flower detection and quantification step.
2.4.2.5. Evaluation metrics. The classification metrics considered for
evaluating the inflorescence segmentation results were: recall,
precision, F1 score and Intersection over Union (IoU). These metrics
are mathematically represented as
TP
Precision =
TP + FP (1)
TP
Recall =
TP + FN (2)
Precision × Recall
F1 = 2 ×
Precision + Recall (3)
TP
IoU =
TP + FP + FN (4)
The IoU metric was calculated for both the “inflorescence” and
“background” classes, and identified as IoUI and IoUB respectively.
Also, the average between IoU of both classes was calculated and
identified as mIoU. To compare the performance of the values tested for
the parameters in the flower detection step (contour filtering and h
value, for SegNet and watershed approaches, respectively), only recall,
precision and F1 score were considered. For evaluating TP, FP and FN
on this step, the flowers centers were manually checked. If a connected
component of pixels segmented as flower class surround the manually
checked flower center, that connected component is considered a true
positive and that flower cannot be detected again. In that sense, if
flower detection method has segmented several pixel groups on the
same flower, only one of them will be considered a true positive while
the rest will be false positives. Also, if a flower center is not surrounded
by any connected component of pixels segmented as flower class, that
flower was considered a false negative.
The linear model estimator relies on a regression approach to obtain
the number of visible flowers. Therefore, the previous described metrics
cannot be applied. The root-mean-square error (RMSE) metric between
the number of manually labelled flowers and the number of visible
flowers estimated was considered instead for this model, and it was
compared to the RMSE obtained between the number of manually la-
belled flowers and the number of detected flowers in the SegNet and
watershed approaches. Also, the normalized root-mean-square error
(NRMSE) was employed. It was calculated as the ratio of RMSE over the
average of the number of manually labelled flowers, expressed as a
percentage.
F. Palacios, et al.
Fig. 5. Data augmentation: a) Original image patch, results of the convolution with a gaussian kernel considering b) σ = 2 and c) σ = 4, and applying d) a contrast
enhancement using CLAHE algorithm.
Fig. 6. Manual labelling of individual flowers: a) Original image patch and b)
the corresponding manually labelled mask, where pixels in red colour re-
presents flower contours and pixels in blue colour represents flower centers.
2.4.2.6. Image processing validation set. A set of 24 full-resolution
(6720 × 4480 pixels) images, six per variety, were manually labelled
to evaluate the performance of the whole algorithm. For each image,
two masks were tagged, one contained the ground truth of the
segmentation of the inflorescences (to evaluate the performance of
6
Computers and Electronics in Agriculture 178 (2020) 105796
the inflorescences extraction step), and the other one contained the
centers of the visible flowers, to evaluate the flower’s detection and
quantification step (Fig. 11).
In addition, the whole set of images acquired from the 96 vines were
processed and the results for each vine were grouped, in order to obtain
a number of flowers per vine, which were then compared to the actual
number of flowers estimated by the algorithm described by Millan et al.
(2017).
3. Results and discussion
3.1. Inflorescences’ extraction
It can be observed in Table 1 that both models yielded similar re-
sults in terms of IoU for the background class (IoUB). However, for all
other metrics the VGG19 model outperformed the VGG16 model, this
being particularly evident for F1 and the IoU. The VGG19 model
overcame the FCN-segmentation model presented by Rudolph et al.
(2019) in terms of IoU, where a mean Intersection Over Union (mIoU)
of 0.876 and a IoU of 0.760 for the inflorescence class were obtained,
while the VGG19 model presented in this work achieved a mIoU of
0.935 and a IoU of 0.871 for the inflorescence class.
Fig. 7. Filtering of flowers segmented by SegNet:
a) Original image patch, b) SegNet raw semantic
segmentation and c) the segmented flower centers
retained after removing pixel groups without
“center” class or with less than 50% of the peri-
meter of their “center”class surrounded by “con-
tour” class pixels. Pixels in red colour represents
pixels of “contour” class and pixels in blue colour
represents pixels of “center” class.
F. Palacios, et al.
Fig. 8. Watershed flower segmentation: a) Original image patch, and b), c) and d) corresponding watershed segmentation masks considering several h values, where
pixels in white colour represent the blocks segmented by the watershed algorithm.
Fig. 9. Patch extraction and overlapping: Two consecutive image patches ex-
tracted from a full resolution image for the segmentation of the inflorescences,
represented in blue and red colour. The overlapping region is represented with
yellow stripes.
3.2. Individual flower detection and quantification
For the VGG19 flower segmentation model, several threshold values
were tested for the flower contour filtering. As shown in Table 2, in-
creasing the filtering led to an enhancement in precision and a decrease
in recall. Of the contour filtering values tested, the highest recall was
obtained when no filtering was applied, while the highest precision was
obtained when the segmented flowers with less than an 80% of the
contour segmented were removed. The balance between precision and
recall is determined by the F1 score. This metric achieved its highest
value when the contour filtering was set at the 50%, yielding a F1 score
of 0.729.
In the watershed approach, several h values from the h-minima
transform were tested for the raw images (see Table 3), and after ap-
plying contrast enhancement using the CLAHE algorithm (see Table 4).
Similarly to the SegNet approach, increasing the h value produced an
increase in precision and a decrease in recall. In general terms for the F1
score, this approach yielded poorer results than the SegNet approach.
Highly similar results were found between the segmentations without
preprocessing and after applying the CLAHE algorithm, being slightly
superior the first one. The best result was obtained at fixing the h value
to 0.05, getting a F1 score of 0.708 (see Table 3).
Comparing the two flower detection methods, the SegNet method
with 50% of contour filtering achieved better results than the wa-
tershed method with 0.05 of h value in terms of F1 score and precision,
while recall was higher for watershed. This implies that the SegNet
7
Computers and Electronics in Agriculture 178 (2020) 105796
method is more precise than the watershed method, but it is also more
restrictive, as it detected a lower number of flowers.
The linear model estimator differs from the previous detection
methods (SegNet and Watershed) on being a regression approach in-
stead of a classification technique. Therefore, RMSE and NRMSE me-
trics were chosen instead, to compare the linear model estimator
against the flower detection methods. The results presented in Table 5
show that the linear model yielded its best result for Cabernet Sau-
vignon variety with a RMSE of 111 flowers, which corresponds to
10.17% of the average of the number of manually labelled flowers, and
a poorer result for Muscat of Alexandria, with a RMSE of 437 flowers
and a NRMSE of 48.95%. For the watershed approach, while the lower
RMSE was obtained for Syrah, 170 flowers, in terms of NRMSE the best
result was obtained for Tempranillo, with a 27.84%, while the lowest
performance was found for Malvasia. The SegNet method yielded the
most favourable outcomes for the Tempranillo variety, with a RMSE
and a NRMSE of 27 flowers and 4.43%, respectively, and performed less
satisfactorily for Verdejo, with a RMSE of 222 flowers and a NRSME of
19%.
In general terms, the SegNet was found to provide the best ap-
proach, as it yielded the best results considering each variety in-
dependently and combining all varieties in the full test set. The dif-
ferences in the performance between varieties could be related to
differences in the morphology and the number of flowers per in-
florescence. In general terms, those varieties with higher number of
flowers per inflorescence (see Table 6), such as Verdejo, Malvasia and
Muscat of Alexandria, were associated with lower performances for all
three detection and quantification methods (see Table 5). Potentially,
higher occlusion effects among flower buttons in those varieties whose
inflorescences are naturally bearing larger number of flowers would
lead to lower performance in flower detection and quantification. The
results in terms of F1-Score in this work, with a value of 0.729, are
similar to the ones obtained by the circular Hough transform approach
presented by Rudolph et al. (2019), who reported a value of 0.752, and
inferior to the results obtained by Grimm et al. (2019), who obtained a
F1-Score of 0.867, where a deep learning semantic segmentation ap-
proach was also applied. In contrast to those works, in which all images
were manually taken, in this study the image acquisition process was
automated, the camera was positioned at a constant distance in order to
capture as much of each vine as possible, and its parameters remained
fixed for all images. That also implied that the focus of the camera
remained constant during the image acquisition, which made it difficult
to obtain clear images of all inflorescences. Likewise, some of the in-
florescences in the images were blurred and their flowers were hardly
distinguishable, which affected the presented results.
3.3. Early yield prediction based on flower number
The results shown in Fig. 12 refer to the high correlation between
F. Palacios, et al. Computers and Electronics in Agriculture 178 (2020) 105796

Fig. 10. Inflorescence segmentation confidence score and thresholding: a) A section of the original image, b) the corresponding confidence score image, where a
lighter colour represents a higher confidence score, and c) final segmentation, where the pixels that exceeded the threshold are represented in purple.

Fig. 11. Masks manually labelled for testing the full processing algorithm's performance: a) A section of the original image and b) and c) the manually labelled masks
containing inflorescences and the flower’s centers, respectively.

Table 1
Inflorescences segmentation performance metrics for VGG16 vs. VGG19 architecture calculated over the image validation set and expressed as the average value of
the metric ± standard deviation in percentage.
Architecture Precision Recall F1-Score IoUI IoUB mIoU

VGG16 0.928 ± 2.90% 0.839 ± 5.65% 0.880 ± 3.49% 0.788 ± 5.58% 0.998 ± 0.09% 0.893 ± 2.80%
VGG19 0.974 ± 1.88% 0.893 ± 5.66% 0.930 ± 3.15% 0.871 ± 5.45% 0.999 ± 0.06% 0.935 ± 2.74%

IoUI: Intersection over Union for “inflorescence” class; IoUB: Intersection over Union for “background” class; mIoU: average between Intersection over Union for
“inflorescence” and “background” classes

8
F. Palacios, et al. Computers and Electronics in Agriculture 178 (2020) 105796

Table 2 Table 6
Flower detection performance metrics for SegNet-VGG19 architecture calcu- Statistical metrics of the number of flowers in each inflorescence per variety
lated over the image validation set and expressed as the average value of the calculated over the validation set of 24 full-resolution images.
metric ± standard deviation in percentage (best result is in bold face).
Grapevine variety Statistical metric
Contour filtering Precision Recall F1-Score
Min Max Average Median Std Deviation
No Filtering 0.567 ± 11.42% 0.827 ± 6.30% 0.663 ± 7.68%
35% 0.696 ± 8.39% 0.768 ± 6.62% 0.726 ± 5.74% Cabernet Sauvignon 15 763 357.04 366 160.67
50% 0.747 ± 7.53% 0.718 ± 7.03% 0.729 ± 5.81% Malvasia 44 771 364.53 361 154.50
65% 0.826 ± 5.81% 0.622 ± 8.05% 0.707 ± 6.42% Muscat of Alexandria 126 776 391.10 383 131.46
80% 0.869 ± 4.39% 0.466 ± 8.82% 0.603 ± 8.17% Syrah 28 489 172.09 169 71.66
Tempranillo 38 478 237.27 226 98.99
Verdejo 6 1331 510.03 524 216.97

Table 3
Flower detection performance metrics for Watershed calculated over the image
validation set and expressed as the average value of the metric ± standard
deviation in percentage (best result is in bold face).
h value Precision Recall F1-Score

0.01 0.334 ± 6.23% 0.942 ± 4.76% 0.488 ± 6.41%

0.05 0.693 ± 13.69% 0.780 ± 14.75% 0.708 ± 7.06%
0.10 0.836 ± 9.01% 0.614 ± 19.10% 0.678 ± 13.58%
0.15 0.901 ± 5.72% 0.462 ± 18.90% 0.580 ± 17.64%
0.20 0.929 ± 3.50% 0.330 ± 15.70% 0.464 ± 17.23%
0.25 0.941 ± 2.63% 0.231 ± 11.42% 0.355 ± 14.50%
0.27 0.945 ± 2.35% 0.200 ± 9.91% 0.318 ± 13.10%
0.30 0.944 ± 2.76% 0.162 ± 7.90% 0.268 ± 10.97%
0.33 0.948 ± 2.89% 0.133 ± 6.19% 0.228 ± 9.00%
0.35 0.946 ± 3.10% 0.118 ± 5.27% 0.206 ± 7.85%

Table 4
Flower detection performance metrics for Watershed with CLAHE preproces-
sing calculated over the image validation set and expressed as the average value
Fig. 12. Correlation between the number of automatically detected flowers by
of the metric ± standard deviation in percentage (best result is in bold face).
the algorithm, using the SegNet-VGG19 for inflorescences’ extraction and
h value Precision Recall F1-Score flower detection steps, and the real number of flowers, obtained by the esti-
mator of Millan et al. (2017), per vine and combining all varieties.
0.01 0.325 ± 5.87% 0.943 ± 4.69% 0.479 ± 6.15%
0.05 0.676 ± 13.08% 0.791 ± 14.30% 0.705 ± 6.78%
0.10 0.829 ± 8.77% 0.627 ± 18.67% 0.686 ± 12.97% the number of flowers detected by the developed algorithm, using the
0.15 0.895 ± 5.40% 0.477 ± 18.63% 0.594 ± 17.18% SegNet-VGG19 for inflorescences’ extraction and flower detection steps,
0.20 0.928 ± 3.47% 0.350 ± 15.52% 0.485 ± 16.86%
0.25 0.939 ± 2.70% 0.242 ± 11.34% 0.371 ± 14.20%
and the total number of flowers estimated by the model of Millan et al.
0.27 0.944 ± 2.59% 0.210 ± 9.80% 0.332 ± 12.86% (2017). The correlation per variety between the two parameters is
0.30 0.945 ± 2.75% 0.168 ± 7.81% 0.278 ± 10.83% shown in Fig. 13, where the R2 metric is close to or above 0.75 in all
0.33 0.946 ± 3.03% 0.138 ± 6.10% 0.235 ± 8.87% cases.
0.35 0.949 ± 2.94% 0.122 ± 5.29% 0.212 ± 7.90%
These results suggest that a linear model can be applied to estimate
CLAHE: Contrast-limited adaptive histogram equalization. the total number of flowers per vine from the number of automatically
detected flowers by the developed algorithm, under conditions of par-
Table 5 tial defoliation of the vines, which calls for a parsimonious approach. In
Flower detection and quantification methods comparative calculated over the this work partial defoliation was conducted just before image acquisi-
validation set of 24 full-resolution images for each variety independently and tion, as described previously. This implied the removal of one or two (at
combining all varieties in the full test set. most) leaves per shoot to favour the visibility of the inflorescences. It
could be thought that this leaf removal around flowering might nega-
Grapevine Method
variety tively impact fruit-set (leading to reduced fruit-set), as it is described in
LM Watershed (h = 0.05) SegNet-VGG19 most early defoliation works (Poni et al., 2006; Tardaguila et al., 2010).
(CF = 50%) However, in these works, the intensity of manual early defoliation
leading to significantly reduced fruit-set, hence yield, was at least four
RMSE NRMSE RMSE NRMSE RMSE NRMSE
to eight leaves per vine (Diago, 2010), and involved lateral removal as
Cabernet 111 10.2% 398 36.6% 103 9.5% well in some cases (Intrieri et al., 2008). Therefore, despite the fact that
Sauvignon the effect of removing one or two leaves per shoot prior to image ac-
Malvasia 238 30.8% 650 83.9% 90 11.6% quisition in fruit set and final yield should be properly assessed, existing
Muscat of 437 48.9% 441 49.5% 105 11.8%
Alexandria
literature about early defoliation suggests that no significant effect in
Syrah 149 27.0% 170 30.9% 73 13.3% yield should be expected. Two cross-validation methods were applied to
Tempranillo 126 20.5% 171 27.8% 27 4.4% test the performance of the linear regression approach for the estima-
Verdejo 181 15.4% 459 39.3% 222 19.0% tion of the total number of flowers using the number of detected
Average 207 25.5% 382 44.7% 103 11.6%
flowers. A 6-fold cross validation, where each test fold corresponded to
Full test set 235 27.7% 417 49.2% 119 14.1%
the whole set of 16 vines from a variety (therefore, training with 5
LM: Linear model; CF: Contour filtering. varieties and leaving the remaining for test), and leave-one-out, where
each test fold corresponded to one vine (therefore, training with 95

9
F. Palacios, et al. Computers and Electronics in Agriculture 178 (2020) 105796

Fig. 13. Correlation between the number of automatically detected flowers by the algorithm, using the SegNet-VGG19 for inflorescences’ extraction and flower
detection steps, and the real number of flowers, obtained by the estimator of Millan et al. (2017), per vine for each variety: a) Cabernet Sauvignon, b) Malvasia, c)
Muscat of Alexandria, d) Syrah, e) Tempranillo and f) Verdejo.

Table 7 validation methods. Averages of 26.4% and 28.5% of NRMSE were

Performance of the actual number of flowers estimation considering two cross obtained for leave-one-out and 6-fold cross validation methods, re-
validation methods for each variety independently and combining all varieties spectively, considering each variety individually, while values of 23.7%
in the full test set. and 24.8% of NRMSE were obtained combining all varieties in a full
Grapevine variety Cross validation method dataset. The worst results were obtained for Tempranillo variety, with
NRMSE values of 47.4% and 55.2%. While the best ones were obtained
Leave-one-out 6-fold for Verdejo variety, with NRMSE values of 16.8% and 17.0%.
The results presented in Fig. 14 shows that there is a strong corre-
RMSE NRMSE RMSE NRMSE
lation between the number of flowers estimated by the linear model
Cabernet Sauvignon 562 17.5% 562 17.5% estimator and the final yield at harvest considering each variety in-
Malvasia 740 32.0% 772 33.4% dividually.
Muscat of Alexandria 484 20.4% 498 21.1%
The results shown in Fig. 14 encourage the development of a
Syrah 286 24.6% 314 27.1%
Tempranillo 522 47.4% 608 55.2% method for the estimation of the total number of flowers. The final yield
Verdejo 796 16.8% 808 17.0% per vine will be related to its number of flowers, its fruit-set rate, and
Average 565 26.4% 594 28.5% the average berry weight. These last two parameters can vary widely
Full dataset 590 23.7% 617 24.8% from one variety to another and are highly dependent on the nutritional
status at flowering (Coombe, 1973), the environmental conditions in
previous and current seasons, mainly light, temperature and soil water
vines and leaving the remaining one for test), were considered. The
availability (May, 2005), and some canopy management operations like
results presented in Table 7 show a similar performance for both
shoot tipping (Krstic et al., 2005) or early defoliation (Diago, 2010).

10
F. Palacios, et al.
Fig. 14. Correlation between the actual number of flowers (estimated by the linear model) and the yield weighted per vine, for each grape variety: a) Cabernet
Sauvignon, b) Malvasia, c) Muscat of Alexandria, d) Syrah, e) Tempranillo and f) Verdejo.
However, the existing knowledge of historical data of fruit-set rates and
average berry weight for a given plot or variety in a certain area from
previous seasons, and the acquisition of additional records year after
year, may be a valuable source of data for yield estimation using the
automated flower counting described in this work.
These strong correlations suggest that final yield can be accurately
predicted per vine very early, near four months before harvest, using
their total number of flowers estimated from the number of flowers
detected in RGB images by an automated method based on computer
vision, assuming that no other environmental factors impair the natural
plant development (such as frost or hail damage). This opens a gateway
to a new earlier vineyard yield estimation approach, which would be
highly useful for grape growers. They could benefit from this knowl-
edge for an early decision making in the vineyard and harvesting
scheduling and logistics.
In this work, no yield prediction tests have been carried out as this
was beyond the scope of this work, but it’s part of on-going research.
11
Computers and Electronics in Agriculture 178 (2020) 105796
Other works have addressed the problem of early yield estimation using
information obtained from images acquired on field using a moving
platform and the correlation between the visible detected fruit and the
actual yield, although not at the flowering phenological stage. Aquino
et al. (2018), using data acquired at pea-size stage, multiplied the
number of berries detected and the average berry weight, calculated
from historical data from the previous season, to obtain the ‘visible’
yield per segment (defined in their work as three consecutive vines).
Then, the visible yield was correlated to the actual weighted yield in
order to obtain a linear model that was used to estimate the actual yield
per segment. They achieved an R2 of 0.74 between the number of de-
tected berries and the actual yield, which is a determination coefficient
similar to the ones presented in this work. Nuske et al. (2014) corrected
the number of detected berries using an equation to estimate de number
of visible berries, and used another equation to relate the visible fruit to
the total fruit and estimate the yield, including historical yield data in
the equation. They analyzed the correlations between the detected
F. Palacios, et al.
berry count and the actual yield per variety, and achieved determina-
tion coefficients (R2) between 0.60 and 0.73, which are also similar to
the ones obtained in this work.
The algorithm developed in this work was able to detect and
quantify the number of flowers on RGB images acquired at night-time
from a moving platform. This system could be improved by processing
images taken during day-time conditions, in order to increase the
working time limitation in commercial applications in the grape and
wine industry. Additional pre-processing and data augmentation tech-
niques should be tested to enhance the algorithm performance on those
images where inflorescences appear blurred.
For the estimation of the total number of flowers, several canopy
and defoliation conditions could be also tested for the analysis of the
influence of the occlusion phenomenon, incorporating this information
into more advanced machine learning models.
4. Conclusions
The results presented show that the algorithm developed in this
system was able to detect and quantify the number of grapevine flowers
in RGB images. These images were acquired under field conditions
using a mobile sensing platform.
The new developed algorithm was composed of two steps. In the
first step, the ROI containing only inflorescences in the image is seg-
mented. In the second step, the flowers are detected and quantified in
the ROI segmented in the previous step. The VGG19 trained model
achieved the best results in both steps
The number of flowers detected by this method proved to be highly
correlated to the actual number of flowers per vine. Finally, a linear
model was trained to estimate the actual number of flowers per vine,
and the estimation was correlated to the final yield at harvest for dif-
ferent variety. The strong correlations between the estimated actual
number of flowers and the final yield proved that the algorithm pre-
sented in this work is highly valuable for an early yield estimation near
four months before harvest.
CRediT authorship contribution statement
Fernando Palacios: Conceptualization, Methodology, Software,
Validation, Formal analysis, Investigation, Data curation, Writing -
original draft, Writing - review & editing. Gloria Bueno: Methodology,
Validation, Resources, Writing - review & editing, Supervision. Jesús
Salido: Methodology, Validation, Resources, Writing - review &
editing, Supervision. Maria P. Diago: Conceptualization, Resources,
Writing - review & editing, Supervision, Project administration. Inés
Hernández: Software, Data curation, Writing - review & editing. Javier
Tardaguila: Conceptualization, Validation, Resources, Writing - review
& editing, Supervision, Project administration, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influ-
ence the work reported in this paper.
Acknowledgements
Authors would like to thank Ignacio Barrio, Eugenio Moreda and
Saúl Río for their help collecting field data.
Funding
Fernando Palacios would like to acknowledge the research founding
FPI grant 286/2017 by Universidad de La Rioja, Gobierno de La Rioja.
Dr Maria P. Diago is funded by the Spanish Ministry of Science,
12
Computers and Electronics in Agriculture 178 (2020) 105796
Innovation and Universities with a Ramon y Cajal grant RYC-2015-
18429.
References
Aquino, A., Millan, B., Gaston, D., Diago, M.-P., Tardaguila, J., 2015a. vitisFlower®: de-
velopment and testing of a novel android-smartphone application for assessing the
number of grapevine flowers per inflorescence using artificial vision techniques.
Sensors 15, 21204–21218. https://doi.org/10.3390/s150921204.
Aquino, A., Millan, B., Gutiérrez, S., Tardáguila, J., 2015b. Grapevine flower estimation
by applying artificial vision techniques on images with uncontrolled scene and multi-
model analysis. Comput. Electron. Agric. 119, 92–104. https://doi.org/10.1016/j.
compag.2015.10.009.
Aquino, A., Millan, B., Diago, M.-P., Tardaguila, J., 2018. Automated early yield pre-
diction in vineyards from on-the-go image acquisition. Comput. Electron. Agric. 144,
26–36. https://doi.org/10.1016/j.compag.2017.11.026.
Badrinarayanan, V., Kendall, A., Cipolla, R., 2017. SegNet: A deep convolutional encoder-
decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell.
39, 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615.
Clingeleffer, P.R., 2001. Crop development, crop estimation and crop control to secure
quality and production of major wine grape varieties. A national approach. In: Final
report to the Grape and Wine Research and Development Corporation. Project n°CSH,
pp. 96/1..
Coombe, B.G., 1973. The regulation of set and development of the grape berry. In: Acta
Horticulturae. International Society for Horticultural Science (ISHS), Leuven,
Belgium, pp. 261–274. https://doi.org/10.17660/ActaHortic.1973.34.36.
Daimary, D., Bora, M.B., Amitab, K., Kandar, D., 2020. Brain Tumor Segmentation from
MRI Images using Hybrid Convolutional Neural Networks. Procedia Comput. Sci.
167, 2419–2428. https://doi.org/10.1016/j.procs.2020.03.295.
Diago, M.P., 2010. Estudio y desarrollo del deshojado precoz como técnica para el control
del rendimiento productivo de la vid (Vitis vinifera L.). Efectos sobre el desarrollo
vegetativo, los componentes de la producción, así como sobre la composición de la
uva y del vino. PhD Thesis. Universidad de La Rioja (Spain).
Diago, M.P., Sanz-Garcia, A., Millan, B., Blasco, J., Tardaguila, J., 2014. Assessment of
flower number per inflorescence in grapevine by image analysis under field condi-
tions. J. Sci. Food Agric. 94, 1981–1987. https://doi.org/10.1002/jsfa.6512.
Diago, M.P., Aquino, A., Millan, B., Palacios, F., Tardaguila, J., 2019. On-the-go assess-
ment of vineyard canopy porosity, bunch and leaf exposure by image analysis. Aust.
J. Grape Wine Res. 25, 363–374. https://doi.org/10.1111/ajgw.12404.
Dunn, G.M.; Martin, S.R. 2003. The current status of crop forecasting in the australian
wine industry. In Proceedings of the ASVO Seminar Series: Grapegrowing at the Edge,
Tanunda, Barossa Valley, South Australia, pp. 4–8.
Dunn, G.M., 2010. Yield forecasting. Australian Government: Grape and wine research
and development corporation. Fact sheet. June 2010. https://www.wineaustralia.
com/getmedia/5304c16d-23b3-4a6f-ad53-b3d4419cc979/201006_Yield-
Forecasting.pdf.
Font, D., Tresanchez, M., Martínez, D., Moreno, J., Clotet, E., Palacín, J., 2015. Vineyard
Yield estimation based on the analysis of high resolution images obtained with ar-
tificial illumination at night. Sensors 15, 8284–8301. https://doi.org/10.3390/
s150408284.
Grimm, J., Herzog, K., Rist, F., Kicherer, A., Töpfer, R., Steinhage, V., 2019. An adaptable
approach to automated visual detection of plant organs with applications in grape-
vine breeding. Biosyst. Eng. 183, 170–183. https://doi.org/10.1016/j.biosystemseng.
2019.04.018.
Herrero-Huerta, M., González-Aguilera, D., Rodriguez-Gonzalvez, P., Hernández-López,
D., 2015. Vineyard yield estimation by automatic 3D bunch modelling in field con-
ditions. Comput. Electron. Agric. 110, 17–26. https://doi.org/10.1016/j.compag.
2014.10.003.
Howell, G.S., 2001. Sustainable grape productivity and the growth-yield relationship: A
review. Am. J. Enology Viticulture 52, 165–174.
Intrieri, C., Filipetti, I., Allegro, G., Centinari, M., Poni, S., 2008. Early defoliation (hand
vs mechanical) for improved crop control and grape composition in Sangiovese (Vitis
vinifera L.). Aust. J. Grape Wine Res. 14, 25–32. https://doi.org/10.1111/j.1755-
0238.2008.00004.x.
Kicherer, A., Klodt, M., Sharifzadeh, S., Cremers, D., Töpfer, R., Herzog, K., 2017.
Automatic image-based determination of pruning mass as a determinant for yield
potential in grapevine management and breeding. Aust. J. Grape Wine Res. 23,
120–124.
Kliewer, W.M., Casteel, T., 2003. Canopy Management. In: Hellman, E. (Ed.), Oregon
Viticulture. Oregon State University Press, Corvallis, OR, pp. 177–184.
Krstic, M., Clingeleffer, P., Dunn, G., Martin, S., Petrie, P., 2005. Grapevine growth and
reproduction: an overview, in: Transforming Flowers to Fruit. In: Proceedings ASVO
Conference, pp. 7–10.
Liu, S., Whitty, M., 2015. Automatic grape bunch detection in vineyards with an SVM
classifier. J. Appl. Logic 13, 643–653. https://doi.org/10.1016/j.jal.2015.06.001.
Liu, S., Cossell, S., Tang, J., Dunn, G., Whitty, M., 2017. A computer vision system for
early stage grape yield estimation based on shoot detection. Comput. Electron. Agric.
137, 88–101. https://doi.org/10.1016/j.compag.2017.03.013.
Liu, S., Li, X., Wu, H., Xin, B., Tang, J., Petrie, P.R., Whitty, M., 2018. A robust automated
flower estimation system for grape vines. Biosyst. Eng. 172, 110–123. https://doi.
org/10.1016/j.biosystemseng.2018.05.009.
Long, J., Shelhamer, E., Darrell, T., 2015. Fully convolutional networks for semantic
segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), pp. 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965.
F. Palacios, et al.
Martin, R.D.S., Dunn, G., 2003. How to forecast wine grape deliveries. Technique report,
Department of Primary Industries.
May, P., 2005. Flowering and fruitset in grapevines. Lythrum Press, Adelaide (Australia).
Meyer, F., 1994. Topographic distance and watershed lines. Signal Process. 38, 113–125.
https://doi.org/10.1016/0165-1684(94)90060-4.
Millan, B., Aquino, A., Diago, M.P., Tardaguila, J., 2017. Image analysis-based modelling
for flower number estimation in grapevine. J. Sci. Food Agric. 97, 784–792. https://
doi.org/10.1002/jsfa.7797.
Millan, B., Velasco-Forero, S., Aquino, A., Tardaguila, J., 2018. On-the-go grapevine yield
estimation using image analysis and boolean model. Journal of Sensors 2018,
9634752. https://doi.org/10.1155/2018/9634752.
Nuske, S., Wilshusen, K., Achar, S., Yoder, L., Narasimhan, S., Singh, S., 2014. Automated
visual yield estimation in vineyards. J. Field Rob. 31, 837–860. https://doi.org/10.
1002/rob.21541.
Otsu, N., 1979. A threshold selection method from gray-level histograms. IEEE Trans.
Syst., Man, Cybernetics 9, 62–66. https://doi.org/10.1109/TSMC.1979.4310076.
Palacios, F., Diago, M.P., Tardaguila, J., 2019. A non-invasive method based on computer
vision for grapevine cluster compactness assessment using a mobile sensing platform
under field conditions. Sensors 19. https://doi.org/10.3390/s19173799.
Poni, S., Casalini, L., Bernizzoni, F., Civardi, S., Intrieri, C., 2006. Effects of Early
Defoliation on Shoot Photosynthesis, Yield Components, and Grape Composition.
Am. J. Enology Viticulture 57, 397–407.
Radhouane, B., Derdour, K., Mohamed, E., 2016. Estimation of the flower buttons per
inflorescences of grapevine (vitis vinifera L.) by image auto-assessment processing.
13
Computers and Electronics in Agriculture 178 (2020) 105796
Afr. J. Agric. Res. 11, 3203–3209. https://doi.org/10.5897/AJAR2016.11331.
Rudolph, R.I., Herzog, K., Töpfer, R., Steinhage, V., 2019. Efficient identification, loca-
lization and quantification of grapevine inflorescences and flowers in unprepared
field images using Fully Convolutional Networks. Vitis: J. Grapevine Res. 58, 95–104.
https://doi.org/10.5073/vitis.2019.58.95-104.
Ruiz-Santaquitaria, J., Pedraza, A., Sánchez, C., Libreros, J.A., Salido, J., Deniz, O.,
Blanco, S., Cristóbal, G., Bueno, G., 2019. Deep Learning Versus Classic Methods for
Multi-taxon Diatom Segmentation. In: Morales, A., Fierrez, J., Sánchez, J.S., Ribeiro,
B. (Eds.), Pattern Recognition and Image Analysis. Springer International Publishing,
Cham, pp. 342–354.
Siam, M., Elkerdawy, S., Jagersand, M., Yogamani, S., 2017. Deep semantic segmentation
for automated driving: Taxonomy, roadmap and challenges. In: 2017 IEEE 20th
International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8.
Simonyan, K., Zisserman, A., 2015. Very deep convolutional networks for large-scale
image recognition. International Conference on Learning Representations.
Soille, P., 1999. Morphological image analysis. Springer, Berlin Heidelberg, pp. 170–171.
Tardaguila, J., Martinez de Toda, F., Poni, S., Diago, M.P., 2010. Impact of Early Leaf
Removal on Yield and Fruit and Wine Composition of Vitis vinifera L. Graciano and
Carignan. Am. J. Enology Viticulture 61, 372–381.
Tello, J., Herzog, K., Rist, F., This, P., Doligez, A., 2019. Automatic flower number eva-
luation in grapevine inflorescences using RGB images. Am. J. Enology Viticulture.
https://doi.org/10.5344/ajev.2019.19036.
Zuiderveld, K., 1994. Contrast limited adaptive histogram equalization. In: Graphics
Gems IV. Academic Press Professional Inc, USA, pp. 474–485.