New and Emerging Digital Technologies for

Digital Agriculture Using Artificial

Intelligence (AI)

Digital Agriculture, Food and Wine Group (DAFW)

School of Agriculture and Food
Faculty of Veterinary and Agricultural Sciences
The University of Melbourne

Assoc. Professor Sigfredo Fuentes

sfuentes@unimelb.edu.au
Dr. Claudia Gonzalez Viejo Dr. Eden Tongson
DAFW Research Group
The University of
Melbourne

What is the difference

between
Precision Agriculture (PA)
and
Digital Agriculture (DA)?
 Digital
The vineyard of the future initiative
Agriculture www.vineyeardofthefutue.com

2008-current
Fuentes, S., & De Bei, R. 2016.
Advances of the Vineyard of the
Future Initiative in viticultural,
sensory science and technology
development. Wine and
viticulture journal (3), 13.

Photo: James Morgan

Photo: Sonja Needs
DAFW Research
Group
Climate Change
Research
DAFW
Other AI research

Automated Urban Monitoring Systems

Fuentes, S., Tongson, E., & Gonzalez Viejo, C. 2021. Urban Green Infrastructure
Monitoring Using Remote Sensing from Integrated Visible and Thermal Infrared
Cameras Mounted on a Moving Vehicle. Sensors, 21(1), 295.
DAFW
Other AI research

Non-Invasive Animal Biometrics for

Welfare Assessment and Traceability
Respiration rate: Eye and body temperature:
Fuentes S., Gonzalez Viejo C., Tongson E.,
Lipovetzky N., Dunshea F.R. 2021. Biometric
and physiological responses from dairy cows
measured by visible remote sensing are good
predictors of milk productivity and quality
through artificial intelligence. Sensors. 21.
Animal ID – Traceability (Deep Learning)
Heart Rate: Video Magnification Analysis
DAFW
Other AI research
Integration of ID with Biometric AI models
AI and Digital Agriculture
for Climate Change
Research

Droughts, frosts, heatwaves Pests and diseases

Fuentes, S., & Jorge Gago. 2021. Book Chapter: Modern

Approaches to Precision and Digital Viticulture. Elsevier
Book: “Improving Sustainable viticulture and winemaking Bushfires and smoke taint
practices" Jan 2022. Compression of phenological
stages / Earlier harvests
DAFW Automatic Data Logs: Detection of Pests and Diseases
App Development

Small Animals Biometrics

Development and Validation of ML models Deployment of ML models

RGB – leaf phenotyping Data Processing
- Image analysis using a
customised Matlab codes.
(Protocol
Leaf phenotyping and
of 20 codes medicinal
Chinese are
plant speciesavailable upon request.
Conditions apply).
Purpose: Automated classification
(through machine learning)

Using an office scanner

• Leaf morphology
• Leaf colour
• Leaf margin fractal dimension

Xue, J., Fuentes, S., Poblete-Echeverria, C., Viejo, C. G., Tongson, E., Du, H., & Su, B. (2019). Automated Chinese
medicinal plants classification based on machine learning using leaf morpho-colorimetry, fractal dimension and
visible/near infrared spectroscopy. International Journal of Agricultural and Biological Engineering, 12(2), 123-131.
 App Development
DAFW
App Development
VitiCanopy App (Free version for
Android & iOS), 10,000+ downloads VitiCanopy App
(Beta version) with
mapping and
History data
options

“The most used viticultural App for Canopy Management”. Ammoniaci, M., Kartsiotis, S. P., Perria, R.,
& Storchi, P. (2021). State of the Art of Monitoring Technologies and Data Processing for Precision
Viticulture. Agriculture, 11(3), 201.

De Bei, Roberta, Sigfredo Fuentes, Matthew Gilliham, Steve Tyerman, Everard Edwards, Nicolò Bianchini, Jason Smith, and Cassandra
Collins. "VitiCanopy: A free computer App to estimate canopy vigor and porosity for grapevine." Sensors 16, no. 4 (2016): 585.
DAFW
App Development
UAV – Visible (RGB) Canopy Architecture Analysis

Mapping aroma profiles of cocoa

• Canopy architecture using RGB

camera (LAI)
• NIR of bean samples
• Established a relationship on
how canopy architecture affect
the variability of different cocoa
aromas (through machine
learning modelling)
DAFW Infrared Thermography
App Development
Data processing
Image analysis using customised
a b
Matlab codes. Available upon
request.

Temperature (o C)

Temperature (oC)
- Thresholding
- Removal of non-leaf material
- Parameters are produced in
Excel

(a) Example of infrared thermal images obtained using reference leaves (middle right) and (b) filtered thermal
image using Twet and Tdry as thresholds. Dark blue colour in the filtered image corresponds to non-leaf or
exposed leaf material exclusion. Colour bar shows temperatures in oC for the Chardonnay trial. From Fuentes et
al. (2012).

Crop Water Stress Index

Tc is canopy temperature (oC) is the actual temperature obtained from the thermal image
𝑇! − 𝑇"#$ Tdry and Twet are the reference temperatures (oC)
𝐶𝑊𝑆𝐼 =
𝑇%&' − 𝑇"#$
 Plant water stress
estimation in
nectarine orchard

UAV-borne remote sensing (thermography) for water

status monitoring

Park, S., Ryu, D., Fuentes, S., Chung, H., O’Connell, M., & Kim, J. (2021). Dependence of
CWSI-Based Plant Water Stress Estimation with Diurnal Acquisition Times in a Nectarine
Orchard. Remote Sensing, 13(14), 2775.
 DAFW
Infrared Thermography
App Development

CWSI
Data processing
• Multiple georeferenced image
Plant water using
stitching stressphotogrammetric
estimation
in nectarine
software orchard
• Image analysis using customised
UAVMatlab
bornecodes.
infrared thermal camera
• Produce a thematic map

Left: Adaptive CWSI map derived from midday UAV

remote sensing of a nectarine orchard.
Right: Detailed adaptive CWSI map with varying
irrigation level (ETc – crop evapotranspiration)

Park, S., Ryu, D., Fuentes, S., Chung, H., Hernández-Montes, E., & O’Connell, M. (2017). Adaptive estimation of crop water stress in nectarine
and peach orchards using high-resolution imagery from an unmanned aerial vehicle (UAV). Remote Sensing, 9(8), 828
DAFW
App Development
Near Infrared Spectroscopy
Smoke Contamination in Grapevine Canopies and
Berries, and Smoke Taint in Wines

Fuentes, S., Tongson, E. J., De Bei, R., Gonzalez Viejo, C., Ristic, R., Tyerman, S., & Wilkinson, K.
(2019). Non-invasive tools to detect smoke contamination in grapevine canopies, berries and
wine: A remote sensing and machine learning modeling approach. Sensors, 19(15), 3335.
 DAFW Hyperspectral imaging
App Development

Hypercube dataset
Also applied in
microscopic scale i.e.
in host-pathogen
interaction

Mishra, P., Asaari, M. S. M., Herrero-Langreo, A., Lohumi, S.,

Diezma, B., & Scheunders, P. (2017). Close range
hyperspectral imaging of plants: A review. Biosystems
Engineering, 164, 49-67.

Example of hyperspectral image acquired from a green leaf. (a) Stack of narrow band sub-images
forming a 3-D hypercube; (b) reflectance spectrum of a particular pixel.
DAFW
Defence Grants
DAFW BioSensory Computer App: Food Beverage and Packaging
Assessment.
App Development Further application with Medicine to Assess PTSD and Trauma

Biometrics:
a) Eye tracking – pupil dilation, fixations
b) Heart rate – rate, amplitude, frequency
c) Body temperature
d) Face expressions (sad, disgusted, contempt, neutral, angry,
happy, surprised); posture tracking, brain waves (alpha, beta,
gamma, attention, meditation, blinking zone
ML modelling to obtain:
Liking
1-FLIR AX8
Emotional response
2-Tablet PC
3-BioSensory App

Fuentes S, Gonzalez Viejo C, Torrico D, Dunshea F. Development of a BioSensory Computer Application to Assess Physiological and Emotional
Responses from Sensory Panelists. Sensors. 2018 Sep 5;18(9):2958.
 SPARKLING WINE

DAFW
bility in time. Data Table 1. Sparkling wines used for the study. Wines were selected based on price,
re contrasted vinification method and origin.
methods, such
hemical analysis Wine Method Label Tirage time (months)
ess wine quality

Robotics and sensors

wn that the 1 Transfer T2 24+
image analysis 2 Champenoise C2 24+
nd cost-effective
3 Champenoise C2 24+
tain parameters

Low-cost E-nose for aroma profile

amics that can be 4 Champenoise C2 24+
tes of sparkling 5 Transfer T1 9-22
m stability
ontent. This
oped within The
nitiative. A video of
6
7
Champenoise
Champenoise
C1
C2
9-22
24+ + detection of smoke taint, insects/diesasse, contaminants.
FIZZEye Robot for foamability, bubble size and sensory
be seen at www.
dpress.com.
8
9
Champenoise
Champenoise
C2
C1
24+
9-22 + UAV + machine learning
ODS
analysis estimation using AI
10
11
Champenoise
Champenoise
C1
C1
9-22
9-22
l
f 11 wines from 25
e robotic pourer
urne Wine Show
rch. All sparkling
Australia.

ion
Zeye-Robot) was
ed to normalise
glasses. The
amber to hold
as lifted by an
tor with a lift
s (Figure 1). The
ontrolled using an Figure 1. Schematic illustration of the robotic pourer (FIZZeye-Robot) and the
grammable board image analysis technique using Matlab®.
y). This program
ouring, either presenting natural effervescence (Liger- to form in the glass (foamability) and its
lly, by activating Belair et al. 2007). Thus, the glasses stability (Prud’homme and Khan 1996).
d pouring can be were etched to provide a continuous flow Several parameters were evaluated
al position for the Image Analysis
of bubbles and avoid random bubbling Sensors for gas release (pouring);
automatically in order to assess
to be tilted up
cal. By pressing • Foam stability
nucleation. The glasses were washed at
45oC for 30 minutes and blow-dried in
ProtoE-nose
foamability and foam stability. Those
parameters were evaluated by using
a first pour of
ed to the glass.
• Lifetime of foam
a dishwasher (Bosch Group, Stuttgart,
Germany). The etching and glass washing
• CO
image analysis algorithms developed
using Matlab, ver. 2013b (Mathworks, 2 Claudia Gonzalez Viejo, Sigfredo Fuentes, Amruta Godbole, Bryce Widdicombe, Ranjith R Unnithan: Development of a low-cost e-nose to
nd pour of 50mL
c bottle size and
• Foam drainability
technique was performed in order to • Alcohol
Inc, Matick. MA, USA). The images were assess aroma profiles: An artificial intelligence application to assess beer quality. Sensors and Actuators B Chemical 01/2020; 308.,
standardise the effervescence process analysed semi-automatically by selecting DOI:10.1016/j.snb.2020.127688
to the Arduino
e accuracy of the
• Bubble size / count
in the experiment. Dimensions and • Temperature (IR)
the foam limits (top and bottom) to
volumetric content of the standard glass account for foam and wine volumes using
bes the image • Wine color (RGB – CieLab)
were mathematically standardised and the normalised algorithm developed for
Fuentes, Sigfredo, Vasiliki Summerson, Claudia Gonzalez Viejo, Eden Tongson, Nir Lipovetzky, Kerry L. Wilkinson, Colleen Szeto, and Ranjith R.
ion and stability
incorporated in the customised code to the standardised glass: the algorithm Unnithan. "Assessment of Smoke Contamination in Grapevine Berries and Taint in Wines Due to Bushfires Using a Low-Cost E-Nose and an
g per sparkling Artificial Intelligence Approach." Sensors 20, no. 18 (2020): 5108.
assess foam morphometrics and volume transforms the set of pixels obtained
automatically Viejo,
from image Fuentes, S., Li, G.,by
C. G., analysis, the line corresponding to the height
Collmann, R., Condé, B., & Torrico, D. (2016). Development of a robotic pourer
ndardisation regardless of the camera used or its of the foam, for a particular image, into
position. constructed with ubiquitous materials,
volume of open
foam. hardware and
For each set of sensors
images, to assess beer foam quality using computer
ss influences the
(Liger-Belair vision and pattern recognitionone algorithms: RoboBEER.
picture was Food half
captured every Research International, 89, 504-513
o avoid glass Image capture and analysis a second. The automated procedure
l standard wine Images from the glass were captured recognises the boundaries of the foam
Condé, Bruna C., Sigfredo Fuentes,
at a rate of one per 0.5 seconds using an
Maeva Caron, Di Xiao, Richard Collmann, and Kate S. Howell. "Development
e glass) were and extracts metrics from this region
were Luigi IPEVO View 2 of a robotic
camera andSunnyvale,
(IPEVO, computer vision method to assess foam quality in sparkling wines." Food Control 71 (2017):
of interest (ROI). The plotted values of
ng glasses. The 383-392.
California, USA). The camera was the volume of foam (mL) versus time
im diameter connected directly to a laptop computer (seconds) are obtained from the data
and images were captured automatically extracted (Figure 2, see page 28). The
 Other techniques: Volatile Organic Compounds (VoC)
Insect Detection in Plants using e-nose + AI ML Regression Models:
LiCOR 6400 physiological
ML Classification Models
parameters as targets:
Detection:
Photosynthesis / Stomatal
Three days after
conductance / Transpiration
Infestation
E-nose to detect (96.5% Accuracy for
herbivore induced plant 4 levels of infestation)
volatiles

ML Regression Models:
Nber Aphids/Plant
R=0.97; b=0.97 (NIR)
R=0.99; b=0.97 (e-nose)

Fuentes, S., Tongson, E., Unnithan, R. R., & Gonzalez Viejo, C. 2021. Early
Detection of Aphid Infestation and Insect-Plant Interaction Assessment in Wheat
Using a Low-Cost Electronic Nose (E-Nose), Near-Infrared Spectroscopy and
Machine Learning Modeling. Sensors, 21(17), 5948.
Other techniques: Near Infrared Spectroscopy (NIR)
NIR for early detection
of pathogens
Verticilium Detection in Plants using NIR + AI

ML Classification Models ML Regression Models:

Detection: LiCOR 6400 physiological
Three days after parameters as targets:
Infestation Photosynthesis / Stomatal
(96.5% Accuracy for conductance / Transpiration
4 levels of infestation)

Mee-Yung Shin, Claudia Gonzalez Viejo, Eden Tongson, Tonya Wiechel, Paul
Taylor, and Sigfredo Fuentes , C. 2021. Early detection of Verticillium wilt of
potatoes using near-infrared spectroscopy and machine learning modeling.
Computers and Electronics in Agriculture, Submitted
 Digital Agriculture
Research
Vision:
Integration -
Deployment

Sigfredo Fuentes, Eden Tongson and Claudia Gonzalez Viejo. 2021. Novel digital technologies implemented in sensory science and consumer perception.
Current Opinions in Food Science. 41: 99-106
Faculty of Veterinary and
Agricultural Sciences
Digital Agriculture, Food and Wine
 New and Emerging Digital Technologies for
Digital Agriculture Using Artificial
Intelligence (AI)

Digital Agriculture, Food and Wine Group (DAFW)

School of Agriculture and Food
Faculty of Veterinary and Agricultural Sciences
The University of Melbourne

Assoc. Professor Sigfredo Fuentes

sfuentes@unimelb.edu.au
Dr. Claudia Gonzalez Viejo Dr. Eden Tongson