S07: Image-based Analysis of Water Content
Change in Tomato Cultivation
Shuto Namba, Junpei Tsuji, Masato Noto
Dept. of Electrical, Electronics and Information Engineering,
Kanagawa University, Yokohama, Japan
s-namba@nt.ee.kanagawa-u.ac.jp, {tsuji, noto}@kanagawa-u.ac.jp
Abstract—Water-stress cultivation is becoming widely used for
deliberately reducing the amount of irrigation and increasing
the sugar content of fruits. Deliberately reducing the amount of
irrigation requires experience and intuition of skilled farmers, so
it is difficult for new farmers to conduct water-stress cultivation.
New farmers need techniques to conduct such cultivation easily.
Under these circumstances, the importance of an irrigation-
control method for automating irrigation should be discussed.
To conduct irrigation control, we must determine the changes in
plant states. In this study, we clarified how much the body-water
content of plants changes due to irrigation. We also designed a
model for representing the body-water content of plants and their
response to irrigation using a hidden Markov model and discuss
whether it can be used for irrigation control. Such technology for
new farmers should be low in cost, so plant reaction to irrigation
is measured using optical flow on the basis of leaf wilt. Based on
the displacement vectors obtained from optical flow, we estimated
the body-water content of plants and evaluated whether it is
possible to determine the irrigation amount with our model.
The results from experiments conducted on the cultivation of
tomato plants revealed that the displacement vectors obtained
from optical flow under specific conditions reflects the change in
body-water content.
Index Terms—water-stress cultivation, irrigation, body-water
content of plants
I. I NTRODUCTION
Advanced cultivation techniques to improve the quality of
crops is spreading in Japan. However, as farmers age, the
agriculture turnover rates will increase, so there is concern
that the techniques of skilled farmers will be lost before new
farmers can inherit them. Therefore, research on the spread
of advanced cultivation techniques has been conducted. Such
studies have attempted to reproduce tacit knowledge of skilled
farmers based on environmental and image data [1]–[3].
Water-stress cultivation, which is done to deliberately re-
duce the amount of irrigation based on the experience and
intuition of skilled farmers, is known as an advanced culti-
vation technique. Worldwide, water stress due to drought is
recognized as the most dangerous stress to plants [4].
In Japan, however, it is used to produce fruits with high
sugar content and improve market value. To reproduce the
tacit knowledge of advanced cultivation techniques in order for
new farmers to easily inherit them, studies on estimating water
stress have been conducted. There are methods for measuring
soil-moisture content [5], [6] and body-water content of plants
directly [7] and indirectly [2], [3], [8]. However, how skilled
farmers carry out irrigation based on the estimated water
stress has not been clarified. We must reproduce water-stress
cultivation techniques carried out by experienced farmers for
new farmers to inherit them. The ultimate goal of our research
is to develop an automatic irrigation-control system that can
be used for water-stress cultivation. Therefore, we must clarify
the relationship between plants and irrigation.
Plants have an internal state called body-water content,
and changes in such content cause water stress. Unless we
clarify changes in the body-water content of plants, water-
stress control will remain difficult. Therefore, it is necessary to
design a model representing the body-water content of plants
and their response to irrigation. Image-processing techniques
to determine plant responses to irrigation should be passed
down to new farmers. There is no established method for
measuring the body-water content of plants from image data,
so it is necessary to determine the internal state of plants using
image-processing technology to achieve irrigation control.
In this study, we designed a model that involves image-
processing technology for measuring the body-water content
of plants and their response to irrigation. In this paper, we
discuss whether this model can be used for an irrigation-
control system that we aim to develop.
II. R ELATED WORK
A. Overview
The formalization of techniques used by experienced farm-
ers, such as automatic irrigation-control systems, and repro-
ducing these techniques through information communication
technology is known as smart agriculture. Smart agriculture
results in high agricultural productivity, cost reduction, and
safer food and working conditions [9] [10]. It consists of
systems based on the Internet of Things, sensors, cloud com-
puting, mobile computing, and big data analysis. To construct
an automatic irrigation-control system, sensing technology to
determine plant states is indispensable. As sensing technology
develops, controlling becomes easier with regard to plant
conditions. This section introduces how sensing technology
is used for agriculture.
B. Field Server
A field server is used for controlling and measuring field
information in open-field cultivation [1]. It consists of sensors,
computers, and communication devices as one module. By
setting up many field servers, a wireless network using built-in
wireless LAN is formed. With this wireless network, a field-
monitoring system for collecting data measured at each point
in real time is constructed. It can be used for managing soil-
moisture information and the growth process of agricultural
crops. Since a field server can be used as a communication
infrastructure, development of a new agricultural form is also
expected [11]. However, it is difficult for farmers to maintain
field servers due to their complex structure; thus, long-term
use is an issue.
C. Plant-disease Detection
Due to the recent spread of smartphones, they are expected
to be used for plant-disease detection. Since smartphones
can be easily introduced, disease diagnosis is becoming a
technology that anyone can use. As a result of deep learning
using a public dataset of diseased and healthy plant leaves, it
has become clear that a trained model can achieve an accuracy
of 99.35% for a test set [12]. However, estimation accuracy is
low in an environment that differs from that used for training.
Disease diagnosis is also used as a function of agricultural
robots [13]. Therefore, further development of precise sensing
technology is expected.
D. Estimation of Water Stress from Soil-moisture Content
Water stress is known to occur due to an imbalance between
the amount of water absorbed by plants and the amount of
water released. The soil-moisture content is closely related to
the amount of water absorbed by plants and measured using a
soil-moisture meter. A soil-moisture meter has a tensiometer
[5] for measuring moisture tension, which indicates the ability
of soil particles to absorb water, and a dielectric soil-moisture
meter [6] for measuring the volume occupied with water in a
certain volume of soil. Tensiometers are suitable for measuring
the amount of water that plants can use. However, such a
water-stress-assessment method has a trade-off of accurate
measurement but increased cost. Thus, easy introduction to
cultivation environments is difficult.
E. Estimation of Water Stress by using Pressure-chamber
Method
One method for assessing water stress from directly measur-
ing the water content in plants is the pressure-chamber method
[7]. It is a method of applying pressure to leaves collected as
samples and measuring the water from the leaves as the water
potential. The water potential indicates the moisture-holding
power of a plant and is used as an indicator of water stress.
Therefore, the pressure-chamber method can be used to assess
water stress with high accuracy. However, this method requires
collecting leaves, which puts a burden on farmers and increases
cost. There are also problems with its introduction to actual
cultivation environments in terms of operation.
F. Estimation of Water Stress by Stem-diameter Change
One method for assessing water stress by indirectly measur-
ing the body-water content of plants is to use stem diameter
[8], [14]. Changes in stem diameter strongly correlate with
the water potential, and it has been shown that such changes
are effective as an indicator of water stress. In addition,
the linear variable displacement transducer (LVDT) used to
measure stem diameter measures the contour of an object, so
it can be used to assess water stress nondestructively, without
contact, and continuously. However, as the stem hardens with
age and gradually stops changing, correlation between stem-
diameter change and water potential decreases with time. In
long-term cultivation, there is a limit to evaluating water stress
when measuring at the same location. Farmers need to change
the location of measuring instruments at regular intervals.
Furthermore, an expensive measuring device, e.g., an LVDT,
is required to measure minute changes in stem diameter. Thus,
it is expensive and difficult for ordinary farmers to introduce
easily.
G. Estimation of Water Stress by Using Leaf Wilt
Research using image-processing technology has been con-
ducted for estimating water stress. Such technology involves
using a camera that is easy to introduce, even for new
farmers, without using expensive sensors [2] [3]. Since image-
processing technology targets the appearance of plants, water
stress is estimated from the leaf wilt.
Studies on estimating water stress by monitoring the pro-
jected plant area by focusing on leaves have been reported [2].
When controlling the addition of water stress, it is desirable
that such stress be detected as soon as possible. From produc-
ing images of projected areas set to the horizontal plan 0, 30,
60, and 90◦ , it was found that water stress can be detected
more quickly in a projected area taken from 90◦ . Introduction
into an actual cultivation environment is difficult since the
image background is a blue sheet.
Optical flow, which is conducted for motion detection in
image processing, has been reported as a suitable technique in
actual cultivation environments [3]. There are correlation [15]
and gradient [16] methods for optical flow detection. With
the gradient method, it is assumed that the brightness of the
corresponding point between images does not change because
of object movement. It is possible to detect even a small
change in the motion of an object to be tracked. According
to the study conducted by Kaneda [3], water stress can be
estimated from wilted leaves using optical flow.
III. P ROPOSED M ODEL
A. Plant Model for Irrigation-control System
Controlling irrigation based on estimated water stress is
indispensable for new farmers to carry out water-stress cul-
tivation. When considering the passing on of techniques to
new farmers, the cost of establishing an irrigation-control
method should be low. We used image-processing technology
to measure without leaf wilt using expensive sensors for water-
stress cultivation and optical flow that can be used to detect
the withering of small leaves.
Several studies have proved that water stress on plants can
be estimated from leaf wilt. It is thought that controlling leaf
movement due to irrigation may lead to effective management
of water stress on plants. The following three items are
necessary to make irrigation control on plants possible under
water stress.
1) To design a model representing the internal state of
plants and their response to irrigation.
2) To observe plant response to irrigation by using a sensor
system.
3) To determine the appropriate amount of irrigation based
on information from the sensor system.
The proposed model based on the above items is described
below.
B. Model Design
Regarding item 1), the amount of water content in plants can
be considered as a factor for determining the state outside the
plant. The amount of water content in plant changes over time,
and the current body-water content is determined from the past
body-water content and amount of irrigation. Our plant model
is a hidden Markov model. Let Xt be the body-water content
of plant at time t, Ut the amount of irrigation at time t, and
Yt leaf wilt appearing the plants, Xt and Yt can be formulated
as follows.
Xt = f (Xt−1 , Ut−1 , wt−1 ) (1)
Yt = g(Xt , εt ) (2)
The notations wt−1 and εt are stochastic variables that define
stochastic factors.
C. Measurement with Sensor System
Regarding item 2), we focused on leaves by using agricul-
tural image data, so we considered the relationship between
Yt and the image data It from observing Yt . It is difficult to
directly obtain the information of Yt from It , and only partial
information of Yt is obtained.
It = P roj(Yt ) (3)
Furthermore, optical flow uses the difference between image
data It−1 and It , the following expression is obtain.
rt = Opt(I0 , I1 , . . . , It ) (4)
Let rt be the reaction of plant by irrigation. Therefore, the
information of Xt at each time is represented by rt through the
filter by g, P rog, Opt, and Xt can be estimated by calculating
rt .
D. Determination of Appropriate Irrigation
Regarding item 3), on the basis of Ut to maintain an
appropriate Xt , rt has the following problems.
• It is not known whether rt can be used to detect plant
response to irrigation.
• There is no known appropriate design method of rt .
We designed rt independently as an index for indirectly
measuring body-water content under irrigation conditions such
as the amount and timing of irrigation.
For the remainder of this paper, we refer to rt as the
recovery trend index (RTI), which is calculated using the
displacement vectors obtained from conducting optical flow at
times t, time t + ∆. The case in which Yt is moving upward
is considered a state in which plant leaves are recovering.
Therefore, the RTI is required as follows. A set of whole
displacement vectors is defined as Vt , and the RTI is defined
as follows with regard to Vt using the number of upward-
direction vectors of displacement.
#{v ∈ Vt | 0 ≤ arg v < π}
rt = (5)
#Vt
We can say that it is possible to conduct irrigation control
using rt by indicating that it is effective. This study will enable
a new farmer to inherit a skilled farmer’s experiences and
intuition.
E. Observation Environment
We used a greenhouse kit (Length：100 cm×Width：100
cm×Height：200 cm) and light-emitting diodes (LEDs) for
plant cultivation, as shown in Figs. 1 and 3 (growing is
possible even in an environment using LEDs [17]).
FP
FP
7RPDWR FP
6FUHHQ &DPHUD
FP
Fig. 1. Greenhouse kit (top view)
Images are taken with a web camera (Logicool C270)
installed so that the distance to tomato plants are 60 cm.
The images include the entire plant, as shown in Fig. 4. The
images are purple because of the LEDs. We found from the
actual cultivation that optical flow is affected by noise from
the LEDs. Therefore, to alleviating such noise, a black screen
was set as the background, as shown in Fig. 4.
F. Calculating RTI Using Optical Flow
Figure 2 show flow until obtaining the RTI. The image taken
at time t was regarding It , and that taken at time t + ∆ was
regarding It+∆ . Since we processed delta as ∆ = 0.5, the
result of conducting optical flow on the It and It+0.5 images
is denoted as Vt,t+0.5 . We then updated Vt,t+0.5 sequentially
and calculated the RTI. We repeated optical flow at 0.5-hour
intervals from 8:00 to 24:00 each day. Because optical flow is
calculated based on temporal and spatial changes in in light
and shade between images, motion other than the wilting of
leaves may be captured. Therefore, we decided to remove
 LPDJHV RSWLFDOsRZ OLQHFKDUW
57,

‫؞؞؞‬
,W
9WW UWW


,W
9WW

57,
UWW
 WLPH>KRXU@

,W 9WW UWW


‫؞؞؞‬

‫؞؞؞‬
Fig. 2. Calculation flow

FP
/('V
FP

&DPHUD

FP

7RPDWR

Fig. 4. Environment for image capturing

Fig. 3. Greenhouse kit (side view)

the displacement vectors with magnitude 3 or less the Vt
beforehand and calculated the RTI. The calculated RTI values
are shown in the line chart.
IV. E XPERIMENT
A. Overview
The proposed model for estimating the change in Xt as-
sesses whether the RTI reflects such changes due to irrigation.
We conducted experiments to assess whether the RTI is
effective as an index to reflect the change in the Xt of tomato
plants.
To measure change in the Xt , we needed to determine the
necessary irrigation time before conducting the experiment.
Therefore, we included a day without irrigation to determine
the change in leaves when there was no irrigation. In con-
firming the change in Yt during irrigation, it is important to
determine whether Xt decreases or increases. Irrigation was
not carried out between 8:00 and 24:00 for the day without
irrigation, which was photographed with the web camera. The
line chart in Fig. 5 shows the displacement vectors, in which
leaf wilt moved upward, from the Vt on the day without
irrigation.
It is evident from Fig. 5 that leaf wilt did not move upward
from 10:30 to 12:00 and 13:00 to 22:00 in the absence of
irrigation. Therefore, these irrigation conditions clarified the
molecule of rt
time [hour]
Fig. 5. Day without irrigation
change in Xt from irrigating at 12:00. The following three
experimental conditions were applied on the basis of the
irrigation at 12:00.
• The condition under which irrigation was carried out
once.
• The condition under which irrigation was carried out
more than once.
• The condition under which the irrigation amount varied.
To determine whether RTI can be used to reflect the change in
Xt due to Ut , we carried out irrigation once. Since irrigation
can be carried out multiple times in water-stress cultivation,
we also carried out irrigation twice. Finally, because it is not
practical to provide the same amount of water, we not only a
used a fixed Ut but also varied Ut . We assessed whether RTI
is valid under each condition.
B. One-time Irrigation
We clarified whether change in Xt can be confirm from the
RTI when irrigation was conducted at 12:00. To determine if
RTI can be used to reflect the change in Xt due to irrigation,
we compared the conditions of irrigation and without irriga-
tion. Figure 6 shows the RTI values with a Ut of 200 ml at
12:00 and when not irrigating.
The graph indicates that the RTI increased after irrigation,
suggesting that it is possible to reflect changes in Xt based
on displacement vectors.
C. Multiple Irrigation Times
We then assessed whether the RTI can be used to reflect the
change in Xt when increasing the number of irrigation times.
We used a Ut of 200 ml at 11:00 and 14:00 and 200 ml at
12:00 and 18:00 on another day. The results are shown in Fig
7.
Compared with no irrigation, we confirmed that RTI can be
used to reflect the change in Xt after carrying out irrigation
once. On the other hand, for two irrigation times, 14:00 and
18:00, the RTI could not be used to measure this change
because it did no change.
57,
WLPH>KRXU@
Fig. 6. One time irrigation
57,
WLPH>KRXU@
Fig. 7. Multiple irrigation
D. Divided Irrigation
We compared the RTI values when irrigation was carried
out with varied Ut . To determine whether the difference in Ut
causes change in Xt . We prepared Ut of 200, 300, 400, and
500 ml, and the results of irrigating in 30-minute intervals
from 12:00 to 16:00, where a marked change occurred, are
shown in Fig. 8.
As a result of preparing four Ut , the RTI showed that the
leaves recovered quickly and grew larger as Ut increased. This
confirms that RTI can be used to measure even changes in Ut .
V. D ISCUSSION
We conducted irrigation at 12:00 so that we could compare
the RTI with that without irrigation and confirmed whether
there was a change (Section IV-B). There was a difference in
RTI between the two conditions, suggesting the possibility of
reflection the change in Xt with the RTI. Irrigation in a state
in which leaves wilt is a case in which Xt decreases and the
change in Xt due to irrigation is easy to confirm.
We also carried out irrigation from a state in which Xt
was thought to be increasing and clarified whether changes in
leaves can be reflected using the RTI (Section IV-C). It became

57,
WLPH>KRXU@
Fig. 8. Divided irrigation
clear that it is difficult to reflect such change due to multiple
irrigation times by just calculating RTI. Thus, we considered
the reasons the RTI cannot be used to reflect changes in Xt .
Since rt was designed independently, it is difficult to assess
whether it is optimal for reflecting the change in Xt . However,
it is obvious from the results we obtained thus far that rt can
be used to reflect Xt if done during the first irrigation of at
least one day.
We determined that it is difficult to reflect the change in Xt
through multiple irrigation times with the RTI, we used four
Ut at 12:00 and clarified whether each Ut caused a change in
Yt afterwards (Section IV-D).
VI. CONCLUSION
We proposed a plant model for developing an irrigation-
control system for water-stress cultivation. To automate irri-
gation carried out by skilled farmers, we discussed whether
irrigation control can be conducted on the basis of our plant
model. As a result, the state in which the body-water content
of plants is thought to be decreasing can be used to determine
the change in body-water content due to irrigation. The RTI
changed due to the difference in irrigation amount when
irrigation control was conducted, which suggests that a pattern
of irrigation amount can be used. However, irrigation must
be carried out more than once since the RTI cannot be used
to reflect changes in body-water content in a state in which
this content is considered to be increasing. The accuracy of
RTI can be improved using environmental information such as
temperature and humidity. In addition, when irrigation control
is conducted on the basis of estimated water stress, we do
not clarified the relationship between water stress and RTI. A
further study on estimating water stress and on RTI should be
conducted.
R EFERENCES
[1] T. Fukatsu and M. Hirafuji, “Field monitoring using sensor-nodes with
a web server.” JRM, vol. 17, no. 2, pp. 164–172, 2005.
[2] K. Takayama and H. Nishima, “Early detection of water stress in tomato
plants based on projected plant area,” Environmental Control in Biology,
vol. 45, no. 4, pp. 241–249, 2007.
[3] Y. Kaneda, S. Shibata, and H. Mineno, “Multi-modal sliding window-
based support vector regression for predicting plant water stress,”
Knowledge-Based Systems, vol. 134, pp. 135–148, 2017.
[4] N. E. Williams and A. Carrico, “Examining adaptations to water stress
among farming households in Sri Lanka’s dry zone,” Ambio, vol. 46,
no. 5, pp. 532–542, 2017.
[5] L. Richards and W. Gardner, “Tensiometers for measuring the capillary
tension of soil water,” Journal of the American Society of Agronomy,
vol. 28, no. 5, pp. 352–358, 1936.
[6] G. C. Topp, J. Davis, and A. P. Annan, “Electromagnetic determination
of soil water content: Measurements in coaxial transmission lines,” Water
Resources Research, vol. 16, no. 3, pp. 574–582, 1980.
[7] J. Boyer, “Leaf water potentials measured with a pressure chamber,”
Plant Physiology, vol. 42, no. 1, pp. 133–137, 1967.
[8] M. Gallardo, R. Thompson, L. Valdez, and M. Fernández, “Response of
stem diameter variations to water stress in greenhouse-grown vegetable
crops,” The Journal of Horticultural Science and Biotechnology, vol. 81,
no. 3, pp. 483–495, 2006.
[9] F. TongKe, “Smart agriculture based on cloud computing and IoT,”
Journal of Convergence Information Technology, vol. 8, no. 2, 2013.
[10] H. Channe, S. Kothari, and D. Kadam, “Multidisciplinary model
for smart agriculture using Internet-of-Things (IoT), sensors, cloud-
computing, mobile-computing & big-data analysis,” Int. J. Computer
Technology & Applications, vol. 6, no. 3, pp. 374–382, 2015.
[11] M. Hirafuji, “Creating comfortable, amazing, exciting and diverse lives
with CYFARS (CYber FARmerS) and agricultural virtual corporation,”
in Proc. of the Second Asian Conference for Information Technology in
Agriculture, 2000, pp. 424–431.
[12] S. P. Mohanty, D. P. Hughes, and M. Salathé, “Using deep learning for
image-based plant disease detection,” Frontiers in plant science, vol. 7,
pp. 1–10, 2016.
[13] S. Arima, S. Shibusawa, N. Kondo, and J. Yamashita, “Traceability
based on multi-operation robot; information from spraying, harvesting
and grading operation robot,” in Proceedings of 2003 IEEE/ASME
International Conference on Advanced Intelligent Mechatronics, vol. 2,
2003, pp. 1204–1209.
[14] Z. Meng, A. Duan, D. Chen, K. B. Dassanayake, X. Wang, Z. Liu,
H. Liu, and S. Gao, “Suitable indicators using stem diameter variation-
derived indices to monitor the water status of greenhouse tomato plants.”
PLoS ONE, vol. 12, no. 2, pp. 1–15, 2017.
[15] T. Camus, “Real-time quantized optical flow,” Real-Time Imaging, vol. 3,
no. 2, pp. 71–86, 1997.
[16] M. Tao, J. Bai, P. Kohli, and S. Paris, “Simpleflow: A non-iterative,
sublinear optical flow algorithm,” in Computer Graphics Forum, vol. 31,
no. 2, 2012, pp. 345–353.
[17] G. D. Massa, H.-H. Kim, R. M. Wheeler, and C. A. Mitchell, “Plant
productivity in response to led lighting,” HortScience, vol. 43, no. 7, pp.
1951–1956, 2008.