Addressing Data Imbalance in Plant Disease

Recognition through Contrastive Learning

Bryan Chung
2024 IEEE 3rd International Conference on AI in Cybersecurity (ICAIC) | 979-8-3503-8185-6/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICAIC60265.2024.10433841

The Loomis Chaffee School

Connecticut, United States
bryan_chung@loomis.org

Abstract—The following study introduces a novel framework B. Previous Methods

for recognizing plant diseases, tackling the issue of imbalanced
datasets, which is prevalent in agriculture, a key sector for many
economies. Plant diseases can significantly affect crop quality
and yield, making early and accurate detection vital for effective
disease management. Traditional Convolutional Neural Networks
(CNNs) have shown promise in plant disease recognition but
often fall short with non-tomato crops due to class imbalance
in datasets. The proposed approach utilizes contrastive learning
to train a model on the PlantDoc dataset in a self-supervised
manner, allowing it to learn meaningful representations from
unlabeled data by maximizing the similarity between images
based on disease state rather than species. This method shows
a marked improvement in accuracy, achieving 87.42% on the
PlantDoc dataset and demonstrating its superiority over existing
supervised learning methods. The agnostic nature of the model
towards plant species allows for universal application in agri-
culture, offering a significant tool for disease management and
enhancing productivity in both existing farms and future smart
farming environments.
Index Terms—Plant Disease Recognition, Data Imbalance,
Contrastive Learning, Machine Learning, Convoluted Neural
Network, Plant Disease, Plant Science
I. I NTRODUCTION
A. Problem Definition
Agriculture is the backbone of innumerable economies
and is pivotal in ensuring food security across the globe.
One of the most significant challenges faced by the agri-
cultural sector is the prevalence of plant diseases, which
can have catastrophic effects on plant quality and yield.
Early and accurate detection of plant diseases is essential
for effective disease management and preventing the
spread of infections. However, current means of identi-
fying plant diseases, which often rely on the expertise of
farmers, are time-consuming and inconsistent by human
nature.
In recent years, computer vision and machine learning
technologies have shown noteworthy promise in numer-
ous fields, one of which is agriculture. Among such tech-
nologies, CNNs (Convolutional Neural Networks) have
emerged as a powerful tool for image recognition tasks
[1]. CNNs, which can extract complex patterns from
images, possess phenomenal potential to revolutionize
plant disease recognition. This research aims to explore
and develop a CNN-based model for accurate, acceler-
ated, and accessible plant disease diagnosis, which could
significantly contribute to not only traditional agricul-
tural practices but also novel forms of farms.
To address the aforementioned problem, several re-
search studies have explored the use of CNNs, which
have demonstrated comparable performance in image
processing. In the early stages of research, Tm et al.
utilized the LeNet-5 architecture to classify ten disease
categories of tomato, achieving a 94.8% test accuracy
on Plant Village Dataset [2] [3]. Later, Karthik et al.
used a Residual CNN to achieve a 98% accuracy on the
same dataset while classifying 4 diseases [4]. Nithish
Kannan et al. applied transfer learning on the ResNet-
50 model to achieve a 97% accuracy on tomato leaves
[5] [6]. More recently, Zhao et Al. once again used the
ResNet architecture to achieve a 96.81% accuracy on
the Plant Village Dataset, with each image placed in
one of ten categories [7]. Finally, Bedi et al. proposed a
hybrid model using autoencoders with a CNN to achieve
a 98.38% accuracy on the same dataset [8].
Despite the success of these methods, they often yield
unsatisfactory results when applied to non-tomato crops.
This can be attributed to a class imbalance problem
within the dataset, where more than 33% of the training
samples consist of tomato leaf images as shown in Figure
1.
Fig. 1. A Distribution of Plant Classes in a Sample Plant Dataset.
Due to such imbalances in datasets and the narrow
scope of each model primarily identifying only tomato
leaf diseases, it is practically difficult to apply the afore-
mentioned models in a farm where many crops and
leaves coexist.
To address this issue, a much more varied dataset con-
taining a comparable number of pictures for many plants
that the model may encounter is necessary. However, the

Authorized licensed use limited to: Bahria University. Downloaded on March 10,2024 at 05:20:16 UTC from IEEE Xplore. Restrictions apply.
process of collecting a dataset will be time-consuming
and costly due to the diverse nature of plants and their
respective diseases.
C. Proposed Method
To address the aforementioned problem, this paper
proposes a novel crop-type agnostic plant disease recog-
nition framework. As aforementioned, in the widely used
PlantVillage Dataset, there are many imbalances regard-
ing the types of plants and their respective diseases.
Hence, this paper will use the PlantDoc dataset, which
still faces imbalance issues, but offers more diversity in
the labeled category of diseases.
The biggest challenge is to prevent the model from
being biased to the imbalances of the dataset. This paper
proposes the application of contrastive learning to a
CNN model via transfer learning, which prevents the
model from relying heavily on the imbalanced labeled
dataset but rather begins by initially training in an unsu-
pervised manner.
The process begins by training a CNN through con-
trastive learning, where the model learns to extract
meaningful representations from unlabeled data, lever-
aging the inherent structures and relationships within the
dataset. In this process, the model is directed to map
similar samples closer and dissimilar samples farther
in high dimensional space, using cosine similarity as a
metric.
Once the pre-training is over, transfer learning is
applied on the CNN model. The model is fine-tuned
using now labeled data from the PlantDoc dataset like
a typical CNN [9]. By incorporating contrastive learning
and transfer learning, the proposed framework attempts
to develop a plant disease recognition model that is
less dependent on an imbalanced, labeled dataset. This
approach allows for a more general application of a
plant disease recognition framework, as the model is not
limited by variations in image states, plant types, and
disease categories.
The paper will explain some of the key ideas used in
the proposed approach in section two, then move on to
the third section with more specific details of how such
concepts are applied in this specific model. Finally, in
section four, the paper will discuss the results of the pro-
posed model, and end with the fifth section summarizing
the implementations, findings, and implications.
II. BACKGROUND
A. Image Classification
Authorized licensed use limited to: Bahria University. Downloaded on March 10,2024 at 05:20:16 UTC from IEEE Xplore. Restrictions apply.
Fig. 2. Classical CNN Architecture. Adapted from [10]
As shown above, the model goes through a process
of training where it learns to map input images into
their categories via a series of convolutional, pooling,
and fully connected layers. During training, the model
optimizes its parameters by repeating such processes and
by using an optimization algorithm, such as gradient de-
scent, and a loss function, such as cross-entropy, to min-
imize the errors between the predicted and true labels.
This research aims to apply such image classification
techniques to successfully categorize various types of
plant diseases.
B. Contrastive Learning
Contrastive learning is a technique in machine learn-
ing that focuses on distinguishing similar and dissimilar
vectors in multi-dimensional space. The essence of con-
trastive learning is to train a model in such a way that
representations, or embeddings, of similar vectors are
pulled closer together, while those of dissimilar vectors
are pulled farther apart. One widely-used metric for
evaluating the similarity of two embeddings is the cosine
similarity, which outputs a value in [-1, 1], where it
returns a value close to 1 if two vectors are similar, and
a value close to -1 if two vectors are different. To quan-
tify the performance of a model, a loss function called
the Normalized Temperature-Scaled Cross-Entropy Loss
(NT-Xent Loss) is often employed. This function com-
bines cosine similarity with softmax to output a proba-
bility array. The function then computes a cross-entropy
loss between the predicted and actual labels, giving a
measure of how different they are. Minimizing the loss
drives the model to produce representations that are
grouped together if they are similar and scattered if they
are not.
Fig. 3. Example Training Process of a Contrastive Learning Model. Adapted
from [11]
 This research utilizes contrastive learning to enable
the trained network to effectively cluster similar feature
spaces within the same plant disease categories.
C. The Class Imbalance Problem
The class imbalance problem is a common challenge
in machine learning, particularly in classification tasks.
It arises when the distribution of classes in the training
dataset is highly imbalanced, with one or more classes
being significantly underrepresented compared to others.
This imbalance can adversely affect the performance of
classification models, as they tend to favor the majority
class and struggle to accurately predict minority class
instances.
Various approaches have been proposed in the litera-
ture to address this problem. Some research studies have
focused on sampling techniques, such as undersampling
the majority class, oversampling the minority class, or
generating synthetic samples. Other methods involve
modifying the original images using data augmentation
techniques.
One relatively new approach to handling imbalance
classification is the idea of contrastive learning, as men-
tioned in 2.2. As a contrastive learning model trains itself
based on the relationships between the representation
of each image rather than explicitly optimizing losses
for class-based objectives, it offers a level of immunity
against class imbalances.
This research addresses the imbalance classification
problem via contrastive learning, not restricting the
model by the limited nature of plant datasets, but en-
couraging it to find relationships between each image.
It will prove that contrastive learning successfully sup-
presses the problem in section 4 through comprehensive
experiments.
III. P ROPOSED P LANT D ISEASE R ECOGNITION
F RAMEWORK
This section presents the methodology used for con-
trastive learning-based plant disease recognition using
convolutional neural networks. The proposed approach
aims to help models find better representations of plant
diseases and enhance their performances in accurately
classifying many types of plant diseases. The proposed
plant disease recognition framework comprises two main
steps: contrastive learning and supervised learning.
A. Phase I: Contrastive Learning
Fig. 4. Contrastive Learning Process in Proposed Approach.
Authorized licensed use limited to: Bahria University. Downloaded on March 10,2024 at 05:20:16 UTC from IEEE Xplore. Restrictions apply.
Figure 4 demonstrates the first main portion of the
proposed architecture—the contrastive learning part. In
this figure, the feature extractor F(.)—a Convolutional
Neural Network(CNN)—takes in input image I ∈ R3 (a
three-dimensional vector) and outputs a one-dimensional
feature map m ∈ R . The feature map is again inputted
to a projection header P(.), which is a two-layer Neural
Network(NN), that outputs an embedding vector z ∈ R,
also of one dimension. Here, we define P(.): m → z.
This overall operation ultimately converts a plant image
I into an embedding vector z, which encodes high-level
features extracted from the plant, such as shapes, colors,
or textures.
To find better representations for plant disease fea-
tures, the research, inspired by SimCLR which pro-
posed a self-supervised learning algorithm for con-
trastive learning with visual representations, assumes
that embeddings from plants with similar diseases are
similar [12]. Cosine similarity is used to measure the
similarity between two embedding vectors, zi and zj , as
shown in the following equation.
zi · zj
S i,j = (1)
|zi |1 · |zj |1
Then, the NT-Xent loss function, which utilizes the
softmax function and the cross-entropy loss function is
applied to the model, which updates its parameters based
on the measured loss of the two embeddings’ similarity.
ezi
σ(zi ) = PK (2)
zj
j=1 e
n
X
− yi · log yˆi (3)
i=1
Ultimately, this process maximizes the similarities be-
tween two images of plants with the same disease. With
such processes, the model can learn to categorize similar
diseases together regardless of the plant species, signif-
icantly reducing pertinent bias, while focusing on the
disease itself to find better representations which results
in higher accuracy. The effectiveness of the proposed ap-
proach will be further proven in section 4 with numerical
results comparing this method with other approaches.
B. Phase II: Supervised Learning
Fig. 5. Supervised Learning Process in Proposed Approach.
 After the contrastive learning phase, the feature extrac- Despite these challenges, the PlantDoc dataset’s di-
tor is transferred to train in a new supervised manner. versity and comprehensiveness make it a valuable re-
The feature extractor again takes in input image I and source for training a plant disease recognition model.
returns feature map m, identical to the earlier process. The dataset’s wide coverage of plant species and disease
Then, however, the feature map is inputted into a Neural types allows the model to learn a broad range of features,
Network, functioning as a plant disease classifier, that thereby enhancing its generalization ability and applica-
returns a vector outlining the probabilities for a plant bility to a wide range of scenarios.
having each disease. Here, a map PDR: m → ŷ is To evaluate the performance of the proposed model,
defined. a 5-fold cross-validation was employed. In this setup,
With this multi-step approach, the model performs no- the dataset was randomly divided into five equal-sized
tably better than one starting from scratch, as the model subsets. The model was then trained and validated five
is already capable of finding adequate representations times, each time using a different subset for validation
that maximize their ability to categorize plant images and the remaining four subsets for training. This ap-
into disease categories. Consequently, the model is less proach ensures that every data point is used for both
biased. training and validation, thereby providing a more robust
and reliable evaluation of the model’s performance.
IV. E XPERIMENTAL R ESULTS
B. Evaluation
A. Dataset
1) Comparison with State-of-the-Art models:
The PlantDoc dataset, which was used in this study,
is a comprehensive collection of 2,598 images across
TABLE I
13 plant species and 27 classes (17 disease classes and ACCURACIES OF STATE - OF - THE - ART MODELS WITH AND WITHOUT
10 healthy classes) [13]. The dataset is unique in its TOMATO IN THE DATASET.
diversity, covering a wide range of plant species and
disease types, which makes it an ideal choice for training Architecture
Accuracy
a robust and versatile plant disease recognition model. All plants All without tomatoes
However, the PlantDoc dataset is not without its chal- VGG19 0.7248 0.6534
lenges. One significant issue is its imbalanced classes,
MobileNetV2 0.7506 0.6723
where some classes are relatively overrepresented. This
imbalance can potentially skew the model towards over- Xception 0.7501 0.6875
represented classes, thereby affecting its performance on HRNet32 0.7639 0.6925
underrepresented classes. Figure 6 shows the distribution DenseNet121 0.7707 0.6930
of each subclass in the PlantDoc dataset, clearly illustrat- Proposed Method 0.8342 0.7964
ing the class imbalance problem.
It can clearly be seen that the models with the pro-
posed contrastive learning phase consistently outperform
models without it. The significant increase in perfor-
mance can be attributed to the unbiased CNN itself,
which is able to generate meaningful representations of
plant images, only focusing on the necessary parts of the
plant, not relying on plant types.
2) Architecture Replacement:

TABLE II
R ESULTS AFTER A RCHITECTURE R EPLACEMENT

Accuracy
Architecture
w/o proposed w/ proposed

VGG19 (Simonyan et al. 2015) 0.7248 0.7647 (+3.9%)

MobileNetV2 (Sandler et al. 2018) 0.7506 0.7858 (+3.5%)
Xception (Chollet 2016) 0.7501 0.8014 (+5.1%)
HRNet32 (Wang et al. 2020) 0.7639 0.8176 (+5.3%)
DenseNet121 (Huang et al. 2018) 0.7707 0.8291 (+5.8%)
Resnet50 (He et al. 2015) 0.7731 0.8342 (+6.1%)

To demonstrate the model-agnostic nature, the pro-

Fig. 6. Distrubution of Plant Classes in PlantDoc Dataset. Adapted from [13] posed architecture has been tested across many well-

Authorized licensed use limited to: Bahria University. Downloaded on March 10,2024 at 05:20:16 UTC from IEEE Xplore. Restrictions apply.
known, existing CNN models. Table 2 depicts the perfor-
mances of respective CNN models with and without the
proposed contrastive learning phase. Across all models,
the implementation of contrastive learning significantly
and consistently increased their performances. The im-
provement is amplified in especially more robust models,
like DenseNet121 and Resnet50.
3) t-SNE Evaluation:
(a) (b)
Fig. 7. Applications of t-SNE on Plant Image Embedding (a): Before and
(b): After Applying Transfer Learning
Figure 7 represents the result of applying t-distributed
stochastic neighbor embedding(t-SNE) to the embed-
dings the feature extractor outputted, essentially re-
ducing each representation of plant images in high-
dimensional space to a more lower, more comprehen-
sible dimension. 7.a and 7.b illustrate the vectors of
each plant image without and with contrastive learning,
respectively. It can be observed that with contrastive
learning, embeddings of the same categories represented
by same colors are more clearly clustered.
4) Confusion Matrix:
Fig. 8. Confusion Matrix of the Proposed Method.
Figure 8 shows a confusion matrix for evaluating the
proposed architecture’s accuracy across different classes.
Shades of darker blue represent accuracy values closer to
1.
Authorized licensed use limited to: Bahria University. Downloaded on March 10,2024 at 05:20:16 UTC from IEEE Xplore. Restrictions apply.
5) Data Augmentation:
TABLE III
DATA AUGMENTATION E VALUATION
Model Accuracy
Vanilla Model 0.8342
Horizontal Flip 0.8531 (+1.89%)
Color Jitter 0.8543 (+2.01%)
Random Perspective 0.8114 (-2.28%)
Grayscale 0.8196 (-1.46%)
Gaussian Noise 0.8439 (+0.97%)
Brightness 0.8033 (-3.09%)
Proposed Method 0.8724 (+3.82%)
Various data augmentation techniques have also been
tested with the proposed method. The proposed method
noticeably outperforms all forms of common augmenta-
tion methods.
C. Applications
The proposed plant disease recognition model, lever-
aging the power of contrastive learning and convolu-
tional neural networks, holds substantial promise for
real-world applications. Its ability to accurately identify
a broad spectrum of plant diseases, irrespective of the
plant species, makes it a versatile tool in the agricultural
sector. One of the most prominent areas where this
model can make a significant impact is smart farming.
Smart farming represents a paradigm shift in agricul-
ture, integrating advanced technologies to enhance the
quality and quantity of agricultural produce. This mod-
ern approach to farming employs a range of technolo-
gies, including Internet of Things (IoT) devices, drones,
and machine learning algorithms, to optimize farming
practices. The proposed plant disease recognition model
fits seamlessly into this technological landscape, offering
a valuable tool for crop health monitoring.
In a smart farming setup, IoT devices equipped with
cameras can be deployed across the farm to continuously
monitor crop health. The images captured by these de-
vices can be processed by our plant disease recognition
model, providing real-time detection of any signs of
disease. This immediate detection allows farmers to take
swift action, such as applying targeted treatments to
affected plants, thereby preventing disease spread and
minimizing crop loss.
The framework’s robustness against class imbalance
further enhances its applicability in diverse farming envi-
ronments. In a farm with a wide variety of crops, certain
types of plants or diseases may be underrepresented in
the training data. However, the model’s use of contrastive
learning ensures it remains effective even in these sce-
narios, making it a universally applicable tool for smart
farming.
Moreover, its model-agnostic nature allows for its use
on less robust hardware, in essence allowing low-budget
farmers to still take advantage of the algorithm.
 In essence, the proposed plant disease recognition [9] D. Singh, N. Jain, P. Jain, P. Kayal, S. Kumawat, and N. Batra,
model can revolutionize smart farming practices. Provid- “Plantdoc: A dataset for visual plant disease detection,” in Proceedings
of the 7th ACM IKDD CoDS and 25th COMAD, CoDS COMAD
ing farmers with accurate, real-time information about 2020, (New York, NY, USA), p. 249–253, Association for Computing
their crop health enables more efficient and sustain- Machinery, 2020.
able farm management. This, in turn, may contribute [10] X. Kang, B. Song, and F. Sun, “A deep similarity metric method
to increased agricultural productivity and food security, based on incomplete data for traffic anomaly detection in iot,” Applied
Sciences, vol. 9, p. 135, Jan. 2019.
underlining the model’s potential to transform the future [11] E. Tiu, “Understanding contrastive learning,” Jan 2021.
of agriculture. [12] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework
for contrastive learning of visual representations,” 2020.
V. C ONCLUSION [13] D. Singh, N. Jain, P. Jain, P. Kayal, S. Kumawat, and N. Batra,
“PlantDoc,” in Proceedings of the 7th ACM IKDD CoDS and 25th
This research proposes a novel plant disease recog- COMAD, (New York, NY, USA), ACM, Jan. 2020.
nition framework that leverages the capabilities of
contrastive learning and Convolutional Neural Net-
works(CNNs). The model was designed to mitigate the
inherent bias towards overrepresented classes prevalent
in most plant image datasets—thus enhancing its ability
to accurately classify a variety of plant diseases, To
demonstrate its effectiveness, the model was trained
and tested on the PlantDoc dataset. Despite significant
class imbalances in the dataset, the model demonstrated
excelling performance in recognizing and classifying
plant diseases, one superior to state-of-the-art methods.
The study further established the general applicability
of the proposed method by performing additional tests,
including the removal of tomato-related data—the most
abundant species in plant disease datasets. The results
of these tests highlighted the robustness of the model
and its ability to adapt to various scenarios, reinforcing
its suitability for real-world applications, like the smart
farm.
VI. ACKNOWLEDGMENTS
I would like to thank Ting Chen, Simon Kornblith,
Mohammad Norouzi, and Geoffrey Hinton for their ex-
tensive work on SimCLR, which established a frame-
work for contrastive learning of visual representations.
This project took significant references from their work.
R EFERENCES
[1] K. O’Shea and R. Nash, “An introduction to convolutional neural
networks,” 2015.
[2] P. Tm, A. Pranathi, K. SaiAshritha, N. B. Chittaragi, and S. G.
Koolagudi, “Tomato leaf disease detection using convolutional neural
networks,” in 2018 Eleventh International Conference on Contemporary
Computing (IC3), pp. 1–5, 2018.
[3] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning
applied to document recognition,” Proceedings of the IEEE, vol. 86,
no. 11, pp. 2278–2324, 1998.
[4] K. R., H. M., S. Anand, P. Mathikshara, A. Johnson, and M. R., “At-
tention embedded residual cnn for disease detection in tomato leaves,”
Applied Soft Computing, vol. 86, p. 105933, Jan. 2020.
[5] N. K. E., K. M., P. P., A. R., and V. S., “Tomato leaf disease detection
using convolutional neural network with data augmentation,” in 2020 5th
International Conference on Communication and Electronics Systems
(ICCES), pp. 1125–1132, 2020.
[6] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” 2015.
[7] S. Zhao, Y. Peng, J. Liu, and S. Wu, “Tomato leaf disease diagnosis
based on improved convolution neural network by attention module,”
Agriculture, vol. 11, p. 651, Jul 2021.
[8] P. Bedi and P. Gole, “Plant disease detection using hybrid model
based on convolutional autoencoder and convolutional neural network,”
Artificial Intelligence in Agriculture, vol. 5, pp. 90–101, 05 2021.

Authorized licensed use limited to: Bahria University. Downloaded on March 10,2024 at 05:20:16 UTC from IEEE Xplore. Restrictions apply.