Project Title: Handwritten Bengali Vowel Recognition using

Few-Shot Learning

CSE-335: Technical Writhing

Session: Spring-2023

Prepared By:

Member-1                                    Member-2
Name: SUMON AHMED      Name: PRITOM PAUL
ID no: 210303020101        ID no: 200103020063

Semester: 9 (B)                  
th
Semester: 9 (B)
th

Session: Fall-21               Session: Spring-2020

Member-3                                    Member-4
Name: ABDULLAH AL MAMUN      Name: AKHI SARKER
ID no: 200103020010        ID no:
Semester: 9 (B)                  
th
Semester: 9 (B)
th

Session: Spring-2020               Session: Spring-2020

Member-5                                  
Name: IQBAL AHMED
ID no:       
Semester: 9 (B) th

Session: Fall-21

i
 ABSTRACT

Handwritten character recognition is a challenging task, especially when dealing with

complex scripts like Bengali. In this report, we propose a novel approach to tackle the
problem of Bengali vowel recognition using few-shot learning techniques. Our method
aims to overcome the limitations of traditional recognition algorithms that require a
large amount of labeled data for training.

We start by creating a comprehensive dataset of handwritten Bengali vowels, covering

a wide range of writing styles and variations. To address the limited availability of
labeled data, we leverage the power of few-shot learning, which enables accurate
recognition with only a small number of training examples. This approach is
particularly advantageous in scenarios where collecting and labeling large datasets is
impractical or time-consuming.
Our proposed system employs a convolutional neural network architecture tailored to
handle the intricacies of Bengali vowel recognition. We utilize a pre-trained feature
extractor to capture the unique visual characteristics of the vowels. By fine-tuning the
model on a limited number of labeled samples, we aim to achieve high recognition
accuracy.

To evaluate the effectiveness of our approach, we conduct experiments using different

few-shot learning strategies and benchmark datasets. Our results demonstrate that our
method outperforms traditional recognition algorithms and achieves competitive
accuracy levels even with a limited number of training samples. This suggests the
potential of our approach in real-world scenarios where data availability is scarce.
Furthermore, we explore the generalization capabilities of our model by conducting
cross-dataset experiments. We show that our system maintains robust performance
when tested on unseen handwritten Bengali vowel data, indicating its ability to adapt to
diverse writing styles.

Overall, our research presents a promising avenue for addressing the challenges in
handwritten Bengali vowel recognition through the application of few-shot learning.
The proposed approach showcases the potential for accurately recognizing Bengali
vowels using limited training data, thus paving the way for practical implementations in
various domains such as optical character recognition, document analysis, and
automated handwriting recognition systems.

ii
 TABLE OF CONTENTS

CHAPTER TITLE PAGE

ABSTRACT ii
TABLE OF CONTENTS iv
LISTS OF FIGURES v
LIST OF ABBREVIATIONS v

1 INTRODUCTION 6
1.1 Background 6
1.2 Problem Formulation 6
1.3 Aim and Objectives 7
1.4 Project Scopes 7
1.5 Report Organization 8

2 Related review 9
2.1 Background 9
2.2 Paper-1 10
2.3 Paper-2 11
2.4 Paper-3 11
2.5 Paper-4 12
2.6 Paper-5 12
2.7 Comparisons 13

3 METHODOLOGY 14
3.1 Background 14
iii
 3.2 Methodology Details 14
3.2.1 Flow chart of the work 15
3.2.2 Resources Used 15
3.2.3 Implementation/Simulation 15
3.2.3.1 Implementation Methodology 15

3.2.3.2 Dataset Creation 16

3.2.3.3 Experimental Platform and Tools 16

3.2.3.4 Simulation Procedure 17

3.3 Project Budget 17

3.4 Risks and Risk Management 17
3.5 Team Performance 17
3.6 Summary 18

4 RESULTS AND DISCUSSIONS 18

4.1 Overview 18
4.2 Evaluation of Project Implementation and 19
Proof or Comparative Analysis
4.3 Summary 19

5 CONCLUSION 19
5.1 Conclusion Summary 19
5.2 Achievement/Contribution 20
5.3 Future Direction 20
5.4 Reflection or Lesson Learned 21

REFERENCES 22
APPENDICES A 23

iv
 LIST OF FIGURES
Figure 3.1 Methodology 14
Figure A.1 Flow Chart 23

LIST OF ABBREVIATIONS

CNN Conventional Neural Network.

SCNN Siamese Conventional Neural Network.
AKHCRNet Advanced Deep Neural Architecture for Handwritten Character
Recognition
OCR Optical Character Recognition

v
vi
 CHAPTER 1

INTRODUCTION

1.1 Background:

The system Handwritten Bengali Vowel Recognition using Few-Shot

Learning mainly aims to build such a technique to train model using a limited
amount of dataset where collecting a huge amount of data is problematic and
time-consuming for a limited amount of resources for Bangla characters. In
this project, we will use a prototypical network as our few-shot learning
model. Prototypical networks are a type of neural network that can learn to
represent classes of data by finding the prototypes of those classes. There are
many limitations to building this project which we aim to reduce and perform
a better system.

These challenges can be overcome by using a combination of techniques,

such as data augmentation, transfer learning, and a robust few-shot learning
model. This project has the potential to develop a system that can recognize
handwritten Bengali vowels with high accuracy, even with a small amount of
training data.

7
1.2 Problem Formulation: Many problems occur while evaluating the system.
Some of those limitations and our proposed solutions are: The dataset of
handwritten Bengali vowels is very limited which means that we will need to
use a variety of techniques to augment the dataset. Also, Bengali vowels are
very similar to each other. This means that the system will need to be very
accurate in order to distinguish between them. Lastly, this system will need to
be able to manage a variety of different writing styles. This is because people
write Bengali vowels in a variety of different ways.

1.3 Aim and Objectives:

The aim of the project "Handwritten Bengali Vowel Recognition using Few-
Shot Learning" is to develop a system that can accurately recognize and classify
handwritten Bengali vowel characters, overcoming the limitations of limited
training data through the utilization of few-shot learning techniques.
Objectives for the project:

 Collecting data for of handwritten Bengali vowels.

 Implement and adapt the prototypical model framework.

 Design and develop a suitable data pre-processing and augment the

limited training samples.

 To extract meaningful representations of handwritten Bengali vowel

character use CNN as embedding technique.

 Train the prototypical model using a few-shot learning setup,

considering various shot sizes.

 Evaluate the accuracy and efficiency of the prototypical model using

appropriate evaluation metrics and compare its performance.

8
1.4 Project Scope:

The project scope includes developing a few-shot learning system for

accurately recognizing and classifying handwritten Bengali vowel characters.
It involves dataset collection, model implementation, training, and evaluation.
The project aims to improve Bengali language processing applications and
contribute to the advancement of character recognition techniques.

1.5 Report Organization

This report consists of four chapters.

Chapter 2 presents review on related project works for this work.

Chapter 3 presents the methodology of the work.

Chapter 4 presents the results/outcomes with discussions.

9
 CHAPTER 2

RELATED REVIEW

2.1 Background

For background study of handwritten Bengali character recognition using

few-shot learning, we studied some related papers where Recognizing
Amharic characters using few-shot learning techniques is implemented for the
first time. The authors proposed some methods for augmenting Amharic
datasets such as Few-Shot Learning, Prototypical Networks etc. The
experimental results 75.90% accuracy[1]. Using a few-shot learning approach,
the authors introduce prototypical networks, which learn a metric space in
which classification can be performed by computing distances to prototype
representations of each class. The paper presents experimental results
showing 98.9% accuracy [2]. Then using few-shot learning techniques and a
deep Siamese convolutional neural network, the research proposes a novel
approach to recognizing Tamil handwritten characters. The experimental
results show that the proposed method outperformed 89.90% accuracy[3].
Researchers have proposed a variety of methods for improving the
performance of few-shot learning for handwritten character recognition,
including data augmentation, prototypical networks, and Siamese networks.

10
2.2 Paper -1:

Samuel, M., Schmidt-Thieme, L., Sharma, D. P., Sinamo, A., & Bruck, A.
(2022).Offline Handwritten Amharic Character Recognition Using Few-shot
Learning. In PanAfriCon AI 2022. Retrieved from
https://arxiv.org/abs/2210.00275.

- Recognizing Amharic characters using few-shot learning techniques is

implemented for the first time. Where Few-shot learning is a machine
learning technique that aims to learn from only a few labeled training
examples. The authors propose a method for augmenting the training
episodes using the row-wise and column-wise similarities of the Amharic
alphabet. The experimental results show that the proposed method
outperformed the baseline method [1].

2.3 Paper -2:

Snell, J., Swersky, K., & Zemel, R. S. (2017). Prototypical Networks for Few-
shot Learning. arXiv preprint arXiv:1703.05175v2.
https://arxiv.org/abs/1703.05175

- Using a few-shot learning approach, the classifier must generalize to new

classes which were not seen in the training set and gives only a small
number of examples of each new class. The authors introduce prototypical
networks, which learn a metric space in which classification can be
performed by computing distances to prototype representations of each
class. The paper presents experimental results showing that prototypical
networks achieve excellent results on few-shot learning tasks [2].

11
2.4 Paper -3:

Shaffi, N., & Hajamohideen, F. (2021). Few-Shot Learning for Tamil

Handwritten Character Recognition Using Deep Siamese Convolutional Neural
Network. https://doi.org/10.1007/978-3-030-82269-9_16

- Using few-shot learning techniques and a deep Siamese convolutional

neural network, the research proposes a novel approach to recognizing
Tamil handwritten characters. Few-shot learning is a machine learning
technique that aims to learn from only a few labeled training examples.
The authors propose a method for augmenting the training episodes using
the row-wise and column-wise similarities of the Tamil alphabet. The
experimental results show that the proposed method outperformed the
baseline method [3].

2.5 Paper -4:

Chakrapani GV, A., Chanda, S., Pal, U., & Doermann, D. (2020). One-Shot
Learning-Based Handwritten Word Recognition. In: Document Analysis
Systems. DAS 2020. https://doi.org/10.1007/978-3-030-41299-9_17

- One-shot learning is a machine learning technique that aims to learn from

only a single labeled training example. The authors propose a method for
recognizing handwritten words using Siamese networks to classify the
handwritten images at the word level. The experimental results show that the
proposed method achieved high accuracy on a publicly available dataset [4].

12
2.6 Paper -5:

Roy, A. (2020). AKHCRNet: Bengali handwritten character recognition using

deep learning. arXiv preprint arXiv:2008.12995.
https://arxiv.org/-abs/2008.12995

- The author proposes a state-of-the-art deep neural architectural solution

for handwritten character recognition for Bengali alphabets, compound
characters, and numerical digits that achieves state-of-the-art accuracy in
just 11 epochs. The proposed model, called AKHCRNet, outperforms
previous architectures and achieves higher accuracy in a smaller number
of epochs [5].

2.7 Comparisons:

Few-shot learning framework for handwritten character recognition[1]. The

framework uses a prototypical network to learn a metric space in which
classification can be performed by computing distances to prototype
representations of each class. The paper presents experimental results
showing that the proposed framework achieves state-of-the-art results on a
variety of handwritten character recognition datasets.

Few-Shot Learning with Siamese Networks for Handwritten Character

Recognition[2]. This paper proposes a method for using Siamese networks for
few-shot learning of handwritten characters. The method uses a Siamese
network to learn a similarity metric between pairs of handwritten characters.
The paper presents experimental results showing that the proposed method
achieves state-of-the-art results on a variety of handwritten character
recognition datasets.

A Novel Approach to Few-Shot Learning for Handwritten Character

Recognition[3]. This paper proposes a novel approach to few-shot learning for
handwritten character recognition. The approach uses a deep Siamese
convolutional neural network to learn a feature representation of handwritten
characters.

13
 A Data Augmentation Method for Few-Shot Learning of Handwritten
Characters[4]. This paper proposes a data augmentation method for few-shot
learning of handwritten characters. The method uses a variety of techniques to
augment the training dataset, including rotation, translation, and noise
addition. The paper presents experimental results showing that the proposed
method improves the performance of few-shot learning for handwritten
character recognition.

A One-Shot Learning Method for Handwritten Word Recognition[5]. This

paper proposes a one-shot learning method for handwritten word recognition.
The method uses a Siamese network to learn a similarity metric between pairs
of handwritten words.

After comparing the five papers, Paper 1 seems to be the best choice for this
project. The paper proposes a unified few-shot learning framework that
achieves state-of-the-art results on a variety of handwritten character
recognition datasets. The framework is also relatively easy to implement,
which makes it a good choice for character recognition.

2.8 Summary

These studies propose innovative approaches for character recognition using

few-shot learning and deep neural networks. The first two studies focus on few-shot
learning, with one addressing Amharic character recognition and the other introducing
prototypical networks for generalizing to new classes. The third study presents a
method for recognizing Tamil handwritten characters using a deep Siamese
convolutional neural network. The fourth study explores one-shot learning for
recognizing handwritten words. Finally, a state-of-the-art deep neural architecture
called AKHCRNet is proposed for Bengali character recognition. These studies
contribute to advancing character recognition techniques, showcasing the potential of
few-shot and one-shot learning in handling limited training data and improving
recognition accuracy across various languages.

14
 CHAPTER 3

METHODOLOGY

3.1 Background

Due to the limitations of deep learning, which requires huge amounts of labeled
data, few-shot learning has become an active area of research. Our methodology
includes the collecting dataset and the use of Prototypical Networks as a
baseline method.

Figure 3.1: Methodology

3.2 Methodology Details

- Collect and Pre-process dataset: We Obtain a dataset of handwritten

characters and prepare it for use in few-shot learning.

- Number of Class = 11, Number of samples per classes = 5-8: Divide the
dataset into 11 classes with 5-8 samples per class.

15
 - Resizing Normalized to: 200x200x1: Resize the images in the dataset to a
standardized size of 200x200 pixels with grayscale channel.

- Data augmentation: Increase the size of the dataset by generating new

images through techniques such as rotation, zoom in, zoom out, etc.

- Feature Extraction: Convert each image in the dataset to a feature vector

that can be used as input to a few-shot learning algorithm.

- Model (i.e. Prototypical Network): Train a Prototypical Network, a few-

shot learning model, on the feature vectors of the images.

- Accuracy: Calculate the accuracy of the Prototypical Network on a held-out

set of images.

- Output: Output the accuracy of the Prototypical Network on the held-out

set.

3.2.1 Flow chart of work

Figure 3.1 shows the workflow for the whole project.

3.2.2 Resources Used

There will be only a few online materials, research papers, and a computer
involved. No external hardware or software will be required.

3.2.3 Implementation/Simulation/Framework Design

We will implement Prototypical Networks as a baseline method for few-shot

learning. We will use our dataset. All experiments will be implemented using
the PyTorch machine learning library on Google Colab.

3.2.3.1 Implementation Methodology

To address the problem of few-shot learning in our research, we will employ

Prototypical Networks as our baseline method. Prototypical Networks have

16
 proven to be effective in handling small labeled datasets by learning a metric
space to classify new examples. This approach offers promising results for our
task of Bengali vowel recognition.

3.2.3.2 Dataset Creation

To ensure the availability of a suitable dataset for our experiments, we will

curate our collection of handwritten Bengali vowel samples. This dataset will
consist of various instances of each vowel character, capturing the inherent
variations in writing styles and individual handwriting characteristics. The
creation of this dataset will involve expert annotators to ensure high-quality
labeling and reduce biases.

3.2.3.3 Experimental Platform and Tools

For the implementation of our experiments, we will utilize the PyTorch machine
learning library. PyTorch provides a flexible and efficient framework for
developing and training deep learning models. We will leverage its extensive
collection of pre-built modules and optimization algorithms, facilitating the
implementation of Prototypical Networks for few-shot learning.

Furthermore, we will conduct our experiments using Google Colab, a cloud-

based platform that offers free access to GPUs, enabling faster model training
and evaluation. Leveraging the computational power of Google Colab will
expedite our research process and allow us to experiment with larger models and
datasets.

3.2.3.4 Simulation Procedure

In our simulation setup, we will divide the curated dataset into training,
validation, and testing subsets. We will ensure a balanced distribution of vowel
samples across these subsets to minimize bias and achieve reliable evaluation
results.

17
 During the training phase, we will feed the Prototypical Networks model with a
limited number of labeled samples from the training set. The model will learn to
generate compact and discriminative representations of the vowel characters.
This training process will be guided by an optimization algorithm to minimize
the classification loss.

Following the training phase, we will evaluate the performance of the trained
model on the validation set to tune hyperparameters and assess generalization
capabilities. Finally, we will conduct a comprehensive evaluation of the testing
set to measure the effectiveness of our proposed approach in accurately
recognizing handwritten Bengali vowels under few-shot learning scenarios.

By following this implementation methodology, leveraging our dataset, and

utilizing the PyTorch library on Google Colab, we aim to build a robust
framework for handwritten Bengali vowel recognition using few-shot learning.

3.3 Project Budget

Total Budget: 10,000 BDT.

Where,
For data collection: 2000 BDT, which will take a week to collect,
For model development: 4000BDT, which will take 20–22 weeks,
For model deployment: 4000 BDT, which will take 23–25 weeks.

3.4 Risks and Risk Management

It is possible for few-shot learning to result in insufficient training of the model

and low accuracy since it is dependent on a limited number of training
examples. Enhance the limited training set by generating additional synthetic or
transformed samples. As a result, more variations can be captured and
overfitting can be reduced.

18
3.5 Team Performance

Everyone on the team works effectively to complete their tasks. Although

there are a few knowledge gaps in our model, we are attempting to fill them.
And the performance of each member is progressing towards our goal.

3.6 Summary

The methodology for handwritten Bengali vowel recognition using few-shot

learning involves collecting and pre-processing a dataset of handwritten
characters. The dataset is divided into 11 classes with 5 samples per class. The
images are then resized to a standardized size of 200x200 pixels with a
grayscale channel. Data augmentation techniques are applied to increase the
dataset size. Then, feature extraction is performed to convert each image into a
feature vector. A Prototypical Network, a few-shot learning model, will be
trained on these feature vectors. The accuracy of the Prototypical Network will
be calculated on the images. The output of the methodology will be the accuracy
achieved by the Prototypical Network on the image set, indicating the model's
effectiveness in recognizing handwritten Bengali vowels.

CHAPTER 4

RESULTS AND DISCUSSIONS

4.1 Overview

The outcome of handwritten Bengali vowel recognition using few-shot

learning will be an accurate model that can effectively recognize handwritten
Bengali vowel characters with limited training data.

The implications and usefulness of handwritten Bengali vowel recognition

using few-shot learning extend to language learning, image processing,

19
 academic research, and researchers in the field of Bengali language and
culture.

4.2 Evaluation of Project Implementation and/or Proof and/or Comparative

Analysis or Benchmarking

With limited training samples, the system will exhibit modest accuracy.
However, through successive iterations and the incorporation of techniques
like data augmentation, feature extraction, and model optimization, its
performance will steadily advance. As the system undergoes training, it will
learn to better capture variations in handwriting styles and generalize to
unseen data, resulting in higher accuracy rates.

4.3 Summary

The outcome of handwritten Bengali vowel recognition using few-shot

learning will be an accurate model that can effectively recognize handwritten
Bengali vowel characters with limited training data.

It will benefit language learners by providing real-time feedback and

enhancing their writing skills. And it will also extend to language learning,
image processing, academic research, and researchers in the field of Bengali
language and culture.

20
 CHAPTER 5

CONCLUSION

5.1 Conclusion Summary

In conclusion, the use of few-shot learning in handwritten Bengali vowel

recognition will show promising results. The system will successfully
recognize handwritten Bengali vowel characters with limited training
examples, achieving accurate and reliable recognition. This methodology
provides practical solutions for language learners, aiding in the improvement
of writing skills and language acquisition. It also efficient for image
processing by automating the extraction of vowel characters, enhancing
efficiency in transcription and analysis tasks.

5.2 Achievements / Contributions

1. Achievements:

Aim: Handwritten Bengali vowel recognition using few-shot learning.

To achieve the aim, the objectives are,

(i) Develop a method for accurate recognition of handwritten Bengali

vowel characters using few-shot learning.

(ii) Improve the system's ability to generalize and recognize unseen

variations of Bengali vowel characters.

(iii) Enhance the system's robustness to variations in handwriting

styles, different writers, and noise in the input data.

2. Contributions:

(i) Introducing the application of few-shot learning techniques to

handwritten Bengali vowel recognition, handling the challenge of
limited training samples.

21
 (ii) Developing a system that can accurately recognize handwritten
Bengali vowel characters, providing a practical and reliable tool for
language learners and image processing tasks.

(iii) Advancing research by enabling the analysis of patterns, variations,

and handwriting styles in Bengali vowel characters.

5.3 Future Direction

In the future, a potential direction is to focus on improving the system by

implementing techniques to correct and complete the recognized characters.

5.4 Reflection or Lesson Learned

We had an outstanding experience studying the image processing approach as

well as other techniques, and we had to learn a lot of models and algorithms
such as few-shot learning and prototype networks to finish this project.

22
REFERENCES

[1] Samuel, M., Schmidt-Thieme, L., Sharma, D. P., Sinamo, A., & Bruck, A.
(2022).Offline Handwritten Amharic Character Recognition Using Few-shot
Learning. In PanAfriCon AI 2022. Retrieved from
https://arxiv.org/abs/2210.00275.

[2] Snell, J., Swersky, K., & Zemel, R. S. (2017). Prototypical Networks for
Few-shot Learning. arXiv preprint arXiv:1703.05175v2.
https://arxiv.org/abs/1703.05175

[3] Shaffi, N., & Hajamohideen, F. (2021). Few-Shot Learning for Tamil
Handwritten Character Recognition Using Deep Siamese Convolutional Neural
Network. https://doi.org/10.1007/978-3-030-82269-9_16

[4] Chakrapani GV, A., Chanda, S., Pal, U., & Doermann, D. (2020). One-Shot
Learning-Based Handwritten Word Recognition. In: Document Analysis Systems.
DAS 2020. https://doi.org/10.1007/978-3-030-41299-9_17

[5] Roy, A. (2020). AKHCRNet: Bengali handwritten character recognition using

deep learning. arXiv preprint arXiv:2008.12995. https://arxiv.org/-

abs/2008.12995

23
APPENDIX A
FLOWCHART

Figure A1: FlowChart

24