Online Handwritten Recognition System for Mixed

Style Characters

Thesis Work Presented By:

Rakesh Ahuja

Under the Supervision of :

Mr. Vikas Kumar
Asstt. Prof. MIT Moradabad
Department of Computer Science & Engineering
.
 Road Map
 Introduction to Thesis Problem Area
 Online and Offline Handwriting Recognition
 Technique Used
 Original Elastic Matching
 Iterative Elastic Matching
 Recognition System
 Experiments
 Discussion of Results
 Future Possibilities of Extensions
27/05/2019 2
 Handwriting Recognition Problem
 Task of transforming a language
represented in its spatial form of
graphical marks into its symbolic
representation

27/05/2019 3
 Thesis Problem
 Development of a Character Recognition
System
 Online Handwritten Data
 Writer-independent

 Characters taken from Unconstrained

Natural Handwriting
 Presegmented Characters

27/05/2019 4
 Potential Applications
 Handheld Devices
 Small size
 Natural Applications
 Examples: Note-taking, Making Annotations
etc.
 Large Symbol Natural Languages
 Examples: Chinese, Kanji etc.
 Easy Data Entry.

27/05/2019 5
 Online Vs Offline Handwriting
Online Handwriting
 Data is captured at the
time of writing.
 Data requirement is few
hundred Bytes.
 Adaptation is easier.
 Some preprocessing
step are not required
27/05/2019
Offline Handwriting
 Data is captured at
previous point of time.
 Data requirement is few
KB.
 Adaptation is relatively
difficult to achieve.
 More preprocessing is
required.
6
 Technique Used
 Statistical Input
 Hybrid statistical-structural
 Structural
Model
 Model Matching
 Elastic Matching
 Deformation model
 Euclidian Distance
 Operations: Substitution, deletion, insertion

27/05/2019 7
 Elastic Matching (contd.)
 Finds an alignment between the events of two
variable length patterns

   
etAA 1 , etBB 1 , etAA  2 , etBB  2 ,..., etAA  L  , etBB  L  
Where
Pattern 
A  e1A , e2A ,..., eNAA 
and
Pattern B  e B
1 , e2
B
,..., e B
NB 
and A
t 1  t  2   ...  t  L  , L 
A A
NA
t B 1  t B  2   ...  t B  L  , L  NB
27/05/2019 8
 Original Elastic Matching
 Aims minimization of distance between two
patterns   D  i  1, j   
   
 min  D  i  1, j  1  j  2 
   
  D  i  1, j  2   
 
D  i, j   d  i, j    
 
 min 
 D  i  1, j  

  j2
 
 D  i  1, j  1 


 
 min  D  i  1, j  j  1 
where  
1  i  N A ,1  j  N B
Dist  A, B   D  N A , N B 
27/05/2019 9
 Elastic Matching (contd.)

27/05/2019 10
 Iterative Elastic Matching
 d  i, j   
 It take following form:    
min d  i, j  1  j  m  1
   
Input   d  i , j  2   
 
DI  i, j '   DI  i  1, j    
 

min  d  i , j  

Model   j  m 1 
 
 d  i, j  1 

 
min d  i, j  j  m 
 where DI 1,1  d 1,1  

is chosen among  j, j  1, j  2
'
 and j
 j ' work as j in next iteration

27/05/2019 11
 Euclidian Distance in String
Matching
 Euclidian distance between two sample points:

( xiA  x Bj )2  ( yiA  y Bj )2  c(iA   jB )2 (1)

i A represent the tangent of sample point in A and

 jB represent the tangent of sample point in B.

27/05/2019 12
 Recognition System
Unknown
Pattern

Training Recognition 8

Training Model Base

Set

Overview of the System Architecture

27/05/2019 13
 Recognition System (Contd.)
• Handwriting Data
Distribution of Test Patterns

564, 14%
1696, Uppercase
42%
Lowercase
Digits
1825,
44%
20 Writers

27/05/2019 14
 Recognition System (Contd.)
• Segmentation

27/05/2019 15
 Recognition System (Contd.)
• Preprocessing
 Size Normalization
 Translation
 Duplicate points removal

27/05/2019 16
 Recognition System (Contd.)
 Recognition
 Nearest Neighbor Classifier
 Distance between Patterns – Iterative Elastic Matching
 Distance between points – Squared Euclidean Distance
d (i, j )  ( xi  x j )2  ( yi  y j )2  c(i   j )2

27/05/2019 17
 Experiments
 Experiment1 (Writer-dependent, Category-specific)
 Writer-wise Uppercase (20 model bases)
 Writer-wise Lowercase (20 model bases)
 Writer-wise Digit (20 model bases)
 Experiment2 (Writer-dependent, Category-free)
 Writer-wise Combined (Uppercase, Lowercase and Digit) (20 model
bases)
 Experiment3 (Writer-independent, Category-specific)
 All-Writers Uppercase (1 model base)
 All-Writers Lowercase (1 model base)
 All-Writers Digit (1 model base)
 Experiment4 (Writer-independent, Category-free)
 All-Writers Combined (Uppercase, Lowercase and Digit) (1 model base)

27/05/2019 18
 Result
Summarized chart for Recognition Result

ModelSet
used
 Upper Case Lower Case Digits
Test
category

Individ
All All All
Indivi- Individua Individ Individua Individ ual
All writer All writer All writer
dual l writer ual l writer ual Writer
Coefficient writer all writers all writer all
writer all writer all Writer all
s UC categor LC categor Digit categ-
UC category LC category Digit categor
y y ory
y

0 86.03 83.73 92.33 86.56 85.15 83.78 89.21 85.04 92.90 85.82 98.58 88.65

25 87.26 85.85 93.99 89.86 87.01 86.08 91.67 88.71 91.49 83.33 98.40 85.99

50 86.03 84.31 93.57 89.68 86.68 85.81 91.01 88.05 88.12 81.03 98.22 84.39

75 85.08 83.55 93.28 89.21 86.30 85.32 90.90 88.00 87.94 78.37 97.87 81.21

100 84.84 83.37 93.21 88.91 85.97 84.88 90.95 88.05 87.06 76.00 96.81 80.67

27/05/2019 19
 Effect of coefficient on Recognition Accuracy

Results (Contd.)
Recognition Vs Coefficient Recognition Vs Coefficient
100
100 All Writers Lower
Recognition(%)

Recognition (%)
95
95 Case
All Writers UC All writer All
90 90
Category
85 All Writer All 85
80 Category
80
75
0 25 50 75 100 75
0 25 50 75 100
Coefficient Coefficient
Recognition Vs Coefficient

100
Recognition %

95 All writer Digit

90
85 All writer all
80 categ-ory
75
0 25 50 75 100

27/05/2019 Coefficient
20
 Results (Contd.)

Category Fix Vs Category Free

Recognition Rate %

98.4
100
96 93.99
91.67
92 89.86 88.71
88 85.99
84
80
UC LC Digit UC LC Digit

Category fix Category free

Category

27/05/2019 21
 Results (Contd.)
Writer Dependent Vs Writer Independent
100
Recognition success (%)

96

92 89.86
88.71
88 85.85 86.08 85.99
83.33
84

80
Dependent

Dependent
Independent

Independent

Independent
Dependent
Writer

Writer

Writer
Writer

Writer

Writer
UC LC Digit

Category

27/05/2019 22
 State of Art Vs Our Result
State of Art Vs Our Result
Recognition(%)

92
90
88
86 State of Art
84 Our Result
82
UC LC Digit
State of Art 86.56 85.04 88.65
Our Result 89.86 88.71 85.99
Pattern Category

27/05/2019 23
 Future Work
 Improved Preprocessing
 Equidistant Resampling
 Smoothing
 Training Set Reduction
 Improved Efficiency
 Improved Accuracy
 Writer Adaptation

27/05/2019 24
Suggestions?

Questions?
Thank you
 References
[1]C. Bahlmann, H. Burkhardt, “The Writer Independent Online
Handwriting Recognition System frog on hand and Cluster Generative
Statistical Dynamic Time Warping”, IEEE Transactions on Pattern
Analysis and Machine Intelligence, Vol. 26, No. 3, March 2004.
[2]R. Plamondon, S.N. Srihari, “On-line and Off-line Handwriting
Recognition: A Comprehensive Survey”, IEEE Transactions on Pattern
Analysis and Machine Intelligence, Vol. 22, No. 1, January 2000.
[3]Cheng-Lin Liu, Stefan Jaeger, Masaki Nakagawa, “Online Recognition
of Chinese Characters: The State-of-the-Art”, IEEE Transactions on
Pattern Analysis and Machine Intelligence, Vol. 26, No. 2, February
2004.
[4]Vikas Kumar, Suneeta Agarwal, “ Template based Writer-Independent
Online Character Recognition System using Elastic-Matching, IIIrd
International conference on Intelligent System, Sep 2005

27/05/2019 27
References
[5]Scott D. Connell, “Online Handwriting Recognition using Multiple Pattern
Class Models”, Ph.D. Thesis, Michigan State University, 2000.
[6]Charles C. TAPPERT, “The State of the Art in On-Line Handwriting
Recognition”, IEEE Transaction on Pattern Analysis and M/C Intelligence,
Volume 12, No.8 , August 1990.
[7]“Anoop M.Namboodiri, Anil Kumar Jain”, Online Handwritten Script
Recognition, Vol 26, No. 1, january 2004.
[8]”Scott D. Connell, Anil Kr. Jain “Writer Adaptation of Online Handwriting
Models”, Department of Computer Science and Engg., Michigan State
University, East Lansing, MI 48824.
[9]Dr.Suneeta Agarwal,Vikas Kumar, “Online Character Recognition”,
Proceeding of the Third International Conference on Information
Technology,2005.
[10]Vikas Kumar,Rakesh Ahuja, “Online Handwriting Recognition System using
Elastic Matching”, IEEE conference on ISIT, Aug-2006

27/05/2019 28
 3 4’
B
2
Example: 3’
Input Pattern A={(1,1),(2,1),(3,2),(4,3)}
1 A
Model Pattern B={(1,1),(2,2),(3,3)} 1’ 2’

4’,3
Value 3
1
returned
3.3
2
i,j 2
3’,2 3’,1
3’,3
di j 1 1 2.3

1 1 1 1 1
2.3
2’,2 2’,1 2’,2 2’,1 2’,1
2’,3
1 1 1 1 1
2.3

2.8 1.4 0 1.4 0 0 1.4 0 0

0
1’,3 1’,2 1’,1 1’,2 1’,1 1’,1 1’,2 1’,1 1’,1 1’,1
2.8 1.4 0 1.4 0 0 1.4 0 0 0
27/05/2019 29
 3
B 4’
2
Example: 3’
Input Pattern A={(1,1),(2,1),(3,2),(4,3)}
1 A
Model Pattern B={(1,1),(2,2),(3,3)} 1’ 2’

 Start with i=1 and j’=1 and DI(1,1)=d(1,1)

i j DI(i-1,j) + min {d(i,j), d(i,j+1), d(i,j+2)} j’ DI(i,j’)
--------------------------------------------------------------------------------------
1’ 1 0 (DI(1’,1))
2’ 1 0 +1 {1 1 2.3} 1 1 (DI(2’,1))
3’ 1 1 +1 {2.3 1 1 } 2 2 (DI(3’,2))
4’ 2 2 +1 {2.3 1} 3 3 (DI(4’,3))

Dist(A,B)=DI(4’,3)=3
Aligned points: {(1’,1), (2’,1), (3’,2), (4’,3)}

27/05/2019 30
 Model for Statistical Pattern Recognition

27/05/2019 31