ISSN No.

0972-6454

Achievements of the TDIL

Resource Centres
• Human Machine Interface
Systems (HUMIS)
• Knowledge Resources (KR)
• Knowledge Tools (KT)
• Language Engineering (LE)
• Localisation (L)
• Translation Support
Systems (TSS)
• Standardisation
 Patron-in-Chief Technology Treading on Multi-lingualism
Arun Shourie, Hon'ble Minister Over 95% of India’s population cannot work in English, hence there is need for shift
Ministry of Communications & Information Technology from English-centric computing to multilingual computing. Technology development
for Indian Languages was initiated at individual level during early 1960’s & ’70s;
(Government of India) and later on Government sponsored R&D projects during 1980-1990s and further
ashourie@nic.in accelerated these activities in mission mode since 2000. Innovation is key in the
growth of new technologies and incubation of these technologies is essential for
Patron productising them. Incubating innovations in Indian language technology follows
‘S’-curve model with four growth prospectives : Improving Core Capabilities,
K.K. Jaswal, Secretary
Collaborating Capabilities, Developing New competitive Capabilities, and Creating
Department of Information Technology Revolutionary Changes.
Ministry of Communications & Information Technology Technology development for Indian Languages may be categorized in A-B-C Technology
(Government of India) phases:
6, CGO Complex, New Delhi-110003 During 1976-1990 (A-Technology Phase), focus was on Adaptation Technologies;
secretary@mit.gov.in abstraction of requisite technological designs and competence building in R&D
institutions. [This may correspond to phase-1 of the ‘S curve’ model – Improving core
Associate Patron capabilities].

• S. Lakshminarayanan, Additional Secretary During 1991-2000 (B-Technology Phase), focus was on developing Basic Technolo-
gies- generic information processing tools, interface technologies and cross-compatibil-
sln@mit.gov.in
ity conversion utilities. TDIL(Technology Development for Indian Languages)
programme was initiated. [This may correspond to phase-2 of the ‘S’ curve model –
Advisor collaborat-
Incubating Innovations - ‘S’ curve model : Technology
• Y.S. Bhave, Joint Secretary & Financial Advisor Development for Indian Languages ing capa-
bilities]
ysbhave@mit.gov.in
During
Contents July 2003, vk"kk<+ 10 2 0 0 1 -
2005
1. Calendar of Events 2 (C-Technol-
ogy Phase),
2. Reader’s Feedback 4 focus is on
3. TDIL Vision 5 developing
Creative
4. About TDIL Programme 7 Technologies
5. Achievements of the TDIL Resource Centres in the con-
text of con-
RCILTS for Hindi & Nepali (IIT, Kanpur) 13 vergence of
computing,
RCILTS for Gujarati (MS Univ., Baroda) 25
communi-
RCILTS for Marathi & Konkani (IIT, Mumbai) 33 cation and content technologies. Collaborative technology development is being en-
couraged. During this period Resource Centres for Indian Language Technology Solu-
RCILTS for Punjabi (TIET, Patiala) 43
tions are set up. These create a virtual organizational Language Technology environ-
RCILTS for Bengali (ISI, Kolkata) 51 ment. [This may correspond to phase-3 of the ‘S’ curve model - Developing new capa-
bilities].
RCILTS for Oriya (Utkal Univ. & OCAC, Bhubaneswar) 69
During 2006- 2010, focus may be on Demonstrative technologies for newer products
RCILTS for Assamese & Manipuri (IIT, Guwahati) 83 or services. Futuristic projects such as Intelligent Cognitive Systems and Speech to
RCILTS for Kannada (IISc, Bangalore) 95 Speech Translation systems are being conceptualized.

RCILTS for Telugu (Univ. of Hyderabad, Hyderabad) 109 Monitoring of the TDIL programme relied on ZOPP (Objectivity Oriented Project
Planning) workshop for building consensus on approach and knowledge sharing, peer
RCILTS for Malayalam (C-DAC, Thiruvananthapuram) 135 review and regional clustering for closer interaction of technology sharing. Once the
RCILTS for Tamil (Anna Univ. Chennai) 149 technology is developed by a particular centre, it is shared with other centres as well.
Clustering of Regional Resource Centres enables peer competition and collaboration for
RCILTS for Urdu, Sindhi & Kashmiri (C-DAC, Pune) 167 innovation and product-oriented development. Basic Information Processing Kit,
RCILTS for Sanskrit, Japanese & Chinese (JNU, Delhi) 181 OCR, Text-to-Speech and messaging system will shortly be available for all Indian
languages. On-line Machine Aided Translation is available from English to Hindi.
6. TDIL Associates Achieve Honours…Congratulations ! 190 Other Translation pairs are also being developed, Language Technology Business Meet
7. Resource Centres Technology Index 192 (LTBM 2001) provided a forum for closer interaction between acdmia, industry and
government to facilitates innovation, incuvation and proliferation. Technology
handshake were encouraged; 41 technology handshake were signed. International
Editorial Team community has also recognised India’s growing expertise in multilingual computing.
Om Vikas omvikas@mit.gov.in Research groups from France, Germany, U.K., Japan etc. had approached Indian
scholar for possible R&D collaboration.
P.K. Chaturvedi pkc@mit.gov.in
Pardeep Chopra pchopra@mit.gov.in This issue of VishwaBharat@tdil consolidates the technologies developed so far mainly
in the Resource Centres under the TDIL programme. We hope it will be highly useful to
V.K. Sharma vkumar@mit.gov.in the Research Groups in academia as well as to the industry for integrating these
Manoj Jain mjain@mit.gov.in technologies into newer products and services.
S. Chandra schandra@mit.gov.in omvikas@mit.gov.in

ISSN No. 0972-6454 has been granted to VishwaBharat@tdil. Google search engine refers to the contents of this journal.
WebSite : http://tdil.mit.gov.in
 çèkku ea=h
Prime Minister
MESSAGE

Information Technology has brought about the

revolution that has transformed commodities, people
and their relations. Our sages envisioned the world as
a Global Village: “Vasudhaiv Kutumbkam”. That
vision is poised to become a reality as communities
and countries network and join up. What we need is the right mindset
to impel us towards the state of “Sarve Bhavantu Sukhinah”.

The opportunities that today’s knowledge economy offers enable

us to make the knowledge and innovation available far and wide,
without any barriers of nationality, language, culture, caste and creed.
IT with such a humanitarian perspective will lead to a society where
peace and harmony obtain. I hope that with the success of the
Technology Development for Indian Languages Mission programme,
India will emerge as a pioneering hub of multilingual computing and
provide appropriate technology solutions for overcoming linguistic
and cultural barriers, and thereby ensure knowledge for all.

(A.B. Vajpayee)

New Delhi
July 29, 2003
 1. Calendar of Events
! Workshop on Spoken Language Processing, Tata Institute
of Fundamental Research, Mumbai, January 9-11, 2003.
Website : http://speech.tifr.res.in
! Indo Wordnet workshop, CIIL Mysore, Jan. 14-15, 2003.
Website : http://www.ciil.org
! Second International workshop on Technology Development
in Indian Languages (IWTDIL-2003), Kolkata,
Jan. 22-24, 2003.
Website : http://isical.ac.in/-cvpr/events/iwtdl.htm
" Text Retrieval Conference, Conducted by National
Institute of Standards and Technology (NIST)
with support from ARDA, U.S.A., February 17-21,
November 2003.
Website : http://trec.nist.gov
! National Workshop on Application of Language
Technology in Indian Languages, CALTS, University
of Hyderabad, Central University, Hyderabad,
March 6-8 2003.
E-mail : sapworkshop@yahoo.com
! 13th International Workshop on Research Issues on Data
Engineering Multilingual Information Management
(RIDE-MLIM’ 2003), Hyderabad, India, March
10-11, 2003.
Website : http://www.iiit.net/conferences/ride2003.html
! First National Indic-Font Workshop at PES Institute of
Technology, Bangalore, 26-30 March, 2003.
E-mail : vijay@ekgaon.com
" EACL 2003-11th Conference of the European Chapter
of the Association for Computational Linguistics Agro
Hotel, Budapest, Hungary, April 12-17, 2003.
Website : http://www.coli.uni-sb.de/eacl/eacl03.php3
" EACL 2003 workshop on Computational Linguistics for
South Asian languages- Expanding synergies with Europe.
Agro Hotel, Budapest, Hungary, April 14, 2003.
" The 12th International Conference on Management of
Technology, International Association for Management
of Technology (IAMOT), Nancy, May 13 – 15, 2003.
Website : http://www.iamot.org/IAMOT2003/cfpl1.html
! WILTS-2003, Utkal University, Bhubaneswar, May
15-17, 2003.
Website : http://ilts-utkal.org/wilts-2003.htm
" Human Language Technology Conference (NAACL),
Edmonton, Canada, May 27- June 1, 2003.
Website : http://www.sims.berkeley.edu/research/
conferences/hlt-naacl03/dates.html
" 3rd International LRC 2003, Localisation Summer School,
University of Limerick, Ireland, 3-6 June, 2003.
Website : http://lrc.csis.ul.ie/
" Athabascan Languages Conferences, Arcata, California,
USA, 5 – 7 June, 2003.
Website : http://www.uaf.edu/anlc/alc/
2
! Summer School in Universal Networking Language, IIT
Mumbai, 5-13 June, 2003.
E-mail : pb@cse.iitb.ernet.in
" 8th Biennial Conference of the International Association
for Language Learning Technology - IALLT 2003,
University of Michigan, USA, Conference Workshop : June
17 – June 18, 2003, Conference Sessions : June 19 – June
21, 2003.
Website : http://www.lsa.umich.edu/lrc/iallt/
" Terminology & Localization Conference/Workshops,
Terminology Summer Academy, Kent State University,
Institute for Applied Linguistics, Kent Ohio 44242, June
17-19, June 20-21, 2003.
Website : http://appling.kent.edu/ResourcePages/TSA-
2003/TSAWeb/TerminologyAndLocalizationHome.html
" NLDB 2003, 8th International Conference on Application
of Natural Language to Information System, from June
23 – 25, Burg (Spreewald), Germany.
Website : http://dbis-conference.informatik.tu-cottbus.de/
nldb2003/
" Association for Computational Linguistics (ACL 2003),
Sapporo Convention Center, SAPPORO, Japan
July 7-12, 2003.
Website : http://www.ec-inc.co.jp/ACL2003/
ACL 2003 - Associated Conferences
" EMNLP2003: The Eighth Conference on Empirical
Methods in Natural Language Processing, Nara Institute
of Science and Technology, Japan.
" 6th International Workshop on Information Retrieval with
Asian Languages (IRAL2003), National Institute of
Informatics, Japan.
ACL 2003 - Associated Workshops
" WS-1 Multilingual Summarization and Question
Answering - Machine Learning and Beyond.
" WS-2 Natural Language Processing in Biomedicine.
" WS-3 The Lexicon and Figurative Language.
" WS-4 Multilingual and Mixed-language Named Entity
Recognition: Combining Statistical and Symbolic Models.
" WS-5 The Second International Workshop on
Paraphrasing: Paraphrase Acquisition and Applications.
" WS-6 Second SIGHAN Workshop on Chinese Language
Processing.
" WS-7 Multiword Expressions : Analysis, Acquisition and
Treatment.
" WS-8 Linguistic Annotation : Getting the Model Right.
" WS-9 Workshop on Patent Corpus Processing.
" WS-10 Towards a Resources Information Infrastructure.
" 2003 Conference on Empirical Method in Language
Processing (EMNLP 2003), Sapporo, Japan from July
11 – 12, 2003.
Website : http://www.ai.mit.edu/people/mcollins/emnlp03
" JHU summer workshop on Language Engineering,
Baltimore, Maryland, USA, July 14 to Aug.22, 2003.
Website : http://www.clsp.jhu.edu/workshops
! International Conference on Information processing and
Resources on the Internet (“Tamil Internet 2003”)
Chennai, Tamilnadu, India, July 31 - 3 August, 2003.
Website : http://www.infitt.org/ti2003/
" The XVIth International Conference on Historical
Linguistics, University of Copenhagen, Denmark from
11th – 15th August 2003.
Website : http://www.hum.ku.dk/ichl2003
" Adaptation of Automatic Learning Methods for Analytical
and Inflectional Languages, Vienna, Austria, August
18-22, 2003.
Website : http://www.logic.at/esslli03
" Workshop on “Adaptation of Automatic Learning Methods
for Analytical and Inflectional Languages” (ALAF’03),
Vienna, Austria, Aug.18-22, 2003.
Website : http://www.logic.at/esslli03
" The 4th Celtic Linguistic Conference, Selwyn College,
University of Cambridge, UK From 1-3 September, 2003.
Website : http://privatewww.essex.ac.uk/~louisa/celtic/
clc4.html
" Eurospeech 2003 – Switzerland (Interspeech 2003), 8th
European Conference on Speech Communication &
Technology, September 1 – 4, 2003 – Geneva, Switzerland.
Website : http://www.symporg.ch/eurospeech/
" Generative Approaches to Language Acquisition, Utrecht,
Netherland, from 4th – 6th September, 2003.
Website : http://www-uilots.let.uu.nl/conferences/Gala/
gala_2003_homepage.htm
" 36th Annual Meeting on Applied Linguistics, University
of leeds, from 4th – 6th September, 2003.
Website : http://www.baal.org.uk/baal03call.htm
" An International Conference on Text Speech and
Dialogue, Czech Republic, September 8 – 11, 2003.
Website : http://www.kiv.zcu.cz/events/tsd2003/
" International Conference RANLP – 2003 (Recent
Advance in Natural Language Processing) Borovets,
Bulgaria from 10 – 12 September 2003.
Website : http://www.lml.bas.bg/ranlp2003/
" LISA Workshops in Boston, Boston U.S.A. from 12-18
September, 2003.
Website : http://www.lisa.org/events/2003boston/index.html
! 3 Days National Seminar on Telugu Language, Center for ALTS,
Univ. of Hyderabad, Hyderabad from 17-19, Sept., 2003.
Website : http://www.cil.org/announcement/ltt2.html
" Machine Translation Summit IX, New Orleans, USA,
September 23-28, 2003.
Website : http://www.mt-summit.org
" Conference on Language Policy and Standardization,
Iceland, 4th October, 2003.
Website : http://www.ismal.hi.is/Radstefna2003ENS.html
" 2nd International Semantic Website Conference,
Workshop on Human Language Technology for the
Semantic Website and Website Services, Sanibel Island,
Florida, USA, From 20 – 23 October 2003
Website : http://gate.ac.uk/conferences/iswc2003/
" 8th Annual LRC Conference, Localisation Research Center,
University College Dublin, from 17-19 November, 2003.
Website : http://Irc.csis.ul.ie/
" LangTech2003-The European Forum for Language Technology,
Paris, France, from 24th & 25th November 2003.
Website : http://www.lang-tech.org/
" 6th International Conference of Asian Digital Libraries
(ICADL) 2003, Kuala Lumpur, Malaysia from 8-11
December 2003.
Website : http://www.ftsm.ukm.my/ICADL2003/
! ICON-2003: International Conference on Natural
Language Processing, Central Institute of Indian Languages,
Manasgangotri, Mysore, Dec.18-21, 2003.
Website : http//www.iiit.net/conferences/icon2003.html
Unicode Related Meetings & Events
" INCITS/CT22,Washington, DC,USA Aug.18 2003.
" UTC # 96/ L2 # 193, Pleasanton, CA, USA, Aug. 25-28
2003.
" IUC # 24, Atlanta GA, Sept. 3-5 2003.
" IRG #22 Guilin, China.
" SC22 Plenary, Oslo, Norway, Sept. 15-19 2003.
" Unicode Workshop, Hosted by Technology Development
for Indian Languages (TDIL) Programme of Department
of Information Technology, Govt. of India, at Park Hotel,
New Delhi, India, Sept 24-26, 2003.
" WG20, Hosted by Microsoft & The Unicode Consortium,
Mountain View, CA-USA, Oct 15-17, 2003.
" SC2/WG2 # 44 Hosted by Microsoft & The Unicode
Consortium, Mountain View, CA-USA, Oct 20-23, 2003.
" UTC#97/L2 # 194 Annual Members Meeting, Hosted
by John Hopkins University, Baltimore, MD,
Nov 4-7, 2003.
Website : http://www.unicode.org/unicode/timesens/
calendar.html
Note : ! Indicates conferences in India.
3
 2. Reader’s Feedback
 “…Thank you very much for participating in the
Universal Library Million Book Project and for
hosting our delegation to India. I enjoyed the
presentations, discussions, collegiality, and food.
I was particularly moved by Dr. Om Vikas’s
research and spoke with him afterwards. He gave
me a copy of one of his papers. That paper and
the other publications you provided in the packet
are impressive. The brass Shiva you gave me as a
gift is in an honored place in my office. I
appreciate your generosity and contribution to
achieving noble goals. God bless you …”
-Denise A. Troll,
Carnegie Mellon University (U.S.A)
Website : http://www.library.cmu.edu
 “…Thank you very much for your letter dated
March 24, 2003. I am delighted with the
information contained therein concerning the
Unicode Consortium Committee meeting on
March 4-7, 2003 at Microsoft Center, Mountain
view, California.
My interest had always been to preserve and
develop the diversity that India represents in its
culture, languages, traditions and much else. That
why I always supported efforts that would leads
to possibilities in this direction. I am particularly
pleased that the initiatives you have taken over
the past quarter of a century, since the whole
programme relating to Indian languages
computers and information technology was first
taken up in the Department of Electronics when
I was Secretary, are now flowering. We have come
long way since then, with considerable success,
for which a lot of credit need to be given to you.
I am delighted to have had the opportunity to be
supportive. My principal interest today is in the
UNL and the approach that this would be
complementary to a great deal of what has already
been done under the TDIL.
Thank you for sending me the “Annals of Indian
Language Computing” and Vishwabharat@tdil
(Newsletter)…”
-M.G.K. Menon, Eminent Scientist,
Dr. Vikram Sarabhai Distinguished
Prof. of ISRO, Dept. of Space
E-mail : mgkmenon@ren02.nic.in
4
 “…Here in Bolivia, we are only recently reaching
sufficient agreement in an alliance of organisations
from civil society, universities, and the private
sector, to begin to plan a National Strategy for
Communications & Education. A vital element
in this, of course, is an indigenous presence, which
makes up over 70% of the Bolivian population,
and the strategy planning of multi-lingual
information processing.
In this light, we are interested to know :
(i) if there is some written documentation you
could send us on how you began your enormous
venture to transform Indian Information
Technology?
(ii) and as we are still at the stage of preparing the
broad outlines of a planning strategy, we wonder
if you have any articles on your experience in
hindsight, particularly concerning the
management of priorities at this earliest stage.
At the same time, we wish to thank you for sending
us copies of your journal, Digital Unite
and Knowledge for all, which inspire us
constantly…”
-Juan de Dios Yapita,
Instituto de Lengua y Cultura Aymara
E-mail : ilca@megalink.com
 “…I appreciate your work on “English to Hindi”
translation. I have tried using it for the first time
and it seems quite impressive. I cannot write hindi
properly so this would be a helpful tool (especially
to write to my parents). It’s done very well and
throws in exceptions very nicely. I was trying to
use your hindi editor and sometimes totally got
stuck while trying to find something. Also some
stuff is hard to delete (i.e. if I made a mistake).
Few more things that would make life more easy
would be to have a picture or a text file
explaining/showing the keyboard mapping to the
hindi fonts, and having an ability to get a hardcopy
of the typed stuff. Thanks again, Cheers…”
-Ashutosh Jha, Osellus Inc. Toronto
E-mail : ashutosh@osellus.com
 3. TDIL Vision
TDIL Vision 2010
(TDIL : Technology Development for Indian Languages)
A B C Technology Development Phases
India was long aware of the technological changes and the
local constraints and development of Language
Technology in India may be categorized in three phases:
● 1976-1990 : A-Technology Phase: Focus was on
Adaptation Technologies; abstraction of requisite
technological designs and competence building in
R&D institutions.
● 1991-2000 : B-Technology Phase: Focus was on
developing Basic Technologies- generic information
processing tools, interface technologies and cross-
compatibility conversion utilities. TDIL (Technology
Development for Indian Languages) programme was
initiated.
● 2001-2010 : C-Technology Phase: Focus is on
developing Creative Technologies in the context of
convergence of computing, communication and
content technologies. Collaborative technology
development is being encouraged to realise.
Vision statement
Digital unite and knowledge for all.
Mission statement
Communicating & moving up the knowledge chain
overcoming language barrier.
Objectives
● To develop information processing tools to facilitate
human machine interaction in Indian languages and
to create and access multilingual knowledge resources/
content.
● To promote the use of information processing tools
for language studies and research.
● To consolidate technologies thus developed for Indian
languages and integrate these to develop innovative
user products and services.
Major Initiatives
● Knowledge Resources
(Parallel Corpora, Multilingual Libraries/Dictionaries,
Wordnets, Ontologies)
● Knowledge Tools
(Portals, Language Processing Tools, Web based Tools)
● Translation Support Systems
(Machine Translation, Multilingual Information Access,
Cross Lingual Information Retrieval)
● Human Machine Interface System
(Optical Character Recognition Systems, Voice
Recognition Systems, Text-to-Speech System)
● Localization
(Adapting IT Tools and solutions in Indian Languages)
● Language Technology Human Resource Development
(Manpower Development in Natural Language
Processing)
● Standardization
(ISCII, Unicode, XML, INSFOC, etc.)
TDIL Programme Goals
Short Term Goals
● Standardization of code, font, keyboard etc.
● Fonts and basic software utilities in public domain.
● Corpora creation and analysis.
● Content creation tools.
● Language Technology be integrated into IT curricula.
● Collaborative development of Indian language lexical
resources.
● Writing aids (Spell checks, grammar checks and text
summarization utilities).
● Sharing of standardized lexware &development of
lexware tools.
● Training programs on ILT awareness, lexware
development, and computational linguistics.
Medium Term Goals
● Indian language speech database
● Multilingual, multimedia, content development with
semantic indexing, classical and multi font and
decorative fonts, off-line/on-line OCR.
● Cross lingual information retrieval (CLIR) tools.
● Human speech encoding
● Speech Engine : Speech recognition, specific speech
I/O.
● Indian language support on Internet appliances.
● Understanding and Acquisition of languages,
knowledge representation, gisting and interfacing.
● Distinguished achievement awards for M.Tech/MCA/
Ph.D. level in Indian Language Technologies.
● Machine aided translation: English to Indian languages,
among Indian languages, Indian languages to English
and other foreign languages.
● On line rapid translation, gisting and summarization.
Long Term Goals
● Speech-to-speech translation.
● Human Inspiring Systems.
5
 Language Technology Map
Resource Centres & CoIL-Net Centres
for Indian Language Technology Solutions

Resource Centres

CoILNet Centres

TIET(P)

IIT(R)
IGNCA (D)
JNU(D)

BV(B) IIT(K)
IIT(G)
BHU(V)

MSU(B) BIT(R)
IIITM(G) ISI(K)

GGU(B)
UU(B)
IIT(M) OCAC (B)
CDAC (P)

UOH(H)

Resource Centres

MSU (B) = MS University, Baroda (Gujarati)

CoIL-Net Centres
CDAC (P) = Centre for
Development of Advanced
Computing, Pune
BIT (R) = Birla Institute of
Technology, Ranchi
IIT (K) = Indian Institute of
Technology, Kanpur
IIITM (G) = Indian Institute
of Information Technology &
Management, Gwalior
BV (B) = Banasthali
Vidyapeeth, Banasthali
BHU (V) = Banaras Hindu
University, Varanasi
IIT (R) = Indian Institute of
Technology, Roorkee
IGNCA (ND) = Indira Gandhi
National Centre for the Arts,
New Delhi
GGU (B) = Guru Ghasidas
University, Bilaspur
IISc(B) AU (C) = Anna University, Chennai (Tamil)
UU (B) = Utkal University, Bhuwaneshwar (Oriya)
AU(C) ISI (K) = Indian Statistical Institute, Kolkata
(Bangala)
JNU (D) = Jawaharlal Nehru University, New Delhi
(Sanskrit, Japanese, Chinese)
CDAC(T) UOH (H) = University of Hyderabad, Hyderabad
(Telugu)
IISc (B) = Indian Institute of Science, Bangalore
(Kannada)
IIT (K) = Indian Institute of Technology, Kanpur
(Hindi & Nepali)
IIT (M) = Indian Institute of Technology, Mumbai
(Marathi & Konkani)
IIT (G) = Indian Institute of Technology, Guwahati
(Assamese, Manipuri)
CDAC (T) = Electronic Research & Development
Centre of India, Thiruvananthapuram (Malayalam)
CDAC (P) = Centre for Development of Advanced
Computing, Pune (Urdu, Sindhi, Kashmiri)
OCAC (B) = Orissa Computer Application Centre,
Bhubaneswar (Oriya)
TIET (P) = Thapar Institute of Engineering &
Technology, Patiala (Punjabi)

6
 4. About TDIL Programme
Technology Development for Indian
Languages Programme (TDIL)
-An Overview
1. Need of the programme
The world is in the midst of a technological
revolution nucleated around Information and
Communication Technology (ICT). Advances in
Human Language Technology will offer nearly
universal access to information and services for more
and more people in their own language. Today 80
% of the content on the Web is in English, which is
spoken by only 8% of the World population and
only 5% of Indian population. In a multilingual
country like India, with 18 official languages & 10
scripts it is essential that information processing and
translation software should be developed in local
languages and available at low cost for wider
proliferation of ICT to benefit the people at large
and thus pave the way towards “Digital Unite and
Knowledge for all” and arrest the sprawling Digital
Divide.
2. Impediments
● Lack of industry involvement due to constrained
demand;
● Sub-critical and Un-sustained demand in States;
● Negligible software tools and re-usable
components in public domain
● Plurality of Internal Codes: ISCII-88, ISCII-91,
UNICODE, and other propriety codes;
● Content Glyph-coded, not (ISCII) character-
coded;
● No font Standard;
● Non-availability of Human resources in the
domain of Language Technology in both
Computational Linguistics (CL) and knowledge
Engineering (KE)
● Reluctance of Academia to focus on productising
the R&D.
3. TDIL Mission
In this context, a number of initiatives have been
taken towards development of software, tools and
human–machine Interface systems in Indian
Languages under the TDIL programme.
Vision
Digital unite and knowledge for all.
Mission
Communicating without language barrier & moving
up the knowledge chain.
Objectives
● To develop information processing tools to
facilitate human machine interaction in Indian
languages and to create and access multilingual
knowledge resources/content.
● To promote the use of information processing
tools for language studies and research.
● To consolidate technologies thus developed for
Indian languages and integrate these to develop
innovative user products and services.
3.1 Focus Areas
Translation Systems
• Machine aided Translation system (MAT)
• Parallel corpora, Lexware, Multilingual
dictionaries, Wordnet, speech databases
• Speech-to-Speech Translation system
Human Machine Interface systems
• Optical Character Recognition
• Speech Recognition
• Text to Speech system
Language processing and Web tools
Word processors, spell checkers, converters and
portals
Localization
Adapting IT tools and solutions in Indian
languages.
Standardization
UNICODE, XML, ISCII (Indian Standard
Code for Information Interchange), INSFOC
(Indian Standard Font Code), INSROT (Indian
7
 Script to Roman Transliteration), Standard for
Lexware format, etc.
Evaluation and Benchmarking
Evaluation and testing of the prototype
technologies against benchmarks, product
compliance testing against specifications and
validations of standards.
4. Technology Development Phases
Development of Language Technology in India may
be categorized in three phases:
1976-1990 : A-Technology Phase
Focus was on Adaptation Technologies; abstraction
of requisite technological designs and competence
building in R&D institutions.
1991-2000 : B-Technology Phase
Focus was on developing Basic Technologies- generic
information processing tools, interface technologies
and cross-compatibility conversion utilities. TDIL
(Technology Development for Indian Languages)
programme was initiated.
2001-2010 : C-Technology Phase
Focus is on developing Creative Technologies in the
context of convergence of computing,
communication and content technologies.
Collaborative technology development is being
encouraged to realise.
5. Program Management Structure
All the above activities are being implemented
through various Resource Centres and Localization
(CoIL-Net) Centres. The programme is being
technically monitored by TDIL Working Group
consisting of experts drawn from academia, industry,
R &D organizations and Government. The Resource
centres are divided into regional clusters for peer
review of the progress of the technologies and
solutions developed by them.
Resource Centres
The Department has thirteen Resource Centres for
Indian Language Technology Solutions at various
educational institutes and R&D organizations who
have developed several important tools and
technologies for Indian Language support.
8
CoIL-Net (Content Creation & IT Localisation
Network
CoIL-Net programme is formulated encompassing
a vision for all pervasive socio-economic
developments by proliferating use of language
specific IT based content, solutions and applications
for bringing the benefits of IT revolution to the
common citizen; requisite technology development
and consequently bridging the existing digital divide
in the Hindi speaking states of India. The objective
is to provide a boost to IT localization based socio-
economic developments and help bridge the existing
digital divide by appreciably improving IT
penetration and awareness levels, using Hindi as
medium of delivery, in the Hindi speaking states of
MP, Chattisgarh, UP, Uttaranchal, Bihar, Jharkhand,
and Rajasthan.
CoIL-Tech
The MAIT Consortium on Innovation & Language
Technology (CoIL-Tech) since its inception in
September 2001, has been actively coordinating
various activities with the Industry and the TDIL
(Technology Development in Indian Languages)
Department of the Ministry of Communications
& Information Technology. The consortium today
has active participation from both Indian and MNC
companies with a focus to promote industry
participation in collaborative R&D in language
technology, to coordinate Open Source Software
supporting Indian languages, to evolve consensus
on standards, benchmarks, and certification of LT
products, to collectively interface with government
and academia, to conduct market surveys, organize
technology shows, promote technology transfers and
expand market collectively.
6. Major Achievements of the TDIL Programme
Optical Character Recognition (OCR)
Optical Character Recognition is an indispensable
tool for digitizing the content and is essential for
development of Knowledge networks such as
digital libraries. Optical Character Recognition
(OCR) technology offers the facility to scan and
store the printed text. There are three essential
elements to OCR technology - scanning,
recognition and then reading text. Initially, a
camera scans a printed document. OCR software
 then converts the images into recognized
characters and words. The OCRs developed are
being tested and benchmarked independently by
STQC.
OCR with more than 97% accuracy has been
developed for seven Indian Languages viz Hindi,
Marathi, Bangla, Tamil, Telugu, Punjabi,
Malayalam.
The OCR technologies for Assamese, Oriya,
Malayalam and Gujarati scripts are in the
advanced stages of development.
Spell Checkers
Spell checker is the language specific software
component used in word processing, OCR post
processing and text processing applications. Spell
Checkers for Hindi, Marathi, Bangla, Tamil,
Telugu, Punjabi, and Malayalam have been
developed. The Spell checkers for other Indian
Languages are in advanced stages of development.
Machine Aided Translation System (MAT)
MAT system provides rough translation from a
source language (say English) into a target
language (say Hindi). This output may be post
edited with specially designed tools for enhancing
the translation accuracy.
Anglabharati English to Hindi Translation
Support System has been developed by IIT
Kanpur (Resource Centre for Indian language
in Hindi and Nepali language) and the same is
available in Public Domain at http://
anglahindi.iitk.ac.in
The utility of text to speech synthesis has also been
integrated with Anglabharati and works on the
Linux platform.
The Machine Translation system for Indian
Languages to Hindi and Hindi to English are in
the process of development.
Support on Indian Language LINUX Operating
System (INDIX)
Localization Linux operating system at x-windows
level has been done. INDIX (Localized LINUX)
operating system has been developed by NCST,
Mumbai.
It is now possible on Linux to give file names,
domain names in Devanagari. Indian language
support on Linux in open domain will also ensure
Indian Languages support on all Open Source
Tools Kits. INDIX supports GNOME desktop
environment (of LINUX) to enable applications
to create, edit and display contents in Indian
Languages. The R&D for supporting other Indian
Languages have been initiated.
Standardization
Unicode Standards are widely being used by the
Industry for the development of Multilingual
Software. Unicode Standard 3.0 includes standard
code sets for Indian scripts based on ISCII-1988
document. Some modifications are necessary to
incorporate in the Unicode Standard for adequate
representation of Indian Scripts.
Department of Information Technology,
Ministry of Communications & IT, is the voting
member of the Unicode Consortium. Proposed
changes in the existing Unicode Standards has
been finalized in consultation with respective State
Government and Indian IT Industry. These have
been published in TDIL Newsletter
Vishwabharat@tdil.
The Indian Scripts Font Code (INSFOC) to
overcome the problem of interoperability of
various software products developed by different
vendors has been completed for Hindi,
Malayalam, Gurmukhi and Gujarati. The work
is in progress for other Indian languages.
An “Indian Script to Romanization Table
(INSROT)” has been worked out to facilitate no-
Hindi users to work in Hindi. Efforts are also on
to Standardise the Indian Lexware Format.
Parallel Corpora (Gyan Nidhi)
A parallel corpus is an important input for MAT
system. One Million pages Parallel Corpora is
presently in the process of development. The
Graphic User Interface for viewing the multiple
Language Corpora has also been developed. The
aim is to develop parallel corpora of One million
pages. So far 450, 000 pages parallel corpora has
been developed (as on May 31, 2003).
9
Bi-lingual Dictionaries
Bi-lingual electronic dictionaries are the basic
linguistic resource which is useful for any NLP
related research development and testing progress.
These are useful in the development of Word
processors in Indian Languages. English – Hindi,
English – Telugu, English- Tamil, English –
Kannada, English- Bangla, Bangla-Bangla,
English-Punjabi, English-Oriya, English-
Malayalam bilingual dictionaries have been
developed by RCs.
IT Localisation Clinic
Workshops were organised by DIT during
November 2002, to accelerate the project
deliverables such as Content development in
Hindi, Creation of Repository of language
technology products, Trainers Training
programmes, IT Localisation Clinics, IT
Curricula for Schools, Website Development in
Hindi. Installation of test beds in the domains of
e-governance (Land Records, Agri-Net, Query
System), e-education (Schools, ITIs), e-business
(Small Business), e-tourism, e-health.
TDIL Portal
The TDIL website is bi-lingual (English and
Hindi) and contains information about the TDIL
Programme, it’s initiatives and achievements. It
provides access to Indian scriptures, standards
(Indian scripts, keyboard layout, font layout, etc),
articles and reviews. The unique services provided
by the website includes E-mail in Hindi and
Online Machine Translation from English to
Hindi and vice versa.
The website also provides downloadable softwares
and tools in Indian Languages viz. Plug-in such
as Akshar for Windows (Plug-in for MS-Word),
Shrilipi Bharti (Plug-in with keyboard driver
having Devanagari fonts), Indian Language Word
Processors, NLP tools, NLP Resource for
Windows/Linux.
10
Quarterly on Indian Language Technologies
The VishwaBharat@tdil is a quarterly Newsletter
of Indian Language Technologies, which
consolidates in one place information about
products, tools, services, activities, developments,
achievements in the area of Indian Language
software. It serves as a means of sharing ideas
among technology developers. This creates
awareness in the society regarding the availability
of language technology resources. It is issued
quarterly is widely circulated
Eight issues viz. Jan 2001, May 2001, Sept.
2001, Jan 2002 , May 2002, July 2002, October
2002, Jan 2003 published. All the issues Accessible
through TDIL Web site and all the issues are
available in a single CD.
7. Targets for 2003-04
● Integration and Deployment of OCR, TTS and
Document processing technologies
● Initiation of development of Machine aided
Translation System between English and Indian
Languages
● Linux Operating System enabled with Indian
languages support
● Enforcing UNICODE revision and Vedic code
standardization
● Initiation of Intelligent Cognitive system
(KUNDALINI) project for bringing synergy in
traditional Sanskrit Shastras and Information
Technology.
● Cross Lingual Information Retrieval for
Universal Access
● IT Localization Clinics for incubation of IT
technologies
● 16 Bit Unicode compliant Hindi fonts for public
domain.
8. New Projects Initiated
Intelligent Cognitive System
A new project in the domain of Intelligent
Cognitive system named Knowledge
 UNDerstanding & Acquisition of Languages,
INferencing and Interpretation (KUNDALINI)
is being initiated with the following objectives:
• To develop methodologies and tools for
knowledge representation, extraction,
mining, gisting, inferencing and
interpretation
• To develop knowledge frameworks and access
mechanisms based on Indian tradition
• To develop Sanskrit based Networking
Language as Machine Translation Interlingua.
(Concept based Networking Languages :
CNL)
Gyanaudyog (YÉÉxÉÉätÉäMÉ)
Encouraged by the success of the Gyanaudyog
Workshop held on April 7 –9, 2003 a new project
Gyanaudyog is being initiated to promote Small
Office & Home Entrepreneurship in the area of
Information Technology for catalyzing IT
enabled services (ITES), specifically, Content
Creation, Content localization & application
software localization, Remote customer
interaction Services, Computer Aided Design
with support for Technology Mentoring, Financial
Support guidance and Market information.
Anusrijan (+xÉÖºÉÞVÉxÉ)
There is addition of over 25 Million pages in
R&D in the field of Science &Technology. It is
planned to prepare books/monographs in local
language on emerging areas of Information &
Communications Technology (ICT). These will
be available for translation into other Indian
Languages so as to create averseness about recent
development to promote innovation and
entrepreneurial aptitude.
SOHE - Ganak Bharti (ºÉÉä½äþ-MÉhÉEò ¦ÉÉ®úiÉÒ)
A new project named as Ganak Bharti is being
initiated to promote development and
deployment of Low cost PC with Indian
Language support and Open Source Software for
small business men, women at home and children
aiming at catalyzing the growth of Small Office,
Home and Education (SOHE) environment.
Evaluation and Benchmarking of the Indian
Language Technology Tools
Testing, Evaluation and Benchmarking of the
Indian Language Technology tools is essential for
wider acceptance of the Indian Language
technology products. In view of this,
Standardization Testing and Quality Certification
(STQC) division of DIT has been designated as
the third party for evaluation of the language
technologies tools and products developed TDIL
programme as per international standard.
9. Digital Library Initiative of India
Digital libraries are a form of Information
Technology in which social impact matters as much
as technological advancements. Future knowledge
networks will rely on scalable semantics, on
automatically indexing the community collections
so those users can effectively search within the
Interspace of a billion of repositories. Just as the
transmission networks of the Internet are connected
via switching machines that switch packets, the
knowledge networks of the Interspace will be
connected via switching machines that switch
concepts. Connectivity and training continue to
be the principal barriers to integrating the global
network of libraries.
Among a number of major DL initiatives in USA is
the Universal Digital Library (UDL) project that
aims at creating a free-to-read, searchable collection
of one million books, primarily in the English
language, available to everyone over the Internet.
This project involves participation from many
countries including USA, Australia, India China,
Egypt, Srilanka . The funding agency is National
Science Foundation (NSF),USA, with a funding of
US $ 4.0 Million
The contribution form various industries of USA is
US $ 6.0 million. The overall coordination of the
project is done by Carnegie Mellon University
(CMU), USA.
India participates in the UDL project and makes
efforts to put e-content of vast Indian knowledge base
in the Indian language as much as possible. The
Indian activity of Digital Library is coordinated by
Indian Institute of Science, Bangalore
11
9.1 Objectives
● To digitize and index the heritage knowledge.
● To provide a test bed that will support other
research domains such as scanning techniques,
optical character recognition etc
● Inter-ministerial collaboration for e-Content.
● To promote life long learning in the society (A
necessity of the Knowledge-based society).
● To promote collaborative creativity and building
up knowledge teams across borders.
● Involvement of Resource Centers for Indian
Language Technology Solutions and COILNet
Centers to digitize and web-enable contents in
Indian Languages.
9.2 Participating Institutions of UDL program
● Indian Institute of Science, Bangalore (Overall
coordination)
● Anna University, Chennai, TamilNadu
● Arulmigu Kalasligam College of Engineering,
Madurai,Tamil Nadu
● Goa University, Goa
● Indian Institute of Information Technology,
Allahabad, Uttar Pradesh
● City and State Central Library, Andra Pradesh
● Indian Institute of Information Technology,
Hyderabad, Andra Pradesh
● Shanmugha Art, Science Technology & Research
Academy, Thanjavore, TamilNadu
● Sri Sri Sharda Peetam, Sringeri Mutt, Sringeri,
Karnataka
● Tirumala Tirupati Devasthanam, Tirupati,
AndraPradesh
● Maharasthra Industrial Development
Corporation, Mumbai, Maharashtra
● University of Pune
● Centre for Development of Advanced
Computing, Noida
12
● Kanchi University, Kanchi, Tamil Nadu
● Indian Institute of AstroPhysics, Karnataka
● Indira Gandhi National Centre for Arts, New Delhi
● Rashtrapathi Bhavan, New Delhi
● Punjab Technical University, Punjab
9.3 Targets for 2003-04
● Development and Deployment of on-line
Multilingual Machine Translation system, OCRs
in Indian Languages and Universal Dictionary
Programme through TDIL programme.
● Establishment of Regional Mega Centres at the
institutions with proven capability of scanning
at least 5000 pages per day. The Mega-centres may
be created with State government’s initiative for
meeting operational costs.
● Up gradation of the connectivity of the Mega-
Centres of scanning with 2Mb/s connectivity.
● Creation of “ Digital Library Act” – Provision for
Tax deduction etc for the purpose of providing a
Digital version on web for public benefit.
● Creation of 4C ( Consortium for Compensation
for Creative content)
● Evolution of Project Management mechanism
and productivity measurement strategy of the
Digital Library Centres and criteria for setting
up of Mega Scanning Centres.
● Setting up of Inter-ministerial committee to
integrate Digital Library efforts in India
● Evolution of National Mission on Digital Library
in consultation with Prof. M.G.K.Menon.
● Participation with lead role in
− WSIS (World Summit on Information Society)
organized by ITU at Geneva in Dec 2003)
− UN Decade of Literacy (2003-2013)
− TWA (Third World Academy)
● Promotion of Spin-off technology from Digital
Library programme.
- TDIL Programme Team
 Resource Centre For
Indian Language Technology Solutions – Hindi, Nepali
Indian Institute of Technology Kanpur
Achievements

Department of Computer Science & Engineering

Indian Institute of Technology Kanpur-208016 India
Tel. : 00-91-512-2590762 E-mail : sgd@iitk.ac.in
Website : http://www.gitasupersite.iitk.ac.in
http://www.cse.iitk.ac.in/users/langtech
 RCILTS-Hindi, Nepali
Indian Institute of Technology, Kanpur
Introduction
In 1995, Department of Electronics, Govt. of India,
sanctioned a grant-in-aid for implementation of the
project titled “Machine Aided Translation from
English to Hindi for standard documents (domain
of Public Health Campaign) based on
ANGLABHARTI approach” for which ER&DCI
(with its office at Lucknow and now moved to
NOIDA) was associated for implementation and
commercialization of this software on a PC platform
in the domain of public health campaign. The
ANGLABHARTI software already developed by
IITK on SUN system was used in this project and
was implemented (re-engineered) on PC under
Linux jointly by IITK and ER&DCI under the
supervision of IITK (R.M.K. Sinha, Ajai Jain). In
1996, IITK also designed and developed an
Example-based approach for Machine Aided
Translation for similar (Indian languages) and
dissimilar (English and Indian Languages) under the
leadership of Professor R.M.K. Sinha. This approach
has been named as ANUBHARTI approach. A
system to translate from Hindi to English has been
implemented based on ANUBHARTI approach by
IITK(R.M.K. Sinha, Ajai Jain and Renu Jain).
Currently, AnglaHindi, the English to Hindi MAT
based on Anglabharti methodology, which accepts
unconstrained text, has already been made available
to the users and is very well received. AnglaUrdu
which is based on AnglaHindi has also been
demonstrated. HindiAngla, the Hindi to English
MAT based on Anubharti methodology, has been
demonstrated for simple sentences and further work
is going on to handle compound and complex
sentences. The current research at IITK is focused
towards development of more efficient machine
translation strategies with user friendly interfaces for
these systems. Another dimension of diversification
for future, is to cater to all other Indian languages
by implementing AnglaSanskrit, AnglaBangala,
AnglaPunjabi, and so on; SanskritAngla,
BangalaAngla, PunjabiAngla, and so on; and
14
HindiSanskrit, HindiBangala, and so on; based on
hybridization of Anglabharti and Anubharti
methodologies.
1. Machine Translation
Chief Investigator: Dr. R.M.K. Sinha
Co-Investigator: Dr. A. Jain
Our work on machine translation started in early
eighties when we proposed using Sanskrit as
interlingua for translation to and from Indian
languages (See the paper on “Computer processing
of Indian languages and scripts - Potentialities and
Problems”, Jour. of Inst. Electron. & Telecom.
Engrs., vol.30,no.6, 1984). This was further
elaborated in CPAL-1 paper presented at Bangkok
in 1989.
Later in 1991, the concept of a Pseudo-Interlingua
was developed which exploited structural
commonality of a group of languages. This concept
has been used in development of machine-aided
translation methodology named ANGLABHARTI
for translation from English to Indian languages.
Anglabharti is a pattern directed rule based system
with context free grammar like structure for English
(source language). It generates a ‘pseudo-target’
(Pseudeo-Interlingua) applicable to a group of
Indian languages (target languages) such as Indo-
Aryan family (Hindi, Bangla, Asamiya, Punjabi,
Marathi, Oriya, Gujarati etc.), Dravidian family
(Tamil, Telugu, Kannada & Malayalam) and others.
A set of rules obtained through corpus analysis is
used to identify plausible constituents with respect
to which movement rules for the ‘pseudo-target’ is
constructed. Within each group the languages exhibit
a high degree of structural homogeneity. We exploit
the similarity to a great extent in our system. A
language specific text-generator converts the
‘pseudo-target’ code into target language text.
Paninian framework based on Sanskrit grammar
using Karak (similar to case) relationship provides
an uniform way of designing the Indian language
text generators. We also use an example-base to
identify noun and verb phrasals and resolve their
semantics. An attempt is made to resolve most of
the ambiguities using ontology, syntactic & semantic
tags and some pragmatic rules. The unresolved
ambiguities are left for human post-editing. Some
of the major design considerations in design of
Anglabharti have been aimed at providing a practical
aid for translation wherein an attempt is made to
get 90% of the task done by the machine and 10%
left to the human post-editing; a system which could
grow incrementally to handle more complex
situations; an uniform mechanism by which
translation from English to majority of Indian
languages with attachment of appropriate text
generator modules; and human engineered man-
machine interface to facilitate both its usage and
augmentation. The translation system has also been
interfaced with text-to-speech module and OCR
input.
This project also received funding from TDIL
programme of Govt. of India during 1995-97 and
2000-2002.
The English to Hindi version named AnglaHindi,
of Anglabharti machine aided translation system has
been web-enabled and is available at URL: http://
anglahindi.iitk.ac.in
The technical know-how of this technology has been
transferred on a non-exclusive basis to ER&DCI/
CDAC Noida for commercialization.
A system for translating English to Urdu, named
AnglaUrdu, has also been developed using our
AnglaHindi system and Urdu display software of
CDAC, Pune.
In 1995, we developed another approach for MT
which was example-based. Here the pre-stored
example-base forms the basis for translation. The
translation is obtained by matching the input
sentence with the minimum ‘distance’ example
sentence. In our approach, we do not store the
examples in the raw form. The examples are
abstracted to contain the category/class information
to a great extent. This makes the example-base smaller
in size and further partitioning reduces the search
space. The creation and growth of the example-base
is also done in an interactive way. This methodology,
named ANUBHARTI, has been used for Hindi to
English translation and further details of this
approach can be seen in the Ph.D. thesis of Renu
Jain.
The Anubharti approach works more efficiently for
similar languages such as among Indian languages.
In such cases the word-order remains the same and
one need not have pointers to establish
correspondences.
Currently, we are working towards developing an
Integrated Machine-aided translation system (with
funding from TDIL programme of Govt. of India,
2003 onwards) hybridizing the rule-based approach
of Anglabharti, example-based approach of
Anubharti, corpus/statistical based approaches to get
the best out of these approaches. This is also being
explored to be used for translation engine of speech
to speech translation system.
In parallel, we are also developing MAT system for
Hindi to English translation system, HindiAngla,
based on our Anubharti approach with funding from
COILNET project of Govt. of India (2001
onwards). AnglaHindi and HindiAngla have been
used to demonstrate the two way reverse translation
for simple sentences.
2. Speech to Speech Translation
The speech to speech (S2S) translation requires a
tight coupling of the automatic speech recognition
(ASR) module, MT module, and the target language
text to speech (TTS) module. A mere interfacing of
ASR, MT and TTS modules does not yield an
acceptable S2S translation. S2S requires an
15
integration of these modules such that the hypotheses
are cross verified and appropriate parameters get
generated. In our environment, it has to cater to bi-
lingual (Hindi mixed with English) speech with
commonly encountered Indian accent variations.
The MT also needs be a chunk translator with
multiple translation engines. Our investigations are
directed to domain specific applications in Indian
environment.
Some Relevant Publications
• R.M.K. Sinha, Towards Speech to Speech
Translation, Key-note presentation at Symposium
on Translation Support Systems (STRANS2002),
March 15-17, 2002, Kanpur, India.
3. Lexical Knowledge-Base Development
Lexical knowledge base is the fuel to the translation
engine. It contains various details for each word in
the source language, like their syntactic categories,
possible senses, keys to disambiguate their senses,
corresponding words in target languages, ontology
and word-net information/linkages. We are also
working towards development of Indian language
wordnet named ShabdKalpTaru in association with
Dr. Om Vikas and Dr. Pushpak Bhhattacharya.
Some Relevant Publications
• Renu Jain and R.M.K. Sinha, ‘On Multi-lingual
Dictionary Design’, Symposium for Machine
Aids for Translation and Communication
(SMATAC96), New Delhi 1996.
16
• R.M.K. Sinha, K. Sivaraman, Aditi Agrawal,
T. Suresh and C. Sanyal, ‘On logical design of
multi-lingual lexicon for machine translation’,
Technical Report TRCS-93-174, Department
of Computer Science and Engineering, IIT
Kanpur, 1993.
4. Optical Character Recognition
Work on Devanagari OCR was carried out with
TDIL, Govt. of India, sponsored project named,
DEVDRISHTI, on Recognition of Handprinted
Devanagari script. The investigations were carried
on in developing new features and in integrating
decision making taking into account large variations
in shape. Further, an automated strategy for training
for construction of prototypes and confusion
matrices, from true ISCII files was developed. This
had to be very much distinct from their Roman
counterpart due to script composition being
involved in case of Devanagari script. This work
was further expanded incorporating blackboard
model for knowledge integration in Ph.D. thesis of
Veena Bansal titled “Integrating Knowledge Sources
in Devanagari Text Recognition”
Some work has also been carried out on On-line
character recognition for Roman using handwriting
modeling. Investigations on on-line isolated
Devanagari characters have also been carried out
and further investigations are in progress on
the subject.
(Editorial Comment : This technology was declined
for test & evaluation by STQC)
5. Transliteration
Transliteration among Indian scripts is easily
achieved using ISCII (Indian Script Code for
Information Interchange). ISCII has been designed
using the phonetic property of Indian scripts and
caters to the superset of all Indian scripts. By
attaching an appropriate script rendering
mechanism to ISCII, transliteration from one
Indian script to another is achieved in a natural way.
However, transliteration from Indian script requires
use of heuristics to convert the non-phonetic script
to its probable intended spoken form before it could
be transliterated. Similarly, transliteration from an
Indian script to Roman requires using a standardized
mapping table to easily readable. In our work on
transliteration, we have suggested heuristics and
tables. Several other workers have come up with their
own suggestions. Recently, TDIL has come up with a
standardization of this table called INSROT which uses
only lower case letters to facilitate standard search.
Some Relevant Publications
• R.M.K. Sinha, ‘Computer processing of Indian
languages and scripts - potentialities and
problems’, Jour. of Inst. Electron. & Telecom.
Engrs., vol.30,no.6, 1984,pp. 133-49.
• R.M.K. Sinha and B. Srinivasan, ‘Machine
transliteration from Roman to Devanagari and
Devanagari to Roman’, Jour. of Inst. Electron.
& Telecom. Engrs., vol.30, no.6, 1984,
pp 243-45.
6. Spell-Checker Design
For Indian scripts, there is a very loose concept
of a spelling. Writing in Indian scripts is a direct
mapping of the inherent phonetics and you write
as you speak. There are geographical variations
in the spoken form and so the spellings vary. Our
approach to design of a spell checker is to develop
an user error model for each class of user where
the source of error may the due to incorrect
phonetics, inaccurate inputting or other
influences. The spell-checker uses this error-model
in making suggestions for the error.
7. Knowledge Resources
Introduction
The seeds of this work were planted in the pre-
Internet days with a project undertaken by Dr. T.V.
Prabhakar, Indian Institute of Technology (IIT)
Kanpur, funded by the Chinmaya International
Foundation (1989). A DOS version of Swami
Chinmayananda’s book, The Holy Geeta was
hyperised and published as Geeta Vaatika (1992),
perhaps the first electronic book in India. After the
emergence of Internet standards, Geeta Vaatika was
redone in HTML (1996).
As the World Wide Web grew, the Government of
India (Department of Electronics) funded a project
that continued this work. Work began on the Gita
Supersite , which included multiple commentaries
and translations of the Bhagavadgita. A website was
designed and built, with the programming (business
logic) initially all on the client side.
Resource Centres for Indian Language Technology
Solutions were established throughout the country.
Under one such Resource Centre established at IIT
Kanpur, work on the Gita Supersite has continued.
The technology was extensively reworked and the
content was converted into a database, with all the
business logic on the server side. Currently work is
going on to convert the data into a font-
independent ISCII database, streamline the
programs, improve the audio content, add many
more commentaries on the Bhagavadgita and
provide additional features on the site.
17
Meanwhile, the idea of building Heritage Websites
related to Indian philosophical texts emerged. A
series of websites were planned, including the
Upanishads (to include 12 major Upanishads with
Sankara’s commentaries and translations in English
& Hindi), Brahma Sutra, Complete Works of
Sankara, Ramcharitmanas and the Yoga Sutra.
The experience of building websites in Indian
languages was shared with others and a bi-lingual
site was designed and built for the Uttar Pradesh
Trade Tax Corporation , Government of India. A
site on the life and works of the contemporary sage,
Paramhans Rammangaldasji was also built. Moving
in another direction, an all-Hindi site on disease-
information and health, Bimari-Jankari was created.
The following sites were developed under the TDIL
scheme:
Gitasupersite : http://www.gitasupersite.iitk.ac.in
Brahmasutra : http://www.bramsutra.iitk.ac.in
Yogasutra : http://www.yogasutra.iitk.ac.in
Complete Works of
Adi Sankara : http://www.sankara.iitk.ac.in
Ramcharitmanas : http://www.ramcharitmanas.iitk.ac.in
Upanishads : http://www.upanishads.iitk.ac.in
Minor Gitas : http://www.gitasupersite.iitk.ac.in/
minigita/index.html
Kavi Sammelan : http://www.kavya.iitk.ac.in
Munshi Premchand : http://www.munsipremchand.iitk.ac.in
Bimari-Jankari : http://www.bimari-jankari.org
U P Trade Tax : http://upgov.up.nic.in/tradetax
Paramhans Ram
Mangal Das Ji : http://www.rammangaldasji.org
Short write ups on some of the above sites are
given below:
7.1 Gitasupersite
On the Gita Supersite, one can view the entire
Bhagavadgita in its original language (Sanskrit) in
any of ten Indian language scripts (assamese, bengali,
18
devanagari, gujarati, kannada, malayalam, oriya,
punjabi, tamil and telugu) or in Roman
transliteration. The Supersite also contains Classical
and Contemporary Commentaries on the
Bhagavadgita, with translations in Hindi and
English.
The Gita Supersite has been designed to open
Multiple Windows, so that one can view multiple
translations and/or commentaries on the
Bhagavadgita simultaneously. A Two-Book option
for comparative study is also available. The Search
facility on this Supersite enables a search for the
occurrence of any word in the original text of the
Bhagavadgita.
The Gita Supersite is available for Windows, Unix/
Linux and Mac Platforms, with web browsers that
support frames, JavaScript and Java (such as Netscape
Navigator 4.0/ Internet Explorer 4.0 or higher
versions). Users will not need to download fonts
because Dynamic Fonts have been used on this
website.
The Audio of the chanting of the Bhagavadgita
shlokas by Swami Brahmananda of Chinmaya
Mission Bangalore is also available on this website.
Texts are included in the Gitasupersite are:
Mool Slokas [Sanskrit Verses] of the Bhagavadgita
in all major Indian Language Scripts
Hindi translation - Swami Ramsukhdas
Hindi translation - Swami Tejomayananda
English translation - Swami Gambhirananda
English translation - Dr. S Sankaranarayan
English translation - Swami Sivananda
Sanskrit Commentary - Sri Abhinavagupta
English translation of Sri Abhinavagupta’s Sanskrit
Commentary - Dr. S Sankaranarayan
Sanskrit Commentary - Sri Ramanuja
English translation of Sri Ramanuja’s Sanskrit
Commentary - Swami Adidevananda
Sanskrit Commentary - Sri Sankaracharya
Hindi translation of Sri Sankaracharya’s Sanskrit
Commentary - Sri Harikrishandas Goenka
English translation of Sri Sankaracharya’s Sanskrit
Commentary - Swami Gambhirananda
Hindi Commentary - Swami Chinmayananda
Hindi Commentary - Swami Ramsukhdas
English Commentary - Swami Sivananda
Sanskrit Commentary - Sri Anandgiri
Sanskrit Commentary - Sri Jayatirtha
Sanskrit Commentary - Sri Madhvacharya
Sanskrit Commentary - Sri Vallabhacharya
English translation - Swami Adidevananda
Sanskrit Commentary - Sri Madhusudan Saraswati
Sanskrit Commentary - Sridhra Swami
Sanskrit Commentary - Sri Vedantadeshikacharya
Venkatanatha
Sanskrit Commentary - Sri Purushottamji
Sanskrit Commentary - Sri Neelkanth
Sanskrit Commentary - Sri Dhanpati
A sample page from the site is shown below
7.2 Brahma Sutra
The Brahma Sutra of Badrayana is one of the
Prasthana Trayi, the three authoritative primary
sources of Vedanta Philosophy. No study of Vedanta
is considered complete without a close examination
of the Brahma Sutra.
It is in this text that the teachings of Vedanta are set
forth in a systematic and logical order. The Brahma
Sutra consists of 555 aphorisms or sutras, in 4
chapters, each chapter being divided into 4 sections
each. The first chapter (Samanvaya: harmony)
explains that all the Vedantic texts talk of Brahman,
the ultimate reality, that is the goal of life. The second
chapter (Avirodha: non-conflict) discusses and
refutes the possible objections against Vedanta
philosophy. The third chapter (Sadhana: the means)
describes the process by which ultimate
emancipation can be achieved. The fourth chapter
(Phala: the fruit) talks of the state that is achieved in
final emancipation.
Indian tradition identifies Badrayana, the author
of the Brahma Sutra, with Vyasa, the compiler of
the Vedas. Many commentaries have been written
on this text, the most authoritative being the one
by Adi Sankara, which is considered to be an
exemplary model of how a commentary should
be written.
7.3 Complete Works of Adi Sankara
Adi Sankara, the 9th century philosophical giant of
India was both an intellectual genius and a prolific
writer. In his brief life-span of 32 years, he composed
over 30 original works on vedanta, wrote
authoritative commentaries on 11 Upanishads, the
Brahma Sutra, Bhagavadgita, and other major texts,
and also created inspiring devotional hymns to
various gods and goddesses.
19
 directly to the verse — doha (or sortha), chaupai,
sloka or chhanda — of your choice.
2-Book View: Open two copies of Ramcharitmanas
simultaneously, for a comparative study of different
kaandas of the text.
Word Search: Alphabetic Search for the occurrence
of any word in Ramcharitmanas
Verse Search: Search for verses in Ramcharitmanas,
using the first few words of the verse
Power browse: This option is for Power Users of
Ramcharitmanas who wish to get an overview or
This website is perhaps the first online repository of quickly browse through the dohas and chaupais of
the Complete Works of Adi Sankaracharya. The texts any kaanda of the text
can be read in the original Sanskrit in any one of
Download: Get printer-friendly chapters of
11 Indian language scripts. The texts can also be
Ramcharitmanas
downloaded for printing, making this vast,
invaluable resource easily accessible to users all over Tulsidas: Read about Tulsidas, the author of
the world. Ramcharitmanas
7.4 Ramcharitmanas Related Links: Annotated Links to related sites
The Ramcharitmanas, the 16th century masterpiece 7.5 Upanishads
written by Goswami Tulsidas, is the story of Lord
The Upanishad site consists of all the major
Rama. The text is an unparalleled combination of
upanishads given below :
devotion and pure non-dualistic philosophy. This
website is an attempt to use contemporary Savasya Mandukya’s karika
technology to facilitate and enhance the study of Kena Taittiriya
this ancient scripture. Some of the features available
on this website are: Katha Aitereya
Prasna Svetashvatra
Mundaka Brihadaranyaka
Mandukya Chandogya

Read the book : Read Ramcharitmanas with a For each Upanishad we have several commentaries
unique, user-friendly interface. Navigation through and translations as given below. For a detailed list of
the book can be linear, using the ‘next’ and ‘previous’ available translations and commentaries for each
buttons. Or, you can use the Navigation Bar to go Upanishad please see the appendix.

20
7.6 Kavi Sammelan
This is the first Virtual Kavi Sammelan on the Web.
On this site, you can “create” a Hindi kavi sammelan,
choosing from a database of around 100 poems.
Selections for your kavi sammelan can be made
based on the mood [rasa] or metre [vidha] of the
poems and the poet(s) whose poems you wish to
include. Video and audio recordings of the poems
are available, so you can ‘see’ and ‘hear’ the poets
recite their own poems.
Poems in five moods and five metres are available
here. The five moods are: vira rasa, hasya/vyanga,
shringar rasa, shant rasa, vividha rasa. The five metres
are: geet, ghazal, doha, chhanda mukta, muktak. 13
poets have been featured on this website. These poets
are: Gopaldas Neeraj, Govind Vyas, Dharmpal
Awasthi, Madhup Pandey, Buddhinath Mishra,
Urmilesh, Kailash Gautam, Shiv Om Ambar, Surya
Kumar Pandey, Surendra Dube, Vineet Chauhan,
Kamal Musaddi, Suresh Awasthi.
Audio recordings of the poets talking about what
poetry means to them is also available. A detailed
biography and a photo-gallery of each poet has been
put up. Other features include book-reviews,
interviews and articles.
An introductory article on the History of Hindi Kavi
Sammelans has been specially written by Dr.
Upendra for this website. Listen to Dr. Upendra
summarise the development of kavi sammelans over
the years.
7.7 Bimari-Jankari
Bimari-Jankari is a medical website created for the
benefit of Hindi-speaking people. Special efforts
have been made to simplify both the language and
the concepts that are explained on the site. The major
idea behind this website is to supplement a doctor’s
function. By reading about medical conditions and
diseases, a patient (or patient’s family and friends)
would understand their own situation, and therefore
be in a better position to cooperate with the doctors’
advice and prescription.
Navigation or moving within the website has been
simplified to make it easy to search for
information on any disease. Under each disease,
information has been provided in an easy to
understand, question-answer format. To enable
the user to understand the disease process better,
the related functional anatomy and physiology
has been briefly explained.
Images/illustrations have been profusely used
throughout the site. Most of these images have been
prepared specially for this website. Some of the
images have been obtained, with permission from
WHO/TDR and CDC.
7.8 Paramhans Ram Mangal Das Ji
Sri Paramhansa Ram Mangal Das ji (1893-1984)
was an extraordinary sage of our times. He was
blessed with a divine vision by virtue of which he
could communicate with saints from the past.
Over 2000 saints and gods belonging to all
religions of the world, visited him and gave him
their messages, which he transcribed. These
transcribed messages (over 3500 in number) are
in the Avadhi dialect and run into four volumes
called the Divya Granths.
This website contains these Divya Granths,
together with other works of Sri Ram Mangal Das ji.
21
The messages have been uniquely indexed and can Vinod Pushpanjali Rajeshwar Dvekota
be read chronologically or alphabetically. A topic
index is also to be included very soon. A photo Abstract Chintan–Pyaaz Shankar Lamichhane
gallery, as well as some audio and video recordings
of Sri Ram Mangal Das ji are in the pipeline. A
simple Avadhi-Hindi dictionary is also under
preparation.

8. Technical Issues

Some of the major features of these sites are:

The server side is written in PHP

The content is stored in MySQL in ISCII (as

Creation of this website has been an exercise in against ISFOC)
making a format using which the works of the
innumerable great sages of our times can be On the fly transliteration into ISFOC for any
preserved as part of our rightful heritage. Indian language.

7.9 Nepali Texts Search in all Indian languages.

On the fly PDF generation in all Indian

The Nepali site has the following original texts:
languages.
Basai Leelabhadhur Shatri
Chanting of the Gita Shlokas.
Bhanubhaktko Ramayana Bhanubhakt Acharya
Continuous Play of the Gita Shlokas.
Kunjani Laxmi Prasad Devkota
Architectural Diagram of the Gitasupersite and its
Langda ko Sathi Lain SinghVadadail sister sites

Maaitghar Lain SinghVadadail

Moona Madan Laxmi Prasad Devkota

Ritu Vichar Lekhnath Paudwal

Tarun Tapsi Lekhnath Paudwal

22
9. Appendix - Details of Texts/Commentaries for
each of the Upanishads
#ISAVASYA
Mool Mantra [Sanskrit Verses] of the Upanishad
English Translation - Swami Sivananda
English Commentary - Swami Sivananda
English Translation - Swami Gambhirananda
Hindi Translation - Gita Press Gorakgpur
Hindi Commentary - Harikrishandas Goenka
Sanskrit Commentary - Sri Shankaracharya
Hindi Translation of Sri Shankaracharyas Sanskrit
Commentary - Gita Press Gorakhpur
English Translation of Sri Shankaracharyas Sanskrit
Commentary - Swami Gambhirananda
#KENA
Mool Mantra [Sanskrit Verses] of the Upanishad
English Translation - Swami Sivananda
English Commentary - Swami Sivananda
English Translation - Swami Gambhirananda
Hindi Translation - Gita Press Gorakgpur
Hindi Commentary - Harikrishandas Goenka
Sanskrit Commentary - Sri Shankaracharya
Hindi Translation of Sri Shankaracharyas Sanskrit
Commentary - Gita Press Gorakhpur
English Translation of Sri Shankaracharyas Sanskrit
Commentary - Swami Gambhirananda
#KATHA
Mool Mantra [Sanskrit Verses] of the Upanishad
English Translation - Swami Sivananda
English Commentary - Swami Sivananda
English Translation - Swami Gambhirananda
Hindi Translation - Gita Press Gorakgpur
Hindi Commentary - Harikrishandas Goenka
Sanskrit Commentary - Sri Shankaracharya
Hindi Translation of Sri Shankaracharyas Sanskrit
Commentary - Gita Press Gorakhpur
English Translation of Sri Shankaracharyas Sanskrit
Commentary - Swami Gambhirananda
#PRASNA
Mool Mantra [Sanskrit Verses] of the Upanishad
English Translation - Swami Sivananda
English Commentary - Swami Sivananda
English Translation - Swami Gambhirananda
Hindi Translation - Gita Press Gorakgpur
Hindi Commentary - Harikrishandas Goenka
Sanskrit Commentary - Sri Shankaracharya
Hindi Translation of Sri Shankaracharyas Sanskrit
Commentary - Gita Press Gorakhpur
English Translation of Sri Shankaracharyas Sanskrit
Commentary - Swami Gambhirananda
#MUNDAKA
Mool Mantra [Sanskrit Verses] of the Upanishad
English Translation - Swami Sivananda
English Commentary - Swami Sivananda
English Translation - Swami Gambhirananda
Hindi Translation - Gita Press Gorakgpur
Hindi Commentary - Harikrishandas Goenka
Sanskrit Commentary - Sri Shankaracharya
Hindi Translation of Sri Shankaracharyas Sanskrit
Commentary - Gita Press Gorakhpur
English Translation of Sri Shankaracharyas Sanskrit
Commentary - Swami Gambhirananda
#MANDUKYA’s KARIKA
Mool Mantra [Sanskrit Verses] of the Upanishad
English Translation - Swami Sivananda
English Commentary - Swami Sivananda
English Translation - Swami Gambhirananda
Hindi Translation - Gita Press Gorakgpur
Hindi Commentary - Harikrishandas Goenka
Sanskrit Commentary - Sri Shankaracharya
23
Hindi Translation of Sri Shankaracharyas Sanskrit Sanskrit Commentary - Sri Shankaracharya
Commentary - Gita Press Gorakhpur English Translation of Sri Shankaracharyas Sanskrit
English Translation of Sri Shankaracharyas Sanskrit Commentary - Swami Madhavananda
Commentary - Swami Gambhirananda
10. The Team Members
#TAITTIRIYA
R.M.K. Sinha rmk@iitk.ac.in
Mool Mantra [Sanskrit Verses] of the Upanishad T.V. Prabhakar tvp@iitk.ac.in
English Translation - Swami Sivananda Harish C. Karnick hk@iitk.ac.in
T. Archna archna@iitk.ac.in
English Commentary - Swami Sivananda Murat Dhwaj Singh murat@cse.iitk.ac.in
English Translation - Swami Gambhirananda Rajni Moona ajnimoona@cse.iitk.ac.in
Madhu Kumar madhuk@cse.iitk.ac.in
Hindi Translation - Gita Press Gorakgpur
Amit Mishra mamit@iitk.ac.in
Hindi Commentary - Harikrishandas Goenka Md . Masroor masroor@iitk.ac.in
Sanskrit Commentary - Sri Shankaracharya Mahaluxmi luxmi@iitk.ac.in
Rajeev Bhatia rajeevkb@iitk.ac.in
Hindi Translation of Sri Shankaracharyas Sanskrit
Commentary - Gita Press Gorakhpur Others Who Helped
Dr. Vineet Chaitanya vc@iiit.net (VC) was the driv-
English Translation of Sri Shankaracharyas Sanskrit
ing force behind the Geeta Vaatika, as well as the
Commentary - Swami Gambhirananda
inspiration for the Gita Supersite. Now he watches
#AITEREYA our activities from IIIT Hyderabad, and is still one
Mool Mantra [Sanskrit Verses] of the Upanishad of the few who truly understand the spirit behind
this work.
English Translation - Swami Sivananda Nagaraju Pappu, the first one, wrote 100,000 lines
English Commentary - Swami Sivananda of c-code for the initial versions of Geeta Vaatika.
English Translation - Swami Gambhirananda His DOS version had more features than the cur-
rent HTML one!
Hindi Translation - Gita Press Gorakgpur A part from the current team, those who have con-
Hindi Commentary - Harikrishandas Goenka tributed to the growth of these websites in a major
way include:
Sanskrit Commentary - Sri Shankaracharya
K. Anil Kumar, Anvita Bajpai, Ashutosh Sharma,
Hindi Translation of Sri Shankaracharyas Sanskrit Gita Pathak, K. Ravi Kiran, Rohit Patwardhan,
Commentary - Gita Press Gorakhpur Samudra Gupta, Shrikant Trivedi and
English Translation of Sri Shankaracharyas Sanskrit Tripti Singh.
Commentary - Swami Gambhirananda Courtesy: Prof Sanjay Dhande
Indian Institute of Technology
#SVETASHVATRA
Department of Computer Science & Engineering
Mool Mantra [Sanskrit Verses] of the Upanishad Kanpur - 208 016
English Translation - Swami Sivananda (RCILTS for Hindi & Nepali)
Tel : 00-91-0512-2597174, 2598254
English Commentary - Swami Sivananda E-mail : sgd@iitk.ac.in
Hindi Commentary - Harikrishandas Goenka Editorial Comment : Because of very large number of publications
related to the above article by the Resource Centre & constraint
#BRIHADARANYAKA
of space, we could not include the publication details here these
Mool Mantra [Sanskrit Verses] of the Upanishad have already been listed in the April 2003 issue of VishwaBharat.
For getting the publications please contact Prof. Sanjay Dhande/
English Translation - Swami Madhavananda Prof. R.M.K. Sinha, IIT(K).

24
 Resource Centre For
Indian Language Technology Solutions – Gujarati
The Maharaja Sayajirao University of Baroda, Vadodara
Achievements

Department of Gujarati, Faculty of Arts

The Maharaja Sayajirao University of Baroda, Vadodara-390002, India
Tel. : 00-91-265-2792959 E-mail : rciltg@satyam.net.in
Website : http://msubaroda.ac.in/rciltg/
 RCILTS-Gujarati
Maharaja Sayajirao University, Baroda
1. Knowledge Resources
(Corpora, Parallel Corpora, Multi Lingual Digital
Dictionaries)
1.1 Gujarati WordNet (Lexical Resource)
Gujarati WordNet, a Lexical Resource, is in
the process of development. RC at MSU
is working in close collaboration with and
through able guidance of Professor Pushpak
Bhattacharya of RC at IIT, Mumbai. The Data
Entry Interface for entering the synsets is build.
Presently above 500 synsets are already been
developed.
Further work on preparing the synsets and entering
into the database is going on. Basic Semantic
Relationships are also been established like
Hypernymy, Hyponymy between the synsets formed
and entered in the database. Further work is going
on for establishing other Semantic relationships
between the synsets. The Web Interface is already
under construction to facilitate the access of an On-
Line Lexical Database of Gujarati to be available
over the Internet.
2. Knowledge Tools
(Portals, Language Processing Tools, fonts,
morphs analyzer, spell checker, text editor,
basic word processor, code conversion utility, etc.)
26
2.1 Portal
RC for Gujarati has a web site in English hosted at
http://msubaroda.ac.in/rciltg/ . The Gujarati version
of the same will be available soon. The web site
contains the information about the software
developments at RCILTG. The full text of the poems
and prose of well known Gujarati authors is planned
to be displayed on the web site. A manuscript of
19 th Century classic “Saraswatichandra” by
Govarthanram Madhavram Tripathi, in the author’s
own handwriting is located and digitized and would
be put on the web for the interested researcher soon.
2.2 Multilingual Text Editor
Unicode based Multilingual Text Editor which was
required since long for Indian Languages, has been
developed at the RC for Gujarati.
The important feature of this Text Editor is that it
fulfils the necessity of Universal storage format
(Unicode) for Indian Languages. The Indian
languages that are given support by this Text Editor
are Bangla, Gujarati, Gurmukhi, Hindi, Kannada,
Malayalam, Marathi, Oriya, Sanskrit, Tamil and
Telugu along with English as International language.
To make typing ease, assistance is given to the user
by providing an onscreen INSCRIPT keyboard
layout for all languages supported. The Text Editor
supports the basic file features like creating new file,
opening existing file and saving file. It is efficient
enough to open file of around 64KB. Also the basic
text editing features are provided like cut, copy, paste,
undo, redo, find, find next, replace and select all.
The user can use any of the system fonts for writing
text. The styles that are supported by Text Editor
are font name, font size, bold, italic and underline.
The sorting facility is also available based of word
sort or line sort. The support for three different Look
and Feels viz., Motif, Windows and Metal of Java
are provided to the user. The Gujarati Spell Checker
is also bundled with this Text Editor.
Current Focus
• To increase the efficiency of Text Editor.
Further Plans
• E-mail support for Indian Languages.
• ISCII compatible Import/Export facilities.
• More formatting features to be provided.
2.3 Code Converter
We have the programs which can convert the codes
between Unicode and ISCII.
2.4 Gujarati Spell Checker
The Gujarati Spell Checker is in its last stage of
development. It is based on a Morphological
Analysisof Gujarati language in order to increase the
intelligence of the Gujarati Spell Checker.
Morphological Analyzer covers the analysis of
Nouns and Verbs of the language.
The suggestion generation is very effective. The
number of suggestions that will be generated will
not be greater then ten. All the non-gujarati words
along with different symbols are ignored and are
not checked. The Gujarati Spell Checker is
integrated along with Multi-Lingual Text Editor for
the convenience to user.
Currently the software is under testing and very soon
the accuracy in percentage of Morphological Analysis
will be available.
Current Focus
• Rigorous testing to be done, using large corpus,
to find all the faults in the spell checker and
morphological analyzer.
• Improving the algorithms for the correctness and
efficiency of the spell checker.
Further Plans
• Increasing the size of the root dictionary covering
the maximum possible words of the language.
• Making an independent spell checker which can
be used on Unicode or ISCII compatible text.
• More and more testing to increase the correctness
of the spell checker and also improving the
suggestion generation.
3. Translation Support Systems
(Machine Translation, Multi Lingual Information
Access, Cross Language Information Retrieval)
3.1 Machine Translation
The work on this front has already started and
initially we have planned to be domain specific
27
and we have selected weather related text. A group
of two is working on this area. Basic Universal
Word Lexicon format is being studied.
Out of all the approaches we have selected to go
for Interlingua (UNL) based approach for this
purpose. The team has started the exploring the
various resources available on the UNDL
Foundation website. The specifications for
enconverter and deconverter are being studied.
The UNL proxy has been downloaded and tested
with the sample documents.
4. Human Machine Interface Systems
(Optical Character Recognition Systems, Voice
Recognition Systems, Text to Speech Systems)
4.1 OCR for Gujarati
A working prototype of Gujarati OCR which
is capable of doing end to end task i.e. converting
an image into an equivalent text, has been
produced. The OCR employs the template
matching technique, the same as the Telugu
OCR developed at RC for Telugu uses,
for recognition and nearest neighbour for
classification. The Gujarati OCR differs in
many ways from the Telugu one due to the
special features like the necessity for zone
separation.
The program, which can be invoked from
command line, takes a gray image, scanned in
color or grayscale, in any of the popular image
formats like JPEG, GIF, PNG, BMP and after
processing it produces the output as Plain Text
file in Unicode format. The input image is
assumed to be skew-corrected, without any images
and tables, having only one column. We have a
converter which coverts Unicode to equivalent
ISCII and vise-versa ready, so we can produce the
ISCII output as and when required.
Rigorous testing of the OCR for recognition rate
etc. is being initiated and the initial results are
quite good. Work is in progress for fine tuning
28
various modules in order to improve the
recognition accuracy.
Current Focus
• In the test we have seen some misrecognition,
so we are currently testing the performance
of Discrete Cosine Transform as a feature
extractor.
• The programs are getting reorganized to in the
Object Oriented way.
Future Plans
• Developing/Adapting skew-correction algorithm.
• Developing the post-processing routines.
• Thorough testing of the OCR and necessary fine
tuning to take it to the required accuracy of 97%
or above.
• Integrating with the OCRized spell-checker.
• Developing GUI for the system with inbuilt Text
Editor.
4.2 Text To Speech (TTS)
The work on TTS has already been started at RC-
Gujarati. A team of three students has explored the
available TTSs such as FreeTTS, IBMJS in English
and Dhvani and Naarad in Hindi. All of these are
the implementation of Java Speech API
specifications. We have made a significant progress
also in this direction.
While starting this work, we had divided our task in
the following two major subtasks :
1. Developing a program which generates the
output, which can be input to the existing
speech engine. E.g. IBMJS takes phonetic
transcription of the text using IPA codes, to
produce the sound.
2. Developing a core speech engine with native
accent. We are adopting the concatenation based
approach.
Task Completed So Far
• A program is now available with RC-Gujarati
which allows user to type in Gujarati text, using
INSCRIPT keyboard layout and then it
converts the string into its equivalent IPA string
and feeds it to the IBM speech engine, which
in turn produces the sound represented by given
IPA string.
Current Focus
• Exploring the phonological properties of
language, extracting the basic phonemes from a
continuous speech and design of basic speech
engine.
Future Plans
• Development of preprocessor to handle language
specific details.
• Developing speech engine and integrating with
preprocessor.
5. Localization
(Adapting IT tools and solutions in Indian
language(s)- IT localisation clinic interaction with
state government for possible solutions in Ils).
5.1 Language Technology Human Resource
Development
(Manpower Development in Natural Language
Processing - specialized training programs)
RC for Gujarati is trying to develop man power
by providing the resources to final year students
from the relevant background like Mathematics,
Computer Science, Electrical Engineering,
Electronics Engineering, Computer Application
etc. for doing their projects with RC. Two (one
ME (Electrical) and one M.Sc. (Statistics))
students have already completed their projects on
study and development of various techniques for
Gujarati OCR.
At presents Five students are working for their
final year projects. Three of which are working
on TTS technologies and two are working on
development of UNL based machine translation
support system.
Apart from this RC, Gujarati has trained three
physically challenged persons in Gujarati data entry
using inscript keyboard and now they are doing the
data entry for RC and for which they are being paid.
6. Standardization
(Contribution towards UNICODE,
Transliteration Tables(s), XML, lexware format, etc)
6.1 UNICODE
UNICODE for Gujarati have been studied and some
additions and corrections have been suggested. They
are as follows :
• Correction in the shape of AVAGRAH
• Addition of Gujarati Currency sign, Symbols for
¼, ½ , ¾ .
• Addition of two rendering rules for combination
of Ja with Dependent vowel sign E and EE
The collation sequence for Gujarati alphabets is not
same as the sequence of the characters in the
UNICODE chart. Hence the correct collation
sequence have been finalized.
All the above modifications and suggestions were
published in the April,2002 issue of Vishwabharat
magazine of TDIL.
In addition to above the RC for Gujarati has worked
closely with Mr. Abhijit Dutta of IBM, Delhi and
helped him in standardizing on the collation
sequence for Gujarati and the set of Glyphs for
developing the fonts.
Apart from development of basic technology RC
for Gujarati has, as per the original proposal
accepted by the Ministry, developed a strong
Knowledge base in Gujarati, available on
multimedia CDs and on our website. This
knowledge base includes the following;
29
1. Select Gujarati classics in print and
manuscripts, including a rare mss of the 19th
century classic, Sarasvatichandra in the
author’s handwriting, with many changes he
has made incorporated in the digitized form
on CD.
2. Cultural History and Geography of Gujarat
on CD, useful to researchers in one set and
to tourist in a second set.
3. Multimedia CDs on Rural Health Problems
and Preventive Care.
4. Language self-learning CDs useful to students
(learning Sanskrit, English etc.) and second
generation NRIs and new Researchers
abroad.
7.2 Products Which can be Launched
At present RC has a multilingual text editor ready
to launch. The detail of which is given above.
7.3 IT Services
RC for Gujarati has developed the following services
which it can provide to the State Government and
the Industry in the region:
• Multimedia CDs on Language Self-learning
(“Learn Gujarati through English”, “Learn
Sanskrit through Gujarati”, “Learn English
through Gujarati” etc.) could be of use to
learners in Gujarat and overseas, including the
NRIs. In opur interaction with him, these CDs
have been found of interest to Dr. M.N.
Cooper of Infosis Ltd. Pune.

7. Products which can be launched and the
service which the Resource Centre can provide to
the State Government and the Industry in the
region.
7.1 Technical Skills
• Multimedia CDs on Rural Health and Adolescent
Heath, developed in collaboration with
Tribhuvandas Foundation, Karamsad and its
Director, Dr. Nikhil Kharod, M.D., could be of
use to the Departments of Health and Education,
Gujarat Government and to the Panchayats.

• Services on Library Knowledge-base, giving

The RC for Gujarati has built technical skills
detailed information on books in print, 19th and
related to Image Processing and Pattern
20th century journals and medieval manuscripts
Recognition, Audio signal processing, Speech
Synthesis, Unicode based text processing and
rendering and implementation of these using
Java. These can readily be shared, with proper
agreement, with researches and industry in the
region. This knowledge can even be transferred
to the new generation by teaching some courses
in these areas at the nearby colleges.

RC-Gujarati has developed Multilingual Text

Editor which can be used to type in Unicode.
With the help of this, we could propagate the
use of standard encoding storage of data in
Unicode or ISCII. In turn this would lead to
the situation where the data can be transferred
from one machine to the other without available at some libraries in Gujarat and Mumbai,
transferring the fonts. This could be much use on CDs and our Website, could be of much use
in Government offices where such data transfer to Universities, Colleges and other institutions
is a daily routine. of research and teaching in India and abroad. The

30
 libraries with which we have entered into service
agreements have undertaken to provide
information through email, fax and surface mail
to users who would have access to this knowledge
base through our Website.
8. IT Services - Multimedia CDs
8.1 Learn Gujarati Through English
A multimedia tutorial is designed that helps an
enthusiast, learning Gujarati language though
English. The tutorial is based upon the concept of
Dr. Jagadish Dave who teaches Gujarati in Europe.
He has used this methodology to teach Gujarati to
the persons who doesn’t have any connection to
Gujarat and for that matter India.
The main features of the CD are:
• Novel approach for teaching Gujarati by
exploiting similarity of shapes for remembering
various characters.
• Starts with a familiar graphic resembling a
Gujarati letter and shows the manufacture of
related letters from it.
• For example, starting with the symbol ‘S’ which
is a character in English as well as Gujarati, the
Gujarati characters with related shapes DA, KA,
PHA, HA, THA are derived as morphological
modifications of the basic symbol.
• The related audio commentary figuratively
explains the modifications of the basic symbol.
• The pronunciation of Gujarati words in which
the character is embedded at various positions also
accompanies the presentation.
• After introducing a set of alphabets the learner is
lead to meaningful word formation involving
those alphabets.
• Introduction of grammar through simple
sentences instead of complicated rules.
Future Plans
• A practical conversation involving those words
suitable for handling a real life situation is to be
provided at a regular intervals
• At the end of the presentation a list of common
words, their meanings and their pronunciation
is to be added.
• The less common grammar rules are to be added.
• A video of some practical situations is to be
incorporated.
8.2 Adolescent Health
RCILTG organized a two days workshop on
Adolescent Health with Tribhuvandas foundation of
Anand on the above topic. About 40 teachers
participated in the workshop. The workshop was
aimed at adolescent health and how teachers can
scientifically disseminate the knowledge about the
changes occurring in male and female body during
this age.
This CD is a gist of ideas and suggestions, came out
after a few long sessions of brain-storming. The
information presented here is the work of Dr. Nikhil
Kharod (M.D. Pediatrician).
The features of this CD are:
• Introduction of sensitive issues from a scientific
view point.
• Accompanying pictures, sketches, diagrams and
tables giving medically proved information about
the topic.
• A detailed description of nutrients, their source
and disease caused in case of their deficiency.
• Description of the Body cycle of male and female.
• A quiz with key answers which gives, the user,
and an idea about his /her knowledge in this area.
Our future plan includes the production of a series
of learning language CDs. Work on the CD “Learn
Sanskrit through Gujarati” has already begun. Next
to follow would be a set of CDs linking Gujarati
with other Indian languages and with English. There
is a good marketing possibility for this series.
31
Initial work has been completed on production of a
set of CDs with multimedia presentation on Cultural
History and Cultural Geography of Gujarat. This
series would be useful for Primary and Secondary
Schools and also for tourism.
One CD on “Rural Health” has been produced,
focusing on Adolescent Health. Similar workshops
on other areas on Rural Health would be conducted
to produce more CDs on other aspects of health in
rural Gujarat.
been located by our RC. The mss, in the author’s
handwriting, has now been digitized and made into
a CD with an appropriate navigation.
Note: RC for Gujarati has trained three physically
challenged former students, Kum. Rina Phadia, Shri
Ganpat Solanki and Arun Paladgaonkar, in data
entry and has used their paid services for its data
entry work for Wordnet and CD production.
9. The Team Members

8.3 Bibliography of Books Prof. Sitanshu Mehta pi_rcmsu@satyam.net.in

Prof. S. Rama Mohan srm-cc@msubaroda.ac.in
In close collaboration with some important research
Mr. Jignesh Dholakia jignesh-rcilt@msubaroda.ac.in
libraries of Gujarati and Mumbai, the RC for
Gujarati has worked on preparing on-line Mr. Mihir Trivedi mihirktrivedi@rediffmail.com
bibliography of books and journals available at these Mr. Irshad Shaikh ishaikh16@yahoo.com
centres. Also on-line lists of critical and creative items
Courtesy: Shri Sitanshu Y. Mehta
(essays, fiction, poetry, drama, culture-related
M.S.University of Baroda
information) published in 19th and early 20th century
Department of Gujarati
Gujarati journals, has been digitized. The Libraries
Faculty of Arts
have agreed to provide photo-copied of the
Baroda – 390 002
knowledge-base items, at no-profit level, to the users
(RCILTS for Gujarati)
of the RC’s website.
Tel : 00-91-265-2792959
8.4 Sarasvatichandra E-mail : rciltg@satyam.net.in

Original manuscript of the 19th century classic,

Sarasvatichandra, by Govardhanram Tripathi, which
had a great impact on cultural life of Gujarat, has

32
 Resource Centre For
Indian Language Technology Solutions – Marathi & Konkani
I.I.T., Mumbai
Achievements

Computer Science and Engineering Department

Indian Institute of Technology, Mumbai-400076 India
Tel. : 00-91-22-25722545 Extn. 5879 E-mail : pb@cse.iitb.ernet.in
Website : http://www.cfilt.iitb.ac.in
 RCILTS-Marathi & Konkani
Indian Institute of Technology, Mumbai
Introduction
The Resource Center for Technology Solutions in
Indian Languages was set up at IIT Bombay in the
year 2000. The center concentrated on developing
niche technologies for Indian languages with special
focus on the languages of Western India, in
particular Marathi and Konkani. The main theme
of the research has been development of (i) lexical
resources like wordnets, ontologies and semantics-rich
machine readable dictionaries (MRD), (ii) interlingua
based machine translation softwares for Hindi and
Marathi and (iii) Marathi search engine and spell
checker. Over the years the center has built its
strength and reputation in its chosen areas and has
obtained national and international visibility. In
the paragraphs that follow we describe the
technologies already developed and in the process
of being developed, and also the impact the center
has made in the area of information technology
localization.
1. Lexicon and Ontology
1.1 Lexicon
The lexicon is being prepared in the context of
interlingua based machine translation. The
interlingua is the Universal Networking Language
(UNL) (http://ww.unl.ias.unu.edu) which is a
recently proposed interlingua. The heart of the UNL
based MT is the universal word (UW) dictionary.
UWs are essentially disambiguated English words
and stand for unique concepts. Our lexicon contains
about 70,000 UWs and approximately 200
morphological, grammatical and semantic attributes.
These entries are linked with Hindi headwords. The
lexicon is available on the web (http://
www.cfilt.iitb.ac.in/dictionary/index.html). UW-
Hindi Lexicon is required for enconversion
(analysis) and deconversion (generation) processes.
The lexicon can also be used as a language reference
for English and Hindi. We are now concentrating
on completing the language coverage and
standardizing the lexicon (standard restrictions and
semantic attributes).
34
1.2 Ontology
An ontology is a hierarchical organization of
concepts. Domain specific ontologies are critical
for NLP systems. We have classified Nouns as
animate and inanimate. Further sub-classification
divides animate nouns as flora and fauna to cover
plants and animals. The inanimate category is sub-
classified as object, place, event, abstract entity etc.
Concept nouns indicating time, emotion, state,
action etc. have also been incorporated. Verbs have
been divided into do, be and occur categories, with
further sub-classifications verbs of action, verbs of
state, temporal verbs, verbs of volition, etc.
Adjectives have been classified as descriptive
indicating weight, colour, quality, quantity, etc. and
also as demonstrative, interrogative, relational and
so on. Adverbs are categorized according to time,
place, manner, quantity, reason, etc.
Statistics of Ontological Categories
We have 22 broad categories for Noun, Verb,
Adjective and Adverb. Under these categories, we
have 57 sub-categories for all word classes. The noun
class has 55 sub-sub and 29 sub-sub-sub categories
(vide the table below):
1. No. of Categories of Noun, Verb, Adjective 22
and Adverb
2. No. of Sub-categories of Noun, Verb, 58
Adjective and Verb
3. No. of Sub-sub Categories of Noun 55
4. No. of Sub-sub-sub Categories of Noun 29
Total No. of Categories 164
2. The Hindi WordNet
The Hindi wordnet is an on-line lexical database.
The design closely follows the English wordnet.
The synonym set {Gar, Gar, , gaR h } and {Gar
Gar
Gar, pirvaar
pirvaar}- for
example- can serve as an unambiguous
differentiator of the two meanings of {Gar Gar
Gar}. Such
synsets are the basic entities in the wordnet. Each
sense of a word is mapped to a separate synset in
the wordnet. All word sense nodes are linked by
a variety of semantic relationships, such as is-a
(hypernymy/hyponymy) and part-of (meronymy/
holonymy).

The lexical data entered by linguists and

lexicographers are stored in a MySQL database. The
web interface (www.cfilt.iitb.ac.in) gives the facility
to query the data entered and browse the semantic
relations for the search word. The interface also
provides links to extract semantic relations that exist
in the database for the search word. So far
approximately 10000 synsets have been entered. This
corresponds to about 30,000 words in Hindi. There
is a morphological processing module as a front end Figure 3: Online web interface of the Hindi WordNet
to the system.

Figure 1: Snapshot of Data entry interface for entering Figure 4: Snapshot of web interface for the query result
synset data for the word “janmadata”

3. The Marathi WordNet

Since Marathi and Hindi share many common

Figure 2: Snapshot of Data Entry interface for entering
semantic relations for noun category
features, it has been possible to adopt the Hindi
wordnet as the basis for Marathi wordnet creation.
Hindi and Marathi originate from Sanskrit and,
therefore, the t%sama (words taken directly from
mother language, e.g. gait, kRit) and the td\Bava (a
word which has evolved organically from the
mother language, e.g. gaRh ! Gar, kma- ! kama) words
in both the languages are often same in meaning.
Also, both the languages have the same script-
Devanagari.
In the construction of the Marathi wordnet, a
Hindi synset is chosen and mapped to the

35
corresponding Marathi synset. For instance, for
the Hindi synset {kagad kagad
kagad, kagaja
kagaja}- standing for paper-
the Marathi synset is {kagad kagad
kagad}. A gloss is added,
which explains the meaning of the word, and an
explanatory sentence is inserted. For the current
example, the gloss is baaMbaU va gavat yaaMcyaa lagadyaapasaUna tyaar
kolao jaaNaaro patL pana jyaavar ilahIlao vaa Caplao jaato and”%yaanao
kao-yaa kagadavar maaJaI sahI GaotlaI”. So far, 3400 synsets
have been entered. This corresponds to about
10,000 words in Marathi. Our aim is to cover all
the common Marathi words. As the Marathi
wordnet is based on the Hindi wordnet, all the
semantic relations become automatically
inherited. An additional benefit to accrue is the
creation of a parallel corpus.
4. Automatic Generation of Concept Dictionary
and Word Sense Disambiguation
A Concept (UW) Dictionary is a repository of
language independent representation of concepts
using special disambiguation constructs. A system
has been developed for automatically generating
document specific concept dictionaries in the
context of language analysis and generation using
the Universal Networking Language (UNL). The
dictionary entries consist of mappings between
head words of a natural language and the universal
words of the UNL. The manual effort in
constructing such dictionaries is enormous,
because the lexicographer has to identify the
disambiguation constructs and the semantic
attributes of the entries. Our system, which
constructs the UW dictionary (semi-)
automatically, makes use the English wordnet and
processes the part of speech tagged and partially
sense tagged documents. The sense tagging step is
a crucial one and uses the verb association
information for disambiguating the nouns. The
accuracy of the word sense disambiguation system
is 70-75% and that of the DDG (document
specific dictionary generator) is 90-95%.
The DDG makes use of a rule base for producing
the dictionary. It is based on the principle of expert
systems, doing inferencing and providing
explanations for the choices made with respect to
the restrictions and semantic attributes.
36
Figure 5: stages in automatic dictionary generation
5. Hindi Analysis and Generation
5.1 Hindi Analysis (Enconversion)
The analysis engine uses the UW-Hindi dictionary
and the analysis rule base. The dictionary contains
headword, universal word and its grammatical and
semantic attributes. The process of analysis depends
on these semantic and grammatical attributes. There
are 41 relations defined in the UNL specification.
The system is capable of handling all of them. The
analysis system can deal with almost all the
morphological phenomena in Hindi. At present,
there are 5500 rules dealing with morphosyntactic
and semantic phenomena. The system has been
tested on corpora from the UN, the MICT and the
agricultural domain.
5.2 Hindi Generation (Deconverion)
The generation process also uses the UW-Hindi
dictionary and the generation rule base. The dictionary
is the same for both analysis and generation purpose,
only the rule base is different. The generation rule
is formed from the grammatical and semantic
attributes as well as the syntactic relations. Some of
the features of the system are:
• Matrix-based priority of Relation has been designed
to get the correct syntax.
• Extensive work has been done on Morphology.
• The system has gone through the UNL of corpora
from various domains with satisfactory results.
• There are almost 6000 rules for syntax planning
and morphology of all parts of speech.
6. Marathi Analysis
Marathi is a natural language of Indo-Aryan family.
The textual communication uses the Devanagari 8. Project Tukaram
(nä´ÉxÉÉMÉ®Ò) script. The analysis and generation of
Tukaram Project provides the entire collection of
Marathi makes use of the Marathi-UW lexicon and
Saint Tukaram’s Abhangs in browsable and searchable
the rules for the Marathi language grammar. There
format. Tukaram’s abhangs are a household name in
are about 2100 rules to handle Marathi verb
phenomena which are listed below: Maharashtra. It is possible to convert the existing
version to a stand-alone system, which can be
Sr. No. Phenomenon Description installed on a single machine. The abhangs are
1 Present tense Time w.r.t. writer available for public viewing on http://
2 Past tense Time w.r.t. writer www.cfilt.iitb.ac.in (also see the picture below).
3 Future tense Time w.r.t. writer Browsability is provided at chapter and verse levels.
4 Event in progress Writer’s view of aspect Users can move from one chapter or verse to another
5 Complement Writer’s view of by using links at the bottom of the webpage.
reference
6 Passive voice Writer’s view of topic The search engine for Tukaram Project is made
7 Imperative mood Writer’s attitude compatible with both Linux and Windows.
8 Ability of doing Writer’s viewpoint Tukaram’s Abhangas were keyed in Akruti font and
something were later converted to ISFOC font. ISCII encoding
9 Intention Intention about is used by the Tukaram search engine.
something or to do
something To enable the users to use the Tukaram search engine
10 Should To do something as a from both Windows and LINUX, we chose the
matter of course XDVNG font for the input technology while
11 Unreality Unreality that keeping ISFOC for display purposes only. Clients
something is true or can key in their queries through phonetic English
happens designed by Sibal (1996). Clients’ input queries
typed in phonetic English are displayed at clients’
7. Speech Synthesis for Marathi Language terminal in Marathi using the XDVNG font.
The aim of this project is to design speech synthesis Phonetic English to XDVNG mapping is available
system to speak out Marathi text written in as JavaScript from http://www.sibal.com/sandeep/
Devanagari script. Concatenative speech synthesis jtrans. Since JavaScript is used for mapping, it is easy
model is used to achieve this goal. Ours is an to integrate it in a page.
unlimited vocabulary system. It employs basic units,
i.e., vowels and consonants as the basis for A query in XDVNG is sent to the server at IIT
constructing words and sentences. The database of Bombay and is converted into ISCII at the server.
basic speech units, required in this approach, is few For this conversion, an XDVNG to ISCII converter
hundred kilobytes as against other approaches where provided by IIIT Hyderabad is used after
the size is almost tens of megabytes. We are able to modifications. The features of Project Tukaram are
get intelligible speech with this approach. A graphical summarized in table below:
user interface for the system has been developed. It Font and Package used for Akruti in MS
provides features for drawing waveforms and for Original Script Word
amplifying the output speech signal. It also provides,
help for displaying Marathi text through English Number of converted HTML 165
keyboards. chapters
Currently we are working in rendering newspapers Number of converted HTML 4614
as speech is taking shape. On-line reading of Marathi verses
newspaper Maharashtra Times is being carried out
as an illustration of the technology. Font used in HTML display pages ISFOC

37
Total number of words excluding 2,12,442 The N-grams method has been developed based on
HTML tags a rank order of short letter sequences. A rank ordered
list of the most common short character sequences
Number of distinct words used for 34,773
the Indexing is derived from the training document set. A rank
ordered list is prepared for every language under
Keyboard mapping at client Phonetic English consideration. Each such list is called a profile.
Query Display Technology at XDVNG Language identification is done by measuring the
Client closeness of the test document to each of these
profiles. The profile, which is closest to the test
Result Display Technology at ISFOC document, gives the language of the test document.
Client Figure 1 shows the data flow diagram of the classifier.
Encoding used for Indexing ISCII We have extended this approach for Devanagari
script, as the conventional approach was not giving
Database Server MySql desired accuracy.
Languages used for Converters Lex and C Letter Approach: In this approach, each letter in
Language used in Search Engine JAVA Devanagari is treated as a character in an N-gram.
That means an N-gram may contain more than “N”
Language used in Client Side JavaScript
characters. On the other hand, an N-gram will be
exactly “N” (possibly half ) letters in Devanagari
script.
Conjunct Approach: This is a variant of the previous
approach. We draw motivation for this approach by
describing Conjuncts. Indian scripts contain
numerous such conjuncts, which essentially are
clusters of up to four consonants without the
intervening implicit vowels. The shape of these
conjuncts can differ from those of constituting
consonants. For example, <SUÉ has a conjunct SUÉ. It
comprises of two consonants, SÉ and U. The
consonant SÉ is considered a half letter. In the
Conjunct approach, a conjunct is considered as a
single letter.
9. Automatic Language Identification of Documents
using Devanagari Script Results: From experimental results, it could be
observed that although the common words
The problem of language identification has been
technique is computationally cheaper, the N-gram
addressed in the past. There are existing methods
techniques are more accurate. Also, it was clear that
like unique letter combinations, common words, and
our extensions to N-grams work very well than the
N-grams technique. A subtle issue here is the length
conventional N-grams approach. The N-gram
of the text available for classification. Many methods
approach was very much suitable for another reason.
require longer texts for language identification. Also,
some methods require some a priori linguistic The documents used in experiments had textual
knowledge. Another issue arises when the test set errors. As N-grams method is quite robust to textual
for the classification program contains some semantic errors, the results were largely satisfactory. In our
errors. The classification technique should be robust approach, although the developing the method itself
in such a way that these errors do not affect the required some linguistic knowledge, the method is
accuracy of the classifier. completely automatic and does not require any fine-

38
tuning as required in case of common words
approach.
10. Object Oriented Parallel and Distributed Web
Crawler
The vastness and dynamic nature of the WWW has
led to the need for efficient information retrieval. A
crawler’s task is to fetch pages from the web. The
crawler starts with an initial set of pages, retrieves
them, extracts URLs in them and adds them to a
queue. It then retrieves URLs from the queue in
some specific order and repeats the process. We have
designed and implemented an object oriented web
crawler. Our crawler has 4 components: Graph
component, UrlManager, Buffer and Document
Processor. The crawler has a 4-tier architecture with
the storage file system at the lowest layer and crawl-
buffers at level 4. The architecture is shown in figure
below. The Graph component is at level 2 and it
interacts with the file system. The UrlManager and
Document Processor are at level 3. This layer forms
the business logic layer. The crawler architecture is
parallel and distributed. It uses a CORBA
environment and runs on a Linux Cluster.
User Level Storage System
A search engine requires large amount of storage
space for storing the crawled pages. The existing
databases and file systems are not specialized to
handle storage of structures such as web graphs. We
are building storage systems for efficient storage and
retrieval to this requirement. In the storage system,
webpages correspond to the nodes of the graph and
the hyperlinks correspond to the edges. Graph
traversal methods can be applied for crawling
purposes.
In the current version of the storage system, Berkeley
Database (BDB) over Linux has been used as the
backend for storing webpages. An advantage of
BDB is that the library runs in the same address
space and no inter-process communication is
required for the database operation. Secondly,
because BDB uses a simple function-call interface
for all operations, there is no query language to
parse.
The storage system provides a set of API for
application programs such as crawlers. The APIs
help keeping users’ applications independent of
implementation of the storage system. The
storage systems recognizes primitives such as nodes
which represent web pages and graphs which
represents a specific WWW graph. A storage API
encapsulates read and write functions. It provides
functions such as create graph, store and read a
webpage, parse the page and get outlinks.
11. Designing Devanagari Fonts
The following three types of ‘Arambh’ family fonts
are created:
• 8-bit true type font (.ttf )
• 16-bit Unicode bitmap ascii font (.bdf )
• 16-bit Unicode true type font (.ttf ).
Though there are many Devanagari fonts
available, no single font contains all glyphs
required for displaying documents such as
Damale’s grammer and Saint Tukaram’s abhangas.
Hence a new font is being created to handle the
additional requirements. The font called 8-bit
Arambh, is a true type font, which covers all
required glyphs. In this font we get the scalability
feature of true type fonts. Two types of 16-bit
Arambh Unicode fonts developed are ArambhU
(bdf ) font and ArambhU (ttf ) font. Each glyph in
BDF is represented as a matrix of dots. These fonts
are rendered faster than TrueType fonts. Each
glyph in the TrueType font is represented as
outline curve thus facilitating scalability.
39
12. Low Level Auto Corrector
We have built a low level syntactic checker and
autocorrector based on a classification of low-level
syntactic errors. The tool uses a set of rules to detect
and correct the errors. It was used in project Tukaram,
and over 300 errors were detected and automatically
corrected. The kinds of errors handled are visible
and invisible errors due to multiplicity, ordering,
mutual exclusion and association properties.
Types of Low Level Syntactic Errors
Below, a classification of low-level syntactic errors
in a document written in an Indian Language is
provided with a few examples. Reasons of these
errors may be attributed to violations of the
constraints of multiplicity, ordering and
composition of ligatures, especially of the matra
characters.
Multiplicity
Typically, ligatures such as matra, halant and anuswar
are used only once in a single letter composition.
While typing, it may be possible that they get typed
twice and the result is still not visible in the text in
an Indian Language font. Such errors can be
classified as invisible errors, while some remain
visible in the display.
Invisible Errors
For example, in the case of an extra anuswar in letter
+ÆÆ the source in ITRANS is given by aM, whereas,
aMM also produces +Æ. The Devanagari characters
seen in the previous sentence were actually produced
by two different encodings. While the result of this
error is not visible in the display, this could be a
cause of concern for an application such as a pattern
matching algorithm, an index or a converter.
Invisible errors may also occur in the case of multiple
ukar and matra ( Ù and ä) characters. For example, in
{ÉÚ an extra ukar is present but it is invisible, or
overlapped.
Visible errors
While some ligatures overlap in display and the errors
remain undetected by mere human inspection of
the display, some errors may be detectable by display
40
inspection. This is possible due to positioning
adjustments done on ligature placements.
For example, an extra kana in Marathi, such as in
letter {ÉÉ, is visibly detectable if it occurs more than
once since the additional ligature is right shifted and
either displayed as a kana, or taken as a separate
character. An example of the later case is encoding
paaa in ITRANS, which is displayed as {ÉÉ+
Ordering
In some cases, more than one vowel can occur in a
single letter. For example, an anuswar and a kana in
Marathi as in letter {ÉÆ É may be typed in
Akruti_Priya_Expanded in two ways, as hebe (where
h stands for { , e for É and b for anuswar), or as heeb.
While the first choice results in an unaesthetic
placement of anuswar on top of the first vertical
line, the second choice gives the desired result. With
a smaller font size, the differences may get
overlooked, but with an enlarged font size, they are
easily detected.
Mutual Exclusion
In Indian Languages, the composition rules are also
specified for mutually exclusive vowels. For
examples, an ukar and a kana cannot occur in one
character. Similarly, a velanti and an ukar or a kana
cannot occur in one letter. Whereas, it is possible
for a kana and a matra to occur together in one
letter.
Association
Certain consonants do not accept rafar (i.e., Ç ) or
specific conjuncts. For example, a combination of
³ and Ç is invalid. Similarly, it is not possible to
combine certain consonants such as YÉ with other
consonants, or ½ with {É.
13. Font Converters
Following is the list font converters have been
developed by IIT Bombay.
• IITK to ISCII
• ISCII to IITK
• Akruti_Priya_Expanded to ISCII
• SHUSHA to ISCII
• IITK to SHUSHA
• DV-TTYogesh to ISCII
• ISCII to DV-TTYogesh
14. Marathi Spell Checker
Under this ongoing project, a stand-alone
spellchecker is being built for Marathi. The
spellchecker will be available to spell check
documents in a given Encoding.
From the CIIL (Central Institute of Indian
Language) corpus, 12,886 distinct words have been
listed. Similarly other Marathi texts at the center
are being used to build a basic dictionary. A
morphological analysis is being carried out on the
collection of words. For example, an automatic
grouping algorithm identified 3,975 groups out of
12,886 distinct words. First word is usually the root
word. Thus, there are approximately 4000 root
words from Marathi corpus. A manual proof reading
will be done on these results. The morphology will
be enriched.
A motivation behind the stand-alone spellchecker is
that it can be used without an editor through a
packaged interface, or it can be integrated with other
compatible applications such as OCR.
Content creation tools
The Marathi content placed on the CFILT website,
viz. Saint Tukaram’s Abhangas, Soundarya Meemansa
and Damle’s Marathi grammar have been typed using
Akruti Software along with MS Word. Akruti
provides three keyboard layouts: Typewriter, English
Phonetic and Inscript. MS Word has the facility to
convert the Word files into web pages. We use this
convert option. The web pages created by MS Word
have a lot of Internet Explorer specific style sheet
tags and attributes. Due to this the appearance of
the Devanagari text on the web page is different in
Internet Explorer on Windows and Netscape
Navigator on Linux. Hence we remove those style
sheet tags from the web pages by using a Java
program. The files are then uploaded to the web
server.
For Marathi search engine Marathi webpages were
crawled from the web. The webpages crawled were
in DV-TTYogesh, DV-TTSurekh font. These font
codings of the webpage text was converted into ISCII
coding. An inverted index is used to store the words
and their attributes like whether the word appears
in bold, italic, whether it appears in the title of the
webpage and other attributes. MySql is used for
storing the inverted index. The user interface uses J-
Trans, which is a JavaScript code. The user types the
query in phonetic English. When the user clicks the
“search” button, the query words are looked up in
the inverted index. The documents which contain
all the query words are noted. Relevancy of a
document is calculated according to the attributes
of the query words in that document. For example:
Bold or italic words are considered more important
than non-bold or non-italic words. The documents
are then sorted in descending order of their
relevancy. The results are then displayed to the user.
Thus the user gets the most relevant Marathi
webpages.
15. IT Localization
IT awareness
There was a meet in October, 2001 with the media
to apprise them of the technology development
efforts going on in India and at the Resource Center
in IIT Bombay on Indian Languages. Leading
newspapers like Times of India, Maharastra Times,
Loksatta, Lokamat, Sakaal sent their representatives
to this meet. The systems developed in the IITB
center were demonstrated and presented. The
newspapers carried reports on this.
Technology solutions
IIT Bombay is providing its expertise for developing
wordnets for Indian languages (please details below).
We recently taught the fundamentals of wordnet
building in the Indo-wordnet workshop at CIIL
Mysore. Licensing agreement for transferring the
lexical data, the user interface and the API for the
Hindi wordnet is being worked out with advice from
the Ministry.
Interaction with State Government
The interaction is mainly with the Marathi
Rajyabhasha Parishad which is the authorized body
of the Maharastra state government for
developments related to the Marathi language.
41
Similarly interaction has been going on with the 17. The Team Members
Government of Goa for the Konkani language. We
Arti Sharma arti_sharma80@yahoo.com
conducted a very successful Language technology
conference in Goa called the International Ashish F. Almeida ashishfa@cse.iitb.ac.in
Conference on Knowledge and Language (ICUKL) in Deepak Jagtap deepak@cse.iitb.ac.in
Goa with generous support from the MICT-TDIL, Gajanan Krishna Ranegk rane@cse.iitb.ac.in
in which the state of Goa participated very actively
Lata G Popale lata@cse.iitb.ac.in
and supported the event wholeheartedly.
Jaya Saraswati jayas@cse.iitb.ac.in
16. Publications Laxmi Kashyap laxmi_kashyap@hotmail.com
Dipali B. Choudhary, Sagar A. Tamhane, Rushikesh Madhura Bapat madhurasbapat@rediffmail.com
K. Joshi, A Survey Of Fonts And Encodings For Indian Manish Sinha manish@cse.iitb.ac.in
Language Scripts, International Conference On Prabhakar Pandey pande@cse.iitb.ac.in
Multimedia And Design (ICMD 2002), Mumbai,
Roopali Nikam roopali@cse.iitb.ac.in
September 2002.
Shraddha Kalele s_mahapurush@yahoo.com
Shachi Dave, Jignashu Parikh and Pushpak Shushant Devlekar suSAMwa@yahoo.com
Bhattacharyya, Interlingua Based English Hindi
Sunil Kumar Dubey dubey@cse.iitb.ac.in
Machine Translation and Language Divergence, to
appear in Journal of Machine Translation, vol 17, Satish A Dethe satishd@cse.iitb.ac.in
2002. Vasant Zende cukl2002@cse.iitb.ac.in
Veena Dixit veena@cse.iitb.ac.in
P. Bhattacharyya Knowledge Extraction from Texts in
Multilingual Contexts, International Conference on M.Tech students
Knowledge Engineering (IKE02), Las Vegas, USA, Dipali B. Choudhary dipali@cse.iitb.ac.in
June 2002. Nitin Verma nitinv@cse.iitb.ac.in
Hrishikesh Bokil and P. Bhattacharyya Language Sagar A. Tamhane sagar@cse.iitb.ac.in
Independent Natural Language Generation from Ph.D. student
Universal Networking Language, Second Debasri Chakrabarti debasri@cse.iitb.ac.in
International Symposium on Translation Support
Systems, IIT Kanpur, India, March, 2002. Courtesy: Prof. Pushpak Bhattacharya
Indian Institute of Technology
Dipak Narayan, Debasri Chakrabarty, Prabhakar Department of Computer Science & Engineering
Pande and P. Bhattacharyya An Experience in Mumbai- 400 076
Building the Indo WordNet- a WordNet for Hindi, in (RCILTS for Marathi & Konkani)
First International Conference on Global WordNet, Tel: 00-91-22-25767718, 25722545
to be held in Mysore, India, January, 2002. Extn. 5479, 25721955
Shachi Dave and P. Bhattacharyya Knowledge E-mail : pb@cse.iitb.ernet.in
Extraction from Hindi Texts, Journal of Institution
of Electronic and Telecommunication Engineers,
vol. 18, no. 4, July, 2001.

P. Bhattacharyya Multilingual Information Processing

Using Universal Networking Language, in Indo UK
Workshop on Language Engineering for South
Asian Languages (LESAL, Mumbai, India, April,
2001.

42
 Resource Centre For
Indian Language Technology Solutions – Punjabi
Thapar Institute of Engineering & Technology, Patiala
Achievements

School of Mathematics and Computer Applications (SMCA)

Thapar Institute of Engineering & Technology
(Deemed University) Patiala-147001
Tel. : 00-91-175-2393382 E-mail : punjabirc@yahoo.com
Website : http://punjabirc.tiet.ac.in
 RCILTS-Punjabi
Thapar Institute of Engineering &
Technology, Patiala
Introduction
The Resource Centre for Indian Languages
Technology Solution-Punjabi was established in
April 2000 at Thapar Institute of Engineering &
Technology working in TDIL Programme of
Department of Information Technology. The
Resource Centre is supported by the Ministry of
Communications and Information Technology
(MCIT) with the aim of working for technology
development of Punjabi and also to provide access
to millions of people across the globe. The main
task before the resource centre has been the
promotion of Punjabi language so as to extend
the benefit of knowledge and awareness across the
globe. It has since evolved into a multi-faced
research center. Before the Punjabi Resource
Centre was set up, very little work had been done
for computerization of Punjabi, even though large
number of Punjabis settled in USA, UK and
Canada have been using computers for long. The
only work done was development of Punjabi fonts
and some Punjabi websites. There was no Punjabi
spell checker, Punjabi sorting utility, electronic
dictionaries or Gurmukhi OCR. In the span of
three years, we have developed for Punjabi lexical
resources, content creation tools, Gurmukhi OCR
and uploaded Punjabi text on web. The details of
the work done during the last three years is as
follows:
fact there are numerous words which are
spelled in more than one way, some are
being written in 3-4 different ways. After
discussions with Punjabi scholars it was
decided to retain multiple spellings for the
commonly used words and a spell checker
for Punjabi has been developed taking
Harkirat Singh’s Shabad Jor Kosh as the base
and additional words were taken from the
dictionaries published by Punjabi University
and the State Language Department and the
corpus developed by CIIL, Mysore.
For generating the suggestion list, a study
was conducted to discover the most
common errors made by Punjabi typists. A
list of similar sounding words and
consonants was also complied and a
suggestion list using this knowledge was
generated based on reverse edit distance. It
was found that the right suggestion is
presented as a default suggestion in majority
of the cases. In such a case, the user needs
only to confirm the default suggestion and
proceed with the next error. Otherwise, the
user needs to scroll through a list of

1. Products Developed

1.1 Spell Checker : A spell checker is a basic

necessity for composing text in any language.
Till now, no spell checker was available for
Punjabi as a result people had to waste lot of
time proof reading the Punjabi text. This
problem was more severe for publishers and
writers residing abroad who used computers
to type Punjabi material, as even Punjabi
dictionaries are not easily available to them.
To make the matters worse, there has been
no standardization of Punjabi spellings. In Fig.1 : Spell Cheker

44
 suggestions and pick one as the right one.
The spell checker supports text typed in any
of the popular Punjabi fonts as well as ISCII
encoded files. The Punjabi spell checker is
now complete and MOU is soon going to
be signed with M/S Modular Systems, for
Transfer of Technology of the spell checker.
Output of the Spell Checker is shown in
Fig. 1.
1.2 Font Converter : The major development
in computerization of Punjabi before the
setting up of the Punjabi Resource Centre
was creation of Punjabi fonts. As a lot of
Punjabis reside in USA, Canada and UK, so
a large variety of fonts were developed. In
the absence of any standardization, each font
had its own keyboard layout. The net result
is that now there is a chaos as far as the
Punjabi language in electronic form is
concerned. Neither can one exchange the
notes in Punjabi as conveniently as in
English language, nor can one perform
search on texts in Punjabi. This is so because
the texts are being stored in font dependent
glyph codes and the glyph coding schemes
for these fonts is typically different for
different fonts. To alleviate this problem, a
font conversion utility has been developed
at the Resource Centre. The utility supports
more than sixty different Punjabi fonts and
thirty two keyboard layouts. A user may type
text in any font and using this utility he can
later on convert it to any other font. This
utility will be a great help to publishers,
writers and people exchanging text in
different fonts.
1.3 Sorting Utility : Sorting and Indexing is one
of the basic necessities of the database
management system such as maintenance of
student’s records or arranging a dictionary
in alphabetic order. But unfortunately there
does not exist any software for automatic
sorting of Gurmukhi words and all such
work has to done manually. The collating
sequence provided by UNICODE or ISCII
is not adequate, as it is not compatible with
the traditional sorting for Gurmukhi words.
Gurmukhi, like other Indian languages has
a unique sorting mechanism. Unlike
English, consonants and vowels have
different priorities in sorting. Words are
sorted by taking the consonant’s order as the
first consideration and then the associated
vowel’s order as the second consideration.
In addition there is another complication.
The properly sorting “characters” in
Gurmukhi often requires treating multiple
(two or three) code points as a single sorting
element. Thus we cannot depend on
character encoding order to get correct
sorting instead we have to develop using
sorting rules of Gurmukhi, collation
function which converts the word into some
intermediate form for sorting. After
discussions with linguists and studying in
detail the alphabetic order in Punjabi we have
developed a general sorting algorithm which
works on text encoded in any popular
Gurmukhi font or ISCII.
1.4 Bilingual Punjabi/English Word Processor:
A Bilingual Punjabi/English Word processor
has been developed keeping in view the
difficulties being faced by the users while
working on the currently available word
processors. It was found that as far as Punjabi
is concerned no word processor which
addresses the word processing requirements
typical for Punjabi language, such as Punjabi
spell checker, Punjabi thesaurus and
dictionary, virtual Punjabi keyboards and
Punjabi sorting and font conversion utilities,
has been developed. In this direction, efforts
were made a Punjabi word processor Likhari
was developed. Likhari supports word
processing under the windows environment
and allows typing and processing in Punjabi
Language through the common typewriter
keyboard layout. It has MS-Word
compatible features and commands. It
provides a number of features that make the
use of Punjabi Language on a computer easy
and provides a number of tools to increase
the efficiency of the user. These tools include
45
 Bilingual Spell Checker with suggestion list,
Onscreen keyboard layouts with
composition reference for Punjabi Language
typing, bilingual search and replace, sorting
as per the language, alphabetical order,
technical glossaries and onscreen Bilingual
dictionaries. The main features of Likhari
are :
1.5 Gurmukhi OCR : Optical Character
• Very simple user interface
• Online active Keyboard for users who
do not know how to type in Punjabi.
• Choice of Phonetic, Remington and
Alphabetic Keyboard layouts with
composition reference.
• Bilingual Spell Checker for Punjabi
and English.
• Bilingual Search and Replace.
• Support for sorting the text in English
or Punjabi as per the language
alphabetical order.
• Support for more than 60 commonly
used Punjabi fonts.
• Support for features like Tables,
Numbering, Bullets, Character
& Paragraph formatting, Page set
up, Print Preview, Header and Footer
etc.
Fig 2 : A Screen Shot of Punjabi WordProcessor Likhari
46
• Online Technical English-Punjabi
glossary.
• Support for. ISCII, .TXT, DOC, .RTF
and .HTML file formats.
• Extensive help at various levels to make
it easy for the user to learn.
Recognition (OCR) is the process whereby
typed or printed pages can be scanned into
computer systems, and their contents
recognized and converted into machine-
readable code. The text, which the machine
can read, has great advantages over that
merely displayed as an image, since it can be
edited, exported to other programs and
indexed for retrieval on any word. If one
needs to electronically store and manipulate
large amounts of text or printed matter such
as newspapers, contracts, letters, faxes, price
lists or corpus development, one will find
OCR programs can save lots of effort. For
the first time a complete OCR package for
Gurmukhi script has been developed. The
OCR has a recognition accuracy of more
than 97%. The OCR can automatically
detect and correct skewed pages and can also
detect page orientation. The OCR can
recognize multiple Punjabi fonts and sizes.
The main features of the Gurmukhi
OCR are:
Recognition Accuracy
Recognition accuracy is around 97% for
books, photocopies and medium degraded
documents and around 99.5% for laser print
outs and good quality documents.
Import Image
B/W Images, 24-bit color images, 256
grayscale images.
Output File Format
ISCII / text file encoded in any one of the
popular Punjabi fonts.
 Fonts Supported two sides of the Punjab, which prevents
cultural and literary exchanges — there is
All non-decorative Gurmukhi fonts.
ignorance on both sides about developments
Proofing in contemporary prose and poetry on the
other side. To break this wall, it is necessary
On-screen verifier
to develop transliteration programs which can
Additional Features automatically convert Gurmukhi text to
Shahmukhi and vice-versa. In this direction
Inbuilt Spell checking facility. Automatic the Punjabi Resource Centre has collaborated
skew detection and correction (skew range – with the Urdu Resource Centre established
5 to +5 degrees). Upside down image auto at CDAC, Pune to develop a computer
detection and correction. program which automatically converts
Gurmukhi text into Shahmukhi. Using this
software, a collection of short stories penned
by Mr. K. S. Duggal in Gurmukhi has been
converted into Shahmukhi (Figs 4-5).

Fig. 3 : A Screen shot of Gurmukhi OCR

1.6 Gurmukhi to Shahmukhi Transliteration:

Punjabi language is used in both parts of
Punjab in India and Pakistan. In East Punjab
( India ) Punjabi is written in Gurmukhi
script. This script was invented by Guru
Fig 4 : A Short Story in Gurmukhi
Angad Dev ji. This is written from left to
right. In West Punjab ( Pakistan ) Punjabi is
written in Persian script , also known as
Shahmukhi script. This is written from right
to left like Urdu and Persian. In East Punjab
many Sanskrit words are absorbed into
Punjabi language and similarly in West
Punjab many Persian and Urdu words are used
in Punjabi. But the essence of Punjabi
language remains same. Both Shahmukhi and
Gurmukhi have been in use simultaneously.
After the Partition, Punjabi on the Indian side
of the border was restricted officially to the
Gurmukhi script. On the Pakistani side it was
restricted to Shahmukhi script. The result is Fig 5 : The Short Story in Gurmukhi of Fig. 4
that a script-wall has come up between the converted to Urdu

47
2. Contents Uploaded on Internet

For benefit of common man, Punjabi literature,

dictionaries and products for free download have
been uploaded on the Resource Centre’s website
(http://punjabirc.tiet.ac.in). The contents
uploaded on the internet are:

2.1 Literature : Punjabi Classics such as

Bullehshah Dian Kafian, Farid De Salok, Heer.
Luna, Chandi di War, Japji Sahib have been
uploaded in Punjabi using Punjabi Dynamic
Fonts. The website contains the detailed
description of these classics. Tool tips of
difficult words have also been provided in
Punjabi. Audio clips of these classics have also
been included.

2.2 Bilingual Dictionaries and glossary : The

following bilingual dictionaries and glossary
have been uploaded at our site.

• Punjabi English On-line Dictionary : A

Punjabi English dictionar y is made
available on the website. The dictionary
has about 40,000 Punjabi words. The
dictionary has sample sentences for
common words and audio clips of
pronunciation of Punjabi words have also
been provided. The Dictionar y is
accessible in both typewriter layout and
phonetic layout. One can search for
complete match or a pattern in the
dictionary.

• English Punjabi On-line Dictionary: An

English-Punjabi dictionary containing
about 45000 words has been made
available on the website.

• Hindi Punjabi On-line Dictionary: A

Hindi-Punjabi dictionary containing
about 45000 words has also been made
available on the website.

• Glossar y of English-Punjabi
administrative terms : A technical glossary
of around 17,000 English-Punjabi
administrative terms has also been

48
 uploaded. A CD has also been developed
and the glossary has been installed on
computers in many offices of Punjab State
Government.
2.3 On Line Teaching of Punjabi : For benefit of
Punjabis settled abroad and others interesting
in learning Punjabi, a web site for On-line
teaching of Punjabi has also been developed.
Work is complete for Gurmukhi Orthography,
Punjabi Pronouncing rules and a limited
vocabulary both in text and pictorial format
along with audio effects is also provided. The
alphabets can be learnt in Animation pattern
as we draw with hands.
2.4 On Line Font Conversion Utility : An online
font conversion utility has been provided.
The user can paste Punjabi text encoded in
any of the supported sixty Punjabi fonts and
convert it to the desired Punjabi font.
2.5 Downloads
• Punajbi Spell Checker – A Punjabi-English
spell checker has been uploaded on the
home page and any user can download and
install the spell checker on his system.
• Punjabi Fonts – Two Punjabi
fonts(LIKHARI_P and LIKHARI_R) for
phonetic and keyboard layouts have been
developed and made available for free
download. The fonts are available in both
True Type and Dynamic formats.
3. Interaction with Punjab State Government
A five day training programme was organised for
staff of State Language Department. They were
given training in word processing, email and
internet. We have been in constant touch with
the Secretary IT, Punjab and a demonstration of
the products developed at our RC was given to
him. The Punjabi wordprocessor and Gurmukhi
OCR were installed on the systems in the office
of Secretary, IT, Punjab. We have been providing
technical support to State Language Department.
Besides providing training to their staff, we have
also set up their web site in Punjabi. A CD of
English-Punjabi administrative terms has been
developed for the Sate Language Department and
the CD has been installed in Patiala DC office
and State Revenue office.
4. Publications
1. G.S.Lehal, “State of Computerization of
Punjabi”, Proceedings Second World Punjabi
Conference, Prince George, Canada (2003).
(Accepted for Publication)
2. G.S.Lehal, “A Gurmukhi Collation
Algorithm”, Journal of CSI. (Accepted for
publication)
3. G.S.Lehal, Chandan Singh and Renu Dhir,
“Structural feature-based approach for script
identification of Gurmukhi and Roman
characters and words”, Document Recognition
and Retrieval X, Proceedings SPIE, USA, Vol.
5010 (2003). (Accepted for publication)
4. G.S.Lehal and Chandan Singh, “A complete
OCR system for Gurmukhi script”, Structural,
Syntactic and Statistical Pattern Recognition,
T. Caelli, A. Amin, R.P.W. Duin, M. Kamel
and D.de Ridder (Eds.), Lecture Notes in
Computer Science, Vol. 2396, Springer-
Verlag, Germany, pp. 344-352, (2002).
5. G.S.Lehal, Chandan Singh and Renu Dhir,
“Automatic separation of Gurmukhi and
Roman script words”, Proceedings Indo-
European Conference on Multilingual
Communication Technologies, Pune, R. K.
Arora, M. Kulkarni and H. Darbari (Editors),
Tata McGraw-Hill, pp. 32-38,(2002).
6. G.S.Lehal and Chandan Singh, “A Post
Processor for Gurmukhi OCR”, SADHANA
Academy Proceedings in Engineering Sciences,
Vol. 27, Part 1, pp. 99-112, (2002)
7. G.S.Lehal and Chandan Singh, “ Text
segmentation of machine printed Gurmukhi
script”, Document Recognition and Retrieval
VIII, Paul B. Kantor, Daniel P. Lopresti,
Jiangying Zhou, Editors, Proceedings SPIE,
USA, Vol. 4307, pp. 223-231, (2001).
49
8. G.S.Lehal and Chandan Singh, “A technique Yoginder Sharma NA
for segmentation of Gurmukhi text”,
Pooja Dhamija NA
Computer Analysis of Images and Patterns, W.
Skarbek (Ed.), Lecture Notes in Computer Zameerpal kaur NA
Science, Vol. 2124, Springer-Verlag,
Dr. Kuljeet Kapoor NA
Germany, pp. 191-200, (2001).
Dr. Devinder Singh NA
9. G.S.Lehal, Chandan Singh and Ritu Lehal, “A
shape based post processor for Gurmukhi Karamjeet Kaur NA
OCR”, Proceedings of 6th International Jaspal Singh NA
Conference on Document Analysis and
Recognition, Seattle, USA, IEEE Computer Manpreet Singh NA
Society Press, USA, pp. 1105-1109, (2001). Rakesh K. Dawra NA
10.G.S.Lehal and Nivedan Bhatt, “A recognition Shallu Kalra NA
system for Devnagri and English handwritten
Kuldeep Kumar NA
numerals”, Advances in Multimodal Interfaces
– ICMI 2001, T. Tan, Y. Shi and W. Gao Baljit Singh NA
(Editors), Lecture Notes in Computer Science,
Aarti Gupta NA
Vol. 1948, Springer-Verlag, Germany, pp.
442-449. (2000). Sunita Sharma NA

11. G.S.Lehal and Chandan Singh, “A Gurmukhi Surjit Singh NA

script recognition system”, Proceedings 15th Courtesy: Prof. R.K. Sharma
International Conference on Pattern Thapar Institute of Engineering & Technology
Recognition, Barcelona, Spain, IEEE Department of Computer Science & Engineering
Computer Society Press, California, USA, Vol Patiala 147 001
2, pp. 557-560, (2000). (RCILTS for Gurmukhi)
Tel: 00-91-175-2393137, 393374, 2283502
5. The Team Members
E-mail : rksharma@mail.tiet.ac.in
Dr. R. K. Sharma rksharma@mail.tiet.ac.in
Dr. G. S. Lehal gslehal@mail.tiet.ac.in

Dr. Rajesh Kumar rakumar@mail.tiet.ac.in

Rajeev Kumar rkumar@mail.tiet.ac.in
Ramneet Mavi ramneet_mavi@rediffmail.com
Deepshikha Goyal deepshikha26us@yahoo.com
Nivedan Bhatt nivedanbhatt@rediffmail.com
Sukhwant Kaur sukhwant2010@yahoo.co.in
Ramanpreet Singh shallukalra@yahoo.co.in
Parneet Cheema NA
Pallavi Dixit NA
Rupsi Arora NA

50
 Resource Centre For
Indian Language Technology Solutions – Bengali
Indian Statistical Institute, Kolkata
Achievements

Computer Vision and Pattern Recognition Unit

203, Barrackpore Trunk Road
Indian Statistical Institute, Kolkata-700108
Tel. : 00-91-33-25778086 Extn. 2852
E-mail : bbc@isical.ac.in
Website : http://www.isical.ac.in/~rc_bangla
 RCILTS-Bengali
Indian Statistical Institute, Kolkata
Introduction
Main objective of this project was resource and
technology development for all aspects of language
processing for Eastern Indian languages particularly,
Bangla. It included Corpus development, Font
generation, Website generation, OCR development,
Information retrieval system development etc. as
well as training of people on Indian language
technology through courses and workshops.
Outcome in Physical Terms
• Corpus of Bangla Document Images, Bangla
Text Corpus in Electronic Form and Corpus
of Bangla Speech Data
• Web site on Eastern Indian Language
Technologies including Bangla Language
Technology design guide
• Bangla Font Package
• OCR System for Oriya and Assamese
• Information Retrieval System for Bangla
Electronic Documents
• Multi-lingual Script Line Separation system
• Automatic Processing system for Handprinted
Table-Form Documents
• Neural Network based tools for printed
document (in eastern regional scripts)
processing
Production Agencies with which Memorandum of
Understanding/link up has been established
MoU has been signed with (i) Webel Mediatronics
Ltd, Kolkata (speech synthesis Technology), (ii)
Centre for Advanced Computing, Pune (Devanagari
and Bangla OCR Technology) (iii) Electronics
Research & Development Centre India, Noida
(Devanagari OCR Technology), (iv) Orissa
Computer Application Centre, Bhubaneswar (Oriya
OCR Technology) (v) Indian Institute of
Technology, Guwahati (Assamese OCR Technology)
52
Link up for Data source has been established with
Bangla Academy.
1. Core Activities
1.1 Web Site Development and The Language
Design Guide
• Creation of a web site on Eastern Indian
Languages and Language Technologies for
information of people interested in TDIL
The name of this site is Resource Centre for Indian
Language Technology Solutions –Bangla. The URL
of this site is: http://www.isical.ac.in/~rc_bangla/. At
present, this site contains details about this MIT
project (Resource Centre for Indian Language
Technology Solutions - Bangla), a brief description
of the products developed at Indian Statistical
Institute, Kolkata. A Bangla design guide is also
introduced in the site so that new technology on
Bangla could maintain a common framework. It
covers details of the origin and development of the
script, the alphabets, character statistics, information
related to the fonts, presentation and storage
consideration, information related to Unicode of
Bangla etc. The local language academy (Bangla
Academy) has been contacted. A prototype of a Web-
based front-end to our spell-checker and phonetic
dictionary has been developed using an evaluation
copy of CDAC’s GIST Software Development Kit
(SDK). Initiative has been taken so that this front-
end can be put on the Internet for public use with
the GIST SDK and iPlugin (or Modular Software’s
Shree Lipi). Issues related to hosting this web site on
another server/Web hosting service are currently
being explored.
1.2 Training Programmes
• Education through Training Programmes and
Workshops
Two International workshops under the banner of
“International Workshop on Technology
Development in Indian Languages (IWTDIL)” were
held respectively during March 26-30, 2001 and
January 22-24, 2003. Some main topics covered in
these workshops were: (a) Machine translation from
English to Indian Language (particularly to Bangla),
(b) Text corpus generation, designing and
annotation, (c) Speech synthesis, processing and
recognition, (d) OCR technology for Indian scripts,
(e) Hand written character recognition, (f )
Document processing and analysis, (g) Some general
features of Indian Languages (e.g, phonology and
acoustics of Bangla phonemes and diphthongs), (h)
Anaphora resolution ellipsis in Hindi and other
Indian Languages. In IWTDIL’01, the first three
days had introductory talks and tutorials on various
areas of language technology and the last two days
featured lectures by international experts on these
subjects. A total of twenty-nine speakers presented
talks at the workshop and there were seventy-five
participants from India and abroad. In IWTDIL’03,
four distinguished scientists from abroad and six
from India delivered lectures to forty-five
participants. A cultural programme consisting of a
play by a renowned theatre group of Kolkata was
also arranged.
One National workshop entitled “Indian language
Spell-checker Design” was organized in July 18-19,
2002. The main theme of the workshop was to
present the work on Spell-checker done by various
Resource Centres and groups in different Indian
languages. The participants demonstrated the Spell-
checker software developed by them. It was
anticipated that a benchmarking method for spell
checker would be evolved out of the workshop.
Twenty-five participant attended/presented lectures
in the workshop.
2. Services
2.1Corpus Development
• Printed Bangla Document Images & Ground
Truth for OCR and Related Research
Some famous Bangla novels and books have been
selected to prepare a Bangla document image corpus
along with ground truth. A brief description of
those selected novels and books are given below:
Maitreya Jatak – This is a famous Bangla
mythological novel written by Bani Basu, a popular
Bangla writer. As far as the linguistic aspect goes,
the overall language of the book is old Bangla.
Polished and chaste form (Sadhu bhasa) of narration
is used through out the book and some archaic forms
(terms mostly derived from Sanskrit and Prakrit
sources) are also used mainly to create an atmosphere
of the period of Goutama Buddha, which the book
depicts. Ananda Publishers, one of the largest
publishers of Kolkata has published this book. The
quality of the printing is excellent. Computer
generated font developed by Ananda Publishers has
been used in printing.
Pratham Alo – This is also a famous Bangla novel
written by famous Bangla writer Sunil
Gangopadhyaya. The incidents of the novel are
nearly one hundred and fifty years old and the theme
is related to Bangla renaissance. The standard Bangla
language has been used for this book. The percentage
of archaic words is much less than that in Maitreya
Jatak. This book is also published by Ananda
Publishers, but using offset printing technology. The
overall printing quality is good but worse than
Maitreya Jatak.
Bhasa Desh Kal – The book is written by Dr. Pabitra
Sarkar, a distinguished author of Bangla Linguistics.
The language of the book is standard Bangla
colloquial. The publisher of the book is Mitra and
Ghosh, Kolkata. Printing quality is moderate and
worse than Pratham Alo.
Upendra Kishore Rachana Samagra – This book is
written by the famous author of juvenile literature
Shri Upendra Kishore Roychowdhuri. Old formal
and polished Bangla language is used in this book.
Here, offset font is used and the book has been
published by Ananda publishers of Kolkata.
Amar Jibanananda Abiskar o Anyanya – Sunil
Gangopadhyaya is the author of this book. The
language of the book is standard Bangla colloquial.
Ananda Publisher has published this book. The
printing quality is good and offset printing
technology is used.
Rajarshi – This is one of the classic novels written
by Rabindranath Tagore. Old formal and polished
Bangla language is used in this novel. Ashoke Book
Agency of Kolkata is the publisher of the book. The
printing quality is not as good as that of Ananda
Publishers.
53
Bangla Choto Galpa - The book is a collection of Table 1
Bangla short stories written by different writers of
old Bangla literature and edited by Rabindranath Name of Books No. of Total No.
Tagore. Old formal language (Sadhu bhasha) is used in Pages of Words
most of the stories. The book is published by Model Scanned
Publishing House. The printing quality is good. Maitreya Jatak 419 2,43,020
Punarujjiban – It is the Bangla version of a novel Pratham Alo 37 18, 833
written by famous Russian novelist Lev Tolstoy. The
book is published by Raduga Prakashan, Moscow. Bhasa Desh Kal 15 5,760
The printing quality is good and modern colloquial
language is used. Upendra Kishore 330 1,38,270

Bhraman Amnibus – The book is written by Sri Amar Jibanananda 21 7,812

Uma Prasad Mukhopadhyaya. It contains detailed Rajarshi 100 39,800
description of different places of Himalayas. Modern
Bangla language is used in this book. It is published Bangla Chotogalpa 260 1,11,800
by Mitra & Ghosh, Kolkata. Offset printing
Punarujjiban 68 25,704
technology is used.
Parakiya – This book is collection of Bangla short Bhraman Amnibus 31 14,198
stories written by both old and modern writers of Parakiya 163 85,412
Bangla literature and edited by Sunil
Gangapadhyaya. The publisher of the book is Amar Debottar Sampatty 29 10,730
Punashcha. Both old formal and modern colloquial Prabasi Pakhi 10 4,020
language are used in different stories of the book.
The printing quality is not so good. Mahabharat Katha 4 1,304
Amar Debottar Sampatty – The book is an auto Total 1,487 7,06,663
biography of Nirod Chandra Chaudhuri, published
by Ananda Publishers. Offset printing technology Those page-by-page scanned images are then fed as
and old formal language are used. input of the Bangla module of the existing Bilingual
Prabasi Pakhi - It is a Bangla story book written by (Hindi and Bangla) Optical Character Recognition
Sunil Gangapadhaya and published by Ananda (OCR) System developed in our department. Each
publishers. Offset printing technology and modern character of those input images are being recognized
colloquial language are used. by this OCR System and are stored as text
documents in 8- bit ISCII format.
Mahabharat Katha – The book is published by
Udbodhan publication of Ramkrishna Mission. The present OCR system gives an accuracy of 96 %
Modern Bangla language has been used in this book to 98 % in character level depending on paper
(Table 1 is the list of books scanned). quality and font style of the books. So, the 4% to
2% error is corrected manually to prepare the text
OCR – A HP Scan Jet flatbed scanner of high
documents, which serves as ground truth.
resolution (300dpi ordinary and gray level spectral
resolution) has been used to get the image A benchmark software is also developed to give a
documents. The images are then saved into specified format to all ground truths generated using
uncompressed TIFF format and are scanned Indian OCR technologies for automatic evaluation
generally using same setting. Minor adjustment of of them (different OCR technologies). The database
brightness and contrast may be done by using Corel formed in this way is shown in the tabular form in
Photopaint software. table1.

54
 Table -2
1. Author’s Name
Iswarchandra Vidyasagar is known as one of the
great social reformers, philanthropist and father of
Bengali prose style of Bengal. He played an
important role in primary education, widow
marriage and women education.

Figure 1: The image is at left hand side and the Name of the Classic & Publisher
corresponding ground truth is at right hand side
Vidyasagar Rachanabali (Tuli – Kalam)
• Development of Bangla Text Corpus in Electronic
Name of Articles (Story, Novel, drama, etc.)
Form including a Bangla dictionary, and several
Bangla classics Akhyanmanjari
Several novels have been entered into the Betal Panchabingshati
computer in ISCII format. The description of
novels, the corresponding authors and total Kathamala
number of words are given in a tabular form Mahabharat
(Table-2). More than 34000 words of a bilingual
(Bangla-English) dictionary have also been Niskritilabh Prayash
entered. A comprehensive corpus for electronic Provaboti Sambhashan
dictionary for Bangla-Bangla (with 65,000 words)
has been constructed and checked. Using the Sakuntala
guidelines of the above two dictionaries a
Sanskrit Bhasa O Sankriti
trilingual (Bangla-English-Hindi) dictionary
creation has started. Till today 17,000 words with Sitar Banobas
meaning and parts of speech as well as other
information has been entered. However, lack of Balyabibaher Dosh
standard size Bangla-Hindi dictionary in printed Ramer Rajjyabhishek
form has created some problem (Bangla-Hindi
dictionaries available in the market are small, Bidhababibaha
containing about 7,000 words only). We have also Jibancharita
designed a prototype of an electronic thesaurus
for Bangla (based on WordNet, a well-known Vrantibilas
electronic resource for the English language). Banglar Itihas
Charitabali
Papers Published Bodhadaya
• Dash, N.S. and Chaudhuri, B.B. (2001) “A No of Words (per Classic)
corpus based study of the Bangla language”.
Indian Journal of Linguistics. 20: 19-40. Sub. Total : 315783

• Dash, N.S. and Chaudhuri, B.B. (2002) 2. Author’s Name

“Corpus generation and text processing”. Bankim Ch. Chattopadhyay is considered the first
International Journal of Dravidian Linguistics. and one of the greatest novelists in Bangla language.
31(1): 25-44. He has written 14 novels besides a large number of

55
essays on various literary and social issues. He wrote work is the epic named ‘Meghnadbadh’.
the ‘Vande Mataram’ song, which played a major
role in Indian Independence struggle. Name of the Classic & Publisher
Madhushudan Rachanabali (Kallol Prakashani)
Name of the Classic & Publisher
Name of Articles (Story, Novel, drama, etc.)
Bankim Rachanabali (Patrajo Publication)
Buroshaliker Ghare Row
Name of Articles (Story, Novel, drama, etc.)
Krishnakumerir Natak
Anandamath Padhyabatir Natak
Bishbriksha
Durgeshnandini No of Words (per Classic)
Kamalakanta
Bigyanrahosya Sub. Total : 54089
Muchiram Gur 5. Author’s Name
Lokrahasya
Samya Computer Dictionary
Bibidha Prabandha (Vol. 1) Name of the Classic & Publisher
Bibidha Prabandha (Vol. 2)
Krishnacharita (Vol 1) B. B. Chaudhuri (Ananda Publishers)
Krishnacharita (Vol 2)
No of Words (per Classic)
No of Words (per Classic)
Sub. Total : 95435
Sub. Total : 320232
6. Author’s Name
3. Author’s Name
Samsad Dictionary
Sarat Chandra Chattopadhyay is probably the most
Name of the Classic & Publisher
popular novelist of Bengal. His novels
sympathetically depicted the weaker section of the S. Biswas et. al. (Sahittya Samsad)
society including woman.
No of Words (per Classic)
Name of the Classic & Publisher
Sub. Total : 167941
Sarat Rachanabali (Sarat Samiti)
Grand Total : 883729
Name of Articles (Story, Novel, drama, etc.)
• Electronic Corpus Of Speech Data
Debdas
Pallisamaj The composition of existing speech databases for
Srikanta English has been studied. Speech data has been
categorized into several classes based on criteria such
No of Words (per Classic)
as sex, age, region of speaker, place of data collection,
Sub. Total : 246032 whether the source of the spoken material is a written
script etc. Based on these, the composition of the
4. Author’s Name database has been designed. All India Radio, Calcutta
has been designated as a potential source of audio
Michael Madhushudan Dutt was a great poet and
material. However, the speech lab, where in the
play writer of Bengal. He has written a few English
controlled environment the data could be generated,
sonnets and poems. He introduced the
is yet to be set up. due to delayed sanction of the
“Amitrakshar” rhyme in Bangali poem. His famous
grant.

56
2.2 Font Generation and Associated Tools consonants without the intervening implicit
vowels. The shape of these conjuncts can differ
• Public Domain Bangla Font Generation from those of the constituting consonants.
Overall font generation and editor development 6. Punctuation: (12)
process can be divided into the following modules:
7. Numerals (10)
1. Designing the exhaustive glyph set.
Converter Program From Font File To ISCII File
2. Converter program from Font file to ISCII file And Vice-versa : The converter program has two
and vice-versa submodules. The first one takes a font encoded string
as input and delivers an ISCII encoded string as
3. Designing a Bangla text editor
output. The second one takes an ISCII encoded
4. Designing a floating Keyboard string as input and gives a font encoded string as
output.
Background Of Designing The Exhaustive Glyph
Set : Every language has its own character set and Bangla Text Editor : Along with the font a Bangla
many of them can be represented within 256 editor has been developed. It supports ISI.ttf font.
characters using individual 8-bit character sets A web version is generated using Bit stream’s web-
including Indic scripts using ISCII code. However font wizard to display text in Bangla within a web
for unification of all these languages recently the browser. The editor provides standard editing
Unicode consortium proposed 16 bit character features such as cut, copy, paste, select, select all, file
encoding scheme. Here characters can be designed operations, bold, italic, underlined text, superscript
at 216 spaces. The Government of India is a member and subscript, left, right and center alignments, find
of the Unicode Consortium and has been engaged and replace specific strings etc. This editor can save
the content in plain text, RTF and in ISCII format.
in a dialogue with the UTC about additional
A floating keyboard can be invoked on demand for
characters in the Indic blocks and improvements to
the new user to get the keying information for
the textual descriptions and annotations. Unicode
writing Bangla in this editor.
is designed to be a multilingual encoding that
requires no escape sequences or switching between Designing a Floating Keyboard : The floating
scripts. For any given Indic script, the consonant Keyboard is illustrated below.
and vowel letter codes of Unicode are based on
ISCII, so they correspond directly. For Bangla,
Unicode 3.0 provides places 0980 to 09FF. One
Bangla True Type Font (called ISI.ttf ) has been
developed in this center which should be compliance
with ISCII and could be upgraded to support
Unicode. The font is designed and generated with
the help of ALTSYS Fontographer 3.0. The following
Bangla orthographic characters can be written with
this font :
1. Vowels (11)
2. Consonants (38)
3. Vowel Matras (10)
4. Halant
5. Conjuncts: Bangla contains numerous conjuncts
(250+), which essentially are clusters of up to four Figure 2:. Standard Bangla DTP typewriter layout

56
• Bangla Spell-Checker
Bangla spell-checker is a tool to detect errors in
Bangla words and correcting them by providing a
set of correct alternatives which includes the intended
word. An errorneous word can belong to one of
two distinct categories, namely, non-word error and
real-word error. Let a string of characters separated
by spaces or punctuation marks be called a candidate
string. A candidate string is a valid word if it carries
a meaning. A meaningless string is a non-word. A
real word error means a valid word but not the
intended one in the sentence. It makes the sentence
syntactically or semantically ill-formed or incorrect.
In both cases, the problem is to detect the erroneous
word and either suggest correct alternatives or
automatically replace it by the appropriate word. In
this spell-checker, only non-word errors are
considered.
Word errors can be classified into four major types
namely, substitution, deletion, insertion, and
transposition error. In Bangla, wrong uses of
characters, which are phonetically similar to the
correct ones is observed. A great deal of confusion
occur in the use of long and short vowels, aspirated
and unaspirated consonants, dental and cerebral
nasal consonant due to phonetic similarity. Another
type of error is typographic error which is caused by
the accidental slip of fingers on the keys which are
neighbours of the intended key.
In this spell-checker, the main technique of error
detection is based on matching the candidate
string in the normal as well as in the reversed
dictionary (following publications may be
referred). To make the system more powerful, this
approach is combined with a phonetic similarity
key based approach where phonetically similar
characters are mapped into a single symbol and a
nearly-phonetic dictionary of words is formed.
Using this dictionary, phonetic errors can be easily
detected and corrected. Here a candidate string
first passes through the phonetic dictionary. If the
word is not found in the dictionary and also failed
to give suggestion then it tries to divide the word
in root part and suffix part by separately verifying
both. If an error is found, the spell-checker
attempts providing suggestions. If it fails, it checks
58
whether the string is a conjunct word generated
by appending two noun word and suffix. Option
for adding new words permanently or temporarily
is provided in the spell checker.
Reference
• B. B. Chauhduri and T. Pal, Detection of word
error position and correction using reverse word
dictionary’, Intl. Conf. On Computational
Linguistics, Speech and Document Processing
ICCLSDP’98, February 18-20, 1998, pp, C41-
C46.
• B. B. Chauhduri, A Novel Spell-checker for
Bangla Text Based on Reversed-Word Dictionary,
Vivek, Vol. 14(4), pp. 3-12, October 2002.
Figure 3: Standard Bangla DTP typewriter layout
with a support of a spell checker
For the spell-checker, several files containing root-
words and suffix words are maintained. The main
dictionary contains about 60,000 root-words and
100,000 inflected words. Noun and verb suffix files
are also used. The spell-checker works fast and the
non-word error are all correctly detected but it makes
about 5% false alarm. This is mainly due to conjunct
words formed by euphony and assimilation as well
as proper nouns in the corpus.
3. Products
Previously Bangla, Devnagari OCR systems were
developed at CVPR Unit of Indian Statistical
Institute and these core technologies were transferred
to Industry for commercialization. The latest
developments are Oriya OCR and Assamese OCR.
These are described below.
3.1 OCR System for Oriya
Purpose
The purpose of this system is to recognize printed
Oriya script automatically.
Summary of the System
In this recognition system, the document image is
first captured using a flatbed scanner. The image is
then passed through different preprocessing modules
like skew correction, line segmentation, zone
detection, word and character segmentation, etc.
Next, individual characters are recognized using a
combination of stroke and run-number based
features, along with features obtained from the
concept of water overflow from a reservoir. These
techniques are discussed in a greater detail in the
following.
System Description
Text Digitization and Noise Cleaning: Text
digitization is done using a flatbed scanner (Model:
HP Scanjet 660 C) at a resolution varying from
200 to 300 dots per inch (dpi). The digitized images
are in gray tone and a histogram-based thresholding
approach is used to convert them into two-tone
images. For a clear document, the histogram shows
two reasonably prominent peaks corresponding to
white and black regions. The threshold value is
chosen as the midpoint between the two peaks of
the histogram. The two-tone image is converted into
0-1 labels where 1 and 0 represent object and
background, respectively. The digitized image shows
protrusions and dents in the characters, as well as
isolated black pixels over the background, cleaned
by a morphological smoothing approach.
Figure 4: Uppermost and lowermost point of
components in a skewed text line
Skew Detection and Correction: When a document
is fed to the scanner either mechanically or by a
human operator, a few degrees of skew (tilt) is
unavoidable. The skew angle is the angle that the
text lines in the digital image make with the
horizontal direction. Skew detection and correction
are important preprocessing steps of document
layout analysis and OCR approaches. Skew
correction can be achieved in two steps, namely (i)
estimation of skew angle, and (ii) rotation of the
image by the skew angle in the opposite direction.
Here a Hough transform based technique is used
for estimating the skew angle of Oriya documents.
It is observed that the uppermost and lowermost
points of most of the characters in an Oriya text
line lie on the mean line and base line, respectively.
The lowermost and uppermost points of characters
in a skewed Oriya text are shown in Figure 4. To
reduce the amount of data to be processed by the
Hough transform, only the uppermost and
lowermost pixels of each component are considered.
First, the connected components in a given image
are identified. For each component, its bounding
box (minimum upright rectangle containing the
component) is defined. The mean width of the
bounding box bm is also computed. Next,
components having bounding box-width greater
than or equal to bm are retained. By thresholding at
bm, small components like dots, punctuation marks,
small modified characters, etc., are mostly filtered
out. Because of this filtering process, the irrelevant
components cannot create errors in skew estimation.
Now, the usual Hough transform technique is used
on these points to get the skew angle of the
document. The image is then rotated according to
the detected skew angle. Font style and size variation
do not affect the proposed skew estimation method.
Also, the approach is not limited to any range of
skew angles.
Line, Word and Character Segmentation: For
convenience of recognition, the OCR system should
automatically detect individual text lines, segment
the words from the line, and then segment the
characters in each word accurately. Since Oriya text
lines can be partitioned into three zones (Figure 5),
it is convenient to distinguish these zones. Character
recognition becomes easier if the zones are
59
distinguished because the lower zone contains only
modifiers and the halant marker, while the upper
zone contains modifiers and portions of some basic
characters.
Figure 5: Zones in an Oriya text line
Text Line Detection and Zone Separation: The lines
of a text block are segmented by finding the valleys
of the projection profile computed by counting the
number of black pixels in each row. The trough
between two consecutive peaks in this profile
denotes the boundary between two text lines. A text
line can be found between two consecutive
boundary lines. After line segmentation, the zones
in each line are detected. From Figure 6 it can be
seen that the upper zone is separated from the middle
zone of a text line by the mean line, and that the
middle zone is separated from the lower zone by
the base line. The uppermost and lowermost points
of the connected components in a text line are used
to detect the mean line and base line, respectively. A
set of horizontal lines passing through the uppermost
and lowermost points of the components are
considered. The horizontal line that passes through
the maximum number of uppermost points (lower
most points) is the mean line (base line). It should
be noted that the uppermost and lowermost points
of the components are previously detected during
skew detection, so these points do not have to be
recalculated during zone detection.
Figure 6 : Projection profile of rows in Oriya text
lines (dotted lines show line boundaries)
60
Word and Character Segmentation: After a text
line is segmented, it is scanned vertically, column-
by-column. If a column contains two or fewer
black pixels, then the scan is denoted by 0, else
the scan is denoted by the number of black pixels
in that column. In this way, a vertical projection
profile is constructed. Now, if in the profile there
exists a run of at least k consecutive 0s then the
midpoint of that run is considered as the
boundary between two words. The value of k is
taken as 2/3 of the text line height (text line height
is the normal distance between the mean line and
the base line). To segment each word into
individual characters, only the middle zone of the
word is considered. To find the boundary between
characters, the image is scanned in the vertical
direction starting from the mean line of the word.
If during a scan, the base line without
encountering any black pixel is reached, then this
scan marks the boundary between two characters.
However, the gray-tone to two-tone conversion
of the image gives rise to some touching characters,
which cannot be segmented using this method.
To segment these touching characters, the
principle of water overflow from a reservoir is
used, which is as follows. If water is poured on
top of the character, the positions where water
will accumulate are considered as reservoirs. Figure
7 shows the location of reservoirs in a single
character as well as in a pair of touching
characters. The height of the water level in the
reservoir, the direction of water overflow from
the reservoir, position of the reservoir with respect
to the character bounding box, etc are noted. A
reservoir whose height is small and which lies in
the upper part of the middle zone of a line is
considered as a candidate reservoir for touching
character segmentation. The cusp (lowermost
point) of the candidate reservoir is considered as
the separation point of the touching characters.
In Figure 7, this position is marked by a vertical
line. Because of the round shape of most of the
Oriya characters, it is observed that such a
reservoir is formed in most of the cases when two
characters touch each other. Sometimes, two or
more reservoirs may be formed. In such cases,
the reservoir close to the middle of the bounding
box for segmentation is selected.
 Figure 7: Water reservoirs in a single and touching
Oriya characters

Feature Selection and Detection: Topological

features, stroke-based features as well as features
obtained from the concept of water overflow
for character recognition are considered. These
features are known as principal features. The
features are chosen with the following
considerations: (a) Robustness, accuracy and
simplicity of detection (b) Speed of
computation (c) Independence of size and
fonts, and (d) Tree classifier design need.
Stroke-based and topological features are
considered for the initial classification of
characters. These features are used to design a
Figure 8: Portion of the tree classifier for Oriya
tree classifier where the decision at each node
characters
of the tree is taken on the basis of the presence/
absence of a particular feature (see Figure 8). Performance
Stroke-based features include the number and
Line Segmentation: Our system identifies individual
position of vertical lines. The topological
text lines with an accuracy of 97.5%.
features used include existence of holes and
their number, position of holes with respect to Word Segmentation: The overall word segmentation
the character bounding box, ratio of hole accuracy of the system is 97.7%. The error rate for
height to character height, etc. In addition, the the inferior documents is 4.2%, whereas for the
concept of water overflow from a reservoir is good-quality documents, it is only 1.2% (these
also used. The reservoirs in a character are figures were calculated based on correctly segmented
identified, and the position of the reservoirs text lines only).
with respect to the character bounding box, the Character Segmentation : The character
height of each reservoir, the direction of water segmentation accuracy of the system is 97.2%. The
overflow, etc. are used as features in the proposed method for separating touching characters
recognition scheme. based on the water reservoir concept is generally
successful.
Platform
Character Recognition: On average, the system
This system is developed in C language in UNIX recognizes characters with an accuracy of about
platform. A WINDOWS based version is also 96.3%, i.e. the overall error rate is 3.7%.
designed.
3.2 Adaptation of Bangla OCR to Assamese
Portability
Background
The system can run in any UNIX and We have already developed an efficient OCR
WINDOWS platform. system for printed documents in Bangla. Since

61
Assamese and Bangla share the same script, this
OCR system can be successfully used for Assamese
documents after some modifications. The
modification is needed mainly in the post-
processing stage when language specific OCR
error correction is needed.

Summary of the System

The segmentation of a document image into lines,

words, and characters, and the recognition of
segmented characters is dependent on the script
only. Thus, the modules of the existing OCR
system for Bangla can be used for Assamese OCR.
However, certain post-processing steps after basic
recognition in order to improve OCR accuracy
are required. For example, the words in the output
of the OCR system may be looked up in a lexicon;
a correctly recognized word will be present in the
lexicon, whereas an incorrectly recognized word
will usually not be found. The incorrect word
can then be replaced by the lexicon-word nearest
to it. This post-processing is obviously language- Figure 9. A sample output from the Assamese OCR
dependent. In this project, we have made the System
necessary modifications in our OCR system so
that it can be used on Assamese documents. Performance

Test pages are selected from three Assamese books.

System Description
The major components of the system are same
as those of Bangla. Details of these components
can be found in the following references.
Modifications are done in the following
modules :
1. Update of Symbol-list: The symbol-list for
Bangla script is updated to contain Assamese
“ra” and “wa” and all the conjuncts involving
these two characters. Other characters remain
unaltered.
2. Formation of the Prototype Library: The
prototype library used in Bangla OCR is
modified by adding new character shapes
found in Assamese script. Character shapes
not appearing in Assamese script are deleted
from the library.
Pages are scanned at 300 dpi. In total, 50 pages are
used in testing phase. Analysis of test results shows a
character-level accuracy of about 95%. Since the font
used for printing Assamese materials are somewhat
different from the fonts used in Bangla, generation
of a new prototype library considering major
Assamese fonts would improve the overall accuracy
of the system.
Design of post-processing module: Post-processing
in Bangla OCR is done using a lexicon of Bangla
language. The same module can be used to do post-
processing for Assamese also. However, a lexicon of
Assamese language is needed for this purpose. This
activity has been taken up by IIT, Guwahati.
Technology Transfer
The source code of the system along with the
technical details has transferred to IIT, Guwahati.

62
Technical Reports/Correspondence

An MCA student, Anirban Mukherjee did work

on this project for his MCA dissertation submitted
to Indira Gandhi National Open University
(IGNOU), New Delhi. Title of his work is
“Development of an Optical Character
Recognition System for Assamese Script”.

References

1. B.B. Chaudhuri and U. Pal, “A complete
printed Bangla OCR system”, Patter n
Recognition, vol. 31, pp. 531-549, 1998.
2. U. Garain and B. B. Chaudhuri,
“Segmentation of Touching Characters in
Printed Devnagari and Bangla Scripts using
Fuzzy Multifactorial Analysis”, IEEE
Transactions on Systems, Man and Cybernetics,
Part C, Vol. 32, No. 4, pp. 449-459, 2002.
3. A. Ray Chaudhuri, A. Mandal, and B. B.
Chaudhuri, Page Layout Analyzer for
Mulitingual Indian Documents, Proc.
Language Engineering Conference, IEEE CS
Press, 2002.
3.3 Information Retrieval system for Bangla
documents
Background
Digital information is available in various forms
like text, image and speech data or multimedia
content. Among them the text information is
considerably abundant and can be easily created.
Passage retrieval from text documents is gaining
momentum over document retrieval for the last
several years. Document ranker returns
documents, which is often infeasible to search and
extract the necessary information from the entire
document. Passage retrieval on the other hand
returns fixed or variable sized text chunks from
document(s) where the information is likely to
reside. This saves both time and effort for
searching from a huge text document corpus.
Figure 10. A sample output from the prototype
developed. The question “what is Pythagoras
theorem” was submitted in Bangla and the system
retrieves passages and ranked as output.
Summary of the System
A prototype n-gram based language identifier for
identifying Indian languages has been developed
[1]. A prototype of natural language text indexer
for passage retrieval has been developed for
Bangla.
System Description
Note that Indian languages can be grouped into
five categories based on their origins: Indo-
European (Hindi, Bangla, Marathi, etc.),
Dravidian (Tamil, Telugu, etc.),Tibeto-Burmese
(e.g., Khasi), Astro-Asiatic (Santhali, Mundari,
etc.) and Sino-Tibetan (e.g., Bhutanese).
Languages within a group share a number of
common elements. For instance, there is a
significant overlap in the vocabulary of Bangla
and other Indo-European languages and are
mutually closer than the profiles for a pair of
languages from two different groups. We have
tested the character level n-gram algorithms for
language identification from a multilingual
collection of Indian language documents. Also, a
prototype of a “English to Bangla” phonetic
transliteration scheme is designed and
implemented for cross-lingual information.
Part of the n-gram distance matrix between every
pair of Indian languages is shown in Table 3.

63
Table 3. The n-gram distance between some major 3.4 Script Identification and Separation From
Indian language Indian Multi-Script Documents
Profile Bangla Hindi Kannada Kashmiri Malayalam Telugu Urdu Background
Bangla 0 16.54 19.42 23.66 20.27 19.08 24.01
Hindi 16.54 0 18.40 23.65 19.74 18.58 24.12 India is a multi-lingual multi-script country, where
Kannada 19.42 18.40 0 23.80 18.11 16.65 24.09 a single document page (e.g., passport application
Kashmiri 23.66 23.65 23.80 0 24.02 23.88 19.54
Malayalam 20.27 19.74 18.11 24.02 0 18.07 24.29 form, examination question paper, money order
Telugu 19.08 18.58 16.65 23.88 18.07 0 24.15 form, bank account opening application form ) may
Urdu 24.01 24.12 24.09 19.54 24.29 24.15 0
contain lines in two or more language scripts. For
A passage detection and ranking algorithm for this type of document page there is a need for
Bangla text has been designed and implemented. separating different scripts before feeding them to
Stop-word-list are common words ignored by search respective OCR systems. The purpose of this system
engines at the time of searching and these words is to identify different script regions of the document
generally do not contain any information. For and hence separate them.
constructing Bangla search engines by combining
statistical and manual methods, about 500 stop Summary of the System
words are identified. The DoE Bangla corpus is used The system works in two stages. In the first stage, it
for this purpose. separates each line in the scanned document page.
The indexer generates an indexed file, which keeps Line segmentation is based on horizontal profile
the record in this fashion: based technique. Secondly, based on the
distinguishing features between different Indian
Document No., Term, Term_weight ( f tD/N D), scripts, it identifies the script for each line in which
Frequency of the term in whole corpus ft , Occurence it is actually written. The identification of a particular
position of the term script from other scripts is mainly based on water
reservoir principle based features, features based on
Document No: The identity number of the contour tracing, profile features etc. These features
document in the whole corpus. are elaborated below. At present, the system has an
Term weight: The ratio of the term frequency for a overall accuracy of about 97.52%.
particular term in a document (f tD) and the System Description
document length in bytes (ND).
Some of the major distinguishing features used to
The occurrence position of the term: The numerical separate different Indian scripts in a document page
value of the position of the term, counted from the from each other, are discussed.
beginning of the document. The position of first
term of the document is one. So for each term ftD Horizontal projection profile: From the following
occurrence positions are listed. Figure 11, it is apparent that there is a distinct
difference among some of the scripts in terms of
A prototype of passage detection algorithm is horizontal projection profile.
developed and a few standard passage-ranking
algorithms have also been tested.
Technical Reports/Correspondence
1. “N-gram: a language independent approach to
IR and NLP”, P. Majumder, M. Mitra, B.B.
Chaudhuri, Proc. International Conference on
UniversalKnowledge and Language (ICUKL-2002),
Goa, India, 2002. Figure 11: Different Indian script lines (from top
to bottom: Devnagari, Bangla, Gurmukhi,

64
 Malayalam, Kannada, English, Tamil, Telugu,
Urdu, Kashmiri, Gujarati, Oriya) with their row-
wise maximum run (left side) and horizontal profile
(right side).
Water reservoir principle based feature: Top
(bottom) reservoir is defined as the reservoir obtained
when water is poured from top (bottom) of the
component. (A bottom reservoir of a component is
visualized as top reservoir when water will be poured
from top after rotating the component by 180°).
Similarly, if water is poured from left (right) side of
the component, the cavity regions of the
components where water will be stored are
considered as left right) reservoirs. This is shown in
Figure 12. Here top, bottom, left and right reservoirs
are shown for the English character X. The water
flow levels of these reservoirs are also shown in this
figure.
Figure 12: Top, bottom, left and right reservoirs are
shown for the character X. Water flow level of reservoir
is shown by dotted arrow.
In some scripts, many reservoirs may be obtained
from a particular side of the characters whereas in
some other scripts many reservoirs may not be
obtained from that side. Thus, this feature is useful
in distinguishing between scripts.
Left and right profile: In this feature each character
is located within a rectangular boundary, a frame.
The horizontal or vertical distances from any one
side of the frame to the character edge are a group
of parallel lines, known as the profile (Figure 12). If
left, right and top profile of the characters in a text
lines are computed, it is observed that there are some
distinct difference in some of the scripts according
to these profiles. Left and right profile of a
Malayalam character is shown in Figure 13.
Figure 13: Left and right profile of a character is shown.
Head-line feature: If the longest horizontal run of
black pixels on the rows of a text line is taken then
such run length (known as Head-line) can be used
to distinguish between scripts with this feature ( like
Bangla ) and those without Head-line ( like English).
Feature based on jump discontinuity: Here jump
discontinuity (a relatively small white run between
two black runs) of a component from a particular
side is considered. Some script’s characters (like in
Telugu, Kannada etc.,) have prominent jump
discontinuties and the occurrence frequency of this
particular feature is successfully used for script
identification. The jump discontinuity for a Gujrati
character is shown in Figure 14.
Figure 14: Example of jump discontinuity feature.
Based on these above major features the system
identifies the script of a particular line.
Script Identification Technique:
• Scanned gray tone document is converted to two
tone image using histogram based automatic
thresholding approach.
• Noise Removal and Skew correction are
performed on this two tone image.
• Line segmentation is performed to the refined
image.
• For each line of the image the particular script in
which the line is actually written is identified based
on a Binary Tree classifier.
• Once the script of the line is identified its region
is marked by the name of the particular script.
• The marked lines are shown in the output image.
65
A part of the Binary Tree classifier is shown in
Figure 15.
Figure 15: Flow diagram of the script
identification scheme.
(Here B =Bangla, D=Devnagari, E=English,
Gu=Gurumukhi, M=Malayalam, Ta=Tamil,
Te=Telugu, G=Gujarati, Ka=Kannada,
U=Urdu and O=Oriya).
Platform
This system is developed in C language in UNIX
platform. An user interface of this system is built in
VC++ 6.0 in WINDOWS2000 platform.
Portability
Both the C version and the VC++6.0 version of this
system can run in any UNIX and WINDOWS
platform respectively.
Performance
The system is tested based on data taken from various
sources like Journals, Newspaper, Synthetic
Document etc. Currently the system has an overall
accuracy of 97.52%. The output generated by the
system when run on a document page is shown in
Figure 16.
66
Figure 16: The output generated by the system
Papers Published
• U. Pal and B. B. Chaudhuri, “Identification of
different script lines from multi-script
documents”, Image and Vision computing, vol.
20, no.13-14 pp. 945-954 2002.
• U. Pal and B. B. Chaudhuri, “Script line
seperation from Indian multi-script documents”
IETE Journal of Research, vol. 49. no.1. 2003.
• U. Pal, S. Sinha, B. B. Chaudhuri, “Multiscript
Line Identification From Indian Documents”, 7th
International Conference on Document Analysis
& Recognition, ICDAR 2003 (In press).
4. Research & Development
4.1 Automatic Processing of Hand-printed Table-
Form Documents
Background
In office environment, thousands of documents
containing tables may be handled while processing
application forms. In Indian context, most of the
cases, these table-form documents contain hand-
printed text (like customer’s name, date, item
details/quantity/price etc.) mixed with printed text
(invoice no., challan no., etc.). Once these forms
are collected from the stockists, customers, or
other business centers, the computer operators
manually enter the data into the computer to
maintain an electronic version of the same. This
manual approach makes this processing time
consuming, tedious and inefficient. Hence, an
automatic approach is called for. Moreover, the
work may be the basis of handwriting recognition
of Indian languages.
Summary of the Project
This project deals with the automatic processing of
hand printed table-form documents. It extracts
different blocks of handwritten information from a
filled-in form. Each such block is segmented into
lines, words and characters. Identification of each
block is followed by tagging (numbering) them
according to the order of their physical placements.
In the final stage of this information extraction
procedure, images of individual hand printed
characters are obtained which should be passed as
inputs to a hand printed character recognition
system.
Figure 17: The binarized image(left) and the result
after the form segmentation (right).
Description of the Work Done
In our approach, extraction of handwritten
information from a filled-in form, does not take
care of the grey values of the pixels but it works on
the binarized image of the input form. In the first
step, horizontal profiles of object pixels help to
segment different horizontal lines of information.
In each such line, one or more blocks are identified
by exploiting the vertical profiles within each
horizontal strip. Within a block, words are identified
by considering the gap between two consecutive
words. Characters in a word are segmented again
by considering the vertical profiles restricted to the
image of the word. This information in a block-
word-character hierarchy is stored in a three
dimensional list.
Testing of the System
We collected several hand-filled-in application form
seeking a job. In Figure 17, result of our form
document processing system is shown.
Technical Reports/Correspondence
An MCA student, Mr. Saikat Das did the project
work for his MCA dissertation submitted to Indira
Gandhi National Open University (IGNOU), New
Delhi. Title of his work is “Recognition of Hand-
written (touching) Bangla characters from Form
Type Documents”.
Another student Prasenjit Mitra to complete DOE
“B” level has started his project work to develop a
system which can automatically process Indian
Money Order Forms. He will take approximately
another 4 months to complete the assignment.
4.2 Research and Development of Neural Network
based tools for printed document (in eastern
regional scripts) processing
Background
Artificial neural network (ANN) based methods
in solving various pattern recognition problems
have several advantages over the conventional/
classical approaches. Recently ANN based
classification approaches have gained tremendous
popularity. They are commonly used in the high
accuracy systems because they perform
satisfactorily in the presence of incomplete or
noisy data and also they can learn from examples.
Another advantage is the parallel nature of ANN
algorithms. On the other hand, we have already
67
developed an efficient OCR system for printed of a larger training set would improve the overall
documents in Bangla. There are obvious accuracy of the system.
justifications to explore suitable supervised/
unsupervised neural network/hybrid models for 5. The Team Members
developing software tools for shape extraction of
S. K. Parui swapan@isical.ac.in
individual printed characters with a view to
character classification and this may in turn make A. K. Datta, M. Mitra mandar@isical.ac.in
the already developed OCR system more U. Pal umapada@isical.ac.in
efficient.
S. Palit sarbani@isical.ac.in
Summary of the Project U. Bhattacharyya ujjwal@isical.ac.in
For extraction of shapes of individual printed U. Garain utpal@isical.ac.in
characters we may consider Self-Organizing T. Pal tama@isical.ac.in
neural network or Vector Quantization
techniques. Instead of using the input character N. S. Dash niladri@isical.ac.in
image for classification purpose we may obtain A. Datta and D. Sengupta
its graph representation consisting of a few nodes
and links between them. Useful topological and Courtesy: Prof. B.B Chaudhary
geometrical features may be easily obtained from Indian Statistical Institute
such a representation and those in turn should Computer Vision and Pattern Recognition Unit
result in better classification accuracy. 203, Barrackpore Trunk Road
Kolkata –700035
Description of the Work Done (RCILTS for Bengali)
Tel: 00-91-33-25778086
We used Topology Adaptive Self-Organizing Extn. 2852, 25781832, 25311928
Neural Network (TASONN) model to obtain the E-mail: bbc@isical.ac.in
graph representation of the input character. We
considered a few structural features of this graph
describing the topology of the character along with
a hierarchical tree classifier to classify printed
Bangla characters into a few subclasses. To
recognize different characters in each of these
resulting subclasses we considered several
geometrical features. Final recognition is
performed using a look-up table of these feature
values.

Testing of the System

Test pages are selected from different Bangla

books. Pages are scanned at 300 dpi. In total, 100
pages are used for training purpose and another
50 pages are used in the testing phase. Analysis of
test results shows a character-level accuracy of
about 98%. Error analysis indicates that since the
font used for printing of different books are
somewhat different from each other, generation

68
 Resource Centre For
Indian Language Technology Solutions – Oriya
Utkal University, & OCAC, Bhubaneswar
Achievements

RC-ILTS-ORIYA
Department of Computer Science & Application
Utkal University, Bhubaneswar, Orissa – 751004
Tel. : 00-91-674-2585518 / 0216 E-mail : sangham1@rediffmail.com
Website : http://www.ilts-utkal.org
&
Orissa Computer Application Centre
Plot No.-N-1/7-D, Acharya Vihar Square
RRL Post Office, Bhubaneswar-751013
Tel. : 00-91-674-2582490/2582850 E-mail : skt_ocac@yahoo.com
Website : http://www.utkal.ernet.in
 RCILTS-Oriya
Utkal University, Bhubaneswar
Introduction
Resource Centers (RC) for Indian Language
Technology Solutions (ILTS), under Technology
Development of Indian Language (TDIL) programs
of the Ministry of Communication and Information
Technology, Government of India, are established
for providing platform to disseminate knowledge
to common man through digital unites. One such
RC is established in Utkal University, Orissa to
handle the issues of Oriya language, the official
language of Orissa. For the effective implementation
of the idea, several technologies, such as Image
Processing, Speech Processing and Natural Language
Processing are merged together towards the
development of hardware and software under the
project for the services of Oriya people. This enable
them to be computer literate and thus to be a
knowledgeable person. RC-ILTS-Oriya works for
the development of tools like
1. Bilingual E-Dictionary English<>Oriya)
2. Oriya Spell Checker
3. Oriya WordNet
4. Oriya Machine Translation System (English –
Oriya)
5. Oriya Optical Character Recognition System
6. Oriya Text-To-Speech System
All these softwares are Copy Righted.
Besides this RC team members are also keen in the
development of softwares like
1. Oriya Speech-To-Text System
2. Oriya Word Processor with trilingual (Oriya,
English, Hindi) Word Processor with spell
checking capacity.
3. Sanskrit WordNet.
4. Jagannath Philosophy
1. Intelligent Document Processing (OCR) for Oriya
Document processing is a need for high-level
representation of the contents present in the
70
document so that the content can be analyzed and
understood clearly and unambiguously. Different
approaches are made for optical recognition of
characters for different languages like English,
Chinese, Japanese and Korean. Very little efforts
have been made for the recognition of Indian
languages. We have made an attempt for the
recognition of alphabetic characters of Oriya
language using novel technique, which helps for
efficient processing of text (document). Machine
intelligence involves several aspects among which
optical recognition is a tool, which can be integrated
to text recognition and text-to-speech system. To
make these aspects effective, character recognition
with better accuracy is needed.
The process of Optical Character Recognition of a
document image mainly involves six phases:
1. Digitization
2. Pre-processing
3. Segmentation
4. Feature Extraction
5. Classification
6. Post Processing
The digitization phase uses a scanner or a digital
camera that divides the whole document into a
rectangular matrix of dots taking into consideration
the change of light intensity at each dot. The matrix
of dots is represented digitally as a two dimensional
array of bits. Each dot can be represented by a single
bit for b/w image (0-black, 1-white) and for color
image, needs 24 bits for its representation. The better
the resolution, the better the image.
Preprocessing involves several activities, which
transform the scanned image into a form suitable
for recognition. These activities are noise clearing,
filtering and smoothing, thinning, normalization,
and skew correction. But for the recognition of
printed characters segmentation plays an important
role in the activities of preprocessing. After noise
clearing phase, it needs individual characters to be
extracted with better approximation so that no
characters should loose its important features during
the process of extraction. So efficient segmentation
algorithms should be employed which will lead to 11. The system tested on documents from book
better recognition. (Bigyandiganta) and also for some old Oriya
books.
During the segmentation phase, the whole image is
The system also integrated to text-to-speech system
analyzed and different logical regions in the image
for Oriya.
are separated. The logical units consist of texts which
has lines thus words thus characters. The error during
isolating characters changes their basic shapes. So
the characters must be properly extracted so that it
will lead to better and accurate representation of
the original character. Many algorithms exist for
extracting characters from an image. But the
problem arises when some characters are connected (An input image in bmp format)
in the document image. So two characters when
connected may be mistaken as a single character,
thus erroneous extraction leading to misrecognition.
In the feature extraction phase, the features of
individual characters are analyzed and represented
in terms of its specialty or uniqueness. These features
help in classification and identification of the
characters. The feature set needs to be small and the
values needs to be coherent for objects in the same
class for better classification.
Technology Involved
1. Literature survey done.
2. Gray tone converted to two-tone image by
dynamic thresholding on intensity.
(The output text in our editor)
3. Skew correction implemented for easy handling
2. Natural Language Processing (Oriya)
of documents.
Natural Language Processing is the technique of
4. Lines extracted from a document using
engineering our language through machine (the
histogram analysis.
computer) by which we can overcome the language
5. Individual characters extracted using region- barrier and the difference between man & machine.
growing and histogram analysis method. With that sincere motive we have taken several steps
6. Matras extracted by region analysis. in assisting the computer system behave like a person
in exchanging our knowledge. Our efforts are
7. Skeletonization for efficient processing, storing converted to product form and are mentioned
(less memory space allocation) and searching. below for the interest of researcher of this field and
8. Connected characters handled by backward and for public interest.
forward chaining of appropriate mask. In short our products already developed or under
9. Features extracted from isolated characters as development are Oriya Machine Translation
well as composite characters like jukta. (OMTrans), Oriya Word Processor (OWP) support
multilingual, Oriya Morphological Analyser
10. Analysis on Multicolored documents by (OMA), ORI-Spell, the Oriya Spell Checker (OSC),
applying Hue, Saturation and Intensity (HSI) Oriya Grammar Checker (OGC), Oriya Semantic
model. Analyser (OSA), ORI-Dic, the bilingual E-

71
Dictionary (English ó Oriya), OriNet (Word-Net 2). The OWP edits OCR (Oriya Character
for Oriya), and SanskritNet (Word-Net for Sanskrit) Recognition) outputs and also the documents of
are under developing phase to produce a complete other editors.
Natural Language Processing System.
2.1 Oriya Machine Translation System (OMTrans)
In OMTrans the source language is English and
target language is Oriya. We have developed a parser,
which is an essential part of Machine Translation.
Our parser is capable in parsing various types of
sentence including complex types of sentence such
as:
I am going to school everyday for learning.
He said that Ram is a hard working boy.
After the parsing phase the real translation is done. Fig 1 : Help option of OWP.
Our translation system is capable in doing the
translation of various types of simple as well as
complex sentences such as:
I am a good boy.
I am going to school everyday.
I will eat rice.
Who is the prime minister of India?
India is passing through financial crisis.
He told that Ram is a good boy.
Our system is doing the translation of sentences
having all types of tense structure. In addition to
that our system is also capable of doing the sense
disambiguation task based on N-gram model such
as:
Fig 2 : Multilingual processing of OWP.
I am going to bank for money. 2.3 Oriya Morphological Analyser (OMA)
I am going to bank everyday.
I am going to bank. I will deposit money Indian languages are characterised by a very rich
there. system of inflections (VIBHAKTI), derivation
In the above examples the meaning of underline and compound formation for which a standard
word (bank) is decided according to the context of OMA is needed to deal with any type of text in
the sentence. Presently it gives very good result for Oriya language. The number of words are being
this type of ambiguous sentences. derived from a given root word by some specific
syntactic rule. Our OMA deals with morphology
2.2 Oriya Word Processor (OWP) (Multilingual) of pronoun, number itself (Nominal, Verbal),
We have developed OWP, which facilitates OSP, Number less and prefix (Fig-3). We have developed
OGC and OSA along with multilingual editing. The and implemented some decision tree of each type
OWP avails phonetic typing for Oriya character with of morphology, by which our OMA is running
help facility (Fig-1). All other basic features are also successfully. It can help to all the applications
available in the OWP like other word processor (Fig- involved in MT, OriNet, OSC, and OGC etc.

72
 2.5 Oriya Grammar Checker (OGC)
The S/W for OGC has been developed to determine
the grammatical mistakes occurring in a sentence
by the help of the OMA and e-Dictionary. It first
parses the sentence according to the rule of Oriya
grammar and checks the grammatical mistakes.
Presently, the OGC functions successfully in our
Editor (fig.-5, fig.-6). The OGC -S/W supports both
the Windows-98/2000/NT and the Linux as well.

Fig-3 : Out put of the OMA for the Oriya derived

word “baHiguDika” in OriNet.
2.4 Oriya Spell Checker (OSC)
The misspelled words are being taken care of
successfully by our OSC. We have developed some
algorithms to perform our OSC in order to find
out more accurate suggestion for a misspelled word.
The searching algorithm of the OSC is so fast that it
processes 170000 Oriya words simultaneously for
each misspelled word. The words are indexed
according to their word length in our word database
Fig.-5 Suggestion of word sequence of OGC in
for effective searching. On the basis of the misspelled
our OWP.
word, it (OSC) matches the number of (i) equal
character, (ii) forward character and (iii) backward
character to give more accurate suggestive words for
the misspelled word. Moreover, it also takes help
from the Oriya Morphological Analyser for
ascertaining the mistakes of derived words. The OSC
functions successfully in our Word Processor. This
S/W supports both the Windows-98/2000/NT and
the Linux as well. Output of the Spell Checker is
shown in Fig- 4.

Fig.-6:- Detection of grammar by OGC

Fig. 4 : Suggestive words for the misspelled word
“bAsitAmAne” of OSC.
2.6 Oriya Semantic Analyser (OSA)
It deals with the understanding procedure of the
sentence for which it takes immense help from the
KARAKA theory of NAVYA NYAYA philosophy,
one of the most advanced epistemological traditions
of India. The OSA determines the semantic position
of Subject (KARTA), Object (KARMA) etc on the
basis of the verb (KRIYA). In other words, it at first
links to the verb of the Verb Table (VT) and from

73
verb it links to the subject. Other KARAKAs are in progress to include Hindi and Sanskrit words in
determined on the basis of these two linkages (fig.- this system for the benefit of the users.
7). It also takes help from OriNet, OMA and OGC
for better understanding. We have worked out 100
verbs in the VT determining their subjective,
objective, locative categories etc with respect to these
verbs. Presently, with these verbs the OSA functions
successfully in our Editor (fig.- 8). The OSA S/W
supports both the Windows-98/2000/NT and the
Linux as well.

Fig. 9: Overview of the Oriya Word “AkAsha.”

Fig-7:- Semantic Grammatical Model (SGM) and

Semantic Extract Model (SEM)
(Here downward arrows represent the SGM and
upward arrows along with down arrow from verb
represent the SEM where nominative case acts as
chief qualifier.) Fig. 10 : Overview of the English Word “delay”.

Fig. 11 : Search Engine (SE) handles misspelled

word and gives suggestion.

Fig. 8:- Detection of semantic by OSA.
2.7 E-Dictionary (Oriya↔English)
It provides the word, category, synonymy and
corresponding English meaning of Oriya as well as
English words. The system is successfully functioning
over 27000 Oriya words and 10000 English words.
Help option also provides the keystroke for each
Oriya character in phonetic form. Search engine
handles the misspelled word and gives some accurate
suggestive words. This S/W supports both the
Windows-98/2000/NT and the Linux. We are also
2.8 OriNet (WordNet for Oriya)-Online lexical
dictionary/ thesaurus.
One of the major problems in the implementation
of Natural Language Processing (NLP) or Machine
Translation (MT) is to develop a complete lexical
database containing all types of information of
words. There are some difficulties in deciding that
what information should be stored in a lexicon and
even greater difficulties in acquiring this information
in proper form. The OriNet system is designed on
the basis of multiple lexical database and tools under
one consistent functional interface in order to

74
facilitate systems requiring syntactic, semantic and per user’s query or other application with the help
lexical information of Oriya language. The system of MA.
is divided into two independent modules. One
At the heart of the OriNet system is the OD, which
module is developed to write the source files
stores the data in simple ASCII format, several easy-
containing the basic lexical data and these files are
to-use interfaces have been devised to cope with
taken as the input for OriNet system. Lexicographer
varied user requirements and the raw data extracted
takes care the major work of this module. Second
from the database are well formatted for display
module is a set of programs by which it accepts the
purpose. MA takes care syntactic analysis of any types
source files, processes it to display for the user and
of lexicon and also provides the root words with
also provides different interface to use other
other grammatical information. X- windows
applications. System has been designed using Object-
interfaces are also designed to access the database
Oriented paradigm according to Oriya language
with different types of options (Fig.-13).
structure with over 1100 lexical entries, which allow
flexibility, reusability and extensibility. It also
provides X-windows interface to access the data from
the OriNet database as per user’s requirement. It
can be widely used in different application like, (i)
Word Sense Disambiguation (WSD) in Oriya
Machine Translation (ii) Oriya Grammar Checker
(OGC) and (iii) Oriya Semantic Checker (OSC).
Moreover, it helps as a lexical resource for Oriya
learners and also for expert scholars who are
involved in the research in NLP. The system also
consists of Oriya Morphological Analyzer (OMA),
which takes care of any type of word such as root
word or derived word and also provides syntactic
information of the word. This S/W supports both
the Windows-98/2000/NT and Linux. Presently, we
Fig. 12:- Architecture of the OriNet System
are adding more and more lexical entries in the
source file and developing different applicable
programs for use in wider range.
Architecture and Output
The architecture of the OriNet system is divided
into five parts (Fig.-12) such as Lexical Source File
(LSF), OriNet Engine (OE), OriNet Database
(OD), Morphological Analyser (MA) and User
Application (UA). The LSF is the collection of
different sorted files according to their syntactic
categories, which are taken as the inputs to the Fig. 13:- Overview of the word bþmþ (good).
OriNet system. The OE is a set of programs, which
2.9 SanskritNet (Word-Net in Sanskrit)-Online
compiles the lexical source files into a database
lexical dictionary/thesaurus.
format that facilitates machine-retrieved information
in proper manner. It also acts as the CPU of the Sanskrit language is the base for most of the Indian
OriNet system. It is used as a verification tool to languages. It has link with foreign languages too.
ensure the syntactic integrity of lexical files. All of To study this language and also to use it for
the lexical files are processed together to build the knowledge enhancement, effective Machine
final OD. It also provides sufficient information as Translation (MT) from one language to the other is

75
necessary for which online lexical resources are
needed. Towards making such resources we have
tried to develop a Word-Net for Sanskrit language
using the Navya-NyAya (xÉ´ªÉxªÉɪÉ) Philosophy and
Paninian grammar. Besides Synonymy, Antonymy,
Hypernymy, Hyponymy, Holonymy and
Meronymy we have also introduced Etymology and
Analogy separately as they play important roles in
Navya-NyAya (xÉ´ªÉxªÉɪÉ) Philosophy, which is the
specialty on this Sanskrit Word-Net for a better
classification of words.
Presently, we have defined and analysed 300 Sanskrit
words (200 nominal words and 100 verbal words)
in the SanskritNet. On the basis of the analysis and
the definition of these words, we are on progress to
design and implement the SanskritNet and other
allied applications as well. Here the prototype model
of the SanskritNet is displayed (Fig.-14).
Fig. 14: - Output of the Nominal word “ºÉÚªÉÇ” (Sun)
in SanskritNet
3. Speech Processing System For Indian Languages
In the age of fast technology, where information
travels at the speed of light, we still rely on feeding
our text inputs in typographical manner. Speech
Processing System is an approach to provide a speech
interface between user and the computer. Basically
our system zeros in on Hindi, Oriya and Nepali
languages at the Resource Centre.
Broadly the system is classified into two sections.
• Text To Speech (TTS)
• Speech To Text (STT)
76
Aim
To design a system/algorithm, which works
efficiently, naturally and utilizes memory as less as
possible.
Progress
We have developed TTS for Oriya and Hindi
language and the TTS for Nepali language is in
progress.
3.1 Text to Speech (TTS)
As the name signifies this system provides an interface
through which a user enters certain text/document
and it is the software that reads it as natural as a
human. The basic approach followed here is, first to
analyse the document (language, font etc.), and then
extract words from the text, try to parse individual
words into vowels and consonants respectively. Then
corresponding to these vowels and consonants
existing (previously stored in the database) “.wav”
files are concatenated and played.
Technologies Behind :
• Creating the wave file database.
For creation of such a database we studied a lot of
recorded words and sentences and try to break them
into vowels and consonants by minute hearing.
Then we analyse those cut pieces and store the
appropriate and generalised form in the database.
• Parse extracted exact words of a given sentence?
The same words in different sentences have different
stress due to its position in the sentence. Appropriate
hidden vowels are detected form the words extracted.
For example considering a word “jc¯“ ( SAMAYA)
in Oriya is parsed as follows :-
jç + @ + c ç + @ + ¯çÆ + @
The format of vowel and consonant break point is
shown in the fig1.
Again considering the same word in Hindi i.e., Þle;ß
(SAMAY) is parsed as follows :-
l~ $ v $ e~ $ v $ ;~ $ v
• Choosing of appropriate ‘.wav’ file from the
database.
Considering the above example, may be the
vowels we get after parsing are the same as ‘@’
(for Oriya) or ‘+’ (for Hindi), but it is not exactly
the same ‘a.wav’ we concat in every case. Thus,
we analyse vowels broadly in three categories as
ma:tra: in
• Beginning
Middle
• End
And this is observed that the duration of ma:tra:s
say ‘@’ here, varies from each other, i.e. ‘@’ in middle
is not the same as that in the end. Again accordingly
we need to get the appropriate “.wav” files from
the database.
As observed in the example
i) The durations of ‘@’(Oriya) are as follows:
Starting ma:tra: - 0.065 sec
Middle ma:tra: - 0.105 sec
End ma:tra: - 0.116 sec
ii) The durations of ‘+’(Hindi) are as follows :
Starting ma:tra: - 0.063 sec
Middle ma:tra: - 0.078 sec
End ma:tra: - 0.020 sec
• It is observed that concatenation of the wave files
is not that natural as expected. This is due to the
certain transitions between the characters in the
actual pronunciation. Thus we are developing a
robust algorithm for the generation of naturalness
in the TTS output.
Applications
# Helpful for the blinds and illiterates.
# Airways and Railways announcement system.
# Initial phase of Speech-to-Speech conversion
system.
3.2 Speech to Text (STT)
The speech processing is a pattern recognition
problem. The recognition of speech is defined as
an activity whereby a speech sample is attributed
to a person on the basis of its phonetic-acoustic
or perceptual properties. In our approach we study
the nature of spoken words by different speakers.
From a continuous sentence the word boundaries
are detected and the nature of utterance of
individual consonants and vowels are marked to
study their behaviour for a particular speaker. We
use word spotting techniques and the variations
of pitch and intonation are marked. The
algorithm makes the use of the properties of F0
contours such as declination tendency, resetting
and fall- rise patterns in the utterance. Once the
patterns are obtained it is mapped with a particular
character and the proper text output is obtained.
The boundary and parameter detection is shown
in fig2. After obtaining the parameters of an
uttered word, they are standardise to maintain the
speech database. We also provide some tolerance
in each parameter, so that little variation in the
utterance doesn’t affect the recognition of the
word for text conversion. Form each word the
characters are obtained and when a speaker utters,
the corresponding character is mapped to the text
and the output is obtained.
wah kitna sunder hai !
´É½þ ÊEòiÉxÉÉ ºÉÖxnù®ú ½èþ !
The stress on the vowels (rising and falling) are
shown in the pitch curve. The spectrograms of the
uttered words show the content frequency and
intensity of utterance. The word boundary is
detected by formant analysis. Though F0 plays a
major role. The cepstral component of the
continuous speech puts major emphasis in the
utterance. Hance taking the word boundary into
consideration the speech database show around 73%
accuracy for hindi and 90% in Oriya speech
recognition.
Application
# Speech password for security in banking sector.
# Foreign sic experts need for speech recognition
& identification of criminals.
# For the blind and illiterate people, provides an
intuitive interface with machine.
77
# Final phase of Speech-to-Speech conversion RCILTS-Oriya
System. OCAC, Bhubaneswar
1. Product Developed at OCAC and the Target
date of its Commercialization
1.1 Oriya Spell Checker
jç @ (starting ma:tra:) Oriya Spellchecker has been developed in Linux
platform. It consists of 60,000 base words. These
base words can be manipulated and stored in
dictionary in a scientific manner. Root words can
also be manipulated. This Spellchecker is
cç @ (middle ma:tra:) incorporated in our Oriya word processor Sulekha.
This project is developed for the smooth use of
novice users.
Here we are concentrating with our suggestion and
¯ç Æ @ (end ma:tra:) checking of words. The checker checks the pattern
Fig. 1 string and suggests to the user more accurate and
required suggestion. Like English Indian languages
are different in their nature. Indian languages are
complex in their nature. For that we have designed
a checker and that checks the pattern string. Words
are stored in side the dictionary file line by line.
Each line is terminated through an escape sequence
of new line character. All these Oriya words are stored
wa ha ki t na sun da r h ai inside a file having extension .txt.
Fig. 2 Features
4. The Team Members Spellchecker consists of several facilities like Prefix,
Suffix, Add, Ignore, Change, Spellcheck, Suggestion
Khyamanidhi Sahoo khyamanidhi@yahoo.com
and Cancel. Suggestion is accurate and match more
Hemanta Kumar Behera kumar_hemanta@yahoo.com closest word to the user requirement. It checks more
Pravat Chandra Satapathy satapathy_pravat@rediffmail.com words within a fraction of seconds. Searching time
is minimum and complexity is less. Spellchecker can
Suman Bhattacharya suman_bh1@yahoo.com
be incorporated in any application for example it
Ajaya Kumar senapati ajayasenapati@rediffmail.com can be incorporated in a word processor or an editor
Pravat Kumar Santi pksanti@rediffmail.com or a browser. It uses ISCII data stored in a file.
Gour Prasad Rout rout_g_prasad@yahoo.com ISFOC for display. Gets the input ISFOC data
displayed the ISFOC data.
Kabi Prasad Samal kabi_prasad@rediffmail.com
Krushna Pada Das Adhikary krushnapada@rediffmail.com
Commercialization
Windows version has been completed and
Courtesy : Prof. (Ms) Sanghmitra Mohanty commercialized and incorporated in LEAP Office
Utkal University of CDAC Linux version Completed
Department of Computer Science & Application
Vani Vihar, Bhubaneshwar – 751 004 1.2 Thesaurus in Oriya
(RCILTS for Oriya) Oriya Thesaurus has been developed in Linux
Tel: 00-91-674-2585518, 254086 platform. It consists of 40,000 base words. These
base words can be manipulated and stored in
E-mail: sangham1@rediffmail.com) dictionary in a scientific manner. Root words can

78
also be manipulated. This Thesaurus is incorporated
in our Oriya word processor Sulekha. This project
is developed for the smooth use of novice users. It is
a tool for correct documentation. Words are stored
in side the dictionary file line by line. Each line is
terminated through an escape sequence of new line
character. All these Oriya words are stored inside a
file having extension .txt.
Features
Thesaurus consists of several facilities like hesaurus,
Replace. Suggestion is accurate and matches closest
word to the user requirement. It checks more words
within a fraction of seconds. Searching time is
minimum and complexity is less.Thesaurus can be
incorporated in any application for example it can
be incorporated in a word processor or an editor or
a browser.
Commercialization
Completed
1.3 Bilingual Electronic Lexicon
The dictionary for administration and official
correspondence has been entered for Oriya-English
and English-Oriya. It is proposed to bring out an
Electronic Dictionary as a commercial product
which will have features such as phonetic key board
and user friendly functions for “add”, “delete”, “view”
etc. After the availability of text to speech technology
in Oriya the same will be integrated to this product
for providing help in correct pronunciation of
words. Lexical inputs from other domains such as
culture, business, science are being collected.
Commercialization
Words have been entered. As a product with more
words will be completed in June 2003.
1.4 Corpus in Oriya
A large corpus has been developed consisting of
nearly 8 million words for analysis and use in spell
checker and thesaurus. Corpus creation has been
taken as a continuing activity as a part of this project.
Commercialization
It will be used for research and use in other products
and is not for commercialization,. (completed)
1.5 Bilingual Chat Server
Oriya Chat is a Standalone Chat Application in
Oriya Language developed in JAVA. After
connecting to the Server, one can chat in one’s
favorite room, send direct messages with emotion
icons to other online users as well as send public
messages.
Features
User Log in to Chat Server with unique ID, Con-
nect to the Server, with or without firewall/proxy,
Phonetic Keyboard Layout to write in Oriya,Send
Oriya text with Emoticons (images), Private and
Public Chat With Friends, Domain specific Pre-
defined Public Rooms.
System Description
Oriya Chat has a client and a server module.
Serversocket and SocksSockets are used for com-
municating with remote machine. SocksSocket is a
CustomSocket that extends Socket to be used with
the firewall or proxy.
Commercialization
Completed
79
1.6 Net Education Varta is an e-mail solution, which enables people to
A Net Education System (NES) is being developed send and receive mail in Oriya language. It uses
using Bilingual Chat Server and Mail Server. The dynamic version of the OR-TTSarala font, created
educational Content is being created for providing using Microsoft WEFT-III. Use of dynamic font
students, public for education on various aspects allows the client to read and compose the mail
through the net. without downloading the Oriya font. The system
has been developed using ASP technology, Java,
Commercialization
JavaScript and HTML. Varta uses default SMTP
The Chat Sever with Interface has been completed.
Server on the Internet Information Server 5.0 of
A full fledged system with content will be completed
Window 2000 Server.
soon.
Features
1.7 XML Document Creation and Manipulation
Most of the basic e-mail features are available in
XML document creation technology in English and
Varta 0.2, which includes : Sending, Replying,
Oriya language has been developed. It incorporates
Forwarding and Reading Oriya Mail using dynamic
users defined tags, structures the document like a
Font, Online Registration of unique user mail
database providing real world meaning of the
account, Password Security with Online Help to
content. This provides more flexibility to the
retrieve forgotten Password, Phonetic keyboard
presentation of the same document in different style
typing in Oriya, Online typing help through a
sheets and same style sheet for different documents.
keyboard map to dynamically select appropriate keys.
Also queries can be processed on the contents of the
XML document. Content on Tourism has been Phonetic Keyboard Engine
created in XML. Agricultural content useful for The Oriya keyboard has been designed as per Indian
farmers are being created. phonetic standard. It is implemented as an applet
in Java language. It is compiled in Java 1.0 JDK so
Commercialization
that it runs in MS Internet Explorer 5.0 onwards.
A few documents have been completed. More work
is in progress to be completed by August 2003.
1.8 Oriya OCR
OCAC is collaborating with ISI Calcutta to develop
and commercialization of Oriya OCR . All the
language inputs has been provided to ISI Calcutta.
The software developed by ISI Calcutta is being
tested by OCAC. OCAC will commercialize the
Oriya OCR with know how from ISI Calcutta.
Commercialization
Work in collaboration with ISI, Calcutta is in
progress. Will be completed in June 2003
1.9 Oriya E-Mail
Commercialization
A majority of Indian population cannot speak, read
Completed
and write in English language. Oriya language based
tools helps the common man to benefit from e- 1.10 Oriya Word Processor with Spell Checker
governance and other applications of Information under LINUX
Technology.
For the first time an Oriya word processor has been
In order to bridge the digital divide it is necessary to developed on Linux operating system. It runs on
provide Internet application such as e-mail and chat any X-windows system. Two Keyboard engines
in Oriya language for use by the common man. (Inscript keyboard engine and Phonetic Keyboard

80
Engine) have been incorporated for the ease of
typing. All the texts will be displayed in
GLYPHCODE. For display of a stored ISCII file,
a converter engine automatically generates
GLYPHCODE equivalent of the file.
Features
Most of the basic editing features available in
standard word processor have been incorporated
which includes : Opening new file, saving a file,
closing window, cut, copy, paste, Single level Undo
and Redo is provided, Keyboard engine for typing
in Inscript layout, Keyboard engine for typing in
Phonetic layout, The ISCII to GLYPHCODE
conversion, The GLYPHCODE to ISCII
conversion, Storing content of a file in plain text as
well as in ISCII format, Other features such as Find,
Find and Replace, Go To Line, File Information, A
Spell Checker incorporated within the word
processor.
Keyboard Engine
Two Keyboard engines have been designed. One
according to Indian script (Inscript) standard.
Another according to Phonetic standard. All the
consonants, vowels and conjuncts and special
symbols etc are displayed in Glyph Code format.
Invalid word combinations according to Oriya
Grammar are disallowed. Pressing the Scroll Lock
Key activates the Keyboard Engine.
Converter Engine
An Export file option is provided to convert 8-bit
glyph code to 8-bit ISCII code format. The 8-bit
ISCII code can also be converted back to 8-bit glyph
code.
Commercialization
Completed
1.11 Computer Based Training (in Oriya)
It is well known that multimedia CDs are highly
effective medium of learning in many disciplines.
The integration of text with hypertext links, speech,
graphics, animation and video in an interactive
mode makes the lessons easy to visualize and
understand for self-learning. Computer Based
Tutorial CDs for school children have been
developed to support the classroom teaching with
interactive self-learning program.
Features
Informative content with animated illustrations,
Interactive quiz for evaluation of subject
understanding.
Description
This project is aimed at the development of tutorial
CDs based on School Curriculum, in Oriya
language. The content is based on textbooks
prescribed by Board of Secondary Education, Govt.
of Orissa. The lessons are illustrated with text, speech,
and animation with facility to navigate through
various chapters using hyperlinks. At the end of a
chapter, the on line quiz helps in self-evaluation by
the student. It is based on Windows 98 Operating
System and Tools used are Macromedia Authorware
6, Macromedia Flash 5.0, Adobe PageMaker 6.5, Adobe
Illustrator 9.0. The minimum System Requirements
for Installation is Pentium II or higher, 128 MB or
more RAM, Internet Explorer 5.5 or more
Commercialization
Two CD have been completed .Write up with colour
brochure enclosed. Work in more subject is in
progress.
81
1.12 Oriya Language Based E-Governance 4. Core Activities
Applications
4.1 Web hosting of Oriya Language Classics
A number of software modules have been
developed for enabling the common citizens to Three well known Oriya classics :
access government information and services in (1) “Chha Mana Atha Guntha” by Fakir Mohan
Oriya and English languages from the Kiosks. A Senapati,
citizen can submit grievance petition; can apply
for one or more certificates ( such as birth, death, (2) “Chilika” by Radhanath Ray and
caste, income etc; down load the application forms (3) “Tapaswini” by Gangadhar Meher - have been
and know the rates of consumer commodities etc. hosted on Internet at http://
These modules have been used as a pilot project www.tdil.utkal.ernet.in
on improving citizens’ access to information in
Kalahandi district of Orissa. 4.2 Hosting of Web Sites of Govt. Colleges in Orissa

Commercialization Web sites have been hosted for the Khallikote College,
Development of a number of applications has been Berhampur www.khallikotecollege.utkal.ernet.in
completed and integrated with the project on and Ravenshaw College, Cuttack.
Citizens’ Access to Information. www.ravenshawcollege.utkal.ernet.in The pages
include information on the college as well as forms
2. TOT done for various projects so far for students feed back. More dynamic information
1. Spell Checking Grammar rules in Oriya Language will be supported by the web sites in future, which
has been Transferred to CDAC. will be undertaken for many colleges in Orissa.
2. Documents on Oriya Language Standards and 5. Products proposed to be developed
Principles have been prepared and for
1. CBT for all classes (std – 4th to 10th)
UNICODE and sent to MIT.
2. OCR
3. The draft for the preparation of National 3. Upgradation of spell checker
Language Design Guide for Oriya has been 4. Grammar module development
prepared and submitted to MIT 5. Machine Translation System (Oriya –English)
6. Net Education System in Oriya Language
4. Oriya Script with Grapheme details have been
provided to ISI, Calcutta for Oriya OCR. 6. The Team Members
3. Training programmes run for the officials for Biswa Ranjan Sahoo biswa_litu@yahoo.com
the State Govt. Sambit Kumar Sahu sam_sahu@yahoo.com
OCAC had earlier trained 70 persons in various Girija Shankar Sarangi _sarangi@rediffmail.com
government offices on Oriya Language Word Sunil Kumar Panda k2_sunil@yahoo.com
Processing using LEAP Office in which the Oriya Sarita Das sarita_das@yahoo.com
Spell Checker has been incorporated. The last Subhendu Kumar Mohanty ohanty_subbu@yahoo.com
formal training course on “ Word processing in Sanjay Kumar Dey sanjay_k_dey@yahoo.com
Oriya Language using Leap Office and ISM “ was Courtesy : Shri S.K. Tripathi
conducted during January 17 - February 12 2002. Orissa Computer Application Centre
in which only four officials attended the course. OCAC Building, Plot No. 1/7-D,
Again a course has been organized for 106 staff of Acharya Vihar Square, RP-O,
the Board of Revenue during 29-07-02 to 3-08-02 Bhubaneswar – 751 013
(first batch) and 5-08-02 to 10-08-02 (second (RCILTS for Oriya)
batch). So far 180 officers have been trained. We Tel: 00-91-674-2582484, 2582490,
are developing course material on a CD for training 2585851, 2554230 (R)
on Oriya Language. E-mail: saroj@ocac.ernet.in

82
 Resource Centre For
Indian Language Technology Solutions – Assamese & Manipuri
Indian Institute of Technology, Guwahati
Achievements

Indian Institute of Technology, Guwahati

Department of Computer Science & Engineering, Panbazar
North Guwahati, Assam-781031 India
Tel. : 00-91-361-2691086 E-mail : tdil@iitg.ernet.in
Website : http://www.iitg.ernet.in/rcilts
 RCILTS-Assamese & Manipuri
Indian Institute of Technology, Guwahati
Introduction
Conceived in the millennium year, the Resource
Centre for Indian Language Technology Solutions
Indian Institute of Technology Guwahati, is a
project funded by the Ministry of Information
Technology of the Government of India. The
main objective of the Centre is to make electronic
information available in native languages mainly
Assamese and Manipuri thereby aiding the
dissemination of information to the larger masses.
This Centre is one among several in the nation
and is equipped with all modern systems and
language related software. Four investigators from
the Indian Institute of Technology Guwahati and
one collaborator from Guwahati University are
presently involved in research in the areas of
Natural Language Processing, Speech Recognition
and Optical Character Recognition Systems. The
Centre is equipped with all modern equipments,
which include several Pentium-III and Pentium-
IV machines, a Linux server and a Windows2000
server, scanner, web camera, printers, digital
camera and audio recording systems. Eight project
personnel, five technical and three linguists
working in joint collaboration with Guwahati
University, man the Centre.
The Centre has developed various language related
tools and technologies over the last two and odd
years. Details of the same follow:
1. Knowledge Resources
1.1 Corpora : A systematic collection of speech
or writing in a language or variety of a languages
forms a language corpus. The two corpora created
by the Centre are-
" Assamese Corpora: Seven Assamese novels
have been transformed into electronic form
to form the base corpus. The salient features
are
1. Number of words : 6,00,000
2. Fonts used : AS-TTDurga (C-DAC font) and
Geetanjalilight (A popular font used in DTP
work).
84
3. Encoding Standard: ISCII.
" Manipuri Corpora : The Manipuri corpora
were received from Ministry of Information
Technology, Govt. of India. This corpora
creation started at Manipur University under
the aegis of Prof. M.S.Ningomba and Dr.
N.Pramodini. The corpora were in LP1
format, which is compatible to Leap Office.
The total word count comes out to be around
3,60,000. The fonts used in the creation of
this corpus are BN-TTDurga, BN-TTBidisha
and AS-TTBidisha. Further investigation to
make it compatible to existing systems is in
progress.
1.2 Dictionaries : Derived from the Latin word
dictio (the act of speaking) and dictionarius (a
collection of words), a dictionary is a reference
book that provides lists of words in order along
with their meanings. Dictionaries may also
provide information about the grammatical
forms, syntactic variations, pronunciation,
variations in spellings, etymology, etc. of a word.
The Centre has developed e-dictionaries, the
details of which are shown in the Table 1.
Dictionaries Root Words Total words
English-Assamese 5000 49,627
Assamese-English 2000 9,015
English-Manipuri 2500 36,778
Manipuri-English 3000 7,228
Table 1. Dictionary details
Each entry in the Dictionary contains
information on – i) the Grammatical form, ii)
Meanings iii) Synonyms iv) Antonyms, v)
Pronunciation of the word (in the from of sound
files) vi) Transliteration in English, vii) Soundex
code and viii) Semantic category. More words are
being added on.
Features
a) Provides information about a word.
b) These dictionaries have been structured to
support the spell checker and machine
translation systems being developed at the
Centre.
1.3 Design Guides
A design guide gives a brief overview of a
particular language. The main topics covered by
the design guide are the consonants, vowels,
conjuncts and matras which form the character
set of the language and numerals, punctuation
marks, month names, weekday names, time zones,
currency, weights and measurements used in that
particular language. Additionally, some linguistic
information such as Phonological features,
grammatical features, the history of the language
and the geographical description of the state where
the particular language is spoken are also included.
The Resource Centre for Indian Language
Technology Solutions, Indian Institute of
Technology Guwahati has designed two design
guides for Assamese and Manipuri.
1.4 Phonetic Guides
Pronunciation is also an important part in
language learning for which we have a Phonetic
guide in the website. This guide describes how a
particular alphabet should be pronounced. The
International Phonetic Alphabet (IPA) is used in
the dictionaries as well as in the description of
the languages (North –East India) available in the
web site.
2. Knowledge Tools
2.1 The RCILTS, IITGuwahati Website (http://
www.iitg.ernet.in/rcilts)
As part of its core objectives the Centre hosts a
website that offers a wide variety of information
ranging from Assamese and Manipuri languages
to Geographical and Cultural issues.
Information hosted may be categorized as:
(i) Linguistic Information: The Northeast is
known for its large diversity in languages.
Presently the website holds information on
sixty-five of the existing languages.
(a) Northeastern Languages : The website
contains information on sixty-five North–
Eastern languages like Assamese, Chokri,
Chakesang, Zeliang, Pochuri, Lotha,
Sangtam, Deuri, Dimasa, Kokborok,
Tintekiya, Koch, Hurso, Miji, Chang,
Khiamngamn, Konyak, Nocte, Phom,
Tangsa, Wanchoo, Hmar, Karbi, Kuki,
Lakher, Manipuri, Mizo, Riang, Khasim
Monpa, Takpa, Tsangla, Sherdukpen, Sulung
(Puirot), Adi, Apatani, Bori, Mishmi,
Mishing, Nishi, Tagin, Liangmaialong with
their classification.
(b) Linguistic Map : Preliminary work
commenced with the compilation and design
of the Linguistic Map of the Northeast. After
acquiring published and unpublished material
on these languages of the Northeast and with
inputs from the Census Department,
Surveyors, Linguistic fieldwork and findings
of Dr. Dipankar Moral, the Linguistic Map
came into being. The website presently has a
coloured updated version of this map. The
size of the font connotes the density of
speakers of a particular language in that area.
Figure :1 Resource Centre’s Home page
85
(ii) On-line Dictionary: A transliterated Assamese
dictionary has been put up on the website. It
carries information on the Meanings, (in
Assamese and English), Grammatical
categories and the Pronunciation.
(iii) Geographic and Demographic information:
Both geographic and demographic
information are made available for each of
the seven Northeastern states of Assam,
Meghalaya, Manipur, Nagaland, Mizoram,
Tripura and Arunachal Pradesh.
(iv) Guides: Guides have been put up on the
website to aid users to understand language
related issues. Two such guides that are
currently are:
(a) Design Guides: Design guides for both
Asamiya and Manipuri that give a general
idea of the respective language together with
certain frequently used sentences has been
hosted to aid a novice get a basic idea about
the language.
(b) Phonetic Guides: Pronunciation is an
integral part of the language learning
mechanism. A Phonetic Guide describes how
an alphabet should be pronounced. The
International Phonetic Alphabet (IPA) has
been used in the dictionaries as well as in the
description of the languages (North–East
India) to convey the manner in which the
words are pronounced. The Phonetic Guide
allows the user to interpret the same correctly.
Figure 2 depicts the manner in which one
can correlate the IPA symbols to the normal
orthography especially for the pronunciation
of vowels using the phonetic guide. Figure 2
depicts the IPA chart for vowels.
Vowels
IPA English Phonetic
Word Representation
i pin pin
e pen pen
ε Pat (in the production of pæt
86
vowels the upper surface
of the tongue is always
convex, hence the result
in tongue height. In this
context it may be mentioned
that Asamiya/ ε /is slightly
higher than English / æ /as
in / pæt / pæt
a farm fam
u too tu
T put pTt
o saw so
N got gN t
Figure 2. Vowel IPA chart
(v) Web based Dictionary: A transliterated
Assamese Dictionary has been put up on the
website. This dictionary is possibly the first
contemporary standard on-line Assamese
dictionary. Moreover, this dictionary is
designed in such a way that it will be helpful
for the non-native speaker. It provides
information on a) English Word b)
Grammatical Category c) Meaning d)
Pronunciation and e) Assamese Meaning.
(vi) Bookmarks: Additional Links to various
Newspapers of Assam have been provided. A
Manipuri News letter titled “Manika” has also
been put up. Further links to the various
tourism related sites of Assam and Manipur,
News Portals and other Resource Centres are
also made available in the website.
2.2 Fonts : The Centre has developed a True Type
Font “Asamiya” using Fontographer software.
This font can be used for typing Assamese text in
Microsoft Word. Fine Tuning of the existing Font
Set has been done where the spacing and Matras
have been adjusted. A Manipuri True Type font
has also been created.
Eõ F G H Iø
»Jô »K÷ L Mõ AÕ
»Oô Pö Qö »Rô S
Figure : 3 Snapshot of Assamese font “Asamiya”
2.3 Spell Checker
A Spell Checker forms a vital ingredient of a word
processing environment. The basic tasks
performed by a spell checker include comparison
of words in a document with those in a lexicon
of correct entries and suggesting the correct ones
when required. The two commonly used methods
for detection of non-words are the Dictionary
look-up and the N-gram analysis. Isolated-word
error correction is achieved by either of the
Minimum edit distance technique, the Similarity
key technique, Rule-based methods, N-gram,
Probabilistic and Neural Net techniques. Context-
dependent error correction methods usually
employ Natural Language Processing (NLP) and
Statistical Language Processing for correcting real-
word errors.
Spell checking strategy for Assamese
The development of a Spell Checker for Assamese
has been undertaken at the Resource Centre, and
code has already been developed in Perl. The Spell
Checker exists in the form of separate modules
for error detection and correction, as well as a
stand-alone system in which all the spell checking
routines have been integrated. The strategy used
is described below.
Non-Word Detection
The non-words are detected by looking up
document words in a dictionary of valid words.
The dictionary used is actually a word list in which
5000 distinct Assamese words have been extracted
from the English-to-Assamese online dictionary
developed by the Centre, and the rest of the words
were taken from a corpus of about 67,000 words.
This means that the look up dictionary contains
around 72,000 words. A hash table has been used
as a lexical lookup data structure. The performance
of a dictionary using a hash table is quite adequate,
even for complex access patterns. Perl has an
efficient implementation of a hash table data
structure, which has been used in the non-word
detection module.
Isolated-Word Error Correction
A three-pronged strategy has been used for
generating suggestions comprising of the
Soundex, Edit-distance and Morphological
processing methods.
Soundex Method
This method maps every word into a key (code)
so that similarly spelled words have similar keys.
A Soundex encoding scheme for Assamese has
been designed based on the encoding scheme for
English which comprises of a set of rules for
encoding words and 14 numerical codes. The
Soundex code of the misspelt word is computed,
and the dictionary is searched for words, which
have similar codes. An example is given below
(Table 2).
Word Soundex code
]»] meaning ]65
affection
Ø‘öçWýÝX meaning Ø‘öç35
independent
Zgõ×c÷Ì^çã_ãcg÷ãTöX meaning Zõ794735
would have analyzed
Table : 2 Assamese words and soundex codes
Edit-Distance Method
Candidate suggestions are obtained by ‘Damerau’s
error reversal’ method. The four well known
error-inducing edit actions are the insertion of a
87
super fluous letter, deletion of a letter, • Text can be copied/cut from an I-leap document
transposition of two adjacent letters and and pasted onto the GUI window.
substitution of one letter for another. In
‘Damerau’s error reversal’ each edit action is
applied to the misspelt string and a set of strings
is first generated. These are checked in the
dictionary to see which of them are valid words
to finally produce the suggestions.

Morphological Processing

Morphological analysis is performed to extract the

root word from the misspelt word as also a list of
valid affixes that can be attached to that root word.
By attaching the affixes that closely match those
of the misspelt word to the root word, a list of
suggestions is generated.

A module for ranking of suggestions, based on

Minimum edit-distance methods has also been
developed.

A run-time snapshot of the Assamese Spell

Checker is shown in Figure 4.

Features of The Stand-Alone Spell Checker
• GUI developed using Perl/Tk, with simple text
editing facilities.
• Currently supports C-DAC’s AS-TTDurga
font.
• Assamese text files can be loaded onto the GUI,
edited, and saved.
• Misspellings can be marked by clicking the
‘Det’ (‘Detect’) and then the ‘Show’ button.
• Selecting the misspelt word and clicking
‘Sug’(‘Suggest’) button can generate
suggestions.
• A facility to add new words to the dictionary
exists.
• ‘Select All/Unselect All’ option available in the
‘Edit’ menu on right clicking on the text, for
selecting/deselecting entire text.
• ‘Copy’/’Cut’ and ‘Paste operations possible for
text within the GUI window.
Figure 4. The Assamese spell checker.
The Soundex encoding scheme for Assamese has
been refined into a more fine-grained one, and now
comprises of 21 numerical codes. Added
functionality has been incorporated into the
Soundex code generator for handling matras attached
to consonants and conjuncts in the first letter
position, as also for khandata/chandrabindu/
anuswar/bisarga attached to consonants, vowels and
conjuncts.
The Edit-distance module now accounts for 2234
distinct letter combinations (matras attached to
consonants/conjuncts, khandata/chandrabindu/
anuswar/bisarga attached to consonants/ vowels /
conjuncts.)
Tests conducted against a corpus of about 67,000
words reveal that the Edit-distance method gives the
best results, followed closely by the Soundex
method. A non-word detection module for
Manipuri has also been developed. Work has also
been undertaken to integrate a spell checking facility
for Assamese in the Microsoft Word environment.

88
2.4 Assamese language support for Microsoft
Word : Word processors like Leap Office provide
the facility of document typing in Indian
languages. Content creation can be done with the
help of these editors by using the inscript keyboard
layout (provided by editors). The Centre has
developed a macro that allows the user to type
Assamese text in Microsoft Word without the need
of available Indian Language Editors.
Technology Description
This Microsoft Word macro maps the inputs to
appropriate glyphs. The macro supports the
Inscript keyboard layout. Typing can be done
using the font “Asamiya” developed by the Centre.
The use of the Inscript keyboard layout facilitates
smooth migration from C-DAC to Microsoft
technologies.
Features
• Inscript keyboard layout
• Template macro
• Supports “Asamiya” font and C-DAC Assamese
fonts (As-ttDurga, As-ttBidisha, As-ttDevashish,
As-ttkali, As-ttAbhijit).
• All the features of MSWord (such as
justification, different font styles and different
font sizes etc.) can be used.
• Documents are stored in glyph form.
Figure 5. Snapshot of Assamese macro with
different font styles.
2.5 Morphological Analyzers : Given any word
or a group of words, a morphological analyzer
determines the root and all other inflectional
forms. Morphological Processing plays a vital role
in the development of Spellcheckers and Machine
Translation Systems. The Centre has developed
morphological analyzers for both Assamese and
Manipuri.
a) Assamese Morphological Analyzer: The
analyzer has been developed for use with the Spell
checker and the Machine translation systems.
Technology Description Stemming technique
forms the base of the Assamese morphological
analyzer. In the above technique, affixes are
added/deleted according to the linguistic rules.
The derived words are verified with the existing
Corpus/Dictionary to treat as valid words.
Features
a) Currently works with Eleven linguistic rules
b) More rules can be added without alteration
in code.
c) Modules are available in the form of API’s
for customization.
b) Manipuri Morphological Analyzer : A design
for developing a Morphological Analyzer for
Manipuri is being investigated. Development
of a root dictionary, a morpheme dictionary
and an affix table has commenced.
Identification of linguistic rules for nouns and
pronouns has also commenced. Further
linguistic rules for other grammatical
categories are being studied.
Technology Description
The Manipuri Morphological Analyzer will be
realized using the same techniques used in the
development of Assamese Morphological
Analyzer.
Features
• Five rules are being used currently
89
• Can be easily upgraded for more complex
patterns
• The modules, which have been developed so
far are available in the form of API’s and can
be used according to the need by other
applications like, spell checker, etc.
• A Graphical User Interface has been developed
(see Figure 6).
Figure 6. Graphical User Interface of Manipuri
Morphological Analyzer
3. Translation Support Systems
3.1 Machine Translation System The term
Machine Translation (MT) refers to the process
of performing or aiding translation tasks
involving more than one human language. It is a
system that translates natural language from one
source language (SL) to a target language (TL).
Technology Description
This MT system developed is basically a rule based
one and relies on a bilingual dictionary. It can
currently handle translation of simple sentences
from English to Assamese. The dictionar y
contains around 5000 root words. The system
simply translates source language texts to the
corresponding target language texts word-for-
word by means of the bilingual dictionary lookup.
The resulting target language words are re-
organized according to the target language
sentence format. In order to improve the output
90
quality, the system performs morphological
analysis before proceeding to the bilingual
dictionary lookup.
Currently it can handle general-purpose simple
translation. It is being upgraded to handle
complex sentences. Efficiency of the system can
be improved by selecting a specific domain.
Currently the rule based Machine Translation
system contains 22 rules. The system has been
tested for more than 250 frequently used simple
sentences. An example of Machine translation
from English to Assamese is depicted in Figure 7.
Figure 7. English-Assamese Machine Translation
System
4. Human Machine Interface Systems
4.1. Optical Character Recognition for Assamese
and Manipuri: Optical Character Recognition
(OCR) is the process of converting scanned images
containing texts into a computer processable
format (such as ASCII, ISCII, and UNICODE
etc.). An MoU to transfer the OCR technology
from the Resource Centre at the Indian Statistical
Institute, Kolkata was signed in August 2002 and
accordingly the same was effected in September
2002.
Technology Description
The system takes a gray level (8bit) TIFF (Tagged
Image file format) image as input. The images
scanned from either Assamese (or Manipuri)
books or from paper documents can be processed
by the OCR software. The current version of the
OCR system produces output in the ISCII format,
which can be viewed or edited using any editor
supporting ISCII.
Features: Table 3 shows the salient features of the
OCR system while Figure 8 shows the associated
GUI.
Features Specifications of Assamese
& Manipuri OCR
Scanning resolution 300-600 dpi
Input image TIFF (8 bit gray level)
Skew detection +5 to -5 degrees
Skew correction +5 to -5 degrees
Font name Assamese: Gitanjalilight,
Luit & AS-TTDurga
Manipuri: Font used by the
publisher, Manipuri Sahitya
Parishad, Imphal
Font size Assamese: 12-28 points
Manipuri: 12-18 points
Test data size Assamese: 600 pages from
books & printed documents
Manipuri: 100 pages from
books.
Template size Assamese: 2600
Manipuri: 1800
Post processing Morphological analysis
Output file format ISCII format
Accuracy Assamese: 95% (without
post processing)
Manipuri: 90% (without
post processing)
Portability/ Windows 98/2000/XP &
Expandability LINUX
Table 3. Specifications of the Assamese/Manipuri
OCR System
Figure 8. GUI of the OCR System
4.2 Speech Recognition System
Automatic Speech Recognition (ASR) for
Assamese, which is concerned with the problem
of recognition of human speech by a machine, is
the core of a natural man-machine interface. A
speaker-dependent continuous speech recognition
system with a list of vocabulary has been
developed for the Assamese language. The
objectives of the system are:
• Assamese and English Spoken Digit
Recognition
• Study of Noise Effects on Assamese
Recognition
• Phonetic Alignment of Assamese Digits
A set of acoustic rules has been formulated from
the acoustic-phonetic features of Assamese and
English that will be able to classify a given sound.
The rules based on these features show an overall
success rate of around 76% for randomly collected
test utterances in Assamese. The classification rate
is least among the sounds falling under the nasal
class (65% success rate). The sounds falling under
the vowel, stops and the fricatives show better
results (80%, 85% and 85% success rate
respectively) while the classification results for
diphthongs (70% success rate) were not so good.
91
The unique Assamese sound of /x/ is found to be
very similar to /s/ acoustically. A comparative
study of acoustic properties of Indian spoken
English and Assamese vowels has been performed
4.3 Interface for e-Dictionary : An interface for
the English-Assamese and Assamese-English
dictionaries has been developed that allows users
to choose between one of the two languages viz.
English and Assamese, enter a word and find its
equivalent in the other language. Based on Client-
Server Architecture, the system allows users to
access the Dictionary Server using a Java applet
opened via a browser.
Technology Description: The basic components
that make up the on-line dictionary are:
4.
1. Client: The client is provided with a
Graphical User Interface (GUI) in the form
of a Java applet shown in Figure 9 and
initiated from the web browser. The user can
invoke the URL of the dictionary and choose
the source language; type the word in the
search window provided in the applet and
click on the Search button. The query from
the user is sent to the server and the results
are displayed at the client-end.
the Dictionary Server has been developed as
a menu-based application that assists the
administrator to maintain the dictionaries,
to stop and restart the server and monitor
incoming requests. The server provides
information on the requests and the actions
taken during run time. The current version
runs on a Windows platform but the can be
ported to a Linux system with minor changes,
making it virtually platform independent.
Whenever a connection to the server is
established, it checks the incoming request
from the client, consults the database,
searches for information on that particular
word after which it acknowledges the request
by serving the information.
Database: The e-dictionaries that comprise
the database were initially developed using
MS Access. These are then converted to
Prolog facts for faster access by the search
engine. The server provides the administrator
an option to open an ODBC (Open
DataBase Connectivity) connection to the
MS-Access database and perform the
conversion thereby uploading the latest
versions of the dictionaries. The current
version of the system supports two (English-
Assamese and vice versa) of the four electronic
dictionaries (viz. English-Assamese, Assamese-
English, English-Manipuri and Manipuri-
English) being created.

Features

a) User Friendly GUI available.

b) Dictionaries can be used to aid machine

Figure 9 Graphical User Interface of the
dictionary
2. Web Server: Apart from the main html page,
this server hosts the Java jar files and serves
the applet classes to the client.
3. Dictionary Server: Coded in Visual Prolog,
translation and/or spell checking
applications.
c) Pronunciations of words are available in the
form of wave files.
d) APIs provided for prospective programmers
to allow them to mould the dictionary
information to suit their custom applications.

92
e) A stand-alone and a web enabled version of
e-dictionaries available.
5. Language Technology Human Resource
Development
5.1 Workshops Conducted
Two workshops were conducted to disseminate
and make people aware of how to use Indian
Languages in IT.
(i) A one-day workshop on “Natural language
Processing” was conducted at the Indian
Institute of Technology Guwahati on the 31st
of March, 2001. The workshop was intended
to act as a forum for promoting interaction
among interested graduate students and
researchers working on natural language
processing and allied areas. The topics that
were addressed in the workshop were :
" Introduction to Computing
" Natural Language Processing
" Linguistics
" Demonstration of language technology
solutions
(ii) A Training Programme on “Web page Design
and Office Automation Using Assamese” was
held at the Indian Institute of Technology
Guwahati on the 15th & 16th of March, 2002.
The Training Programme was designed for
the benefit of Web Designers and people who
work with Assamese Word Processors. With
State Government officials using Assamese as
a medium of communication, the need for
Office Automation in this language is picking
up fast. Likewise, the demand for putting up
information in the native language on the
web is also increasing. This training
programme was aimed at disseminating
techniques to generate e-content in Assamese
and use this language for routine office tasks.
The training programme was specifically
designed to encourage state government
employees to use Assamese for office
automation. Participants of the programme
ranged from employees of Central, State and
Public sector undertakings to Private
entrepreneurs. The course contents of the
workshop were :
" Office Automation
" Web development in Assamese.
" Creation and manipulation of Database
and Spreadsheets in Assamese.
" Hands-On session.
6. Standardization
A draft of the Assamese Unicode Code-Set was
prepared after due consultation with linguists and
the State Govt. and submitted to the Ministry of
Communications & Information technology,
Govt. of India for further evaluation. The
Assamese Code –Set is similar to that of the
Bangali, exceptions being.
• The letter Ra » has Code 09F0 instead of 09B1
and the letter Wa ¾ has code 09F1 in lieu of
09B5.
• An additional letter Khya lù with code 09BB is
introduced.
7. Publications
1. Monisha Das, S.Borgohain, Juli Gogoi,
S.B.Nair, Design and Implementation of a
Spell Checker for Assamese, Proceedings of
the Language Engineering Conference,
LEC2002, December 2002, Hyderabad,
Published by the IEEE Press.
2. Monisha Das, S.Borgohain, S.B.Nair, Spell
checking in MSWord for Assamese,
Proceedings of the ITPC-2003:Information
93
 Technology: Prospects and Challenges in the Nilima Deka nilima_deka@iitg.ernet.in
21st Century Kathmandu, Nepal, May 23-
26, 2003. Juli Gogoi juli@iitg.ernet.in

8. Manika Newsletter Dr. L.Sarbajit Singh sarbajit@iitg.ernet.in

Snapshot of the Manika: e-newsletter in Manipuri Sirajul Chawdhury si_chow@iitg.ernet.in

Courtesy : Prof.Gautam Barua

Indian Institute of Technology
Department of Computer Science & Engineering
Panbazar, North Guwahati,
Guwahati -781 031, Assam
(RCILTS for Assamese & Manipuri)
Tel: 00-91-361-2690401, 2690325-28
Extn 2001, 2452088
E-mail : gb@iitg.ernet.in, g_barua@yahoo.com)

Description

Manika, an e-newsletter in Manipuri, was

launched on the Republic day this year (26th
January 2003) to mark the beginning of a new
electronic information era in Manipur - the Land
of Jewels. The newsletter hosted by the RCILTS
at Indian Institute of Technology Guwahati will
bring news and share knowledge of the heritage
and local innovations in Manipur and help enable
the Manipuris to be well informed of the
electronic age and forge ahead.

9. The Team Members

Prof. Gautam Barua gb@iitg.ernet.in

Dr. S.B.Nair sbnair@iitg.ernet.in

Dr. S.V.Rao svrao@iitg.ernet.in

Dr. P.K.Das pkdas@iitg.ernet.in

Dr. Dipankar Moral dipankarmoral@postmark.net

Samir Kr. Borgohain samir@iitg.ernet.in

Sushil Kr. Deka sdeka@iitg.ernet.in

Monisha Das moni@iitg.ernet.in

94
 Resource Centre For
Indian Language Technology Solutions – Kannada
Indian Institute of Science, Bangalore
Achievements

Indian Institute of Science

Centre for Electronics Design & Technology, Bangalore - 560012.
Tel. : 00-91-80-2932377, 293 3267
E-mail: njrao@mgmt.iisc.ernet.in
 RCILTS-Kannada would enthuse both faculty and students at
Indian Institute of Science, Bangalore engineering colleges to do their mini projects and
main projects in language technology related
Introduction activities.
The Resource Centre for Indian Language This phase of Resource Centre created adequate
Technology Solutions-Kannada at Indian Institute momentum in creating IT tools for Kannada and
of Science chose a very broad spectrum of made a beginning that would ensure that
activities, and undertook several activities related Kannadigas need not feel alien in their own state.
languages other than Kannada. Participation by
a large number of faculty of the Institute enabled 1. Web Sites And Support To Instruction
the Resource Centre to create knowledge bases,
web sites, OCR for Tamil and Kannada, speech 1.1 Kannudi
synthesis for Tamil and Kannada, resource
Kannudi is bilingual web site for the benefit of
information on European history in Hindi for
Kannadigas to learn about their language and
high school children, a bilingual interactive
heritage. Non-Kannadigas will also have an
German-Hindi course, CDs in support of learning
opportunity to know about Karnataka. It is
Kannada at primary and high school level, a
mainly aimed at non-specialists, and provides a
variety of tools for language processing, word net
first level overview of a large number of topics.
in Kannada, and investigate into language
identification. It was also recognized that language
technology segment of the IT industry remained
very small in India for a variety of reasons. Two
important approaches were taken towards
improving this. One is to get some jobs done by
the industry instead of getting them done through
project assistants, and the other one is to make
available many of the basic tools developed in the
public domain. The web site KANNUDI should
meet the long felt needs of Kannadigas to know
about their language and state. A large number
of scholars in various disciplines actively
contributed in making this web site very rich. An
organization like Kannada Sahitya Parishat felt
enthused enough to permit this resource centre
to create a web site on their activities and have it The contents of this site are organized under 11
co-located with KANNUDI. In an attempt to topics: Language, Epigraphs, Literature, Folklore,
make a difference to an important segment of Geography, History, Arts, Classics, Personalities,
population, namely primary and high school Temples and Festivals, and Cultural Societies and
students, Bodhana Bharati series were created, their Activities. Some of the important features
which provide interactive learning of Kannada of Kannudi are:
language as per the state syllabus. Even the process
• The site has more than 2000 pages of content.
of creation sensitized a large number of teachers
to the usefulness of IT tools in education, and • Most of the articles are written by experts in
enabled them to actively participate in creating the respective areas.
the CDs. It was realized that manpower
availability in the area of language technology is • Images have been incorporated at all possible
very poor. The web site LT-IISC is created that places.

96
• Brief life sketches of a large number of
personalities of Karnataka are prepared.
• Several classics of Kannada literature are made
available on the web site.
The web site also hosts the web site of Kannada
Sahitya Parishad.
The contents of the site are continuously
enhanced. The web site address is
www.kannudi.org.
1.2 LT-IISC
This is a web site to meet the needs of students,
teachers, and developers interested and concerned
with language technologies in general and in
particular with emphasis on Kannada language.
The main sections of the web site are Standards,
language resources, office automation, web
technologies, optical character recognition, speech
technologies, machine translation, multi-lingual
issues, applications, and open source software. The
website will have a download section common to
all topics which would also have all the products
generated at IISc. The website also features a
Discussion Forum on each topic with which
interested people can exchange their views
regarding technologies, solve problems (if any),
etc. Information on events, seminars, workshops
in the area of language technologies is also
provided. Annotated links to other websites on
language technologies, language products, and
standards is also given.
The address of the site is www.ltiisc.org.
Publications:
1. Narmadamba, K.: e-learning in Kannada
(Kannadadhalli e Kalike) (Book), 2002,
Kannada Sahitya Parishat.
2. Narmadamba, K.: Kannada and computers in
Samyukta Karnataka, April, 2003.
3. Narmadamba, K.: Speech variation in South
India languages with respect to dialect, emotion
and style, Sadhane, Bangalore University, May,
2003.
1.3 Bodhana Bharathi : Multimedia Educational
CDs for 7th, 8th and 10th Standards
It has been observed that the quality of teaching
Kannada language at primary andhigh school
levels needs considerable improvement. It was,
therefore, decided to create interactive multimedia
learning material, useful to both teachers and
students, in the form of CDs. This CD series are
named as “Bodhana Bharathi”. These CDs are
meant to supplement the text books and classroom
teaching. The framework within which these CDs
are developed is,
1. To improve listening, and writing skills.
2. To enable the students perform better in the
examinations.
97
3. To appreciate the background in which the
lessons were created by the original authors.
4. To make them understand and feel proud about
their nationality and culture, literature, people
and language.
5. To enjoy poems and songs.
Some of the specific aspects of the instructional
material include,
1. Introducing the student to the writers of the
lessons and their works.
2. To make them understand the important points
and the moral of the lessons.
3. Exposing them to new words that they come
across in the lessons.
4. To help them recite poems appropriately.
5. Testing their knowledge in the lessons by giving
them unit tests.
98
The material is divided into three sections
Teacher’s section, Student’s section and a
common section. The teachers section has
information such as lesson plan, objectives and
others. The students section contains extra
questions, new words, etc. The common section
comprises of a preamble, summary of the lesson,
values in the lesson, key points of the lesson
supported by visuals, model question paper with
partial interaction and answers, and meaning of
words with relations,.
1.4 Bilingual Instructional Aid for learning
German through Hindi
Both Hindi and German belong to the Indo-
European family of languages. There are a lot of
similarities between the two in terms of grammar,
vocabulary etc. These two are semantically quite
close to each other. As a consequence it is easier
to explain an unknown German word, phrase or
idiom in Hindi and vice versa than using English.
Moreover with the growing number of joint
ventures and offshore projects there has been
increasing interaction between Germans and
Indians. If the visitors from Germany need to
stay in India for a longer period, then some
knowledge of Hindi would be of great advantage
in making professional, social and cultural
contacts. Similar would be the case of Indian
visitors to Germany. This was the motivation for
developing a bilingual instructional aid for Hindi-
German in the form of a web-based material.
This web site has four sections
Learn Hindi section teaches Hindi alphabets,
vowels, consonants, counting and methods of
introducing oneself.
Traveling to India section gives information on
how to speak at Airport, Restaurant, Shopping
complex, Bank, Post Office, Hospital, Railway
Station, and for Renting a House.
Life in India section has information on culture,
lifestyles, tradition, food etc.
Exercises section provides practicing exercises.
This information can be accessed at
www.mgmt.iisc.ernet.in/~fls/projects and
www.mgmt.iisc.ernet.in/~fls/german
1.5 Information Base in Hindi Pertaining to
German History
The Information Base primarily concentrates on
bringing information about Germany in
perspective, which appeals to Indian school
children. The aim of this project is also to put all
this material into Hindi so as to make the
information accessible to as many school children
as possible. NCERT has now undertaken to
provide all school children access to information
materials both in Hindi and English. This project
made use of the knowledge of the foreign
language experts at the Institute, and evolved
materials in Hindi pertaining to German history.
This was done in conjunction with the prescribed
syllabus.
The Web-site contains information on German
Anthem, Germany in Europe, German Weather,
German Population, World War II, Construction
and Fall of Berlin wall, German Automobiles,
German Beer and Wine, German Culture,
German Economy, German Unification, German
Literature, Political Parties, Social security system,
European Union, Indo-European Languages, and
German Cities-Berlin, Brandenburg, Cologne,
Hamburg, Frankfurt, Muenich, Trier.
2. Knolwedge Bases
2.1 Sudarshana : A Web Knowledge Base on
Darshana Shastras
The main objective was to create a “knowledge
base” containing basic texts, commentaries and
free translations of texts on the Systems of Indian
Philosophy, popularly known as Darshanas, in
multimedia form, and have them web-enabled,
and make them available in the form of CDs.
The target audience is classified into Naïveté,
Students, and Scholars.
All the resource material and the experts on all
Darshanas are identified. All the web sites on
Sanskrit and Darshana-s in particular are surveyed
comprehensively. A white paper on Shad-
darshana-s is created. A series of lectures by one
of the experts on Ishvara in all darshana-s and
“Arthasamgraha” is recorded. The audio files and
transcripted version of these lectures is made
available on the site. Partial translation in English
is available. Nyayakusumanjali and other Sanskrit
99
texts referred for the above discourses are also
made available on-line. Glossary of technical
terms in Nyaya, Vaisheshika and Mimamsa
darshana-s in English and Sanskrit is created. A
relevant search engine for both Sanskrit and
English is also made available

2.3 Indian Aesthetics

A website on Indian Aesthetics in Product Design

has been created with following features:

• Identifies the body of knowledge in Indian

Aesthetics from Literature, Culture and
For any clarifications Dr. N.R.Srinvasa Raghavan
Philosophy.
can be contacted at raghavan@mgmt.iisc.ernet.in
• A Visual Database of Indian Products, Visuals
2.2 Indian Logic Systems
and Examples.
The website“PRAMITI : PRAMana – IT with
• Methods for Designers to enable incorporation
Indian logic” is being developed to provide
of “Indianness” into product design.
comprehensive information on various aspects of
Indian Logic and its applicability to Computer • Learning materials for students in design
Science and Information Technology. The
programs regarding Indian Aesthetics, and
website is designed to be user-friendly by using a
methodology of incorporation into product
concise and objective style. The web address is
designs.
www.pramiti.org.

100
The contents are broadly classified as
“philosophical”, “cultural”, and “pragmatic”
The philosophical section contains extracts of
ancient texts, book reviews, review and research
papers. It has a rich visual database of images from
crafts, artifacts and general visuals. It has a
comprehensive glossary and rich bibliography.
The cultural section has articles on Indian
philosophy and Western Philosophy. Case studies
containing research in the area of product and
visual semantics are also included.
The pragmatic section discusses the learning
material for design students
3. Technologies and Language Resources
3.1. BRAHMI: Kannada Indic Input method,
Word Processor
Some scripts have hundreds of individual
characters, and it is not easy to put all these
characters on a standard keyboard that is designed
for simpler scripts. Input Methods are developed
to provide a better approach for inputting text
for these languages. The Brahmi Kannada Input
Method (BKIM) is one such method which allows
user to give input in Kannada. The BKIM is
developed using Java which provides an input
method framework that enables the user to enter
text directly into the text component. Application
developers who wish to have direct Kannada input
in their applications can make use of the BKIM.
As it is developed in Java, it is also platform
independent. The BKIM uses the KGP keyboard
layout (standardized by Govt. of Karnataka) and
each keystroke is mapped for their corresponding
Unicode characters. It uses Open type fonts.
The Brahmi - Multilingual Word processor is a
demonstration of Brahmi Kannada Input
Method. The word processor has the option of
choosing the input method from among nine
Indian Languages (at present only Kannada is
101
enabled) and English. Apart from normal features tested and are distributed along with Brahmi –
like file open, print, cut, copy, paste, font features, Multilingual Word processor and also individually.
paragraph settings, color changes it also has some People are free to use these fonts in their
unique features such as saving file in different applications and/or for documentation purposes.
encodings (UTF-8, UTF-16, Unicode Big
Endian, etc.), search and replace Unicode text and
email facility. It has an easy access graphical
toolbar and comprehensive help.

3.2 OPENTYPE FONTS: Sampige, Mallige,

Kedage

OpenType Font is a new cross platform font file

format developed jointly by Adobe Systems
Incorporated and Microsoft. Based on Unicode
standard the OpenType format is an extension of
the TrueType SFNT format that can now support
PostScript font data and new Typographic
features.

Open Type offers several compelling advantages:

• A single, cross-platform font file that can be

used on both Macintosh and Windows
platforms

• An expanded character set based on the

international Unicode encoding standard for
rich linguistic support

• Advanced typographic capabilities related to

glyph positioning and glyph substitution that
allow for the inclusion of numerous alternate
glyphs — such as old-style figures, small capitals
and swashes — in one font file
• A compact font outline data structure for
smaller font file sizes
• OpenType fonts are best suited for meeting the
demand for complex script handling and high
quality typography in today’s global publishing
and communication environment.
3.3 Kannada Wordnet : WordNet is an on-line
lexical reference system whose design is inspired
by current psycholinguistic theories of human
lexical memory. The nouns, verbs, and adjectives
of a language are organized into synonym sets,
each representing one underlying lexical concept.
Different relations link the synonym sets.
Synchronic organization of lexical knowledge and
structured organization of words is helpful for
natural language processing.

Although the design of Kannada WordNet has

Three OpenType fonts called “Sampige”,
been inspired by the famous English WordNet,
“Kedage” and “Mallige” are developed. Sampige
the unique features of Kannada WordNet are
and Kedage are text based fonts whereas Mallige
‘Graded antonyms and meronymy relationships’
is a handwritten type of font. The fonts have been
and ‘efficient underlying database design’.

102
Nominal as well as verbal compounding, complex
verb constructions also play a vital role.
There are different organizing principles for
different syntactic categories. Two kinds of
relations are recognized: lexical and semantic in
WordNet. Lexical relations hold between word
forms; semantic relations hold between word
meanings. The basic categories such as nouns,
verbs, adjectives and adverbs are organized into
synonym sets, each representing one underlying
lexical concept. Synsets or the synonymy sets are
the basic building blocks. The synonym set serves
as an identifying definition of lexical concepts.
The semantic relations discussed in the Kannada
WordNet are Synonymy, Antonymy, Hypernymy
–hyponymy, Meronymy – holonymy, Entailment,
troponymy. This WordNet will initially be built
for 2000 words.
3.4 OCR for Tamil
A, Optical Character Recognition (OCR) system
capable of converting multi-lingual manuscripts
to machine readable codes is one of the key steps
in working towards the goal of machine
translation. There are numerous applications that
OCR systems have to offer which are of help in
day-to-day activities of life. These include mass
conversion of existing documents and literature
into electronic format, reading aid for the blind
as a part of text-to-speech converter, automatic
sorting of mails in postal department and
processing of bank documents, machine
transliteration/translation of documents and
literature of other scripts, etc.
A OCR for Tamil script is developed that works
in a multi-font and multi-size scenario. The input
to the system is a scanned or digitized document
and the output is in TAB code. The documents
are expected to contain text only. The process
sequence is given in the following block diagram.
Preprocessing is the first step in OCR, which
involves binarisation, skew detection and
correction. Binarisation is the process of
converting the input gray scale image scanned
with a resolution of 300 dpi into a binary image
with foreground as white and back ground as
black. Suitable techniques have been used to take
care of contrast in the images. The skew angle of
the document is estimated using a combination
of Hough transform and Principal Component
Analysis. Segmentation is done by first detecting
lines, and then detecting the words in each line
followed by detection of the individual characters
in each word. Horizontal and vertical projection
profiles are employed for line and word detection,
respectively. Connected component analysis is
performed to extract the individual characters.
The segmented characters are normalized to a
103
predefined size and thinned before recognition
phase.
The segmented symbols are fed to the classifier
for recognition. Tamil alphabet set contains 154
different symbols. The characters are divided into
some clusters based on domain knowledge, to
reduce the recognition time and a smaller
probability of confusion. This is accomplished by
designing a three level, tree structured classifier
to classify Tamil script symbols.
A line in any Tamil text has three different
segments; upper, middle and lower. Depending
upon the occupancy of these segments, these
symbols are divided into one of the four different
classes. The number of dots present in each
segment contributes a lot for the classification,
which is dependent on the scanning resolution.
This constitutes the first level clustering
The second level of classification based on Matras/
extensions is applied only to symbols which have
upward matras and downward extensions. The
classes are further divided into groups, depending
on the type of ascenders and descenders present
in the character. This classification is feature based
i.e. the feature vectors of the test symbol are
compared with the feature vectors of the
normalized training set. The features used in this
level are second order geometric moments and
the classifier employed is the nearest neighbor
classifier.
Feature-based recognition is performed at the
third level. For each of the groups, the symbol
normalization scheme is different. The dimensions
of the feature vector are different for different
groups, as their normalization sizes are different.
Truncated Discrete Cosine Transform (DCT)
coefficients are used as features at this level of
classification. Nearest neighbor classifier is used
for the classification of the symbols.
The system has been tested on files taken from
Tamil magazines and novels which are scanned at
104
300 dpi (dots per inch). It has been ensured that
the test files contain almost all the symbols present
in the script. An accuracy of over 99% has been
achieved on the training set and 98% on other
samples. The sample size was 100.
[1] K G Aparna and A G Ramakrishnan, “A
complete Tamil Optical Character Recognition
System”, Document Analysis Systems V, 5 th
International Workshop, DAS 2002, Princeton,
NJ, USA, August 19-21, 2002, pp. 53-57.
[2] K G Aparna and A G Ramakrishnan, “Tamil
Gnani – an OCR on Windows, Proc. Tamil
Internet 2001, Kuala Lumper, August 26-28,
2001, pp. 60-63.
3.5 OCR of Printed Text Documents in Kannada
The input to the system is the image of the printed
page obtained by scanning on a flat bed scanner
at 300 DPI resolution and converting the image
into binary by making use of a global threshold
selected automatically for each page. This image
is processed to remove any skew (so that the text
lines are aligned horizontally) using a Hough
transform based technique. Next, the individual
lines in the image and the words in each line are
separated using Projection Profile based methods.
Due to the special characteristics of Kannada
script, separating the individual characters in the
word is not an attractive choice. Hence a novel
segmentation algorithm is developed which
segments words into sub-character level so that
each akshara may be composed of many segments.
A pattern classification method is used which is
based on the Support Vector Machines to assign
a classification label to each segment so obtained.
After labeling individual segments, rules are used
on how aksharas are composed to finally effect
recognition of individual aksharas. The final
output of the system is an ASCII file compatible
with kantex typesetting package for Kannada
which is built around the standard Latex system.

The words are first vertically segmented into three

zones as shown in the figure.
The next task is to segment the three zones
horizontally. The middle zone is the most critical
since it contains a major portion of the letter.

After segmenting the document each segment has

to be recognized to effect final recognition.

Features (set of numbers that capture the salient

characteristics of the segment image) from each
segment are extracted. The characters in Kannada
have a rounded appearance. Therefore features
which can extract the distribution of the ON
pixels in the radial and the angular directions will
be effective in capturing the shapes of the
characters.
The features extracted have to be classified using
a classifier. Here a Support Vector Machine
(SVM) classifier is used.
The output after classification is transformed into
a format which can be loaded into a Kannada
editing package. The input is usually ASCII text
in which aksharas are encoded ASCII strings.
The system was tested on pages scanned from
Kannada magazines, etc. Currently the system
recognizes about 85% of the aksharas correctly.
4. Research
4.1 Automatic Classification of Languages Using
Speech Signals
Automatic language identification is the problem
of identifying the language being spoken from a
sample of speech from an unknown speaker.
Among various approaches for language
identification, the phone recognition offers
considerable promise, as it incorporates sufficient
knowledge of the phonology of the language to
be identified, without incurring the significantly
higher cost of word based approaches. An
approach based on sub-words that does not
require manually labeled data in any of the
languages recognized is used. Particularly, the

105
focus was on the specific architecture termed
Parallel Phone Recognition (PPR), and the system
is referred to as parallel Sub-word Recognition
System.
Research in automatic language identification
requires a large corpus of multi-lingual speech data
to capture many sources of variability within and
across the languages. Among various Indian
languages six languages Hindi, Kannada,
Malayalam, Marathi, Tamil and Telugu were
selected. English as spoken by the same Indian
speakers is the seventh language. For each
language, twenty adult speakers of different age
and gender are selected. Care was taken to ensure
that a speaker was chosen for a particular language, This SWRLM approach uses a single front-end
only if he/she had that language as a native sub word recognizer followed by N back-end LMs
language particularly during childhood. Speech for an N language LID task. The front-end SWR
was collected using a Sennheiser HMD224 noise- can be language dependent or language
canceling microphone and low pass filtered at 7.6 independent. The SWRLM performance will
kHz. The recording protocol was designed to improve if we use language independent single-
obtain Digits and days of the week, Numbers in
word unit inventory which is obtained from all
English, English alphabets, commonly used words
languages in LID task.
in native languages, railway reservation words in
native language and in English, Banking words Results: the table III shows the LID performance
in native and English language. In the elicited of SWRLM on OGI-TS database for each of the
free speech, personal details with name, age, native
front ends for both the cases of MLC. Table IV
language, profession, family, passage reading in
shows the performance of SWRLM on ILDB for
Hindi, English and native languages and etc.
all six front ends and both cases of MLC.
Here three approaches are studied, namely,
Parallel Sub-Word Recognition.(PSWR), From the above tables we note that LID
SWRLM, and Parallel – SWRLM. The following performance on ILDB is much better than on
tables show the performance of PWSR system OGI-TS database for training data. Whereas for
on OGI-TS and ILDB data base with different test data, OGI-TS database gives better results as
scores. compared to ILDB database.

As we know that the sounds in the language to be

identified do not always occur in the one language
used to train the front-end sub-word recognizer.
Thus it seems natural to look for a way to
incorporate phones from more than one language
into a SWRL like systems. Alternatively another
approach is simply to run multiple SWRLM
systems in parallel with the single language SWR
each trained in different languages. Therefore P-
SWRLM uses multiple front-end SWRs with each

106
SWR followed by N number of backend LMs for
a N language LID task.

The database and preprocessing with

parameterization is identical to that in PSWR
approach. The bias removal was done by Zissman
method on Pl. Results of LID accuracy for six
languages Hindi, Kannada, Malayalam, Marathi,
Tamil, Telugu PSWRLM system, with two types
of classifiers Maximum Likelihood Classifier and
Gaussian Classifier are given in the following plots
that compare the performance of LID accuracy

on PSWRLM system for two classifiers i.e., MLC

and GC-MD, and also across OGI databases and
for ILDB database.
The following conclusions may drawn:
• The sub-word approach to LID, developed in
our lab holds good promise for LID among a
small set of languages.
• The LID performance is quite good among
south Indian languages, although they share a
lot of phonetic structure and even vocabulary.
• PSWR approach is more promising than the
SWRLM and PSWRLM approaches to LID.
4.2 Algorithms for Kannada Speech Synthesis
The basic units, namely, CV, VC, VCV and
VCCV have identified and recorded. A
framework has been standardized for creation and
handling of database of spoken basic units. A
synthesis scheme, based on waveform
concatenation of basic units has been attempted.

107
New techniques for pitch detection and
modification, and speech synthesis with emotion
are proposed.
5. Publications
1. R. Muralishankar, K. Suresh and A. G.
Ramakrishnan, “DCT based approaches to
Pitch Estimation”, submitted to Signal
Processing.
2. K. Suresh and A. G. Ramakrishnan, “A DCT
based approach to Estimation of Pitch”, Proc.
Intern. Conf. on Multimedia Processing and
Systems, Chennai, Aug. 13-15, 2000, pp. 54-
57.
3. R. Murali Shankar and A. G. Ramakrishnan,
“Robust Pitch detection using DCT based
Spectral Autocorrelation”, Proc. Intern.
Conf. on Multimedia Processing and Systems,
Chennai, Aug. 13-15, 2000, pp. 129-132.
4. R. Murali Shankar and A. G. Ramakrishnan,
“Synthesis of Speech with Emotions”, Proc.
Intern. Conf. on Commn., Computers and
Devices, Vol. II, Kharagpur, Dec. 14-16,
2000, pp. 767-770.
5. Anoop Mohan, Harish T, A. G.
Ramakrishnan and R. Muralishankar, “Pitch
modification sans pitch marking”, submitted
to Intern. Conf. Acoustics, Speech and Signal
Processing, May 7-11, Salt Lake City, Utah,
USA, 2001.
Courtesy: Prof. N.J. Rao
Indian Institute of Science
Centre for Electronics Design and Technology
(CEDT)
Bangalore – 560 012
(RCILTS for Kannada)
Tel. : 00-91-80-3466022, 3942378, 3410764
E-Mail: njrao@mgmt.iisc.ernet.in,
chairman@mgmt.iisc.ernet.in

108
 Resource Centre For
Indian Language Technology Solutions – Telugu
University of Hyderabad
Achievements

University of Hyderabad
Department of CIS, Hyderabad-500046
Tel. : 00-91-40-23100500 Extn. : 4017 E-mail : knmcs@uohyd.ernet.in
Website : http://www.languagetechnology.ac.in
 RCILTS-Telugu
University of Hyderabad, Hyderabad
Introduction
University of Hyderabad is a premier institute of
higher education and research in India. The University
Grants Commission has selected University of
Hyderabad, among four others in the country, as a
“University with Potential for Excellence”. The
National Assessment and Accreditation Council
(NAAC) has awarded the highest rating of five stars.
University of Hyderabad is the only University in
India to be included among the top 50 institutions
in India under the “High Output - High Impact”
category by The National Information System for
Science and Technology (NISSAT) of the Department
of Scientific and Industrial Research. University of
Hyderabad has been rated the “number one”
University in India in sciences by the Department of
Scientific and Industrial Research.
1. Resource Centre for Indian Language Technology
Solutions
This Resource Centre for Indian Language
Technology Solutions was established by the
Ministry of Communications and Information
Technology, Government of India, at the University
of Hyderabad with a funding of nearly Rs.
100,00,000 spread over three years - April 2000
to March 2003. The project has since been extended
till 30th September 2003 to enable thorough
consolidation of all the work done. Two
departments, eight members of the faculty, 20
to 30 students and research staff at any given
point of time have put in their very best for the
past three years and several products, services and
knowledge bases have been developed. The core
competencies, the data, tools and other resources
developed here during this period will enable
this team to scale new heights in future.
2. Products
2.1 DRISHTI: Optical Character Recognition (OCR)
An Optical Character Recognition (OCR) system
converts a scanned image of a text document into
electronic text just as if the text matter was typed-in
by somebody. Scanned images are much larger in size
compared to corresponding text files. The statement
“A picture is worth one thousand words” is literally
110
true here. Texts occupy less storage space and less
network bandwidth when sent across a
network. Converting images into texts makes it
possible to edit and process the contents as normal
text.
DRISHTI: An OCR System for Telugu and other
Indian Languages
OCR systems can be used to convert available
printed documents into electronic texts without
typing. Since OCR engine can be run day and
night on several computers parallely, we can
generate large scale corpora with less time and
effort. OCR engines can also be used for a variety
of other applications. OCR systems have just started
appearing for Indian scripts. Most of the current
OCR systems for Indian languages are designed
only for printed texts and perform well only on
reasonably good quality documents. However,
research work on hand-written document
recognition is going on.
Drishti is a complete Optical Character Recognition
system for Telugu language. Currently, it handles
good quality documents scanned at 300 dpi with a
recognition accuracy of approximately 97\%. The
system is tested with a number of different fonts
provided by C-DAC and Modular Infotech, and on
several popular novels, laser and desktop printer
generated pages and books. Preprocessing modules that
separate textual and graphic blocks, handle multi-
column text inputs, and skew correction are also
implemented. Drishti is the first comprehensive
OCR system for Telugu.
A truthing tool with facilities for creating ground-
truth information, and to review the ground
 DRISHTI: Truthing Tool DRISHTI: How it works

truth against image data is also implemented. Such

truthing tools are extremely important in objective
and quick evaluation of OCR system performance. Preprocessing Stage
Benchmark standards were proposed in collaboration
Binarization : refers to the conversion of a scanned
with Indian Statistical Institute (ISI) Kolkata, to
256-gray level image into a two-tone or binary (pure
enable uniform and objective evaluation and
black and white) image. A binary image is
performance comparison of OCR systems and
appropriate for OCR work as the image document
subsystems. In addition, several other useful modules
contains only two useful classes of data — the
and library functions that enhance or simplify
background, usually paper, and the foreground, the
adding new features are developed. Initial work is
printed text. It is common to represent the background
also done on touching characters that led to
paper colour by white-coloured pixels and the text
identifying major characteristics and issues in
by black-coloured pixels. In image processing jargon,
addressing the problem. Ours is the only major work
the background pixels have a value of 1 and the
in this area apart from that by ISI, Kolkata for Bangla
foreground pixels have a value of 0. Binarization
characters [4]
has a significant impact because it provides input
Drishti, although designed for Telugu, has been tested to every stage of an OCR system. Drishti provides
with Kannada, Malayalam and Gujarati scripts, with three options — global (the default), percentile based
recognition accuracies over 90\%. Our OCR and iterative method — to achieve desired
technology was transferred to the resource centre for performance on different types of scanned
Gujarati. Work is underway in extending the system documents and scanners.
to Amharic script of Ethiopia.
Skew Detection and Correction : deal with improper
2.1.1 System Overview alignment of a document while it is scanned. The
normal effect is that the lines of text or no longer
Drishti contains three stages: preprocessing stage,
horizontal but at an angle, called the skew angle.
recognition stage and postprocessing stage.
Documents with skew cause line, word and character
Binarization, separation of image regions into textual
breaking routines to fail. Skew also causes reduction
and graphical regions, multi-column detection and
in recognition accuracy. In Drishti, skew detection
skew correction are the major tasks performed in
and correction are done by maximizing the variance
preprocessing phase. Separation of text into glyphs,
in horizontal projection profile.
characters, words and lines, and recognition of
individual glyphs are tasks of the recognition stage. Text and Graphics Separation: refers to the process
Postprocessing comprises combining the recognized of identifying which regions of the document image
glyphs into valid characters and syllables, and spell- contain text and which regions contain pictures and
checking. other non-text information that is not processed by

111
an OCR system. Drishti uses horizontal and vertical
projection profiles for such separation as well as for
many other preprocessing operations (see below). A
horizontal profile is obtained by counting and plotting
the number of text or black pixels in each row of the
image. A vertical profile is obtained by counting the
black pixels in each column of the image. Horizontal
profiles show distinct peaks that correspond to lines
of text and valleys that result from inter-line gaps.
A line of text is revealed by a peak in the
horizontal profile whose width is approximately
the font size. A graphic object, in contrast, is much
larger. The actual shape of the peak is also different
because of higher density of black pixels in a graphics
block. Thus, the profile shapes discriminate between
text and graphical blocks.
Multi-column Text Detection: is done using Recursive
X-Y Cuts technique proposed in [5]. It is based
on recursively splitting a document into rectangular
regions using vertical and horizontal projection
profiles alternately. A different method that allows
recognition of non-rectangular regions is also
implemented but not yet included in Drishti.
The use of horizontal and vertical projection profiles
for all the major preprocessing tasks minimizes
system complexity and allows faster processing of
documents. The preprocessing stages, except
binarization, are not enabled in the basic version of
Drishti, but are available as add-on options.
Recognition Stage
Line, Word, Character and Glyph Separation: is a
very important task as the recognition engine processes
only one glyph at a time. In Drishti, word and glyph
separation are the key steps. Word segmentation is
done using a combination of Run-Length Smearing
Algorithm (RLSA) [8] and Connected-Component
Labelling. Words are combined into lines using
simple heuristics based on their locations. The
performance of RLSA in accurately segmenting
words is very high on good quality text but drops in
the presence of complex layouts and tightly packed
text that is sometimes seen in magazines. However,
the difficulty in applying zoning techniques to
Telugu because of the complex orthography requires
further studies for improvement.
Words are decomposed into glyphs by running the
connected component labelling algorithm again. The
112
glyph separation is extremely accurate and very few
segmentation errors were found in our
experimentation.
Recognition: is based on template matching. A glyph
database containing all the glyphs in the script was
created from high-quality laser-printed text. Each
glyph is scaled to a size of 32 X 32 pixels that forms
a template for recognition and is stored in the
database. When a document is scanned for OCR,
the glyph obtained from the glyph separation step is
scaled to the same size as the template and
matched using fringe distance maps [3] against
each of the templates in the database. The
template with the best matching score is output as
the recognized glyph.
Drishti provides several options and parameters
that affect recognition. The default setup scales the
glyphs using a linear scaling algorithm and matching
is performed using a fringe distance map. Linear
scaling is fast but suffers from problems with complex
shaped glyphs at large font sizes and with small glyphs
at small font sizes. Non-linear normalization was
shown to improve performance[7] by selectively
scaling regions of low curvature. Non-linear
normalization, provided as a user option, gives better
performance on Hemalata and Harshapriya
fonts of C-DAC. Punctuation marks, which are
easily distorted because of their small sizes, are
handled separately without using template
matching. Recognition accuracy is very high for
punctuation marks using a location and stacking
based heuristic developed for Drishti[2].
There are also several ways to modify the basic
fringe distance measure to reflect the
idiosyncrasies of the Telugu script.
Experimentation was done on 18 distance measures
for matching, including 6 new measures, and
the best is chosen for recognition. Details on
the overall recognition process and modifications
for improving recognition accuracy may be found
in [1,2,6,7].
Output: is written into a file. Information about the
location of the glyph with respect to the text baseline,
the type of the glyph, i.e., whether it is a base, maatra,
vottu or punctuation glyph, and the recognized symbol
code are written into the file. Also, there is a facility
to output the k-best matches.
2.1.2 Postprocessing Stage
Assembling Glyphs into Syllables: is one of the most
challenging tasks of Drishti. The complex
orthography of Telugu permits glyph placement all
around the base character and finding syllable
boundaries is a non-trivial task. Currently, Drishti
uses the relative positions and types output from
recognition stage in conjunction with a stacking
heuristic to identify syllables. The heuristic works
correctly except in the case of certain large vottus with
an error rate of about 2%.
Converting Glyph Codes into ISCII: is currently done
by combining a simple table look-up method with
type codes output by the recognition stage.
Improvements are being done to the conversion code
which currently does not work for a limited
number of glyph/punctuation combinations and
consonant clusters. Consequently, the accuracy of
the ISCII output is approximately 5% - 7\% worse
than that of the raw OCR output as on date. The
improved conversion algorithm being developed is
expected to mitigate this problem.
Spell-Checker: is recently added for detecting mistakes
in the ISCII output. The current version recognizes
nearly 98% of the misspelled words (false positive
are 2%).
2.1.3 Salient Features
High Accuracy, Complete OCR System
Drishti is the first and currently the only complete
OCR system for Telugu. Currently, several
binarization algorithms (to select the best for given
set of documents and print quality), text-graphics
separation, multi-column layouts, skew detection
and correction are available as optional plug-ins.
The raw recognition accuracy (i.e., considering the
accuracy of the glyph codes) is currently 97%. The
accuracy of the ISCII output generated from the
Glyph Code-ISCII conversion process is currently
lower because of the errors in identifying syllable
boundaries and assigning ISCII byte codes. It is
already improved in internal testing within the
resource centre and the improved algorithms will be
included in Drishti very shortly.
The basic system was tested by STQC unit of
the Ministry of Communications and Information
Technology, Government of India, and it performed
with an accuracy of about 85% - 87% at the ISCII
level implying a raw accuracy of 93% - 95% without
using any of the preprocessing or postprocessing
routines. On our scanner, Drishti performed with an
accuracy of 96% - 97% on test documents provided
at C-DAC, Noida in September 2002. On our
tests on a number of documents after fine-tuning
the input images for scanner contrast variations
and other effects, Drishti consistently gives higher
accuracies. Currently, preprocessing stage is being
tuned to adapt for scanner differences.
A Unique Collection of Powerful Library Routines
Drishti is completely modular and implemented using
a number of highly useful C-callable library functions
for doing each of its tasks. The result is a main() routine
that is only about 100 lines in length. The complete
OCR system can be created by linking these 100 lines
of code with the powerful, pre-compiled library
routines. The design process permits changing the
functionality of the system by calling different library
function as and when needed.
Visual Truthing Tool
The visual truthing tool based on the proposed
benchmark standards allows easy generation of
ground truth data (including bounding boxes) from
scanned documents. It is a very powerful addition to
the OCR community to test and improve their
systems.
2.1.4 First Workshop on Indian Language OCR
Technology
The first OCR workshop for Indian scripts was
organized by us. All the major groups in India working
on OCR technology participated. The underlying
technologies were discussed in great detail. The
various systems under development were installed
and tested during the workshop to identify the
strengths and weaknesses of various approaches.
The discussion and debate that followed helped all
the centres do make further progress.
2.1.5 Conclusion
It is now possible, using the developed tools and
library functions, to implement a working OCR
system in less than a day. The development of a
working Gujarati OCR system from an absolute
scratch in under two days and subsequent transfer of
technology is a testimony to the design of Drishti.
113
The result of the work on Telugu OCR system at
RCILTS (Telugu) is more than a product or a
technology. It is a powerful set of research and
technology tools and a platform that facilitates rapid
development of OCR technologies and solutions for
Telugu and other Indian languages in the future.
2.2 Tel-Spell: Spell Checker
A Spell Checker consists of a spelling error detection
system and a spelling error correction system. An ideal
spell checker detects all spelling errors and does not
raise false alarms for valid words. It also automatically
corrects all misspelled words. Clearly, real spell
checkers will not be able to match up to such as an
ideal system. Real spell checkers may raise false alarms
for some valid words and may also fail to catch
some wrongly spelled words. Also, practical spell
checkers rarely correct misspelled words
automatically - they only offer a list of suggestions
for the user to choose from. Some spell checkers can
handle cases where an extra space has been typed in
the middle of a word or when two or more words
have been joined together into one. The performance
of a spell checker may therefore be measured in terms
of factors such as
• Percentage of False Alarm
• Percentage of Missed Detection
• Number of suggestions offered
• Whether the intended correct word is included
in the list of suggestions or not
• The rank of the intended correct word in the list
of suggestions
• Whether split and merged words are handled
Developing good spell checkers for Indian languages
has been a challenge. No spell checkers were available
at all for Telugu and a few other Indian languages.
Tel-Spell is the first ever spell checker for Telugu
and it includes both the spelling error detection
and correction components.
2.2.1 How do spell checkers work?
Many spell checkers store a list of valid words in the
language. A given word is assumed to be free of
spelling errors if that word is found in this stored
list. Otherwise it is presumed that the given word is
wrongly spelled. Since no dictionary can be perfect,
114
such a dictionary based approach is bound to produce
some false alarms and some missed detections. A large
dictionary may reduce false alarms but it is also more
likely to increase missed detections since rarely used
words may occur more because of errors in
typing than by intention. The choice of words to
be included in the dictionary is thus critical for the
best overall performance of the spell checker.
When a misspelled word is detected, other words
from the stored list that are similar to the given word
in terms of spelling are given out as suggestions for
correction. A quantitative measure of closeness of
spelling such as the minimum edit distance can be
used to select the words to be included in the suggestion
list.
This description of spell checkers is obviously
a highly over-simplified description. The number
of techniques used for both detection and correction
are large and varied. One may use statistical
techniques such as n-grams. Sequences of characters
that do not occur or occur with various frequencies
are obtained from a training dataset to build a model
of the language. This model can then be used to detect
spelling errors. See Karen Kukich for a good survey
of the various techniques.
2.2.2 Why is it difficult to build a spell checker for
Telugu?
Indian languages in general and Dravidian languages
in particular are characterized by an extremely rich
system of morphology. Words in Dravidian languages
like Telugu and Kannada are long and complex,
built up from many affixes that combine with one
another according to complex rules of saMdhi. For
example, nilapeTTukooleekapootunnaaDaa?
which means something like “Is it true that he is
finding it difficult to hold on to (his words/
something)?”
Telugu is both highly inflectional and agglutinative.
Auxiliary verbs are used in various combinations to
indicate complex aspects. Clitics, particles,
vocatives are all part of the word. Telugu exhibits
vowel harmony - vowels deep inside a verb may
change due to changes at the boundaries of
saMdhi. External saMdhi between whole words
and compounds also occur in the language. See
the references below for more on Telugu
morphology. Suffice it to say that Telugu is one of
the most complex languages of the world as
far as morphology is concerned.
It is therefore not practically feasible to store all
forms of all words directly in a dictionary for
purposes of spelling error detection and correction.
At the same time building a robust morphological
analyzer and generator is an extremely challenging
task. Developing a good spell checker for languages
such as Telugu is thus a very difficult task. No wonder
no spell checkers are available for these languages to
date.
2.2.3 Design of Tel-Spell
Perhaps the best and most thoroughly worked out
morphological analyzer for Telugu is the one we
have developed at University of Hyderabad over the
past 10 years or so. The system has been tested on
large scale corpora and enhancements and
refinements have been going on for years. During
this project, a thorough re-engineering work was taken
up and a new version was developed. The new
version is far simpler, more transparent, portable,
well documented, and conforms to standards.
AKSHARA: Spelling Error Detection and Correction
Research work has also been taken up on developing
stemming algorithms for Telugu. A pure corpus based
statistical stemming algorithm has been developed.
The performance of this stemmer for the spell
checking application has been studied in various
combinations with dictionary and morphology based
approaches. See thesis by Ravi Mrutyunjaya below
for reference.
A lot of empirical work and experimental studies
had to be conducted to come out with the best
combination of dictionary and morphology to develop
the first version of the spell checker for Telugu. A
10 Million word corpus developed by us has been
used to build and test the performance. The
performance of the system has been found to be
satisfactory both in terms of detection and correction
of spelling errors. See the references below for more
details. Our Telugu spell checker technology has been
transfered to M/S Modular Infotech. Ltd. on a
non-exclusive and non-preferential basis for
commercialization. The Telugu spell checker has also
been integrated into our AKSHARA advanced multi-
lingual text processing system.
2.2.4 Error Pattern Analysis
Large scale spelling error data has been obtained from
our 10 Million word Telugu corpus. The raw corpus
as it was typed has been compared with the final
version after three levels of proof reading and
certification by qualified and experienced proof
readers. A number of tools have been developed to
prepare such a data. Quantitative study of spelling
error patterns in Telugu is being conducted. This
will help us to build better spell checkers in future.
2.2.5 Syllable level statistics for spell checking
Since words are large and complex and hence too
numerous in Telugu and proper morphological analysis
is difficult, it would be useful to perform studies at
lower levels of linguistic units. The syllable level is
a natural choice since writing in Indian languages
is primarily syllabic in nature. n-gram models have
been built at syllable level. HMM models have also
been built. These models can be used to detect
spelling errors and to rank the suggestions for
correcting a given word. See the references below for
more technical details.
2.2.6 Further work
A agreement has been entered into with M/S Modular
Infotech. Ltd. For further development and transfer
of spell checkers for Telugu and Kannada.
2.3 AKSHARA: Advanced Multi-Lingual Text
Processor
2.3.1 Why one more word processor?
A systematic study of various available word
processors for Indian languages was performed in
order to choose the best ones for our own use here.
115
 AKSHARA: Advanced Multilingual Text Processor
The study indicated that none of the available
software products were satisfactory. They were slow,
fragile, unreliable, and they broke down when the
data is large. Even very simple operations such as
changing the font size cause the system to crash when
the file is big. There are a number of problems and a
detailed study convinced us that these are not merely
implementation level bugs that can be hoped to be
removed in the future versions. The basic design
philosophies are faulty and short sighted. It appears
that most of the commercial packages have been
designed without thinking beyond type-compose-
print paradigm for using computers as mere type-
writers. Most commercial packages work only
under Microsoft Windows platforms. The better
ones are a bit too costly for most ordinary users.
Adherence to standards is poor and compatibility
across fonts, versions and different packages is a big
problem. This motivated us to start developing our
own advanced multi-lingual text processor named
AKSHARA.
2.3.2 Encoding scheme
AKSHARA encodes texts in a standard character
encoding scheme such as ISCII or UNICODE.
Many commercial systems use font encoded pages
to by-pass the character to font conversion process
- itself a complex step for which there does not
seem to be any fully satisfactory solution so far.
These commercial systems also often use
proprietar y, non-standard fonts with secret
encodings. The documents so created are not texts
at all. There will be serious portability constraints
116
and unfortunate as it is, often the only practical
way to get back some text already in electronic
form encoded in such fonts is to re-type the whole
text! AKSHARA documents are always
character encoded. Mapping to fonts is done only
for the purposes of display and printing - all other
operations are performed on character encodings.
Commercial packages use proprietary and secret
encodings using non-printable control characters
for storing the attributes. In AKSHARA attributes
are included in an open XML style markup
language called Extensible Document Definition
Language (XDL) developed by us. This makes it
easy to convert to and from various other encoding
schemes thereby ensuring highest levels of
portability and platform independence.
2.3.3 Script Grammar
One of the unique features of Indian Language
writing systems is a script grammar. While “akshara”
or syllable is the basic unit of writing, these aksharas
are actually composed of more basic elements such
as vowels and consonants. Not all sequences of
such basic elements are valid and the script
grammar specifies legal combinations. Most
commercial packages do not seem to respect the
script grammar properly. A large percentage of errors
in the corpora developed using other tools earlier has
been found to be due to the inability of these tools
to check and strictly apply the script grammar.
What you see on the screen is not always what is
stored in the file and hence there is no way to check
and correct these mistakes by looking at the documents
on the screen.
A unique feature of AKSHARA is that it understands
the script grammar and warns you if you try to build
ungrammatical syllables. AKSHARA has been
successfully used to clean up all the corpora at CIIL,
Mysore.
2.3.4 AKSHARA is Robust, Reliable and Platform
Independent
AKSHARA is platform independent - you can use it
on MS Windows, Linux and many other platforms.
AKSHARA is also robust and reliable – you can
comfortably work with large documents without
worrying of silly restrictions such as line lengths.
AKSHARA has been successfully used to develop a
10 Million word corpus of Telugu.
2.3.5 Advanced Text Processing Tools this is made possible through our unique WILIO
technology. You will find more on WILIO below.
AKSHARA is an advanced text processing tool -
The web pages in our site
dictionaries, morphological analyzers, spell checkers,
www.LanguageTechnologies.ac.in have been
OCR systems, TTS systems, text processing tools
developed using this technology. For example, you
including searching, sorting etc. are part of
will be able to interactively search bilingual
AKSHARA. Several text processing tools, Telugu
dictionaries from our website.
spell checker and Telugu TTS have already been
integrated. We would also be happy to integrate any 2.3.8 Availability
of the other dictionaries, spell checkers, TTS etc. that
Wondering how much AKSHARA may cost?
other centres may have developed. Full support for
AKSHARA will be available for free and freely
Regular Expressions and Finite State Machines is being
available. Let everybody have the basic Indian
integrated.
Language processing capabilities without restrictions.
AKSHARA: Text Processing Tools 2.4 email in Indian Languages
We have developed technology for composing,
sending as well receiving emails in any combination
of English and other Indian languages. Many other
systems support only sending of mails, receiving mails
would not be as straight forward. This technology
has been integrated into our AKSHARA system. You
will only need a public email account (that supports
POP3 or IMAP protocols) somewhere. Unlike other
technologies, here there will be no dependence on
any other third party web sites. AKSHARA installed
on your local machine will be your email client.
AKSHARA: email client
2.3.6 AKSHARA as an email client
AKSHARA is unique in providing multilingual email
sending as well as receiving facilities. All you need
is a public email account somewhere. While many
other systems allow you to send emails, receiving
mails is not as easy. With AKSHARA there is no
longer any need to depend on any third party sites on
the Internet.
2.3.7 Developing Interactive web pages in Indian
languages
AKSHARA also enables you to develop interactive
web pages in Indian languages and English. Just use
AKSHARA and any web browser to create, edit,
modify and refine your web pages. These web pages 2.5 WILIO: Interactive Web Pages in Indian
will work across platforms and browsers. All your Languages
web pages will still be character encoded, not font
Developing web content in Indian Languages has been
encoded. Thus you will be really building a long
a challenge. None of the browsers understand the
lasting knowledge base. And what more, you can
ISCII character encoding scheme. We may hope that
create interactive web pages - pages into which your
UNICODE compatible browsers and free availability
users can directly type in in Indian languages. All
of UNICODE fonts will mitigate the situation to a

117
large extent in future. As on date, however, not all
browsers support UNICODE, UNICODE fonts are
not readily available for all Indian Languages and
UNICODE scheme is itself still not completely
satisfactory and revisions are going on. We explore
here briefly various alternatives people have tried and
present our own technology that we feel is far
superior to the others.
2.5.1 Text as Pictures
The simplest way to ensure that every client sees
exactly what you want him or her to see is to encode
texts as images. This technology Will work irrespective
of the platform and particular browser the user is using
and whether he or she has the required fonts or not.
However a picture is worth one thousand words (or a
lot more) - both in terms of storage and network
bandwidth required. Clearly, this cannot even be
considered as a solution as there is no text at all.
2.5.2 Font encoded pages
We can have font encoded web pages and expect
the users to have the fonts locally available on their
machines. Unfortunately, fonts are not yet freely
available for Indian languages and most computers
in the country will have no Indian language fonts
at all. Much more importantly, font encoded pages
are not texts at all. There is no font encoding standard
and encoding fonts in proprietary fonts is as good as
encrypting them. Web sites must be viewed as
knowledge bases – long lasting and easily maintainable.
Unlike in the case of languages like English where
character and glyph have a one to one
correspondence, Indian scripts are complex and the
mapping from characters to glyph sequences in a given
font is a complex many to many mapping. Therefore
font encoded web pages are no solution at all.
Dynamic Font Technology
One part of the problem with font encoded schemes,
namely availability of fonts on the client machine,
can be solved by using the so called dynamic font
technology. The basic idea is to send the fonts also
along with the requested documents to the client.
The pages are still font encoded. Not much different
from the previous method. Further,
dynamic font libraries are required - the usual
fonts are not sufficient. One needs to buy tools
to prepare dynamic font libraries. Otherwise you
will have to depend upon some other service provider.
118
Requests for a web pages will require connections
to the service provider’s web site too. Clearly, this
is not a good solution.
2.5.3 Plug-in technology
Then there are plug-ins - add-on pieces of software
that do the character to font conversion on the client
machine. The plug-in will have to be downloaded
and installed only once by a client. The pages will be
character encoded. This sounds like a good solution
but it has not worked well in practice. Plug-ins are
add-ons to browser software and the browsers vary
widely in terms of the support and the details of how
they take on these add-ons. For each browser and in
fact for each version of a browser, a suitable version of
the plug-in will have to be developed. As new
browsers keep appearing in the market, new versions
of plug-ins will need to be developed too. Unless the
browser developers themselves take on the
responsibility of supporting Indian script standards,
this technology is unlikely to be accepted as a good
and permanent solution.
2.5.4 Forget the scripts, use Roman
Of course one may forget Indian scripts and encode
Indian languages in the Roman script. This will
provide complete immunity from platform and
browser variations and font dependencies. Literate
Indian language users who are not comfortable with
the Indian scripts, say, non-resident Indians, will
also be able to use this technology. However,
this cannot be taken as a solution for Indian language
support for web content!.
2.5.5 What is WILIO?
We have endeavored to develop a better technology
which we call WILIO. WILIO permits standard
character encoded web pages to be viewed on any
browser and any operating system. The character
encoded pages are received by the client and mapped
to the required fonts before displaying them.
WILIO is unique in its ability to permit two-
way communication. We can develop interactive web
pages wherein the users can also type-in Indian
language content directly into the browser. The
required keyboard driver etc. are included in WILIO.
Thus one may prepare lessons, ask questions, allow
users to type in their responses, receive and validate
the answerers and get back to the users accordingly.
This will open up a whole new experience with
Indian language web content. The web pages in
our site have been developed using WILIO technology.
For example, look at our dictionary look up services
at our website www.LangaugeTechnologies.ac.in
2.5.6 How WILIO works
WILIO works through a Java Applet. Browsers must
support Java. Most browsers do. In case the Java
plug-in is not installed, it will be automatically
installed after getting user’s confirmation. WILIO also
requires that the fonts are locally available. A few
fonts are available freely in any country and we hope
we will not be far from such a day in India too. We
are making all out efforts to make a few fonts freely
available to everybody for non-commercial use. WILIO
itself is fully integrated into AKSHARA and AKSHARA
will be freely available and available for free.
2.5.7 Security
One of the very useful side-effects of this technology
is document security - WILIO it more difficult for
people to download and print the pages.
2.6 Telugu Corpus
A large, representative corpus is the first and most
essential resource for language engineering research
and development. A corpus is essential for building
language models as well as for large scale testing and
evaluation. Special emphasis was therefore laid on
developing a fairly large corpus of Telugu language,
the language of focus in the current project.
2.6.1 Status before the year 2000
Developing corpora for Indian languages has been
more challenging than it may appear. These days most
publishers use computers at some stage or the other.
Why not simply compile such readily available
material? While it is possible to get some material in
electronic form directly from publishers, DTP centres
and websites, it must be emphasized that there are
no free fonts and the proprietary fonts used by
various groups do not stick to any standard. While
the ISCII national standard for character encoding
has been around for a long time now, most of the
documents continue to be developed using
proprietary fonts embedded in proprietary
commercial software. Thus it is not possible to simply
download and add to the corpus.
Before the year 2000 corpora of only about 3
Million words were available for the major Indian
languages. These corpora were developed with the
support of the Ministry of Communications and
Information Technology, then known as the
Department of Electronics. Even these corpora were
not released for many years for researchers because
of legal and technical problems relating to copy rights.
2.6.2 How large is a large corpus?
Given the rich morphological nature of Indian
languages, it was felt that a mere 3 Million word
corpus would not be sufficient. In order to establish
this fact, we conducted a growth rate analysis of the
available corpus of Telugu. The corpus is split
randomly into equal sized parts and type-token
analysis is performed. A “type” is a particular word
form and each occurrence of that word form
constitutes a “token”. For example, “word” is a type
and there are two tokens of this type in the previous
sentence. Each part of the corpus contributes a set
of types and the cumulative number of types is
plotted on the Y-axis against the size of the corpus
(measured in terms of the cumulative number of
tokens) on the X-axis. The resulting curve depicts the
rate at which new types grow as the size of the
corpus increases. If the curve shows signs of saturation
and tends towards the horizontal, that means that
most of the word forms have already been obtained
and adding more corpus will not add too many new
word forms. As long as the growth rate curve continues
to show a high slope, it indicates that the corpus we
have is insufficient and many of the possible word
forms are yet to be seen even once in the corpus. Given
below is the growth rate curves for all the major
Indian languages for which corpora were available.
2.6.3 Dravidian Languages vs Indo-Aryan Languages
The distinction between Dravidian languages and
Indo-Aryan languages is striking in this figure - there
are many more word forms (types) in Dravidian
languages than in the other Indian languages. While
150,000 to 200,000 word types should be giving
an excellent coverage for northern languages,
Dravidian languages such as Telugu spoken mainly
in the southern parts of India require a much larger
number of word forms. And, more importantly, the
available corpus is not sufficient even to get a clear
idea of how many words are there in the language.
The morphology of these languages is so rich, no one
so far has an idea how many different word
forms are there in the language. One well known
linguist has argued that each verb root in Telugu can
give rise to as many as 200,000 different inflected/
derived word forms (Ref. Dr. G. Uma Maheshwara
Rao, personal communication).
119
What this shows is that techniques which work
well for Indo-Aryan languages may not be applicable
to Dravidian languages. For example, it would be
possible to simply list all forms of all words and use
this for dictionary based spelling error detection
and correction system for Hindi, Punjabi or Bengali
but such an approach cannot not be expected to
produce comparable performance results for say,
Telugu or Kannada. It would thus is not proper to
make out right comparisons of performance of
language engineering products across these classes of
languages. The inherent complexity of the languages
must be factored in when making any comparative
judgements of performance.
2.6.4 Copy Right Issues
Given the earlier experiences of groups developing
corpora, we had to take every care to ensure that
we did not get into copy right problems. At the
same time, we realized that it is not going to be easy
for us to take over the legal copy rights from the
authors or publishers. Hence it was decided that we
will only ask for right of electronic reproduction and
rights for hosting selected works on our web-site,
without asking for a legal transfer of copy rights.
The original copy right holders would continue to
hold their copy rights and would be free to sell,
distribute or transfer their works to any other party at
their will.
It was nevertheless an extraordinary convincing effort
to get the best authors to part with their best works
for our corpus without spending a single pie of money.
It took a tremendous amount of time and effort to
convince the copy right holders that what they and
the country at large gains in the long run will be
much more than the hypothetical loss incurred in
giving us their works free of cost. A variety of
strategies and tactics had to be used but at the end
we have been able to obtain the rights for more than
250 of the best works of the best known writers.
Add to this works for which copy rights have expired
and we have a list of more than 500 books. (It will
be interesting to note that the expectation of the
funding body was 10 good books!).
2.6.5 The Status
A corpus of 225 books adding upto about 30,000 pages
and 9.25 Million words has been completed. The corpus
includes a variety of topics and categories - newspaper
articles, short stories, novels, poetry, classical and
modern writings etc. Each of these works has been typed
120
in using our AKSHARA - advanced multi-lingual text
processor and other such tools and subjected to two
levels of thorough proof reading by qualified and
experienced proof readers and finally certified free of
errors. The entire corpus is encoded in the ISCII/
UNICODE character encoding standard and XML style
annotation scheme is used for meta information.
The growth rate curve of types against tokens for the
12 Million word total corpus of Telugu available now
was conducted recently. The curve shown below
shows clearly that even this corpus is not sufficient -
there is no sign of saturation and the growth rate
has not reduced significantly. We still do not have
even a single occurrence of most of the word forms
although the number of types has already reached
20,000,000.
2.6.6 Tools
A number of tools have been developed to develop,
analyze and manage large scale corpora. Some of these
tools have also been given to other centres. A
comprehensive tool kit is being developed.
We have also developed tools for semi-automatically
decoding any unknown font with minimum effort.
Using this technique, a mapping scheme can be
developed to map the text strings encoded in the
unknown font into an equivalent text string in a
standard character encoding scheme such as ISCII
or UNICODE. In fact several of the widely used
fonts have been decoded and we can now add more
free material such as newspaper articles at ease.
2.6.7 Plans
Now that our OCR system for Telugu and the Telugu
spell checker have reached a level of performance
that makes it suitable for use in content creation,
we hope to be able to develop even larger corpora of
Telugu very soon. A thorough investigation into the
spread across genres and representativeness of the
corpus is also being carried out so that further work
can be fine tuned accordingly despite the non-
availability per se of texts in some of the categories
in Telugu language.
Plans for the future including various levels
of annotation. English-Telugu parallel corpus
development is also being considered.
2.7 Dictionaries, Thesauri and other Lexical
Resources
Dictionaries are the most basic and essential data
resource for any language. Accordingly, we have
developed a number of monolingual and bilingual
dictionaries as detailed below. The dictionaries
are available in the XML format for data exchange
and indexed cleverly for efficient search. Look-up
services are provided from our website using our unique
WILIO technology for OS and browser independent
deployment. All dictionaries are encoded in ISCII/
UNICODE standard character encoding schemes.
Apart from dictionaries, we have also developed
thesauri and more importantly, a tool by which we
can develop a thesaurus of sorts for any language in
just a few minutes from a suitable bilingual
dictionary. Here is a summary of the dictionaries we
have with us. Some of these are already being used
by researchers in other centres.
2.7.1 C P Brown’s English - Telugu Dictionary
Status : Completed; Size : 31,000 plus; Fields : POS,
Meanings, Usage; XML? : Yes; Indexed? : Yes; Web-
enabled? : Yes.
2.7.2 C P Brown’s Telugu - English Dictionary
Status : Completed; Size : 31,000 plus; Fields : POS,
Meanings, Usage, Etymology; XML? : WIP; Indexed?
: WIP; Web-enabled? : WIP.
2.7.3 English - Telugu Dictionary suitable for
Machine Aided Translation
Status : Completed; Size : 37,500 plus; Fields : POS,
Meanings; XML? : Yes; Indexed? : Yes; Web- enabled?
: WIP.
2.7.4 Telugu - Hindi Dictionary suitable for
Automatic Translation
Status : Completed; Size : 64,000 plus; Fields : POS,
Paradigm Class, Meanings; XML? : Yes; Indexed? :
Yes; Web- enabled? : WIP.
2.7.5 English - Kannada Dictionary
Status : Completed; Size : 15,000 plus; Fields : POS,
Meanings, XML? : Yes; Indexed? : Yes; Web- enabled?
: WIP.
2.7.6 Basic Material for English Dictionary
Status : Completed; Size : 6,00,000 plus; Fields : POS,
Frequency; XML? : Yes; Indexed? : Yes; Web- enabled?
: WIP.
2.7.7 English Dictionary
Status : Completed; Size : 80,000 plus; Fields : POS,
Frequency; XML? : Yes; Indexed? : Yes; Web- enabled?
: WIP.
2.7.8 Telugu Dictionary
Status : Completed; Size : 64,000 plus; Fields : POS,
Paradigm Class; XML? : Yes; Indexed? : Yes; Web-
enabled? : WIP.
2.7.9 Kannada Dictionary
Status : Completed; Size : 12,000 plus; Fields : POS;
XML? : Yes; Indexed? : Yes; Web- enabled? : WIP.
2.7.10 Kannada Thesaurus
Status : Completed; Size : 12,000 plus; Fields :
Synonyms, POS, Sense; XML? : Yes; Indexed? : Yes;
Web- enabled? : WIP.
WIP: Work in progress
These dictionaries are all closely linked up with
corpora, morphological analyzers and generators,
spell checkers etc. Cross-validation and refinement
continue on a regular basis.
2.7.11 Tools
We have also developed a number of tools for
developing electronic dictionaries, for efficient
indexing, searching and other such operations on
electronic dictionaries, for formatting in XML or other
standards, for verification and validation, for web-
enabling and offering web based services etc. We
would be glad to host dictionaries developed by other
centres using our platform independent and secure
WILIO technology.
2.7.12 Automatic generation of Thesaurus from
Bilingual Dictionary
We have developed a unique tool that can generate a
thesaurus of sorts for any language in just a few
minutes starting form a suitable bilingual dictionary.
We would be glad to offer this service to any centre
that has a suitable bilingual dictionary. Thesauri
being extremely useful resources yet non existent for
many Indian Languages, we believe the contribution
of this tool is being appreciated very well from all
quarters hope this tool would be very useful.
2.7.13 Technology for hosting dictionaries on the web
We have the unique capability to place Dictionaries
on the web for efficient, secure, platform and browser
independent services. We would be glad to host any
other dictionary developed by any other centre
through our technology from our site.
2.8 Morphology
2.8.1 What is Morphology?
121
Morphology deals with the internal structure of
words. Morphology makes it possible to treat words
such as compute, computer, computers, computing,
computed, computation, computerize, computerization,
computerizable and computerizability as variants of the
same root rather than as different words unrelated
to one another. Morphology makes it possible to store
only the root words in the dictionary and derive other
variants through the rules of the morphology. It helps
us to understand the meaning of related words.
2.8.2 Indian Languages exhibit rich morphology
Morphology plays a much greater role in Indian
languages because our languages are highly
inflectional. While the English verb eat gives rise to
only a few variants such as eats, ate, eaten and eating,
the corresponding verb in Telugu can give rise to a
very large number of variants. Words in Dravidian
languages like Telugu and Kannada are long and
complex, built up from many affixes that combine
with one another according to complex rules of
saMdhi. For example,
nilapeTTukooleekapootunnaaDaa? which means
something like “Is it true that he is finding it difficult
to hold on to (his words/something)?”
Telugu is both highly inflectional and
agglutinative. Auxiliary verbs are used in various
combinations to indicate complex aspects. Clitics,
particles, vocatives are all part of the word. Telugu
exhibits vowel harmony - Vowels deep inside a
verb may change due to changes at the boundaries
of saMdhi. External saMdhi between whole
words and compounds also occur in the language.
See the references below for more on Telugu
morphology. One linguist puts the number of
variants for a single Telugu verb at nearly 200,000!
[G. Uma Maheshwara Rao, Personal
Communication.] The exact number of different
forms that a verb can take in a language like Telugu
is not yet clear. The growth rate analysis described
in the section on corpora clearly shows that the 12
Million corpus available at present is not sufficient
to give us even a single occurrence of many possible
words in the language. While Indian languages
in general are morphologically richer than
languages like English, Dravidian languages
are a lot more complex. The 12 Million word
corpus of Telugu has nearly 20,000,000 different
words and there will be many more as the growth
rate curve indicates. In contrast, the Indo-Aryan
languages have only about 1,50,000 to 2,00,000
122
words forms in all. Dravidian languages
including Telugu, Kannada, Malayalam and Tamil
are among the most complex languages of the
world and can only be placed along with
languages such as Finnish and Turkish. Clearly,
there is no way we can hope to list all forms of all
words in a dictionary. We cannot build a spell
checker, for example, by simply listing all forms
of all words. Morphology is not just useful but
absolutely essential.
2.8.3 Design of Telugu Morphological analyzer
Building a morphological analyzer and generator for
a language like Telugu is thus a very challenging
task. Perhaps the only large scale system built for
Telugu is ours. Our Telugu morphological analyzer
has been built, tested against corpora and refined
over the past 10 years. This system uses a root word
dictionary of 64,000 entries and a suffix list categorized
into a number of paradigm classes. The basic
methodology is to look for suffixes, remove them taking
care of saMdhi changes and then cross checking
with the dictionary. Inflection, derivation, external
saMdhi are all handled. See the references below for
more technical details. There is also a separate
morphological generator that can put together the
roots and affixes to construct complete word forms.
2.8.4 Design of Kannada Morphological Analyzer
We have also developed a Kannada morphological
analyzer and generator using our own Network and
Process Model. A finite state network captures in a
declarative and bidirectional fashion all the affixes,
their ordering and the various combinations
permitted. The process component takes care of
saMdhi changes when affixes are added or removed.
This model makes it possible to develop a
morphological analyzer, test it against a corpus and
then we get a generator of comparable performance
with no extra effort since the same network is used
both for analysis and generation. In this model, a
complete and detailed analysis is made at the level
of each affix.
2.8.5 Tool for developing Morph systems for other
languages
Morphological analyzers and generators for several
languages including Kannada, Tamil, Oriya etc. have
been built using this Network and Process model.
That a good Tamil morphological analyzer and
generator could be built within a week using this
system is a testimony to the quality of design and
implementation of the system. See references below
for more details.
As we build larger and more representative
corpora, further refinements to dictionaries as well
as morphological analyzers and generators will
continue.
2.9 Stemmer
As the above section on Morphology shows, it is
very difficult to build a high performance analyzer or
generator for Dravidian languages such as Telugu. An
alternative short-cut approach that can be used in
practice is stemming. Here a complete and detailed
morphological analysis is not performed. Instead, the
affixes are removed to obtain the root. For example,
the common prefix in the words compute, computer
and computing is comput and hence all these word
forms are reduced by removing the affixes to the
common stem comput. Note that comput is not a valid
linguistic unit at all. Yet, such stemming techniques
are useful and have been used in many areas
including Information Retrieval. Stemming can also
be used as the second line of defence when
morphology fails.
A thorough study of various stemming techniques
have been conducted. Ingenious corpus based
statistical stemming techniques have been developed
for stemming in Telugu. Vowel changes, gemination
etc. need to be taken care of in building a stemmer
for Telugu. The stemmer has been compared with
the full morphological analyzer and various
combinations have been tried out for the purpose
of spelling error detection and correction in Telugu.
See the references below for more technical details.
2.10 Part of Speech Tagging
2.10.1 What is POS tagging?
A dictionary lists all possible grammatical categories
for a given word. The job of a Part of Speech
(POS) tagger is to identify the correct POS for a
given word in context. For example, the word thought
is a verb in the following sentence and a noun in
the sentence that comes next: I have thought about
it from various angles. Suddenly this strange thought
came to my mind.} POS tagging may be at the level
of gross grammatical categories such as verbs and
nouns or, more often, at a more fine grained
level of sub-categorization.
2.10.2 POS tagging techniques for English and
Indian languages
In English like positional languages, the category of
a word can be determined in terms of the categories
of the preceding words. As such Hidden Markov
Models have been widely used. There are several
other techniques too. However, as far as Indian
languages are concerned, many of these sequence
oriented techniques are not very much applicable.
Our languages are characterized by free word order
and hence it does make much sense to depend so
much on previous or following few words. Instead,
our languages are characterized by a very rich system
of morphological inflection and it is here that we get
maximum information about the correct part of
speech of a word. The percentage of words that
occur in some inflected form rather than in the bare
stem form is far more for Indian Languages as compared
to English. Morphology holds the key for POS tagging
of Indian languages. In fact one may even go a step
further and argue that a POS-tagged corpus does not
make much sense. Whenever you process some text,
you will need to perform morphological analysis and
the job of POS tagger will be done there too.
However, developing robust morphological
analyzers for Indian languages in general and
Dravidian languages in particular have been difficult
challenges. The performance of any POS tagger
based on morphology would be limited by the
performance of the morphological analyzer itself.
2.10.3 Degree and nature of lexical ambiguities
A systematic study of the degree and nature of lexical
ambiguities at the dictionary and corpus levels is
being conducted. Appropriate technologies for POS
tagging based on morphology are being developed.
The percentage of words that occur in some inflected
form rather than in the bare stem form is far more
for Indian Languages as compared to English. This
has serious implications for the degree and nature of
lexical ambiguities in running texts.
2.10.4 HMM system for POS tagging
In order to gain deeper understanding of POS
tagging for various languages including English, a
Tri-tag based HMM model has also been built and
tested on the SUSANNE corpus.
2.11 VIDYA: Comprehensive Toolkit for Web-Based
Education
2.11.1 eLearning
eLearning helps to overcome the barriers of distance
and time in learning. Thrust is on learning, not
123
teaching. Students can thus learn whatever they like,
in whatever order they please and at a pace that is
best for them. Instead of teachers, there will only
be facilitators.
There are many tools for eLearning. Only some
of them are comprehensive tools that provide for
the entire gamut of facilities from pre-registration
counseling to maintenance of the alumni database.
Good ones are very costly and most educational
institutions in India will not be able to afford to
buy such tools. Many developments are taking place
in this area but the major usage so far has been
limited to corporate training in Information
Technology and related areas. The focus is on adult
serious learners only. Profit seems to be the main
motive in many cases.
2.11.2 Web Based Education - Technology for Quality
Education
Our view point is very different. Traditional
education is more teacher centric whereas eLearning
is learner centric. The idea is not to choose one or
the other but the right combination of both. Not all
students are mature and serious enough to learn on
their own. Teachers are required to guide, instill
confidence and to inspire. The need is to look at
education at all levels in a holistic sense.
The primary issue in question is quality of education.
We produce very large number of BSc’s and MSc’s but
very few scientists. We produce very large number of
BE and BTech degree holders but very few engineers.
Primary school level is worse. There are several problems
and not all of them can be solved through technology.
The question is what is it that technology can do to
ensure quality education to everybody?
How do we ensure highest quality of education at
all levels without barriers of distance and time? Good
teachers are not always available. Distance and time
are not the only barriers. Cost and language are
bigger and more serious barriers. The ultimate
objective should therefore be to reach out to all
interested students and offer the highest quality of
education without any kinds of barriers - distance,
time, cost or language. Here is where technology
can bring the services of best teachers, best course
materials to every student in a cost effective manner.
Our technologies must be Indian language enabled.
2.11.3 VIDYA - a comprehensive suite of tools
Given this scenario, we started developing a
comprehensive suite of tools for web based education.
124
Our suite is called VIDYA. Indian languages can be
supported. Interactive web content can be created
using our WILIO and AKSHARA technologies.
VIDYA supports inter-student and student-teacher
interaction through email, chat, discussion rooms
and white boards. It encourages collaborative
problem solving and group activities.
VIDYA: Comprehensive suite of Tools for Web Based
Education
VIDYA has the unique facility to link with auxiliary
servers for extra support such as for laboratories.
For example, you may use VIDYA to do
programming in Java, C++, C, and Perl languages
without the need for these compilers on your
machines. Coding, editing, compiling, executing,
archieval are all supported. Similarly, science labs.
And language labs. can be developed. Appropriate
use of multi-media and learning by doing makes
learning a pleasure and has much greater impact
than reading text books and listening to class room
lectures.
VIDYA support a wide variety of testing, evaluation
and reporting facilities. Adaptive testing, navigation
control, timing etc. are supported. Full range of
question types including multiple choice, short
answer and essay type questions are permitted.
2.11.4 Status and Plans
VIDYA has been installed in several centres
including CIIL, Mysore. VIDYA is being regularly
used in University of Hyderabad for teaching courses
at MCA and MTech levels. It has also been used for
offering special courses to reputed industries. A
recent study has shown that it is suitable for
deployment in our distance education programme.
 VIDYA: Interactive Multimedia Content
VIDYA could be used by schools, colleges,
universities, research laboratories etc. for regular
education, continuing education, part-time courses,
in-house training etc. Suitable material can be
developed and shared with others so as to maximize
the impact. In particular, language teaching material
already developed or being developed by various
centres can be linked with with VIDYA to enable
various classes of language learners to get maximum
benefit. We would also be glad to enter into
agreements for further collaborative development
of the tool itself.
2.12 Grammars and Syntactic Parsers
2.12.1 Computational Grammars for Indian
Languages
There are no large scale computational grammars for
any of the Indian languages. Computational grammars
and syntactic parsers are very much required for taking
Indian languages beyond the type-compose-print
paradigm that is holding back the country from
growing beyond using computers as some kind of
type-writers. All language engineering applications
including machine translation, information retrieval,
information extraction, automatic categorization
and automatic summarization would greatly benefit
from syntactic parsers.
2.12.2 UCSG system of Syntax
The UCSG system of syntax was developed by us
to place positional languages such as English on an
equal footing with Indian languages that are
characterized by relatively free word order. A careful
study of both the Western grammar formalisms and
the paaNinian approach to syntax showed that none
of these would be equally suitable for positional
and free word order languages and hence a new
formalism had to be developed. Computational
grammar and Parser has been developed for English
VIDYA: On-Line LABs
and demonstration level systems have also been
developed for Telugu and Kannada. UCSG uses a
combination of Finite State Machines, Context Free
Grammars and Constraint Satisfaction to achieve the
best overall performance. Grammars become simple
and easy to write, parsers become computationally
very efficient and the same basic framework works
for English and other Indian languages. UCSG works
from whole to part, rather than from left to right.
See the references below for more details.
Further development of the UCSG English parser is
going on. A much larger and more informative
dictionary has been built based on the analysis of
large scale corpora. Combination of linguistic and
statistical models are being used to enhance the
coverage and robustness of the system. Plans
include the development of computational
grammars for Telugu and other Indian languages.
2.12.3 Robust Partial Parsing
It is now well recognized that full syntactic parsers
are difficult to build. Hence there is increased
interest in robust but shallow or partial parsing. An
extensive study of parsing technologies have been made
and efforts are on to build a large scale robust partial
parsing system for English. Efforts are also underway,
in collaboration with linguists from CIIL to develop
computational grammars and shallow parsing systems
for Indian languages.
125
2.13 Machine Aided Translation
Automatic or Machine Translation is the one of
the widely known applications in language
engineering. It has been recognized very well that fully
automatic high quality translation in open domains
is difficult to achieve. Either restricted domains of
applications with controlled language usage must
be considered or the translation process has to be
semi-automatic, the man and the machine doing what
they are good at and seeking help from each other in
other areas. Even with such restrictions, ensuring
quality of translation is a very challenging task.
Language is rich and varied in structure as well as
meaning. Since the machine cannot be expected to
“understand” the meaning of the given source
language text in any real sense, the out of the machine
can at best be good, calculated guesses. The situation
is further complicated by the fact that expectations
of users are very high when it comes to translation.
2.13.1 English to Kannada Machine Aided
Translation System
A Machine Aided Translation system was developed
here for the Government of Karnataka for
translating budget speech texts from English to
Kannada. English text is pre-processed and segmented
into sentences. Each sentence is syntactically parsed
using our UCSG English parser. The parsed
sentences are translated to Kannada in a whole-to-
part fashion using the bilingual English-Kannada
dictionary and Kannada morphological generator
developed by us. There is a powerful post processor
that is tightly integrated with the dictionary,
thesaurus, morphology and the translator. A full 150
page text is parsed and translated in just a couple of
minutes on a desktop PC. The output is post-
edited and then sent for final proof reading.
Although this was a very short project with a very
low budget, this project has demonstrated the merits
of a well designed and well engineered product.
There are several unique and powerful features in
this system. This system has become a very good
technology demonstrator and has inspired a lot of
serious work in various directions by several groups
across the country. See www.LanguageTechnologies.ac.
in for more details.
The success of the MAT system for English to
Kannada translation has inspired further work on
dictionaries, morphology of Kannada, robust parsing
and word sense disambiguation. Research work has
126
also been taken up on a number of other specific
topics such as corpus based machine learning
techniques for sentence boundary identification.
The MAT2 system proposed purports to combine
the best of linguistic theories, corpus based machine
learning algorithms and human judgement based on
world knowledge and commonsense to achieve
high quality translations in a semi-automatic setup.
A very useful by-product of this exercise will be a
high quality POS and sense tagged, parsed, aligned,
parallel corpus.
2.14 Tools
We have also developed a number of tools over the
past many years for our own use and some of these
tools could be useful to other groups as well. In fact
some of these tools have already been given to other
resource centres. Here we list some of the important
tools developed by us. It may be noted that not all
these tools were developed within the period of, or
with the support of, this specific project.
2.14.1 Font Decoding
Unlike English, Indian scripts are syllabic in nature.
The units of writing are akshara’s or syllables. The
total number of possible syllables is very large. Thus
fonts are developed using shape units called glyphs
which need to be composed to form complete
syllables. The mapping from syllables to glyph
sequences is complex. There is a proliferation of
non-standard and proprietary fonts. In fact there is
no font encoding standard as yet for Indian languages.
Thus documents encoded in some unknown font is
exactly like a coded message - one needs to do decode
them before they start making any sense. Only
documents encoded in a standard character encoding
scheme such as ISCII or UNICODE can be
considered as text. Font encoded documents re not
texts at all. However a large number of documents
are available only in font encoded forms. Some
companies in fact use this as a means of achieving
some degree of security for their documents.
We have developed a set of tools through which
we can decode any unknown font and map it onto a
standard character encoding scheme such as ISCII
or UNICODE. This is a semi-automatic and
iterative process. With this tool, now it possible to
decode any unknown font and we hope this would
encourage commercial companies to become more
open and follow standards instead of pursuing myopic,
proprietary, and restrictive practices.
Some of the other centres have developed direct
mappings from one font to another. We believe that
the best way to handle font-to-font variations is to
go through a standard character encoding scheme.
All documents must be encoded and processed in a
character encoding scheme and mapping to fonts is to
be used only for the purposes of display and printing.
2.14.2 Web Crawler for Search Engine
A web crawler searches the whole web and builds
up an index of web pages that is structured and
classified in a way that enables search engines to search
the web efficiently. We have developed a basic web
crawler through which such an index can be built.
This tool can also be used to download whole web
sites, for archival of web sites etc.
2.14.3 PSA: A Meta Search Engine
A search engine searches the web for the documents
users request through a short query. There are several
good search engines but there is no single search engine
that is ideal in all cases. A Meta Search Engine accepts
a user query, fires search engines, obtains the results
and presents them to the user. Personal Search Assistant
(PSA) is one such search engine designed and
developed by us here.
PSA accepts user queries, formats them in the
manner required for various search engines and fires
the search engines accordingly. PSA can currently
handle upto eight different search engines
simultaneously. It is possible to work in the
background mode so that users do not need to sit
in front of the machine and wait for results. Status
can be checked up at any given point of time. It is
possible to monitor the network load and adjust
accordingly. Results are collated, duplicates removed
and stored in local database where required. Unlike
a search engine a meta search engine can reside on
local machines and can be customized to suit
individual requirements. Some work has been done
on personalization of PSA. PSA is being used within
University of Hyderabad for several years now.
Plans include Indian language support for PSA.
Queries can then be posed in Indian languages.
Specialized versions of the web crawler can also be
developed to locate web pages in Indian languages
or web pages relating to India.
2.14.4 Corpus Analysis Tools
A number of tools have been developed for corpus
analysis. Some of these tools are being used by other
centres as well. It is planned to organize these tools
in the form of tool kit so that it will become more
convenient for others to use them.
2.14.5 Website development Tools
We have a whole range of tools including our
AKSHARA and WILIO systems to develop interactive
web sites in Indian languages. For example, our
History-Society-Culture portal has nearly 500 pages
of Indian language content developed and hosted
through our technology. Interactive lookup services
of our dictionaries also use these tools and
technologies.
2.14.6 Character to Font mapping Tools
As has been noted elsewhere in this report, mapping
between characters and fonts is a non-trivial process
in Indian languages. Some have used table look-up
method while others have used hand crafted rules.
None of the systems seem to be satisfactory.
Completeness, consistency, robustness, efficiency,
transparency, extensibility, ease of development
are some of the desirable features of such mapping
systems. Given this scenario we have explored the
possibility of developing good mapping systems using
the Finite State Machine technology. Suitable
extensions to the basic technology are proposed for
the purpose.
2.14.7 Dictionary to Thesaurus Tool
Give us any bilingual dictionary and we can give
you a kind of a thesaurus in a couple of minutes.
Our tool can do a clever reverse indexing on any
bilingual dictionary to identify closely related words
for any given word. While this is not exactly a
thesaurus in technical sense, the basic idea is the same
- given one word, to identify other words in the
language that are closely related to it in meaning.
The Kannada thesaurus so developed has been
demonstrated and has been judged to be very useful.
Some researchers are already using this system and
there are plans to do more work on Kannada and
Telugu thesauri. We will be glad to develop thesauri
for any other language given the suitable bilingual
dictionaries.
2.14.8 Dictionary Indexing Tools
Dictionaries need to be indexed for efficient
access. We have developed clever indexing schemes
for efficient indexing of large dictionaries on any
computer. Combinations of TRIE indexing, Hashing,
B-trees, AVL Trees etc. are used.
127
2.14.9 Text Processing Tools
A number of text processing tools for working
with word lists, dictionaries, etc. have been developed
over the past many years. These tools have been
found to be very useful for linguists and
lexicographers too. Some of these tools have been
integrated into AKSHARA.
2.14.10 Finite State Technologies Toolkit
Finite State technologies are increasingly being used
in language engineering as they are simple yet very
efficient. A full toolkit has been developed and tested
on large scale data. Now it is possible to work with
Regular Expressions, NFA, DFA etc. and perform all
the usual operations without having to write any
program code. It is planned to integrate these tools
into AKSHARA.
3. Services and Knowledge Bases
3.1 On-Line Literature
Telugu has a very rich literary tradition dating back
to around 10th Century. In today’s busy world where
reading habits seem to be depleting, here is an
attempt to bring some of the best works of Telugu
at your door steps. Making literature available on-
line means making it available anywhere, anytime.
You will not need to goto a book store or otherwise
order and buy a book. This service is absolutely
free of charge and so you spend no money.
More than 550 of the best works in Telugu have been
enlisted. We have obtained, by a great deal of
convincing effort, the right of electronic
reproduction and web enabling these works from
the respective copy right holders. Not a single pie
was spent to obtain the rights though. About 225 of
these books have been converted into electronic form
and checked by a three stage proof reading and
certification process by qualified, experienced and
professional proof readers. A panoramic selection of
these works will be made available from our web site
through our unique WILIO technology that
guarantees platform and browser independence as
well as some degree of security. It is planned to
add Roman transliterated versions too for the benefit
of those who know Telugu language but are not
very comfortable with the Telugu script.
3.2 History-Society-Culture Portal
India is a uniquely pluralistic society that is the
home of many religions, traditions and cultures. Here
is an attempt to bring to you nearly 500 pages of
128
History, Society and Culture Portal: Temples
authentic material on the society and culture of the
Telugu people and indirectly, on its history too. Get
to know more about temples, music, dance, folk arts
and many more items. Color photographs are included.
All the pages are available through our unique WILIO
technology that works across operating systems and
web browsers. Roman transliterations are also provided
for those who would have difficulties in reading the
Telugu script. Visit www.LanguageTechnologies .ac.in
3.3 On-Line Searchable Directory
Given the importance of networking of individuals
and organizations with overlapping interests, it was
decided to develop an on-line searchable directory
of people and organizations interested in various
aspects of language technology for Telugu. More
than 1200 relevant entries have been developed
and cross checked. The directory is available on
line. A flexible search facility has been included.
Kindly visit www.LanguageTechnologies.ac.in
3.4 Character encoding standards, Roman
Transliteration Schemes, Tools
There is quite a bit of confusion in the country
about the exact nature of character encoding
schemes, fonts, rendering engines, character to font
mapping schemes etc. Many hasty decisions are
sometimes being taken without a full and in depth
understanding of all the issues concerned. Hence a
detailed article was written about the issues involved
in character encoding schemes and related issues. A
version of the article has been published in the
Vidyullipi journal.
3.5 Research Portal
Research and development requires time, effort,
money and other resources but as far as Language
Engineering in India is concerned, the most
important resource required is adequate trained
manpower. Language Engineering is a highly multi-
disciplinary field - it borrows from such diverse
disciplines as Linguistics, Psychology, Philosophy,
Logic, Artificial Intelligence, Cognitive Science,
Computer Science, Mathematics, Statistics and
Physics. Clearly, there are no experts who know
all these areas very well. This multi-disciplinary
nature of the subject makes it so much more
difficult to create quality training materials, books
etc. that are understood by people across so many
disciplines. Indian languages are also characterized by
certain unique features compounding the difficulties
in developing trained researchers and developers.
With this in mind, we have developed research
portals in selected areas of Language Engineering. In
one place, you will find basic and introductory
material, tutorial and survey papers, classified and
structured collection of large number of relevant
research papers, pointers to people, departments,
institutions, conferences and other regular events and
so on. This would substantially reduce the time and
effort needed by newcomers to these research areas.
We could also develop these portals for news,
discussion and debate, collaborative development etc.
3.6 VAANI: A Text to Speech system for Telugu
Text-to-Speech systems convert given text into speech
form. Thanks to the maturity of the techniques and
availability of required tools, it is now possible to develop
minimal TTS systems in months. VAANI, our TTS
system for Telugu, is one such attempt. Di-phone
segmentation based approach has been used. Phonemes
in the language are identified and for each ordered pair
of phonemes, called di-phones, example words are
recorded in speech form. From this raw data, di-
phones are segmented using available tools. It would
then be possible to process this raw data further and
develop a database. To produce speech from any given
text, the text is initially parsed. certain pre-processing
steps are essential to handle numerals, homographs etc.
Then the text is segmented into diphones and the
corresponding speech units are concatenated to produce
speech output. Segmenting at di-phone boundaries
gives better continuity since the variations due to co-
articulation effects are least in the middle of a
phoneme as compared to its ends. Since the number
of phonemes in a language is usually a small and
closed set, this technology also leads to unlimited
vocabulary TTS technology. VAANI is thus an
unlimited vocabulary, open domain TTS system for
Telugu. The system has been tested for intelligibility
both directly and across telephone lines. Prosodic
features such as duration, pitch and intonation can be
added to make the sounds more natural. A substantial
amount of research has also been done on other
competing technologies and we would be able to
deliver high quality unrestricted TTS systems in future.
VAANI has already been into our AKSHARA -
Advanced Multi-lingual Text Processing system.
3.7 Manpower Development
Trained manpower is a critical issue in language
engineering in India. More than 100 students and
staff have worked in the project for periods ranging
from 3 months to three years on various research
and development activities. In the process they have
obtained significant theoretical knowledge as well
as practical skills in language technologies.
The LEC-2002 International Conference on Language
engineering included a full day of tutorials by
distinguished experts from India and abroad. These
tutorials were free for students. Several hundred
students could benefit from these.
The first IL-OCR workshop organized by us here
attracted many interested students and researchers
from across the country. The detailed presentations,
demos. and discussions were very helpful.
The research portals being set up here will be of
value to beginners in language technology research.
A number of articles, technical reports and research
papers have either been published or are being
prepared for wider dissemination of the ideas,
techniques and technologies. Our website is intended
to serve a similar purpose and is being enhanced
and updated accordingly.
A text book on Natural Language Processing with
specific emphasis and examples from Indian languages
is planned.
With this it should be possible now to organize training
programmes on specific topics to identified target
groups. Suitable course material can also be developed
specifically for the purpose.
4. Epilogue
4.1 Strengths and Opportunities
Our strength lies in our core research competence.
Our team has experts from linguistics, statistics,
computer science, artificial intelligence and cognitive
science. Each member of the team has a very rich and
varied experience. What binds us all is the common
research competence. The tools, techniques and
algorithms used in bio-informatics, image processing,
speech recognition and language technology have
129
many things in common. Our team emphasizes
this common core.
We had a one semester long seminar cum discussion
series on Markov Models. This semester we have
semester long seminar cum discussion series on feature
extraction and feature selection. See annexure for
details. Thorough investigation of Word Sense
Disambiguation and Shallow Parsing techniques are
going on. This in depth understanding of the
technologies will enable to do world class research in
various areas and develop quality products and services.
We also have striven to develop large scale linguistic
data resources that are essential for further research
and development. Our centre is perhaps unique in
having developed a 10 million word corpora and more
than half a dozen dictionaries of significant size and
quality. Large and representative data and the right
kinds of tools will enable us to move much faster.
We have also striven to strike a good balance
between the pure research and publications on the
one hand and product development and technology
transfer to meet the needs the society on the
other hand. Many of our results are yet to be published
and we hope to bring out several publications soon.
We have also striven to strike a good balance between
long term and short term goals. Two years ago when
we started off almost from scratch on our OCR system,
other experts felt that as far as content creation is
concerned there is nothing better than simply typing
in texts. Today our OCR is one of the successful
ones in the country and combined with our spell
checker, we will soon be able to develop much large
corpora for Telugu and other Indian languages than
would have been possible otherwise.
We have struggled in this first phase of development,
to overcome the teething problems and look for long
lasting and permanent solutions rather than hop
onto short sighted and immediate solutions. This
approach has payed off and with the data and tools
we have with us now, we hope to be able to move
much faster.
Our future efforts will be in more focused areas of
language and speech engineering. We look forward
to meaningful collaboration with other leaders in the
world for research as well as technology development.
4.2 Outreach
The LEC-2002 Language Engineering conference
organized by us was quite successful and we hope to
be able to organize quality international conferences
130
in future as well. We also organized the first ever
OCR workshop for Indian scripts. This was a trend
setter of sorts. We laid bare our OCR system in full
detail and others followed. It was perhaps for the first
time that different research groups got to know about
each other’s approach is such great detail. The systems
developed by various groups were demonstrated and
tested publicly. The discussion and debate that
followed helped all the centres to make further
progress. We have been the Indian coordinators for
a Indo-French research network in computational
linguistics and we plan to work more closely with other
groups across the world.
Within the country, we have been maintaining close
technical links with a number of organizations
including the Society for Computer Applications in
Indian Languages, Computer Literacy House,
Computer Vignannam, AP Press Academy, CMC,
Telugu University, Telugu Academy, etc. apart from
commercial companies such as C-DAC and
Modular Infotech.
A large number of very distinguished visitors have
visited our labs. and offered their appreciation as well
as very valuable suggestions.
We have been organizing seminars on a regular basis.
So far more than 25 seminars have been organized.
Some of the distinguished speakers include Prof.
Gerard Huet, Dr. Mark Pedersen and Prof. Rajat
Moona.
5. Publications
1. Chakravarthy Bhagvati, Atul Negi, and
B.Chandrasekhar. Weighted Fringe Distances
for Improving Accuracy of a Template
Matching Telugu OCR System. In Proc. of
IEEE TENCON2003, Bangalore, 2003.
2. Chakravarthy Bhagvati, T.Ravi, S.M.Kumar,
and Atul Negi. Developing High Accuracy
OCR Systems for Telugu and other Indian
Scripts. In Proc. of Language Engineering
Conference, Pages 18-23, Hyderabad, 2003.
IEEE computer society Press.
3. R.L.Brown. The fringe distance measure: an
easily calculated image distance measure with
recognition results comparable to Gaussian
blurring. IEEE Trans. System man and
Cybernetics, 24(1): 111-116, 1994.
4. U. Garain and B.B. Chaudhuri. Segmentation
of Touching Characters in Printed Devnagari
 and Bangla Scripts Using Fuzzy Multifactorial
analysis. In Proc. of Int. Conf. on Document
Analysis and Recognition. IEEE Comp. Soc.
Press, Los Alamitos (CA), USA, 2001.
5. G.Nagy, S.Seth, and M.Vishwanathan.A
prototype document image analysis system for
technical journals. Computer, 25(7), 1992.
6. Atul Negi, Chakravarthy Bhagvati and
B.Krishna. an OCR system for Telugu. In Proc.
Int. Conf. on Document Analysis and
Recognition. IEEE Comp. Soc. Press, Los
Alamitos (CA), USA, 2001.
7. Atul Negi, Chakravarthy Bhagvati, and V.V.
Suresh Kumar. Non-linear Normalization to
Improve Telugu OCR. In Proc. of Indo-
European Conf. on Multilingual
Communication Technologies, pages 45-57,
Tata McGraw Hill Book Co., New Delhi,
2002.
8. K.Wong, R. Casey, and F.Wahl. Document
analysis system. IBM J. Research and
Development, 26(6), 1982.
9. K. Narayana Murthy, B B Chaudhuri (Eds),
LEC-2002: Language Engineering Conference,
IEEE Computer Society Press, 2003
10. K V K Kalpana Reddy and Jahnavi A,
“Text to Speech system for Telugu”, MCA
thesis
11. K. Narayana Murthy, “UNICODE: Issues in
Standardization of Character Encoding
Schemes”, Vidyullipi, April 2002
12. K. Narayana Murthy, Nandakumar Hegde,
“Some Issues relating to a Common Script for
Indian Languages”, International Conference
on Indian Writing Systems and nagari Script,
6-7 february 1999, Delhi University, Delhi
13. K Narayana Murthy, “An Indexing Technique
for Efficient Retrieval fro Large Dictionaries”,
National Conference on Information
Technology NCIT-97, 21-23 December 1997,
Bhubaneswar
14. P R Kaushik, K Narayana Murthy, “Personal
Search Assistant: A Configurable Meta Search
Engine”, Proceedings of the AusWeb99 – The
Fifth Australian World Wide Web Conference,
17-20 April 1999, Southern Cross University,
Australia
15. P R Kaushik, “PSA - A Meta Search Engine
for World Wide Web Searching”, M. Tech.
thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 1998
16. P Naga Samba Siva Rao, “Enhancements to
the PSA model for Meta Search Engines”,
M.Tech. thesis, Department of Computer
and Information Sciences, University of
Hyderabad, 1999
17. K. Narayana Murthy, “MAT2: Enhanced
Machine Aided Translation System”,
STRANSS 2002 Symposiu on Translation
Support Systems, 15-17 March 2002, Indian
Institute of Technology, Kanpur
18. K. Narayana Murthy, “MAT” A Machine
Assisted Translation System”, Fifth Natural
Language Pacific Rim Symposium, NLPRS-99,
5-7 November 1999, Beijing, China
19. K. Narayana Murthy, “UCSG and Machine
Aided Translation from English to Kannada”,
Indo-French Symposium on Natural Language
Processing, University of Hyderabad, 21-26
March 1997
20. Kasina Vamsi Krishna, “ Word Sense
Disambiguation: A Study”, MCA thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 2003
21. D Madhusudhana Rao, V V Raghuram, “A
Generic Approach to Sentence Segmentation”,
MCA thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2003
22. K. Narayana Murthy, “Universal Clause
Structure Grammar”, PhD thesis, Department
of Computer and Information Sciences,
University of Hyderabad 1996
23. K. Narayana Murthy, A Sivasankara Reddy,
“Universal Clause Structure Grammar”,
Special issue on Natural Language Processing
and Machine Learning, Computer Science and
Informatics, Vol. 27, No. 1 March 1997
pp 26-38
131
24. K. Narayana Murthy, “Universal Clause
Structure Grammar and the Syntax of
Relatively Free Word Order Languages”, South
Asia Language Review, Vol VII, No. 1 Jan.
1997 pp 47-64
25. K. Narayana Murthy, “Parsing Telugu in the
UCSG Formalism”, Indian Congress on
Knowledge and Language, January 1996,
Mysore, Vol 2 pp 1-6
26. Sreekanth D, “A Statistical Syntactic
Disambiguation Tool”, M.Tech. Thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 1998.
27. Ashish Gupta, “Improvements to the
UCSG English Parser”, M.Tech. Thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 2001.
28. D. Srinivasa Rao and M. Suresh Babu, “Design
of a Web Based Education Tool”, MCA thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 2000
29. B. P. V. Prasad and A. Rambabu, “Design of
VIRAT: A ProtoPUBLISHER Virtual
Authoring Tool for Web-Based Education”,
M.Tech. Thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2003
30. P. Uday Bhaskar and R. Krishna Kishore, “A
Tool for Web Based Education”, M.Tech.
Thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2003
31. Rajesh V Patankar and Tammana Dilip,
“VIRAT: An Authoring Tool for Publishing on
the Web”, MCA thesis, Department of
Computer and Information Sciences,
University of Hyderabad, 2001
32. T. S. Vivek and J. Reddeppa Reddy, “Web
Based Education Tool – Online Testing &
Evaluation Module, MCA thesis, Department
of Computer and Information Sciences,
University of Hyderabad, 2001
33. P. Siva Rama Krishna and G. Nagachandra
Sekhar, “Web Based Education Tool (Instructor
Module)”, MCA thesis, Department of
132
Computer and Information Sciences,
University of Hyderabad, 2001
34. T. Ramesh and T. Phani Raju, “Web Based
Education Tool (Software Lab Module)”, MCA
thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2001
35. G. Murali Krishna and P. Ravi Kumar,
“Generic Framework for Developing Web-Lab
Experiments”, M. Tech.thesis, Department of
Computer and Information Sciences,
University of Hyderabad, 2002
36. G. Vamsidhar, “White Board Application - A
GUI Tool for Web Based Education Tool”,
M.Tech.thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2002
37. M. Narsimhulu, “An Enhanced Architecture
for Web Based Education”, M.Tech. Thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 2002
38. K Vasuprada, K. Narayana Murthy, “Part-of-
Speech Tagging using a Tri-Tag HMM Model”,
Second National Symposium on Quantitative
Linguistics, 28-29 Feb. 2000, Indian Statistical
Institute, Kolkata
39. K. Vasuprada, “Part of Speech Tagging and
Syntactic Disambiguation using stochastic
parsing techniques”, M.Tech. thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 1999
40. Ravi Mrutyunjaya, “Corpus-Based Stemming
Algorithm for Indian Languages”,
M.Tech.thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2003
41. K. Narayana Murthy, “Theories and
Techniques in Computational Morphology”,
Proc. of the National Seminar on Word
Structure of Dravidian Languages, 26-28
November 2001, Dravidian University,
Kuppam pp 365-375
42. K. Narayana Murthy, “A Network and Process
Model for Morphological Analysis/
Generation”, Second International Conference
 on South Asian Languages ICOSAL-2, 9-11
Jan. 1999, Punjabi University, Patiala
43. Anil Kumar, “Morphological Analysis of Telugu
Words”, M.Tech.thesis, Department of
Computer and Information Sciences,
University of Hyderabad, 2003
44. K. Narayana Murthy, “Electronic Dictionaries
and Computational Tools”, Linguistics Today,
Vol 1, No. 1, July 1997 pp 34-50
45. A Sivasankara Reddy, K. Narayana Murthy,
Vasudev Varma, “Object Oriented
Multipurpose lexicon”, International Journal
of Communication, Vol. 6, No 1 and 2, Jan-
Dec 1996 pp 69-84
46. K. Narayana Murthy, “An Indexing Technique
for Efficient retrieval from Large Dictionaries”,
National Conference on Information
Technology NCIT-97, 21-23 December 1997,
Bhubaneshwar
47. Vasudev Varma, K. Narayana Murthy, A
Sivasankara Reddy, “electronic Dictionaries: A
Model Architecture”, National workshop on
Electronic Dictionaries for second language
learners, Bharatiar University, Coimbatore, 29
Sept. to 7 Oct. 1993
48. Surya Kiran Mamidi, “Developing keyboard
and Display drivers for Indian Language
Editors, MCA thesis, Department of Computer
and Information Sciences, University of
Hyderabad, 2001
49. . Maruti Kumar and Y. N. V. Ganesh Kumar,
“Editor for Indian Languages”, MCA thesis,
Department of Computer and Information
Sciences, University of Hyderabad, 2001
50. V S S Neelima, “Enhancements to the
AKSHARA Text Processing System”, M. Sc.
Computer Science thesis, K R R Vignan
degree and PG College for Women,
Hyderabad, 2003
51. K. Narayana Murthy, “UNICODE: Issues in
Standardization of Character Encoding
Schemes”, Vidyullipi, April 2002
52. Karen Kukich, “Techniques for Automatically
Correcting Words in Text”, ACM Computing
Reviews, Vol 24. No. 4 December 1992
53. Narayana Murthy, “Issues in the Design of a
Spell Checker for Morphologically Rich
Languages”, 3rd International Conference on
South Asian Languages - ICOSAL-3”, 4-6
January 2001, University of Hyderabad.
54. K. S. RajyaShree and K. Narayana Murthy,
“Statistical Spell Checking”, Indo-UK
Workshop on LESAL - Language Engineering
in South Asian Languages”, 23-24 April 2001,
National Centre for Software Technology,
Mumbai
55. K. Narayana Murthy, “Markov Models of
Syllables for Spell Checking”, Statistical
Technqiues in Language Processing, 11-12 June
2001, central Institute of Indian Languages,
Mysore
56. Naresh Pidatal and Dhanjay Kumar Singh,
“Spelling error Detection and Correction“,
MCA thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2003
57. Anil Kumar, “Morphological Analysis of Telugu
Words”, M.Tech.thesis, Department of
Computer and Information Sciences,
University of Hyderabad, 2003
58. Ravi Mrutyunjaya, “Corpus-Based Stemming
Algorithm for Indian Languages”, M.Tech.
Thesis, Department of Computer and
Information Sciences, University of
Hyderabad, 2003
59. K. Narayana Murthy, “Design of a Spelling
Error Detection and Correction System for
Telugu”, National Seminar on Language
Technology Tools: Implementation for Telugu,
September 17-19, University of Hyderabad
60. Thesis in the area of Indian Language
Technologies Thejavath Ramdas Naik and
Manoj Kumar Pradhan, “Off-Line printed
Devanagari page Layout Analysis and Script
Recognition”, MCA Thesis, Department of
Computer and Information Sciences,
University of Hyderabad, June 2000.
61. Buska Krishna, “Design and Implementation
of a Telugu Script Recognition System”,
M.Tech. Thesis, Department of Computer
and Information Sciences, University of
Hyderabad, December 2000.
133
62. L. K. Prashant Kumar, “A Study of School of Physics, University of Hyderabad,
Binarization methods for Optical Character May 2003.
Recognition”, M.Sc.(Tech) Thesis, School 6. The Team Members
of Physics, University of Hyderabad,
December 2000. Dr. K. Narayana Murthy, Dr. Arun Agarwal, Dr.
B. Chakravarthy, Dr. S. Bapi Raju, Dr. Atul Negi,
63. D. Vijaya Bhaskar, “Preliminary Study of Dr. G. Uma Maheshwara Rao, Dr. P. Mohanthy,
GOCR to implement OCR for Telugu”, MCA Dr. P. R. Dadegoankar, Raman Pillai Rajesh, K.
Thesis, Department of Computer and Rajini Reddy, P. Muttaiah, K. Naga Sabita, Ganesh
Information Sciences, University of Raju, Maruthi, Naresh, Suresh Kumar, Anand,
Hyderabad, 1998-2001. Kwaja Sirajuddin, B.Navatha, P. RamaKrishna
Prasad, O. Bhaskar, Vaishnavi, K. Balaji Rambabu,
64. Ajay Koul, “A Study of Gradient Based Feature
Suresh Kumar, P. Siva Ramakrishna, M. Surya
Extraction for Telugu OCR”, M.Tech. Thesis,
Kiran, T. Ramesh, Ravi Raj Singh, Venu Gopal
Department of Computer and Information
Sciences, University of Hyderabad, December Courtesy: Prof. K. Narayan Murthy
2001. University of Hyderabad, Dept. of CIS,
Hyderabad -500046 (RCILTS for Telugu)
65. G. Raghava Kumar, “A Study of Nearest
Tel: 00-91-40-23100500, 23100518
Neighbour Computation Techniques and their
Extn. 4017, 23010374
applications”, M.Tech. Thesis, Department
E-mail: knmcs@uohyd.ernet.in
of Computer and Information Sciences,
University of Hyderabad, December 2001. Editorial Comment : Because of very large number of
publications by the Resource Centre & constraint of space
66. Y. Pradeep Kumar and G. Koteswar Rao, “ A we could not include the publication details here. For getting
Tool-kit for Document Image Processing”, the publications please contact Prof. K. Narayan Murthy
M.Tech. Thesis, Department of Computer
and Information Sciences, University of
Hyderabad, January 2002.
67. V. V. Suresh Kumar, “Non-Linear
Normalization Techniques to improve OCR”,
M.Tech. Thesis, Department of Computer and
Information Sciences, University of
Hyderabad, January 2002.
68. Tanuk Ravi and S. Mahesh Kumar,
“Enhancements and Redesign of DRISHTI,
An OCR System for Telugu”, MCA Thesis,
Department of Computer and Information
Sciences, University of Hyderabad, June 2002.
69. Anand Kumar and Arafath Pasha, “Soft-
ware for OCR”, BE Thesis, Department of
Computer Science, 2001-2002.
70. D. Bhavani, C. Madhuri and S. Shesha Phani,
“GUI for an OCR System”, B. Tech. Thesis,
Department of Computer Science, March
2002,
71. I. N. Lekha, “ An Experimental Evaluation of
Fringe Distance Measures for Malayalam
Printed Text OCR”, M.Tech.(CT) Thesis,

134
 Resource Centre For
Indian Language Technology Solutions – Malayalam
C-DAC, Thiruvananthapuram
Achievements

Centre for Development of Advanced Computing

(Formerly ER&DCI(T)), Vellayambalam,
Thiruvananthapuram, Kerala, India.
Tel. : 00-91-471-2723333 Extn., 243, 303
E-mail : contact@malayalabhasha.com
Website : http://www.erdcitvm.org/hdg/mrc.htm
http://www.malayalabhasha.com
 RCILTS-Malayalam
C-DAC, Thiruvananthapuram
Introduction
C-DAC, Thir uvananthapuram, formerly
ER&DCI, Thiruvananthapuram, is one of the
thirteen Resource Centres (Resource Centre for
Indian Language Technology Solutions) set up
across the country by the Ministry of
Communications and Information Technology,
Govt. of India under the TDIL (Technology
Development for Indian Languages) programme.
These thirteen Resource Centres are aimed at
taking IT to masses in their local languages and
cater to all the constitutionally recognised Indian
and some foreign languages. The Language of
focus at C-DAC Thiruvananthapuram is
Malayalam, the official language of the state of
Kerala.
The main objectives of the “ Resource Centre for
Indian Language Technology Solutions –
Malayalam” (RCILTS-Malayalam) are to build
competence and expertise in the proliferation of
Information Technology using Malayalam, the
regional language of the state of Kerala.
Development of Malayalam enabled core
technologies and products would give a
tremendous fillip to IT enabled services in the
state. The comprehensive IT solutions developed
would enable the citizens of Kerala to enhance
their quality of life using the benefits of modern
computer and communications technology
through Malayalam. This will help them to better
understand their own culture and heritage,
interact with government departments and local
bodies more effectively, besides obtaining a host
of other advantages.
Now that the Resource Centre has completed
three years of functioning, we are happy to note
that we have achieved significant progress and
have been able to complete development of all
the expected core deliverables of the project with
in the scheduled time.
136
We have been successful in developing a variety
of tools and technologies for Malayalam
computerization and taking IT to the common
Malayalee in his local language.
Many of the products developed under the
Resource Centre project are first of its kind and
are significant for enabling Malayalam
computerisation. They have got good market
potential in the present scenario of
computerisation and conversion of official
language to Malayalam in the state of Kerala.
Various Government departments have purchased
our “Aksharamaala” software for Malayalam word
processing. Ezuthachan, the Malayalam Tutor, also
have good demand among non resident
Malayalees. We have already sold 25 copies of the
same and are trying for marketing tie up with
some business houses.
The range of our products include:
Knowledge Resources like Malayalam Corpora,
Trilingual ( English-Hindi-Malayalam) Online
Dictionary and Knowledge bases for Literature,
Art and Culture of Kerala.
Knowledge Tools for Malayalam such as Portal,
Fonts, Morphological Analyser, Spell checker, Text
Editor, Search Engine and Code Converters.
Human Machine Interface systems comprising of
Optical Character Recognition and Text to
Speech Systems
Services like E-Commerce application and E-Mail
Server in Malayalam and Language Tutors for
Malayalam and English.
In addition regular training courses are being
conducted as part of Language Technology
Human Resource Development. Also, there is
regular interaction with the Government of
Kerala for providing solutions in the area of
standardization, computerisation of various
Government departments and converting the
official language to Malayalam. We have been
providing consultancy to individuals and
organisations regarding Language Technology
applications. Given below is a detailed description
of the products developed and the achievements
of the Malayalam Resource Centre. Our Resource
Centre developed the following Technologies/
Products.
1. Human Machine Interface Systems
1.1 NAYANA™ - Optical Character Recognition
System for Malayalam
The Malayalam OCR system converts scanned
images of printed Malayalam documents to editable
text. It is a multi font system that works across a
range of font sizes. The system has a recognition
speed of fifty characters per second.
The System consists of a preprocessing module, the
OCR engine and a post processing module.
The block diagram of the system is given in
figure 1.
Figure. 1
The preprocessing tasks performed by the first
module include noise removal, conversion of grey
scale image to binary, skew detection and correction
and line, word, and character segmentation. The
scanned images in grey tone are converted into two-
tone (binary) images using a histogram based
thresholding approach (Otsu’s algorithm). Skew
detection is done using the Projection profile based
technique. After estimating the skew angle the skew
is corrected by rotating the image against the
estimated skew angle.
The OCR engine (Character Recognition
Module) is based on the Feature Extraction
method of character recognition. Feature
extraction can be considered as finding a set of
vectors, which effectively represent the
information content of a character. The features
are selected in such a way that they help in
discriminating between characters. A multistage
classification procedure is used, which reduces the
processing time while maintaining the accuracy.
After passing through different stages of the
classifier, the character is identified and
corresponding character code is assigned. A
training module is incorporated in the OCR
engine to recognize characters, which are different
from normal characters in their shape and style
(example- decorative fonts).
137
In the post processing module, Linguistic rules Supported output format : RTF/HTML/
are applied to the recognised text to correct ACI/ TXT
classification errors. For example, certain
characters never occur at the beginning of a word Character recognition accuracy (%)
and if found so, they are remapped appropriately. Document Type Good quality Bad quality
Similarly, dependent vowel signs can occur only Paper Paper
with consonants or consonant conjuncts; if found
along with vowels or soft consonants, they are Computer Printed 97% 94%
remapped into consonants/conjuncts similar in Document
shape to the vowel sign. Independent vowels
Magazine 92% 90%
occur only at the beginning of a word and if
found anywhere else, they will be mapped into a Newspaper 85% 82%
consonant or ligature having similar shape.
Books 95% 93%
Performance of the OCR
Table1
Developed on VC++ platform, Our Malayalam
OCR runs on Windows 98/2000. It recognises Extensive testing has been done on approximately
50 characters per second and gives an accuracy of 500 pages of different quality printed documents.
97% for good quality printed documents. The Table1 consolidates the results of testing. The
specifications and performance of the system is system has undergone certification testing at
given below. ETDC Chennai.

Skew detection and : - 5 to +5 degree Applications

correction
The Malayalam OCR can be integrated with a
Supported image formats : BMP, TIFF Malayalam Text to speech system to get a Text
Reading System for the visually challenged. Other
Image scan resolution : 300dpi and above. application areas include publishing sector,
content creation, digital library, corpus
Document Type : Single-font single development etc.
size.
1.2 SUBHASHINI™- Malayalam Text to Speech
Supporting Fonts System (TTS)
Fonts Names : CDAC Fonts The Malayalam Text to Speech system
(ML-TTKarthika, SUBHASHINI™ is a Windows based software,
MLW-TTKarthika) which converts Malayalam Text files into fairly
Mathrubhumi Font, intelligible speech output. The software is
Manorama Font, integrated with a text editor having both ISCII
Fonts used by DC and ISFOC support. The editor supports
Books INSCRIPT Key board layout.

Font Size : 12-20 The TTS is based on Speech synthesis by

diaphonic concatenation and consists of the
Font Styles : Normal, BOLD following four modules
Supported Code format : ISCII/ISFOC • Diaphone Database

138
 The concatenation of diaphones corresponding
to the text is done in the Synthesis module and
we get speech output. We are using the MBROLA
speech engine for speech synthesis.

Applications

Text reading systems, announcement Systems, and

systems providing voice interface.

2. Knowledge Tools

2.1 NERPADAM™ - Malayalam Spell Checker

Nerpadam is a software subsystem that can be

integrated with Microsoft word as a macro or the
Malayalam editor stylepad developed by us, to
check the spelling of words in a Malayalam text
file. While running as a macro in word, it
functions as an offline spell checker in the sense
that one can use this software with a previously
typed text file only. Both off line and online

• Text Processing module

• Prosodic Rules Generator
• Speech Synthesiser

The Diaphone database consists of 2500

diaphones segmented from recorded words. All
the commonly used allophones are also
considered.

The text-processing module organizes the input

sentences into manageable lists of words. It also
identifies the punctuation symbols, abbreviation,
acronyms and digits in the input data and tags
the input data. These are then processed and
converted to phonetic language – a language that
the speech engine is able to recognise.

Rules for adding prosody to the speech output

are generated using the Speech corpus. This
includes the pitch and delay informations for
different intonations.

139
checking are possible when it is integrated with
the text editor. It generates suggestions for
wrongly spelt words.

The system adapts a rule cum dictionary-based

approach for spell checking. It incorporates a fully
developed Morphological Analyser for
Malayalam. This module splits the input word into
root word, suffixes, post positions etc. and checks
the validity of each using the rule database. Finally
it will check the dictionary to find whether the
root word is present in the dictionary. If anything
goes wrong in this checking it is detected as an
error and the error word is reprocessed to get 3
to 4 valid words, which are displayed as suggestion.
The user can add new words into a personalised
data base file, which can be added to the
dictionary if required.

Applications

All Malayalam word processing jobs.

2.2 AKSHARAMAALA™ Malayalam Font

Package and Script Manager
The “AKSHARAMAALA” software package
consists of two sub packages, ”MSM“ a Malayalam
Script Manager and “ Vahini”, a Malayalam font
package. This package complies with the standard
INSCRIPT keyboard layout, Phonetic Keyboard
Layout and the ISFOC standard. This package
enables the use of Windows based applications
for Malayalam data processing. Some of the
packages supported by this application are MS
Office, PageMaker, Adobe Illustrator, MS
Frontpage, Macromedia Dreamweaver Corel
Draw, and Lotus Smart Suite. This package is
intended for use under Windows 95, 98, NT,
2000, ME and XP.
The “MSM” package consists of a keyboard
manager, which supports the INSCRIPT
keyboard overlay and a Phonetic keyboard overlay
for the entry of Malayalam characters. The
Manager can optionally switch on the entry of
Malayalam or English characters with the help of
a switchover key. The appropriate key
combinations automatically render the
Malayalam characters, such as conjuncts and soft
consonants. The keyboard manager is designed
to work with Malayalam ISFOC, monolingual as
well as bi-lingual fonts. This manager also supports
the use of Web fonts for data entry. Different
options are provided in the package that turns on
these features.
The “Vahini” package is a collection of Malayalam
fonts that can be used for word processing, web
publishing, data processing etc. and for use in MS-
Windows applications. There are number of high
quality True-type fonts . The fonts comply with
the ISFOC standard and consist of both
monolingual and bilingual fonts. A few web fonts
are also provided for use in web pages. The bi-
lingual fonts are those that contain both
Malayalam and English characters. The webfonts
are monolingual TTF fonts specially designed for
the use with browsers.

140
We have also developed Dynamic (PFR) fonts,
which can be used in web page development so
that the user can view the web content without
the font being actually installed in the machines.
These fonts are platform independent and will
work in both I.E and Netscape.
A Unicode font in open Type format is also
available.
Government of Kerala is considering the
AKSHARAMAALA package to teach Malayalam
Data entry to the beneficiaries of “Akshaya”, a
project aimed at providing Computer literacy to
at least one person per family and also in the E-
Governance applications of the government of
Kerala.
The AKSHARAMAALA package can be used with
standard content creation tools to develop
contents in Malayalam.
MLBW (ISFOC) ↔ ISCII
MLW (ISFOC) ↔ ISCII
MLTT (ISFOC) ↔ ISCII
2.5 ANWESHANAM™ – Malayalam Web based
Search Engine
“Anweshanam” is a directory based search engine,
which searches for Malayalam content and
information in web pages. This solution provides
a Malayalam interface that helps the user to search
for information quickly and easily on the web. It
searches for specific Malayalam keywords and
generates a list of links to web pages containing
the searched information. It searches for specific
content in the pages on the server. This solution
was developed using Java Server pages.
The interface is in Malayalam and includes easy
input facilities with keyboard driver and floating

2.3 Text Editor: A basic Malayalam Text Editor

“STYLEPAD” is developed. It incorporates all the
facilities available in Notepad together with
provision to save Malayalam documents in ISCII
format and read ISCII files.

2.4 Code Converters: Content creation in

Malayalam is accomplished using different fonts
by different organisations/individuals. Most of the
On-line Malayalam Newspapers have their own
proprietary fonts. Lack of standards in font coding
makes data retrieval a difficult task. In order to
alleviate this task, we have developed Font
Converters for most of the commonly used
Malayalam fonts. Given below is a list of the Font
Converters developed by us.

MATWEB ↔ ISCII

THOOLIKA ↔ ISCII

SHREELIPI ↔ ISCII

AKRUTI ↔ ISCII

TULASI ↔ ISCII

141
character map. The keyboard driver supports the
INSCRIPT keyboard with automatic formation
of conjuncts. The application provides facilities
for searching either English or Malayalam
keywords. The result pages contain links to the
web pages containing the searched information
along with their brief description in Malayalam.
A form is also provided whereby new pages and
sites can be added to the database with details of
the content.
The application can be expanded to function as a
multilingual search engine and also as a meaning
based search engine.
This application is beneficial to Multilingual portal
developers and is useful for any one looking for some
particular Malayalam content on the web.
This application is presently being run as a
service on our website and portal,
www.malayalamresourcecentre.org.
142
2.6 Malayalam Portal
We have designed and developed a Malayalam Web
portal, www.malayalamresourcecentre.org alias
www.malayalabhasha.com. Some of the facilities/
contents provided in the portal are:
• Malayalam version of the Constitution of
India.
• A Newspaper “Pradeepam”, published from
Kozhikode.
• A knowledge base of traditional Kerala Art
forms and Culture (Both English and
Malayalam)
• A Knowledge base of Malayalam Literature.
• Malayalam Literary classic “KRISHNAGATHA”
by Cherusseri and the Grammatical classic,
“KERALAPANINEEYAM” by A.R. Rajaraja
Varma.
• Full text of Sanskrit Ayurvedic Classics
Charakasamhita and Susrutasamhita
(transliterated in Malayalam and ten other
languages), along with their Malayalam
interpretations.
• PRAKES (Prakruthi estimate) - An Interactive
software package for estimating the Prakruti
(constitution) of a person based on Ayurvedic
concepts. Available in both English and
Malayalam.
• A database of forms commonly employed by
Govt. of Kerala (City Corporations, Motor
Vehicles, Revenue, Civil Supplies Depts., etc.)
Altogether 76 forms from different
government departments put on the web.
• SSLC (Mathematics and Science) Question
papers and Answers for the last five years in
Malayalam.
• A knowledge base in Malayalam for rubber
cultivators.
• A tourist aid package named “Explore-Kerala”. The
site, http://www.malayalamresourcecentre.org/wap/
can be accessed from WAP enabled
mobile phones.
• Malayalam brochures on Cancer Awareness
In addition, some of the tools/technologies
developed (Sandesam, Anweshanam,
Dictionary and E commerce application) have
been integrated in the portal for
demonstration.
The contents of the website are continuously
being upgraded and enhanced.
The number of visitors to the site has exceeded
20,500 since June, 2002.
143
3. Services
3.1 SANDESAM™ - MALAYALAM E-MAIL
SERVER
Sandesam is a solution for a Web based mail
service in Malayalam. The back-end mail server
comprises of, or is based on the Qmail, Vmailmgr
and Courier-IMAP running on Redhat Linux.
The Qmail server supports the SMTP and POP3
services, the Vmailmgr program performs the
management of user accounts effectively, while
the Courier –Imap server supports the IMAP
service. The web interface is developed using Java
Server Pages and the Java mail API and does not
use Java on the client side. The database server
used is Postgres SQL
This is a lightweight and a fast solution to provide
an easy-to-use web interface in Malayalam. The
storage of mailboxes is based on maildir structure
144
and the mail is read directly from the disks. The
administration of user accounts is done through
the Vmailmgr program. Authentication is also
done via Vmailmgr for both Qmail and Courier
servers. The service gives the user complete access
to his POP3 or IMAP mailboxes via an easy-to-
use web interface.
Some of the facilities provided in the service are
the IMAP support with user manageable folders,
extensive mime support for attachments.
The interface is in Malayalam and includes
easy input facilities with keyboard drivers and
floating character maps. The service provides
facilities for sending and receiving mail in
Malayalam, storage of addresses in address book
with Malayalam names and description.
The service also provides facilities for user
configuration like changing the password, setting
quota for mailboxes etc.
This solution can be expanded to support any
IMAP client mail server running on Linux or
Windows platform.
This solution is beneficial for Small and medium
ISPs, business organizations, Government
Departments Multilingual portal developers.
This solution is presently being run as a service on our
website and portal, www.malayalamresourcecentre.org
with the domain id sandesam.com.
3.2 Malayalam E-com Application
An e-commerce application in Malayalam has
been developed to help computer literates not that
proficient in English to purchase goods online.
This solution is developed using Java Server pages.
This solution is simple using pure html and JSP
and is viewable in any browser.
through the Internet. It is useful for multilingual
portal developers and is beneficial to computer
literates not that proficient in English but familiar
with Malayalam.
The application has been presently employed in our
website and portal www.malayalamresourcecentre.org for
the sale of the products developed at the centre.
4. Knowledge Resources
4.1 Trilingual Dictionary
An Online Trilingual (English-Hindi-Malayalam)
Dictionary is developed. It Contains 50,000 plus
words in each Language. English based Search and
advanced search facilities are implemented. Search
based on the other two languages being
implemented.

The web interface is in Malayalam and basically

contains a window with display of products for
sale with their descriptions in Malayalam. A
shopping cart is provided whereby goods to be
purchased can be added and removed according
to will. The online bill for the cart with price
details can be viewed during the process and then
the purchase can be finalized by filling up and
submitting an order form. The payment can be
made by cheque or DD giving its details in the
order form and sending the instrument by post.

The application also considers inventory details

by displaying available stock, automatically
updating the stock after each purchase and
prompting certain functions on reaching certain
limits. The facilities provided in the web interface
are easy input facilities like keyboard driver and
floating character map for the entry of Malayalam
data and acknowledgement of the purchase order
by e-mail.

The application can be extended to support secure

online payment with credit cards.

The application is useful for small or medium

organizations to increase their sales coverage The main features of the Dictionary are:

145
• Portable in XML format.

• ISCII Based.

• Retrieval based on Parts of Speech

• (POS)

• Word search can be made in all three languages.

(Implementation in progress)

• Description of every word with example.

• Advanced search facilities.

• Extremely fast processing.

The Dictionary can be integrated with any other

application or web portal. It can also be used as
an aid for translation – both Machine aided and
manual.

It helps the study of the concerned languages with

relative ease.

5. Language Tutors

5.1 Ezhuthachan- The Malayalam Tutor

A Malayalam Tutor Package,
“EZHUTHACHAN”, which is aimed at teaching
Malayalam to foreigners and second generation
Keralites living abroad. It is basically a multimedia
package with animations, which show the method
of writing the letters and sound giving the
pronunciation of characters and words. A writing
pad is also provided, where a shaded model of
each character appears and the user can practice
writing over this using the mouse. This application
lists the commonly used expressions in Malayalam
(words for daily use, knowing somebody,
numbers, days, months, colours, animals, birds
etc.), with their pronunciation and English
meaning. The contents are formatted into well
structured chapters. A test module is also provided
at the end. The package also contains an English-
Malayalam Dictionary of 2000 words.
The Honourable Minister of State for Urban
Development & Poverty Alleviation, Shri. O.
Rajagopal (presently Minister of state for defence)
formally released the package on 19th October,
2002. The Chief Executive of Non Resident
Keralites Welfare Association (NORKWA), Shri.
Satish Namboothiripad, IAS received a copy of
EZHUTHACHAN from the Honourable
Minister. NORKWA is considering the
“EZHUTHACHAN” package for their
programme of on-line teaching of Malayalam for
Non resident Malayalee children. We are receiving
lot of enquiries and have already sold some copies
of the software.
5.2 English Tutor
English tutor is a multimedia-based application
intended to help, nursery,/primary school students
or in general, any one interested in learning
English, to learn the basics of English through
Malayalam. The contents of the application are
organised into different modules that comprise
of interactive learning programmes to develop

146

skills for reading and writing alphabets, words and
sentences. The animations, pictures and sounds
teach the association of objects and words along
with their spelling and pronunciation
6. Other Activities
6.1 Providing Technology Solutions
C-DAC, Thiruvananthapuram has entered into
a contract with Govt. of Kerala to be a Total
Solution Provider for IT implementation in
Government. The Government has already
implemented “Project Grameen” for effective
dissemination of IT to grass root level. C-DAC,
Thiruvananthapuram together with Library
Council has setup Community Information
Centres in fourteen districts in Kerala. C-DAC,
Thiruvananthapuram is the executing agency of
the E-Governance projects of Government of
Kerala.
6.2 Interaction with State Government
We have been active member of state-level Expert
Committee for standardisation of Malayalam
Keyboard and Character Encoding. The same
committee has recommended the modifications
to be made in the Malayalam UNICODE. The
Committee has submitted its final report and the
Government have approved it.
Recently Government of Kerala has launched a
programme called “Akshaya” which is aimed at
making at least one person per family in Kerala
computer literate. C-DAC, Thiruvananthapuram
is a member of the committee setup for evaluating
the study material for this programme.
C-DAC, Thiruvananthapuram is Member of the
General Council, IT@School Project of the
Government of Kerala.
6.3 Training Activities
C-DAC, Thiruvananthapuram, together with
Centre for Development of Imaging Technology
(C-DIT) has conducted a two day work shop on
Font Design in April 2002.
The ongoing computerisation of various state
government departments, along with increased
use of Malayalam as official language has created
a need for a large number of computer literate
manpower with proficiency in Malayalam data
processing. A module on Malayalam word
processing tools has been incorporated into the
PGDCA course and other public and corporate
training programmes offered by C-DAC,
Thiruvananthapuram.
6.4 COWMAC
C-DAC, Thiruvananthapuram has formed a
COnsortium of Industries Working in the area
of Malayalam Computing (COWMAC). The
Consortium aims at mutual interaction among
the participating agencies, so that standardisation
in the areas of font coding, IT Vocabulary,
Keyboard overlay, transliteration schemes etc. can
be done effectively. It also aims at avoiding
duplication of development activities by way of
closer interaction between developers.
147
7. Publications 9. Future Plans
1. Malayalam Spell Checker - presented in the We plan to take up development of the following
International Conference on “Universal products as second phase of the Resource Centre
Knowledge and Language” in Goa in Project.
November 2002.
• Online Character Recognition System for
2. Optical Character Recognition System for Malayalam
Printed Malayalam Documents and
• Lexical Resources for Machine Translation
3. Text to Speech System for Malayalam –
presented in the SAP workshop at the Centre • Speech to Text
for Applied Linguistics and Translation • Malayalam Word Net and
Studies(CALTS) in the University of
Hyderabad, in March 2003. • Porting of the various applications developed
to Linux platform.
4. Text Reading System for Malayalam - Selected
for presentation in the International 10. The Team Members
Conference, “Information Technology :
Ravindra Kumar R ravi@erdcitvm.org
Prospects and Challenges in the 21st Century”
in Kathmandu, Nepal in May, 2003. Sulochana K.G sulochana@erdcitvm.org
Jithesh .K jithesh@erdcitvm.org
We have also prepared the final proposal for
modifications to be made in the next version of Jose Stephen jose_stephen@erdcitvm.org
Malayalam Unicode and the Malayalam Design Santhosh Varghese santhosh@erdcitvm.org
guide for TDIL. Shaji V.N shaj@erdcitvm.org

8. Expertise Gained Vipin C Das vipincdas@erdcitvm.org

Keith Fernandes keith@erdcitvm.org
The Malayalam resource Centre consists of a core
team of Design engineers, programmers and Sanker C. S sankercs@rediffmail.com
linguists proficient in natural Language Sunil Babu sunilbabu_ktr@yahoo.com
processing. We have gained expertise in the areas Hari Kumar karamanahari@rediffmail.com
of Image Processing, Speech Synthesis, Font Neena M.S neenams@rediffmail.com
Design, Phonology of Malayalam language and
Praveen V.L praveenvl@erdcitvm.org
Morphological Analysis of Malayalam language.
Mithra R.S. shammymithra@rediffmail.com
We have also built up technical capability in the
Praveen Kumar praveenvijay@rediffmail.com
use of Data base packages like MS Access , XML,
Postgre SQL Scripts, Languages and tools : Hari .K harichithira@yahoo.com
HTML, Java Server Pages, Java , and JavaScript, Dr. V.R Prabhodhachandran Nayar pnayar@email.com
VC++, C++, VB, HTML, DHTML, Diaphone
Dr. Usha Nambudripad shanambudripad@yahoo.co.in
Studio, Macromedia Flash, C-DAC Iplugin &
Leap Office, Macromedia Fontographer and Courtesy: Prof. Ravindra Kumar
Bitstream Webfont wizard 1.0 C-DAC, Vellayambalam
Expertise in configuring server side soft wares are Thiruvananthapuram-695 033
also available at the Centre. (RCILTS for Malayalam)
Tel: 00-91-471-2723333, 2725897, 2726718
Web Servers : Linux Apache Server, Tomcat server E-mail : ravi@erdcitvm.org
Mail Server : Qmail, Send mail

148
 Resource Centre For
Indian Language Technology Solutions – Tamil
Anna University, Chennai
Achievements

School of Computer Science and Engineering

Anna University, Chennai-600025 India
Tel. : 00-91-44-2351265 Extn : 3340
E-mail : rctamil@annauniv.edu
Web Site: http://annauniv.edu/rctamil
 RCILTS-Tamil The following activities have been undertaken for
Anna University, Chennai development of knowledge resources.

Introduction " Online Dictionary

The Resource Center for Indian Language " Corpus Collection Tool
Technology Solutions – Tamil, Anna University
" Content Authoring Tool
has been actively working in the area of language
technology with special emphasis on the following " Tamil Picture Dictionary
areas:
" Flash Tutorial
• Linguistic Tools
" Handbook of Tamil
• Language Technology Products
" Karpanaikkatchi - Scenes from Sangam
• Content Development literature
• Research On Information And Knowledge " District Information
Extraction
" Educational resources
Our website http://annauniv.edu/rctamil
showcases the highlights of these activities. " Appreciation of Tamil poetry
Downloadable demos are available for selected
products. " Geography for You

A detailed report on the activities of the center is " Chemistry for X Standard
presented below. 1.1 Online Dictionary
1. Knowledge Resources Online Dictionary is a language-oriented software
for the retrieval of lexical entries from
monolingual and bilingual dictionaries by many
users simultaneously. It has been designed to
interact with various language-processing tools –
morphological analyzer, parser, machine
translation etc.

The Tamil dictionary contains 20,000 root words.

Each entry in the dictionary includes the Tamil
root word, its English equivalent, different senses
of the word, and the associated syntactic category.
The root words are classified into 15 categories.

The special features of this Online Dictionary are

the following :

• It is an inflectional or derivational dictionary.

Screen shots of Knowledge Resources
That is the search can be based on an inflected
word. Given an inflected word, it calls the
Morphological analyzer to get the root word
and searches the dictionary. In addition to
providing information about the root word

150
 from the dictionary, it also provides the
analyzed information about the inflection.
• It supports search based on syntactic category.
• It also supports search based on parts of a word.
• Tamil equivalent words of given English words
can also be obtained.
Status : Dictionary is available with 20,000 root
words. In effect, since it is an inflectional dictionary,
it is possible to find the meaning of over 1 lakh
words. It is to be enhanced to 35,000 root words.
1.2 Corpus Collection Tool
A corpus with 12 lakh words has been built from
Tamil news paper articles, using a Corpus
Collection tool. This tool automatically collects
articles from different web-sites and stores them
into a corpus database. The corpus database is
organized into a word database and a document
database. The databases have the following
information : the article, category of the article,
key words, title, date, author, source, unique
words and their frequencies. The tool collects
articles available in different fonts, and converts
them to a standard font, before storing them in
the database.
Status : Available for use with 12 lakh words.
Collection is an on-going process.
1.3 Content Authoring Tool
This is a tool used to create and deliver content,
integrating text, sound, video, and graphics. The
design of the tool is generic so that it can be
adapted to any domain. The information can be
organized hierarchically with multiple levels.
Content can be updated at any time, with new
levels being added. There are primarily two
different modes in which this operates – authoring
mode, used by the content developer to organize
and present the content, and the viewing mode
used by the end-user to view the contents. The
authoring mode stores the content in a database,
and the viewing mode serves the pages using JSP.
The user can select the sections that he/she wants
to view.
Status : Testing in progress.
1.4 Tamil Picture Dictionary
This is a picture book for children. It has about
three hundred words - organized as nouns and
verbs. Each word has on associated picture, an
explanation, the English equivalent and a small
poem to illustrate its meaning.
Status : Available on our website with 300 words.
Audio capability is to be added.
1.5 Flash Tutorial
This tutorial in Tamil, provides an easy way to
learn Macromedia Flash, a content development
tool. Flash is the most popular multimedia tool
to enhance or design web pages or CBT packages.
This tutorial contains a set of technical keywords
along with the content. It starts with the basics of
Flash and methodically moves up to an advanced
level. The components available in the toolbox
are explained. Creation of movies and animated
images are also explained. The major concepts like
Tweening and Symbol conversion are discussed
in detail. Snapshots are provided wherever
necessary. Plenty of examples are also provided.
The English equivalent for the keywords are given
within brackets.
Status : Available on our website
1.6 Handbook of Tamil
This e-handbook gives an overview and special
features of Tamil grammar and literature. The
handbook is divided into four sections -
Introduction, Literature, Grammar and others.
In the ‘introduction’ section, history of Tamil
language and details about the Tamil speaking
people are given. The second section provides an
overview of Tamil literature starting from the
Sangam age up to the modern times. The grammar
section outlines the classification schemes used in
Tamil grammar and highlights important features.
The last section provides miscellaneous
information such as names of days, months,
measures, traditional art forms etc. in Tamil.
Status : Available on our website
151
1.7 Karpanaikkatchi - Scenes from Sangam
Literature
This is a visualization of scenes from Sangam
literature. Selected poems depicting various
moods pertaining to Sangam period have been
chosen. The poems, their interpretation and
specially created imagery picturising the mood
conveyed by these poems, have been presented.
Status: Development in progress
1.8 District Information
• Gives statistical, informative and special
features about districts of Tamil Nadu
• User can browse through the information
district wise or subject wise
Status : Under development
1.9 Educational Resources
• Appreciation of Tamil Poetry
• Geography for You
• Chemistry for X Standard
1.9.1 Appreciation of Tamil Poetry
The aim of this package is to introduce the
nuances of Tamil poetry to young children
through music, picture, animation and games.
This package consists of 25 lessons that take the
children from the basics of Tamil poetry to an
advanced level of appreciation. Each lesson
consists of the poem, segmentation of the
compound words, word-by-word meaning,
interpretation of the poem, related poems, and
author information, and exercises.
1.9.2 Geography for You
This package is available both in Tamil and in
English. The aim of this package is to
systematically introduce the concepts of
Geography, catering to students starting from
primary to higher secondary stages. A total of 33
lessons have been organized accordingly. The
topics covered include atmosphere, universe, solar
system, earth, natural regions of the world, and
152
the continents. Images, audio, and animation have
been added to make this appealing to students.
1.9.3 Chemistry for X Standard
This package has been specifically targeted at tenth
standard students. Here again the focus is on
presenting concepts, and it is presented both in
Tamil and English. The package consists of eight
topics – periodic classification, atomic structure,
chemical bonding, phosphorus, halogens, metals,
organic chemistry, and chemical industry. In
addition to innovatively using images, and audio,
the highlight of this package is simulation of
experiments using animation.
2. Knowledge Tools
The following tools have been developed for
language and information processing.
• Language Processing Tools
• Morphological Analyser
• Morphological Generator
• Text Editor
• Spell Checker
• Document Visualization Tool
• Utilities
• Code Conversion Utility
• Tamil Typing Utility
2.1 Language Processing Tools
Tamil like many other Indian languages is a
morphologically rich language. Formation of
words occurs by means of agglutination. Thus
each word consists of a root word and one or
more suffixes. Hence morphological analysis of a
word and morphological generation of a word
are primary and challenging tasks. Hence, the core
of the language processing tools developed are the
Morphological Analyser and Morphological
Generator. They are used to build other language
processing tools and products, like parser,
translation system, spell checker, search engine,
dictionary etc.
• Morphological Analyser
The morphological analyser tool takes a derived
word as input and separates it into the root word
and the corresponding suffixes. The function of
each suffix is indicated. It has two major modules
– noun analyser and verb analyser. A rule-based
approach is used. The number of rules used is
approximately 125. Heuristic rules are used to deal
with ambiguities. A dictionary of 20,000 root
words is used. The design of the analyser is such
that the number of times the dictionary is accessed
is minimized. The steps involved are as follows:
1. Given an input string, it starts scanning the
string from right to left to look for suffixes. A
list of suffixes is maintained.
2. It searches for the longest match in the suffix
list.
3. It then removes the last suffix and determines
its tag and adds it to the word’s suffix list.
4. It checks the remaining part of the word in the
dictionary and exits if the entry is found.
5. According to the identified suffix, it generates
the next possible suffix list.
6. The process repeats from step 2, with the
current suffix list.
Screen shot of Morphological Analyser
Status : Currently, in the testing phase. A
preliminary version with over 70% accuracy is
available. It is being enhanced by adding rules, to
improve the accuracy.
• Morphological Generator
The morphological generator does the inverse
process of the morphological analyser. It generates
words when Tamil morphs are given as input. It
also has two major modules: - noun generator and
verb generator. In the noun section, a root noun,
plural marker, oblique form, case marker and
postpositions are given as inputs. In the verb
section a root verb, tense markers, relative
participle suffix, verbal participle suffix, auxiliary
verbs, number, person and gender marker are
given as inputs. At a time, four auxiliary verbs
can be added to the main verb. In addition to
these, adjectives and adverbs are also handled.
It generates the word using the internal sandhi
rules and the given inputs. It uses 125
morphological rules. For a given verb, with all
possible combinations of tense markers and person
number gender markers nearly 30 forms can be
generated. With the auxiliary verbs added to the
main verb, about 200 verb forms can be
generated. In the same way for a given noun, with
all possible combinations of case markers and
postpositions nearly 68 forms can be generated.
Screen shot of Morphological Generator
Using this generator, an interactive sentence
generator has been designed. Given the root noun
and associative number, case, adjective,
153
postposition, etc., and the root verb and the equivalent letters and checked against the
corresponding auxiliary, tense, adverb, dictionary.
postposition, etc, this tool will generate
morphologically and syntactically correct 3. Adjacent key errors : Adjacent keys are based
sentences. In order to generate a simple sentence on Tamil 99 standard keyboard. In this case,
at least a subject, verb and the tense should be each letter is replaced with adjacent keys and
given. checked against the dictionary.
After the correction phase, all the corrected case
Status : Preliminary version is available. It is being
endings, PNG markers, root words are given to
enhanced by adding rules to take care of special cases.
the suggestion generation process. The spell
2.2 Text Editor checker makes use of the Morphological generator
to generate all possible suggestions. User has the
The Tamil text editor is aimed at naïve Tamil provision to select the suggestion among the list,
users. It is platform-independent ( works on both ignore the suggestion or add the particular word
Windows and Linux machines). It provides basic to the dictionary.
facilities for word processing both in Tamil and
English. It doesn’t have any special file format.
Files of both rich text format and text-only
formats can be created or edited.
Status : Available for use. Can be downloaded from
the web site.
2.3 Spell Checker
Tamil Spell Checker is a tool used to check the
spelling of Tamil words. It provides possible Screen shot of Tamil Spell Checker
suggestions for the wrong words. This spell
checker is based on the dictionary, which contains Status : Version 1.0 is available. Will be enhanced
Tamil root words. At present the size of the by increasing the root-word dictionary size to 35000.
dictionary is 20,000 words.
2.4 Document Visualization
The spell checker uses the morphological analyser
Document visualization aims at presenting
for splitting the given Tamil word into the root
abstracted information extracted from raw data
word and a set of suffixes. If the word is fully
files visually. Instead of forcing a user to read a
split by the morphological analyser, it is assumed
whole document just to grasp certain information,
that the given word is correct. Otherwise it goes
document visualization offers very important clues
for the correction process. The following types
about the document. It is a tool that is capable of
of errors are handled :
visualizing any document.
1. Errors in case endings and PNG markers: In
The tool has various control points for the user
this case, if there are any errors in the case
to specify his options. The tool takes any tagged
endings like il, aal, iTam, uTan, uTaiya, aan,
document as input and extracts the tag
aaL, arkaL, etc …, they are corrected and given
information and uses that for visualization. The
to the suggestion generation phase.
various views presented are scatter plot view, to
2. Errors due to similar sounding characters, (eg. show relative tag positions in a two dimensional
manam, maNam,palam, paLam, pazham, etc.) plane, document zoom view, to show thumb views
: In this case, if any letter in the erroneous word of pages, statistical information view, to show
can have similar sounds, then it is replaced with statistics about input document and synchronous

154
depth view. These views together make up the
entire tool.
Scatter Plot view : Here the document is mapped
on to a graph. This view gives the relevant position
of the tag in a particular page. It normalizes the
tag position according to the display. The position
of the tags is marked with circles of predefined
size and color. Their shapes distinguish nested and
normal tags. Apart from displaying the tags, it
gives the relevant information about a particular
tag in a separate screen, when the user clicks on it
and, if the selected tag is of nested type, it also
drops a line from the starting position to the end
of the tag.
Zoom view : The document zoom view prepares
the intermediate file, which has a proper
alignment of certain number of lines per page.
Each page is shown as a small thumbnail. The user
can click on the appropriate page to view that
page. Search can be done in zoom view.
Statistical Information view : This view presents
the frequencies of tags identified in the input file
in a bar chart form. It is used to show the legend
about the tags present in the document.
Synchronous Depth view : This view projects the
input document onto the 3-dimensional plane.
The whole document is considered to be an object
in 3- dimensional plane. This view helps to get
the picture of how the tags spread over the
document.
Display of Zoom and Scatter Ploter View
Status : A demo version is available. Currently being
tested.
2.5 Utilities
In order to facilitate language processing in Tamil,
a set of common utilities have been developed.
These include Code conversion utility, and Tamil
typing utility.
Code Conversion Utility
There are a plethora of fonts in use on various
web sites in Tamil. Hence a generic code
conversion utility has been developed. Given a
source and a target-coding scheme, this utility will
generate the code for the conversion. This code
can be integrated as part of any language
processing tool or application. Existing standard
coding schemes are already built in. Any other
coding scheme has to be explicitly specified.
Status : Available for use.
• Tamil Typing Utility
This is an API, which indirectly serves as a
keyboard driver. It allows the user to type the
Tamil text in both transliterated mode and using
the Tamilnet99 standard keyboard. This can be
included as part of any language processing
application that requires Tamil input.
Status : Available for use.
3. Translation Support Systems
The following are the components that have been
developed to aid machine translation :
• Tamil Parser
• Universal Networking Language (UNL) for
Tamil
• Heuristic Rule Based Automatic Tagger
3.1 Tamil Parser
The Tamil parser identifies the syntactic
constituents of a Tamil sentence. It outputs the
parse tree in a list form. The free word order
feature of Tamil makes parsing a challenging task,
155
since there is a need to associate and link syntactic
components that are not always adjacent to each
other. This is done by associating positional
information with the words. The parser tackles
both simple sentences having a verb, a number
of noun phrases, and simple adverbs and
adjectives, and complex sentences with multiple
adjectival, adverbial, and noun clausal forms.
The processing uses a phrase-structure grammar
together with a limited amount of look-ahead to
handle free word order. The words of the sentence
are first analyzed using the morphological
analyzer, and the root word along with the part-
of-speech tag and suffix information obtained is
input to the parser. However, the analyzer does
not unambiguously give the part-of-speech
information for all words. In these cases, linguistic
based heuristic rules are used to obtain the
associated tags. Currently, 15 heuristic rules are
used. For sentences with multiple clauses, the
parser first identifies the cue words of the clauses
and syntactically groups the clauses based on the
cue words and phrases.
Screen shot of Tamil Parser
Status : At present it can handle simple sentences, and
complex sentences with a noun clause, with multiple
adjective clauses, with multiple adverb clauses, and
both multiple adjective and adverb clauses.
3.2 Universal Networking Language (UNL) for Tamil
Universal Networking Language (UNL) is an
intermediate representation for inter-lingua based
156
machine translation. The Universal Networking
Language (UNL) is an intermediate language
based on semantic network, and used by machines
to express and exchange natural language
information.
UNL provides sentence-by-sentence
representation of the semantic content. Sentence
information is represented as a hyper-graph having
Universal Words (UWs) as nodes and binary
relations as arcs. Binary relations are the building
blocks of UNL sentences. They are made up of a
relation and two UWs. Each relation is labeled
with one of the possible label descriptors.
Relations that link UWs are labeled with semantic
roles of the type such as agent, object, experiencer,
time, place, cause, which characterize the
relationships between the concepts participating
in the events. This project focuses on developing
the Tamil EnConverter (Tamil to UNL format)
and Tamil DeConverter (UNL format to Tamil).
Both the EnConverter and the DeConverter use
language dependent rules, and a language
dictionary that contains Universal Word (UW)
for identifying concepts, word heading and
syntactical behavior of words.
An EnConverter is a software that automatically
or interactively enconverts natural language text
into UNL. EnConverter is a language dependent
parser that provides synchronously a framework
for morphological, syntactic and semantic
analysis. EnConverter uses the following sequence
of operations:
• Convert or load the rules
• Input a Tamil sentence
• Apply the rules and retrieve the word
dictionary
• Output the UNL expressions
EnConverter analyses a sentence using the word
dictionary, knowledge base, and enconversion
rules. It first uses the morphological analyser to
analyze the given word. It retrieves relevant
dictionary entries from the language dictionary,
operates on nodes in the Node-list by applying
enconversion rules, and generates semantic
networks of UNL by consulting the knowledge
base.
A DeConverter is software that automatically or
interactively deconverts UNL into natural
language text. Tamil DeConverter uses the
morphological generator to generate the Tamil
sentence.
The information needed to construct the UNL
structure and generate Tamil sentence from UNL
structure is available at different linguistic levels.
Tamil being a morphologically rich language
allows a large amount of information including
syntactic categorization, and thematic case
relation to be extracted from the morphological
level. Endings in Tamil words with syntactic
functional grouping and semantic information are
used to identify the correct binary relation. For
syntactical functional grouping, information
about relating concepts like verbs to thematic
cases, adjectival components to nouns and
adverbial components to verbs are needed.
Syntactical functional grouping, has been done
by the specially designed parser, taking into
consideration, the requirements of the UNL
structure.
Status : At present, all the relations have been
identified and simple sentences can be handled.
3.3 Heuristic Rule Based Automatic Tagger
This tagger automatically tags the words of a given
document with parts of speech tags. It uses
linguistically oriented heuristic rules for tagging.
It does not use the dictionary and the
Morphological Analyser.
The part of speech (POS) tagger uses heuristic rules
to find the nouns, verbs, adjectives, adverbs and
postpositions. This is done, by checking the
appropriate morphemes of the words. Clitics are
removed from the word before it goes through
rule verification.
Some of the heuristic rules used are as follows.
1. If a word contains PNG markers and tense
markers, then it is tagged as verb.
2. If a word contains case marker then it is tagged
as noun.
3. Standalone words : Some commonly used
words are categorized and stored in lists. If a
word belongs to a particular list then the word
is tagged with that category.
4. Fill in rule: If the unknown word comes in
between two nouns, then it is tagged as a noun.
5. Verb Terminating: If the unknown word comes
at the end of a sentence, then it is tagged as a
verb. This is due to the fact that normally Tamil
sentences end with a verb.
6. Bigram : The unknown word is identified using
the category of the previous word.
Status : About 83% accuracy is obtained when tested
on a sample corpus.
4. Human Machine Interface Systems
Two packages that serve as human-machine
interfaces, have been developed. They are :
• Text To Speech (Ethiroli)
• Poonguzhali (A chatterbot)
4.1 Text To Speech (Ethiroli)
Ethiroli is a Text to Speech engine for Tamil. The
engine has been so designed that it can be plugged
into any application requiring a text to speech
component. When Tamil text is given as input, it
is pre-processed, and transliterated using linguistic
rules, to a phonetic representation in order to
remove homographic ambiguities. Homographic
ambiguity arises in Tamil since the same character
can have different sounds depending on the
position in which it occurs. For example, the
vallinum letters (è, ê, ì, î, ð, ø) in Tamil have more
than one sound based on their position of
occurrence within a word. Each word in this
transliterated text is then split into its
corresponding phonemes using the cvcc model.
Once the phonemes are identified, the sound files
corresponding to the phonemes are concatenated
and played in sequence to give the speech output.
157
Currently, 4000 syllables are used to generate
Tamil speech using this method.
Status : A preliminary version is available. Efforts
are on to improve the quality of speech produced.
technical terms. The input key is entered with its
response and alternate response in the knowledge
base. The input key can be more than one word
with the ‘+’ separator.

4.2 Poonguzhali ( Chatterbot )

Poonguzhali is a generic Tamil chatterbot that

allows the user to chat with the system on any
technical topic. A chatterbot is an AI program
that simulates human conversation and allows for
Natural Language communication between man
and machine.

A question or a statement is taken as input from

the user. The function of the system is to generate
an appropriate response, based on the context of
the input. The user can choose any existing topic
for conversation and ask questions to the system
in Tamil. Provision for input in transliterated
form is available. The system identifies the
minimal context of the input as to what the user Screen shot of Poonguzhali (Chatterbot )
is trying to ask, no matter in what way the user
frames his question. This is done using a set of Status : A version with knowledge base for two
decomposition rules. The response is then formed domains is available.
using a set of reassembly rules that reside in the 5. Localization
knowledge base. This response is then reframed
to match the way in which the user had framed A complete office suite for processing Tamil
his question. The system is also versatile enough information has been developed. This works on
to initiate a conversation based on the earlier both Windows and Linux platforms. A search
sequence of the conversation, when there is no engine to search Tamil websites has also been
input from the user. developed.

The software offers the facility of an optional • Tamil Office Suite

alternate explanation to the previous one, in case • Tamil Word Processor,
the user doesn’t understand the earlier answer
given by the system. The pronoun references like • Presentation Tool for Tamil,
the pronoun “idhu” in the second question related
• Tamil Database
to the first one are identified. The Yes/No type of
questions and compound technical terms are also • Tamil Spreadsheet
handled.
• Tamil Search Engine
Currently there are two domains available. Any
number of domains can be added. For this 5.1 Tamil Office Suite (Aluval Pezhai)
purpose, the tool provides a separate interface for The Tamil office suite consists of four text
adding domains and for updating the knowledge processing applications, namely :
base. This has three separate options - for entering
the knowledge base, non-technical words and • Tamil word processor

158
 • Presentation tool for Tamil
• Tamil Database
• Tamil Spreadsheet.
This office suite is specifically designed for Tamil
users. However, it can be easily localized for any
language.
• Tamil Word Processor (Palagai)
The Tamil word processor is aimed at Tamil users.
It provides basic facilities for word processing both
in Tamil and English. There are two versions of
the word processor available. One with the spell-
checker and grammar-checker, and another
without the spell checker and grammar-checker.
Both these versions include an e-mail facility. The
grammar-checker checks for person, gender and
number compatibility between the subject and the
main verb of the sentence.
The word processor does not have any special file
format. Files of both rich text format and text-
only formats can be created or edited. HTML
files can be viewed with this word processor. E-
mails can be sent using this word processor if the
current system is connected with the SMTP
server. Email attachments are also handled.
Status :Version 1.0 is available for use.
• Tamil Presentation Tool (Arangam)
Arangam is a presentation tool for Tamil. It
organizes a presentation with slides consisting of
text, pictures and images. The presentation is saved
in a proprietary format.
Once a file is created one can add, remove or
edit slides with common operations like adding
text boxes, pictures and general shapes. There are
three modes of operation:
Edit Mode (Slide)
Preview Mode (Slide Sorter)
Presentation Mode (Slide Show)
The edit mode is the one that allows working on
one selected slide. Preview mode gives a preview
of number of slides (there are different possibilities
2x2, 3x3 or 4x4 slides in one page). Presentation
mode is the regular slideshow.
Screen shot of Tamil Presentation Tool (Arangam)
Status : Version 1.0 is available for use.
• Tamil Database (ThaenKoodu)
The database tool helps a user to store Tamil data
and also provides various means of retrieving it
using queries, forms and reports. The help
document for this tool is provided in bilingual
format (Both Tamil & English). All the data are
stored in MS-Access format.
This tool is divided into five major modules. They
are Database, Table, Query, Forms and Reports
modules. In the Database module, a new database
can be created or existing databases can be edited.
The Table module allows the user to create the
tables in two different ways. One is the design
mode where the table name, field name, data type
and the constraints (null, primary key and so on.)
are to be entered. The other is through the Table
creation wizard, where the sample table names
along with their fields are provided to create the
table. Tables can be modified and the structure of
the table can also be altered. New fields can be
added to the existing table at any time.
The Query module handles queries in three
different modes. In the first, the table name and
field name should be entered to query the table.
The second mode uses the wizard, where the
159
required table name and the corresponding field
name can be selected. The third one is, where the
Tamil query can be entered in the space provided,
to retrieve the result.
The Forms module generates forms. The forms
can be built with the help of a wizard and can
also be saved for later use. Report module allows
the user to generate reports. The report can also
be saved and printed.
Screen shot of Tamil Database (ThaenKoodu)
Status : Version 1.0 is available for use.
• Tamil SpreadSheet (Chathurangam)
Tamil Spreadsheet allows one to easily enter or
edit the data in the desired format, save data, and
view using various charts. Mathematical
expressions are also handled. The data is saved in
the proprietary format (“.tss” Tamil spread sheet
format).
It consists of three modules viz. Worksheet
module, Expression Module and Chart Module.
The default number of worksheets available is
three. More number of worksheets can be added.
The worksheets can be deleted or renamed.
Worksheet module allows the user to enter the
data in the required format. The basic operations
like cut, copy, paste, find, replace, delete sheet,
print the sheet, insert row/column and delete row/
column are available in this tool.
The Expression Module evaluates the expressions.
Addition (Sum), Count, Maximum, Minimum
160
and Average are the functions supported in this
module.
The Chart Module allows the user to draw the
charts, which are visually appealing and make it
easy for users to see comparisons, patterns, and
trends in data. This module represents the data in
the form of pie, horizontal bar and vertical bar
charts. It can be saved as jpeg (.jpg) file format.
The chart title and the scale to draw the charts
can also be customized.
Status : Version 1.0 is available for use.
5.2 Tamil Search Engine (Bavani)
A search engine is a software package that searches
for documents in the Internet dealing with a
particular topic. The Tamil Search Engine aims
at looking up Tamil sites for information sought
by a user. It searches for Tamil words in Tamil
web sites available in popular font encoding
schemes. Currently 22 font-encoding schemes are
supported. An important feature of the search
engine is the integration of the morphological
analyser, so that the keywords are always reduced
to the root form.
The system gathers information from the Internet
by the process of crawling. The information is
gathered and stored in the database. An interface
is provided to the user to enter the query. The
query is analyzed with the information from the
database, and the information that matches the
query is returned to the user.
The following are the important modules handled
in the search engine:
Crawler: This is the kernel part of the design.
Starting from a given URL and domain, the
document will be downloaded and parsed. It will
then retrieve the URLs from that document and
insert these URLs in the URL tree. After parsing
each document, the next URL in the tree will be
taken out for further crawling.
Database: A database will be maintained to store
each word and its corresponding appearance in
all the documents. These words are stored as a B-
tree for efficient indexing. The position of the
word in various HTML tags is maintained and
used for ranking analysis.
Searcher: Searcher accepts queries from the user,
and then splits the queries into words. The given
words are searched in the BTree and finally the
results are sent back to the user after ranking is
done. The ranking is performed in the order of
number of outgoing links, number of
occurrences, description (meta), head and title.
The results are displayed in the order of AND,
OR operations. The document, which has the
maximum number of query words, gets a higher
rank.
Status : Available for use.
6. Language Technology Human Resource
Development
Many activities were undertaken to create
awareness and develop human resources in the
area of Language Technology. The activities can
be grouped as follows:
• Creating awareness among the general public
• Language Technology Training
• Co-ordination with linguistic and subject
experts
• Co-ordination with Government and industry
6.1 Creating Awareness Among The General
Public
One of the priorities of the center has been to
create awareness about the need for content
development among the public. Working towards
this goal, we targeted school and college teachers
and students and the following workshops have
been conducted:
• Workshop on Multimedia Tools for Effective
Teaching: Conducted a two day workshop on
Multimedia Tools for effective Teaching for
college and school teachers in Feb 2001 – 100
teachers attended the workshop (200 person days).
• Workshop on Multimedia Tools for content
creators Conducted a ten-day hands-on
workshop on Multimedia tools for content
creators in May 2001 – 13 teachers attended
the workshop (130 person days)
• Effective Teaching through computer
A presentation was given by RCILTS-Tamil
members at S.B.O.A School, Anna Nagar,
Chennai. 20 teachers attended.
A presentation was given by RCILTS-Tamil
members, at Teachers training school, Saidapet,
Chennai. 100 teacher trainees attended.
• WWW-2001
Organized a Creative writing competition for
school children on Indian cultural aspects in
Tamil and English, titled “ WWW 2001,
Weave your Words to the Web”, in Sep 2001
to create an awareness about content
development. 40 students participated.
• InSight
“InSight – Scientific Thought, the Indian way”,
an Indian sciences appreciation course for
school children, is being conducted every 6
months. (500 person days )
6.2 Language Technology Training
Language technology training has been imparted
in two ways. One is by conducting intensive
workshops to encourage research and
development in language technology, and the
other is by carrying out development projects in-
house.
• Workshop conducted:
• Organized a workshop titled “Corpus-based
Natural Language Processing” targeted at
both computer scientists and linguists
• Pre workshop Tutorials – Dec 14 –16,
2001
• Date: Workshop Dec 17, 2001 – Jan 2,
2002
• No. of participants: 40
161
 • Contents: Paninian Grammar,
Dependency Grammar, Machine
Translation, Statistical NLP, TAG
grammar etc.
• In-house Training
More than 15 project associates have been actively
working in the area of language technology for
the past three years and have gained expertise in
this area. Specifically they are well versed in
computational morphology, syntax, lexical
formats, and font and coding issues.
A lot of enthusiasm and interest has been created
among students to work in the area of language
technology. Over the last three years about 30
major student projects have been carried out in
language technology. The students are both from
under graduate (B.E – 36 students – 12 batches)
and post graduate (M.E and M.C.A – 18 students)
streams. Some of these projects have been further
developed into products.
A number of research scholars are actively
persuing research in this area. Three M.S by
research scholars are working in the areas of
language translation, speech processing, and
information retrieval. Three Ph.D scholars are
working in the areas of information extraction,
Indian logic based knowledge representation, and
automatic software document generation.
6.3 Co-ordination with Linguistic and Subject
Experts
The center has been coordinating with various
experts and agencies both for linguistic expertise
and for content development. Numerous
interactions have been organized for this purpose.
• Seminar conducted by RCILTS-Tamil
members with S.B.O.A School, Anna Nagar,
Chennai, for content development in Tamil
Literature.
• Seminar conducted by RCILTS-Tamil
members with MOP Vishnawa College,
Chennai, for content development in Tamil
Culture.
162
• Discussion with Tamil scholars for effective
content development in classical Tamil
literature.
• Seminar given by Dr.Subaiya Pillai and
Dr.D.Ranganathan at Anna University for
“Sandhi in Tamil”.
• A one day workshop titled “Language and
Computational Issues” for interaction with
Tamil Linguists
Date: Feb 15, 2002
No. of participants: 30
• Discussion with experts from C-DAC, ASR,
Melkote, CIIL Mysore, Annamalai University,
Madras University regarding language
processing tools developed.
• Demonstration of products developed at
various forums.
6.4 Co-ordination with Government and
Industry
The products developed have been demonstrated
to industries working in the area of language
technology. The outcome of this interaction is
two-fold. One is the signing of memorandum of
understanding with industries like Modular
Infotech, Pune, Apple Soft, Bangalore, and
Chennai Kavigal, Chennai for products such as
spell checker, morphological analyser and
generator, and text to speech engine. The other is
the proactive interest shown by Kanithamizh
Changam, a consortium of industries working on
Tamil computing to work in close coordination
with the center.
The office suite will be made available free of cost
to interested users, industries and Government
organizations. It is proposed to publicize the
availability of the products and tools through
interaction with appropriate state government
authorities. Core technology such as morphological
analyser, generator, and spell checker will be made
available after signing of appropriate MOUs. The
educational resources developed will be made
available freely to government schools.
7. Standardization
All the software developed caters to both the
prevailing Tamil Nadu Govt. standard (TAM/TAB
font encoding scheme), and the ISCII standard.
Testing for Unicode support is in progress. The
center, actively co-ordinates with the Tamil Nadu
Govt, in defining the Tamil Unicode scheme.
Available standards for transliteration have been
adhered to, wherever necessary.
All the development work has been carried out
in Java, to provide cross-platform operability. All
the software has been tested on both Windows
and Linux platforms. Code, technical and user
documentation have been done for all the
products.
8. Research Activities
The following research activities have been going
on, in the areas of information retrieval from
documents and knowledge representation.
• Text Summarization
• Latent Semantic Indexing
• Knowledge Representation System Based
On Nyaya Shastra
8.1 Text Summarization
Document summarization plays a vital role in the
use and management of information
dissemination. This project investigates a method
for the production of summaries from arbitrary
Tamil text. The primary goal is to create an
effective and efficient tool that is able to
summarize large Tamil documents. Summarizer
proceeds to produce a meaningful extract of the
original text document using the sentence
extraction model. Lexical chains are employed to
extract the significant sentences from the text
without complete semantic analysis of the original
document. Segmentation is done initially which
helps to maintain the coherence in the generated
summary.
The three important steps involved in
summarization are : Segmentation of the
document, Lexical chain creation and significant
sentence extraction for summary generation.
Tamil WordNet is used as knowledge source for
summarization, which helps to find out the
relations to build lexical chains. It is able to
incorporate the relationship among words and at
the same time account for the various senses of
the words.
Segmentation: Text segmentation is a method for
partitioning full-length text documents into
multi-paragraph units that correspond to a
sequence of sub topical passages. This method is
based on Multi-paragraph segmentation of
expository text. It has now been implemented for
Tamil.
Lexical Chain Creation : Lexical chains are formed
by disambiguating the senses of the words
occurring in the original text and chaining the
related terms together. Though repetition of
words itself carries informational value,
collocation of terms serves to recognize topics
precisely. Ranking of the lexical chains and
selecting the strong chains are done using Length
and Homogeneity index (the number of distinct
occurrences divided by the length.).
Sentence Extraction : Once the chains have been
selected, the next step of the summarization
algorithm is to extract full sentences from the
original text based on chain distribution. There
are methods, which involve choosing the sentence
that contains the first appearance of a chain
member in the text. Depending on the requested
summary ratio, this algorithm can dynamically
change the block size and token-sequence size in
the segmentation step so that the number of
segments recognized is either decreased or
increased. The number of segments in turn
determines the number of lexical chains thus either
increasing or decreasing the length of the resulting
summary. For convenience, the percentage size
of the document is represented in terms of the
number of sentences.
163
Status : A demo version for sports domain is
available.
8.2 Latent Semantic Indexing
Latent Semantic Analysis is a fully automatic
mathematical/statistical technique for extracting
and inferring relations of expected contextual
usage of words in passages of discourse. Latent
Semantic Indexing (LSI) is chosen to index the
documents, for it uses the higher order associations
between the words in a passage or document. This
information is further represented mathematically
for easier manipulation. Tamil language follows
free word order, and Latent Semantic Indexing
does not consider word order to retrieve the
semantic space of the documents. Hence, the
choice of LSI for indexing Tamil documents.
In LSI, the raw text is represented as a matrix in
which each row stands for a unique word and each
column stands for a text passage or other context.
The Matrix plays a vital role in LSI. It is
considered to be the semantic space. Each cell
contains the frequency with which the word of
its row appears in the passage. Singular value
decomposition is applied to the matrix.
The following applications have been done using
the LSI techniques:
Sentence Comparison : The similarity of multiple
sentences in Tamil is assessed. A similarity score
for each submitted sentence , will be computed
and given to the user.
Essay Assessor : This application compares two
essays. The essays are accepted as input, separated,
and converted as vectors. Then the similarity
between them is computed, and given as a score
to the user.
Document Retrieval: Relevant documents are
retrieved using Latent Semantic indexing. In this
application, the query words are the input , the
relevant document is retrieved based on the query.
The result is ranked and presented to the user.
Status : A demo version is available.
164
8.3 Knowledge Representation System Based On
Nyaya Shastra
World knowledge representation for the purpose
of language understanding is a challenging issue
because of the amount of knowledge to be
considered and the ease and efficiency of retrieval
of appropriate knowledge. An important issue in
the design of an ontology dealing with world
knowledge is the philosophy on which the
classification scheme is based. In this work, an
ontology based on Nyaya shastra, an Indian Logic
System has been designed and a knowledge
representation system called KRIL has been
implemented. The methodology is based on the
notion of associating qualities and values to
concepts and adding different kinds of negation,
which brings a new perspective to the
interpretation of knowledge. The knowledge
hierarchy and the techniques to interpret the
knowledge have been adapted from Nyaya shastra.
Association of qualities and values to concepts and
addition of new relationships based on these
associations and new relations between concepts
themselves adds a new dimension to the reasoning
process.
To adapt the classification scheme and reasoning
methodology of Nyaya shastra, description logic
has been extended by adding operators for
concept definition with qualities and for new
types of negation. Hence, the building blocks of
this model are the concept, inherent association
of qualities, values, relationship between concepts
and relationships between concept and qualities.
The representation of knowledge at various levels
includes time variant and invariant components.
The special feature of this model is handling of
prior and posterior absence of concepts through
additional negation operators. This model can be
used as a base for various applications like natural
language processing, natural language
understanding, machine translation etc.

Screen shot of KRIL-system
Status : An implementation of this model to represent
the knowledge about the dairy domain has been
done.
9. Publications
1. G.V.Uma, and T.V.Geetha (2001),
“Generation of natural language text using
perspective descriptors in frames”, IETE
Journal of Research-special issue on
Knowledge and Data Engineering, January-
April 2001, Vol.47 No.1&2 pp43-56.
2. G.V.Uma, and T.V.Geetha, “Softplan a
planner for automatic software
documentation”, National Conference on
Document analysis and recognition,
Mandya, 2001, pp 266-271.
3. G.V.Uma, T.V.Geetha, “Automatic software
documentation using frames and causal link
representation” VIVEK journal – A quarterly
in Artificial Intelligence, 2001, vol.14, No.3
pp 3-13.
4. D.Manjula and T.V.Geetha, “Message
Optimization by Polling for Text Mining”,
National Conference on Document analysis
and recognition, Mandya, 13-14 July 2002,
pp. 199-202.
5. D.Manjula, P.Malliga and T.V.Geetha,
“Semantic based Text Mining”, First
International Conference on Global Word
net, Mysore, 21-25 January 2002, pp. 266-
270.
6. D.Manjula, and T.V.Geetha, “ Distributed
semantic based text mining”, To appear in
the proceedings of the Third International
Conference on Data mining methods and
databases for engineering, finance and other
fields, Bologona, Italy, 25-27 Sep 2002.
7. G.Aghila, Ranjani Parthasarathi and
T.V.Geetha, “Design of conceptual ontology
based on Nyaya theory and Tolkappiam”,
Third International conference on South
Asian Languages (ICOSAL – 3), Jan 4 – 6
University of Hyderabad, Hyderabad, 2000,
pp.2.
8. G.Aghila, Ranjani Parthasarathi and
T.V.Geetha, “Indian Logic Based Conceptual
Ontology Using Description Logics”,
National Conference on Data Analysis and
Recognition, Mandya, 2001, pp.279-286.
9. Devi Poonguzhali, P.Kavitha Noel, N.Preeda
Lakshmi, T.V.Geetha, A. Manavazhan,
“ Tamil Wordnet”, First International
Conference on Global Wordnet, 21-25 Jan
2002, pp 65-71.
10. G.V.Uma, and T.V.Geetha, “A knowledge
based approach Towards automatic software
documentation”, Spring 2002, International
Journal of Trends in Software Engineering
Process Management, Malaysia.
11. G.V.Uma, N.S.M.Kanimozhi, and
T.V.Geetha, “Automatic software
documentation in Tamil”, presented in the
3rd International conference on south Asian
languages ICOSAL-3”, University of
Hyderabad, 1999.
12. Madhan Karky.V, Sudarshanan.S,
Thayagarajan.R., T.V.Geetha, Ranjani
Parthasarathy, and Manoj Annadurai, “Tamil
Voice Engine”, Presented at Tamil Inayam,
Malaysia, 2001.
165
13. P.Anandan, T.V.Geetha, and Ranjani Conference on Universal Knowledge and Lan-
Parathasarathy, “Morphological Generator for guage (ICUKL) – Nov 25-29th, Goa, India.
Tamil” Presented at Tamil Inayam, Malaysia,
2001. 23. G.S.Mahalakshmi, G.Aghila, T.V.Geetha,
“Multi-level Ontology representation based on
14. G.V.Uma, and T.V.Geetha, “A knowledge based Indian Logic system” International Conference
approach Towards automatic software on South Asian Languages (ICOSAL) – 4, Dec
documentation”, Spring 2002, International 3-5, Annamalai University, TamilNadu, 2002.
Journal of Trends in Software Engineering
Process Management, Malaysia. 24. P.Devi Poongulhali, N. Kavitha Noel, R. Preeda
Lakshmi, Manavazhahan and T.V. Geetha,
15. G.V.Uma, N.S.M.Kanimozhi, and T.V.Geetha, “Tamil Text Summarization” International
“Automatic software documentation in Tamil”, Conference on Knowledge Based Computer
presented in the 3rd International conference Systems, Mumbai, India, December 18-21,
on south Asian languages ICOSAL-3”, 2002.
University of Hyderabad, 1999.
10. The Team Members
16. D. Manjula and T.V. Geetha, “ Semantic based
text mining”, of the International conference Dr. T. V. Geetha rctamil@annauniv.edu
on Global WordNet, Pg 266-270, Mysore, Jan Dr. Ranjani Parthasarathi rp@annauniv.edu
21-25, 2002. Dr. K. M. Mehata mehata@annauniv.edu
Dr. Arul Sironmoney asiro@annauniv.edu
17. D. Manjula, A. Kannan and T.V. Geetha Ms. V. Uma Maheswari umam_in@yahoo.com
“Semantic Information Extraction and Query Ms. N. Anuradha anuradha_n_2000@yahoo.com
processing from world wide web”, Bombay, Ms. S. Chithrapoovizhi scpoovizhi@yahoo.com
Dec18-21, KBCS 2002. Ms. J. Deepa devi deepajayaveer@rediffmail.com
Mr. T. Dhanabalan tdhanabalan@yahoo.com
18. D. Manjula and T.V. Geetha, “Message Mr. T. Dhinakaran dhina_t@yahoo.com
Optimization using polling for distributed text Ms. T. Kalaiyarasi kalaivas@yahoo.com
mining,” National Conference on Analysis and Mr. A. Manavazhahan a_manavazhahan@hotmail.com
Recognition, Mandya, 2001. Mr. G. PalaniRajan palaniworld@rediffmail.com
19. P. Malliga, D.Manjula and T.V. Geetha, “ Mr. R. Purusothaman r_purush@hotmail.com
Boostexter for Tamil Document Mr. K. Saravanan ksaranme@yahoo.com
Categorization”, International Conference on Ms. M. Vinu krithiga vinu_krithiga@yahoo.com
South Asian Languages . Mr. V. Venkateswaran venkatjee@indiatimes.com
A. Kavitha avitha_mariyappan@yahoo.com
20. Siva Gurusamy, D. Manjula and T.V. Geetha, M. Kavitha malar_vanam@sify.com
“Text Mining in request for comments
Document Series”, LEC 2002 Language Courtesy: Dr. T.V.Geetha/Ms. Ranjani Parthasarthi
Engineering Conference, Dec 13-15, Anna University, School of Computer Science &
Hyderabad, 2002. Engineering, Chennai - 600 025
(RCILTS for Tamil)
21. T.Dhanabalan, T.V.Geetha, “ UNL – Tel: 00-91-44-22351723,22350397
EnConverter for Tamil”, International Extn- 3342/3347, 24422620, 2423557
Conference on South Asian Languages E-Mail: rp@annauniv.edu
(ICOSAL) – 4, Dec 3-5, Annamalai University,
TamilNadu, 2002.
22. T.Dhanabalan, K.Saravanan, T.V.Geetha,
“Tamil to UNL EnConverter” International

166
 Resource Centre For
Indian Language Technology Solutions – Urdu, Sindhi, Kashmiri
C-DAC, Pune
Achievements

Centre for Development of Advanced Computing

Pune University Campus, Ganesh Khind, Pune-411007
Tel. : 00-91-20-5694000-2 E-mail : mdk@cdacindia.com
Website : http://.parc.cdacindia.com
 RCILTS-Urdu, Sindhi & Kashmiri
Centre for Development of Advanced
Computing, Gist, Pune
Introduction
“Digital unite & Knowledge for all the vision of TDIL”
MC&IT wishes to make a difference to the quality
of life of people of India through its TDIL mission,
and is investing significant amount of resources
towards fulfilling this wish. One such effort was the
establishment of “Resource centers” for design,
development & deployment of tools & technologies
for Indian languages.
“Dissolving the language barrier the vision of
GIST”.
C-DAC has pioneered the Graphics and Intelligence
based Script Technology (GIST) which facilitates
the use of Indian languages in IT. GIST is geared
for the Internet enabled world where all activities
are gradually going online. In its endeavor to stay
abreast with technologies worldwide, GIST has been
adopting the latest concepts to be able to stay tuned
with the changing IT scenario. C-DAC, GIST
through its continuous R&D efforts has to its credit
many innovative products & continues to be leaders
in the language technology field.
MC&IT, through its TDIL Programme has
awarded C-DAC, GIST with the project “Resource
centre” for development of tools & technologies for
Perso-Aarbic languages.
Overall Goal of the Resource Centre
To empower people of India through the use of
Information Technology solutions in Indian
languages.
Project Purpose
To improve the quality of life of people of India
by enabling them to use information technology
in Indian languages.
To develop new products and ser vices for
processing information in Indian languages.
To conduct research in computer processing of
Indian languages.
168
Broad areas of work under the Perso-Arabic
Resource center
GIST workshop for usage of GIST tools for
organizations concerned with development &
deployment of Indian language processing systems.
Creation of a network of persons and organizations
working on IT applications in Perso-Arabic( PA)
languages namely Urdu, Sindhi & Kashmiri.
Development of Language tools & applications in
PA scripts.
Conduction of workshop to discuss on
standardization of PA scripts data & font
representations.
Deployment of the developed technology, products,
& services through a country-wide network.
Services and Knowledge Bases
• Creation of a web site for language tools, solutions
and knowledge available in PA language.
• Imparting training for the usage of PA.
• language applications on IT.
• Establishments of Translation & Subtitling
services in PA languages.
Products
• High quality PA fonts.
• Dictionary, Spell Checker tools in PA languages.
• Word processor package on PA languages.
• Transliteration tool between Hindi & Urdu.
• Prototype of Pocket translator with Urdu support.
• Product for Subtitling, Character Generator,
Teleprompter, DVD in PA scripts.
Activities Undertaken Under The Project
GIST workshop was conducted at C-DAC Pune
from 17th to 22nd July 2000, for usage of GIST
series of Indian Language Technologies, tools and
products for Resource Centers concerned with
development and deployment of Indian Language
solutions. A team of 23 members from 8 Resource
centres have attended.
1. ER & DC, Trivendrum
2. Jawaharlal Nehru University, New Delhi
3. Indian Institute Of Technology, Kanpur
4. Orrisa Computer Application Center,
Bhubneswar
5. Thapar Institute Of Engg. And Technology,
Punjab
6. Anna University, Chennai
7. M. S. University, Baroda
8. Indian Statistical Institute, Calcutta
Covering over five days and total of 12 interactive
sessions, the subjects covered were
1. Building dictionary & Designing spell checker
2. Keyboard driver
3. GIST software development kit
4. Multilingual web tools
5. Script code, Font storage & Design.
During the workshop training kits were distributed
which contained work manuals on following topics;
1. Designing Indian Language Spell Checkers,
2. Myths about Standards for Indian Languages,
3. Enabling Indian Languages on Web,
4. ISCII- Foundations & Future,
5. GIST Software Development Kit,
6. Using GIST SDK in VB,
7. Internet First Steps: Upgrading an Existing
ActiveX Control.
8. Inscript Keyboard layout,
9. Typing in Phonetic Keyboard.
Two CDs on GIST s/w and brochures on various
language products & fonts type face catalogue were
distributed:
1. GIST based S/W Developer CD.
2. GIST Products & Channel Information.
A visit was organized to various departments of C-
DAC, GIST Laboratory, National Multimedia
Resource Centre, and National PARAM Super
Computing facility.
Evolving Storage, Font & Inputting Standards
The main task before C-DAC was to evolve
standards for storage, fonts & inputting for the
Perso-Arabic languages. Development of USCII
(Urdu standard for Information Interchange) was
carried out well before awarding of the project. GIST
Terminals designed & developed by C-DAC are
based on these standards. The support for the GIST
terminals was limited to only the Naskh
implementation of the fonts.
Accordingly with lots of efforts and valuable inputs
from experts all over, C-DAC drafted PASCII
standards for storage, keyboard standards & font
standards. These proposed standards are published
in the TDIL magazine – Vishwa Bharat for
comments / suggestions from experts all over India.
1. The PASCII standard - The Perso-Arabic
Standard for Information Interchange
There are some peculiarities that can be seen in
Perso-Arabic scripts:
These scripts join letters with each other, and
therefore letters have different forms as per their
position in a ligature (the positions are beginning,
middle, ending).
Some shapes do not have middle shape i.e. they do
not join at both the ends. For example alif, vao,
daal, etc. Every letter also has a standalone form
1.1 Characteristics of Proposed Standard
Its an 8-bit standard. Supports letters for Urdu,
Arabic, Sindhi, Kashmiri.
Defines PA alphabets in the upper ASCII leaving
lower ASCII free for English. Bilingual support
possible.
Defines numerals other than ASCII numbers (48
to 57).(This may help supporting both Arabic
Numerals 0-9 and language specific numerals).
Maintains the order of alphabets for perso-arabic
languages. Khate-kasheed is given lower value than
169
alphabets. This will sort the words correctly even
though they have khate-kasheed in between.
Alphabets for different languages are placed in their
ascending order. Letters like “bhey” are not provided
for URDU but kept for languages like SINDHI.
The URDU may make use of “be” and “choTi-hai”
for that.
Minimal erabs are provided. Tanveen ,for example
do-zabar, can be formed with the help of two
consecutive zabar.
Superscripts
Place for superscripts like khaRa-alif is provided.
Place for superscripts for ARABIC is provided.
Place for superscripts like “re-ze”, “ain”, etc. is
provided.
Numerals are placed after erabs and super scripts.
(This is provided only to support display for
language specific numerals and standard numerals
i.e. the ASCII numerals are available).
Only a few control characters are defined.
1.2 Standardization of Urdu Keyboard
Overview
There is no standard keyboard available for URDU
language. There are vendor specific keyboards
available which all differ in their layouts. There are
lot of keyboards designed by different companies
who provide Perso-Arabic support in their
applications. Unfortunately, every company has its
own Keyboard layout that entirely differs from the
other’s Keyboard Layouts. For example Microsoft
has its own Keyboard layouts for Perso-Arabic
languages.
Drawbacks of Available Keyboards (in view of
URDU language)
No standard keyboard available.
All vendor specific keyboards differ in their layout.
Most of the keyboards have been designed for
Arabic.
Although Arabic and Urdu have similar script, they
are both different languages, and hence a Keyboard
170
designed for Arabic may not be that useful for Urdu
language. CDAC has taken this care of languages to
design the keyboard for URDU, SINDHI &
KASHMIRI and tried to come up with an optimum
solution.
(Details of these standards can be found in TDIL
Vishwabharat magazine.)
1.3 Standardisation of Fonts
Characterstics of Perso-Arabic languages:
Urdu has traditionally been written in the Nastaleeq
script. Although the script employs the basic letters
of the language, the rendering of these letters in a
word is extremely complex. The reason for this
complexity is that Urdu text has traditionally been
composed through calligraphy, a medium whose
precepts are based on the aesthetic sense of the
calligrapher rather than on any formula. So great is
the variation in calligraphy that many times it is
difficult to recognize the letters in a constituent
word. This is because, in their calligraphed form,
the individual letters partially or completely fuse into
each other thereby losing their identity. A degree of
fusion is purposely introduced to make the resulting
fused glyph visually appealing.
Another characteristic of the Urdu is the existence
of diacritics. Diacritics, although sparingly used, help
in the proper pronunciation of the constituent word.
The diacritics appear above or below a character to
define a vowel or emphasize a particular sound. They
are essential for removal of ambiguities, natural
language processing and speech synthesis.
A Nastaliq style font – designed, to be used for
printing and display. The font is based on traditional
Nastaliq style written in India
Arabic Font – A high quality Arabic Font is designed
to compliment religious text in Urdu.
Considerations for Digital Font Design
Following is a group-wise list of characters that
were taken into consideration while designing
fonts for the Perso-Arabic languages.
1. Alphabet.
2. Numerals.
3. Special Characters. 2. Script Core Technology & Rule Engine
4. Diacritics. Because of the complex nature of the Nastaliq script
which is written right to left and top to bottom
5. Religious and linguistic Symbols. diagonally, it was required to make certain databases,
6. Control characters. and use them along with the rule engine, to make
display possible. Following are some tools which were
The 16-bit Naskh & Nastaliq Font developed to create these databases.
The Fonts developed by CDAC are 16-bit and are 2.1 Glyph Property Editor
also defined in the User Area of the Unicode range.
The ASCII range is not used and can be used for In Urdu and all other Perso Arabic languages each
different purposes (can be used to English support character has many different shapes depending
for example). upon the position of that character, 1) the stand
alone 2) the starting position 3) the middle position
Includes 4) the ending position etc. So according to the
• all the basic shapes. position of the character and the letters with it its
joining that particular shape is to be displayed. This
• all the starting shapes and variations. is an application that allows to store the data and
the compositional information for various shapes
• all the middle shapes and variations.
in a font file for proper alignment of characters for
• all the ending shapes and variations. display. ( Urdu requires a larger number of glyphs
than other scripts) This takes care of the horizontal
• levels for erabs (short vowels).
and vertical movement for Nashtaliq script.
• complete ligatures.
2.2 Rule Engine
• Beginning ligatures.
A module that containing rules for each shape. The
• Middle ligatures. rules indicate how and which shape will join with
other shape or shapes based on the succeeding and
• Ending ligatures. preceding shapes that occur in Nastaliq text
• missing character glyph. composition. This facilitates more accurate and faster
display of the shape composition in the Nashtaliq
Glyph Standards decided for FONT text composition.
Development
2.3 Testing version of Urdu Editor
Urdu & Kashmiri- Naskh & Nastaliq script
A simple text editor is ready to test the Rule Engine
Sindhi - Naskh script. and check the various rules formed for displaying
• Basic letters (Vowels, Consonants). the glyphs. This editor allows to write the text in left
to right, and handles all keyboard mappings from
• Full shape letters. ASCII to USCII, and Urdu text is displayed.
• Beginning. 2.4 Font Glyph Editor
• Medial. In Nastaleeq script (for Urdu), the shapes join from
right to left, and within a ligature they also shift
• Final.
vertically. This forms a ligature. So in a ligature,
• Ligatures. shapes join right to left and top to bottom. The
vertical positioning information cannot be put in
• Graphic components.
the font itself. Hence it was decided to make a utility,
• Numerals, Signs and Symbols, etc. which will let one set this information in separate file.

171
Example of Nastaleeq joints: (“sehan” => suad baRi- (Ghalib)
he noon):

Purpose (Ghalib Bold)

To select a glyph from the given font and character,
and define on it following information:
1. Starting point (SP): the point where a glyph ends.
This is generally on baseline or zero.
2. Ending point (EP): the point where a glyph joins (Imaad)
with another glyph on its starting point (SP of
another glyph). This point is less than or equal to
the glyph width.
3. Dot points (UP and DP): dot points where a dot
or diacritic mark will be positioned on the glyph.
(Kamran)
2.5 The Font Editor Utility
The Font Editor utility lets us place the dot
information for each glyph. The file thus generated
is used for retrieving dot information at run time.
We’ll call it a dot file.
The following figure depicts a glyph with dot and (Maghribi)
caret information. As shown in the figure, the utility
helps one define following for a glyph:
1. Font Name.
2. Unicode value of the glyph.
3. Glyph Index in dot file (this is the unique id of (Makhdoum)
the glyph).
4. Start and End points.
5. Dot positions.
6. Caret position.
(Murasilat)

(Riyadh)

Fonts : following are samples of fonts developed

172
(Rubai) A software (automatic) tool providing
transliteration from one language to another is
an effective tool which reduces the amount of
efforts required to create large databases of names
in multiple languages.
(Safi) Once a database in one language is created, the
tool can be used to generate the same data for
other languages. This results in considerable
savings in time and money.
It is a useful tool for applications which require
(Saleem) to convert database of names, addresses, phone
numbers or electoral rolls from one language to
another.
This can also be useful for converting entire
databases from one language to another without
(Shibli) re-entering the data. This utility can convert data
in English from a database directly to ISCII.
This will facilitate automated transliteration across
databases.
A software tool for transliterating text from
(Shuja) Devanagari & Gurumukhi (ISCII) to Urdu has
been developed. This tool is also available as a
separate module, and also been implemented in
the Text Editor Test Application, which is
developed.
(Abid) 3.1 Following is a sample of Hindi text
Transliterated into Urdu (UTRANS)

3. Devanagari/Gurmukhi to Urdu Transliteration

“UTRANS ”
A computer based transliteration allows a user to
convert a given text (say, in storage format - ISCII)
Using this automated tool – UTRANS, we have
of language ‘A’ to text in another language ‘B’ (in
transliterated the book “Meri Ekyavan Kavitayen”
some storage format, say ASCII) on the basis of the
written by our Hon. Prime Minister Shri. Atal
phonetic rules that govern languages A and B.
Bihari Vajpayee.

173

3.2 Following is a sample of Punjabi text
Transliterated into Urdu (UTRANS)
4. Dictionary Development Tool
Natural language processing tools such as lookup
dictionaries, thesaurus, spellchecker, grammar
checkers, part of speech taggers, machine aided
translations require language specific dictionaries.
Several other domains such as speech processing,
expert systems require backend language
dictionaries to be able to process and generate valid
outputs. Each of these applications require
dictionaries to be created to cater for the specific
requirements.
The Generic Dictionary Development Tool will
allow users to create dictionaries specific to
application domains. As part of this development
the format of a generic dictionary should
standardized. The tool will provide facilities for
obtaining views of dictionaries that are subsets of
the generic dictionary.
174
This tool, though lot of refinement is required, is
in the form of a standalone desktop utility which
allows the user to create dictionaries containing
words for Perso-Arabic languages. The user is
provided with an interface to define inflection and
root word rules, add suffix, prefix, grammar tags,
domain for a given word entry.
In order to build spellchecker & domain specific
dictionaries following dictionary creation work was
under taken and completed
1. Official Hindi dictionary - Hindi 50,000 words,
equivalent 50,000
2. Urdu to English : urdu words 17000, grammer
tag 17000
3. Hindi Urdu Shabdh Kosh Hindi words 10,000,
Equivalent 10,000
4. Dictionary Urdu words : 10,000 grammer tag
10,000
5. Dictionary of Urdu & Hindi names : 3200,
Equivalent : 3200
6. Technical dictionary Hindi words : 1500, equiva
lent : 1500
7. Muslim names Dictionary : Hindi 1500,
equivalent : 1500
9. Hindi - English - Urdu Hindi words : 8000,
equivalent : 8000
Following is the typical GUI of the dictionary
development tool for URDU.
5. Nashir - Wordprocessor– Publishing Made Easy Table
The Nashir is designed to be easy to use and create A Table control is provided to feed tabular
documents in Perso-Arabic languages, and at the information in the document.
same time powerful enough to layout complete
newspapers and magazines in Urdu, Sindhi, Keyboards
Kashmiri, Arabic and Farsi. Phonetic keyboard supported. A Floating key
Each document of the Nashir consists of a number board will be supported soon.
of pages. On each page of the document, you can
Tool tip
place items like text blocks, graphics, etc.
The Nashir is kind of an Word processor & also Tool tips are provided for the user
best suited for publishing segment. Spellchecker Master Page
support is added.
A master page is provided with each document to
Apart from the base dictionary for spellchecker, provide the user with the header, footer and
various domain specific dictionaries addition is similar settings. Whatever is designed on Master
planned. page appears on the pages.
Salient Features of Nashir
Kerning
1. Support Nastaliq True Type fonts. (presently 2
fonts). Kerning feature is provided to manually adjust a
text. Both Horizontal and Vertical kerning is
2. Supports Naskh fonts. (presently 12 fonts). possible.
3. Fonts for Sindhi, and Kashmiri added. Variable Font Sizes & Colors
4. Supports C-DAC & Phonetic Keyboards.
Use variable font sizes and color.
5. User defined keyboard support available.
Others
6. Drawing objects provided.
The Nashir has a number of other features like
7. OLE Automation supported. Horizontal and Vertical rulers in the GUI,
Other Features dynamic font settings for the Urdu and English
fonts. Indents and paragraph settings, page
Bilingual Support settings, etc.
Supports Urdu, Sindhi & Kashmiri along with
Nashir Screen Shots
English.
Transliteration
Transliteration engine (uTRANS) has been
implemented in the Nashir. One can insert an “.aci”
(ISCII file) file into Nashir and see the transliterated
version in Urdu script (Naskh or Nastaliq). Rule
based transliteration developed for Hindi & Punjabi.
Save As HTML
The user can save the document as HTML page,
and thus Naskh as well Nastaliq scripts can be viewed
on the Internet.

175

Following screen shot shows "Text wrapping around
object" feature of Nashir :
Spellchecker Support in Nashir
The full version of Nashir is now equipped with the
spell checker facility.
The Spellchecker program works as a word-level
error detection and correction (single or multiple
correction) tool. Following consideration were given
while designing the spellchecker module
• Percentage of invalid words pass through it
• Average suggestion rate
• Intended suggestion is given
• Domain the spellchecker is serving
• Dictionary structure
Error Detection/Correction Approaches
1. Non-word error detection
176
2. Isolated word error detection
3. Context based word correction
Non-Word Error Detection
Typographic errors (typing mistake)
Cognitive errors (invalid entry by user)
Phonetic error
Real word error
1. Detection: Block invalid words to pass through it
2. Correction: Suggest alternate valid words that can
pass through it
Error Detection Techniques
Lookup Dictionaries
• Structure : Hash, Tries, etc.
• Dictionary size
• Inflection, euphony, assimilation: (Morphological
Analyzer)
• Domain
Bi-gram / Tri-gram approach (based on conditional
probability approach)
A two dimensional matrix having conditional
probability state of letters occurring next to each
other. Bi-grams are searched and if a bi-gram is not
in the dictionary, then the words are termed
incorrect.
6. The Urdu SDK Controls 3. Copy paste text glyphs into external applications
The GIST Urdu SDK is a software development 4. Use for database applications
tool for Urdu Language on MS Windows.
5. Transliterate (import) Hindi (ISCII) documents
This is a set of software components that enables into Urdu
software developers to facilitate the use of Urdu
6. Use as simple Editor with fixed font and size
scripts with MS Windows applications. Unlike other
solutions, the significant feature of GIST Urdu SDK 7. Use Rich Edit text facilities (e.g. multiple colors,
is that it enables the MS Windows application to font sizes, multiple fonts)
process PASCII data directly.
8. A number of Naskh fonts provided along with
The GIST SDK uses ActiveX Technology from the control
Microsoft and provides a seamless, transparent and
Sample Urdu Active-X control in a form
self contained Indian language layer for data entry,
storage, retrieval and printing in Indian scripts for
your MS Windows applications.
These controls are bilingual and also support English
language. The package contains editing control that
support the complex scripts like Nastaliq. Following
is a listing of controls that are part of the Urdu SDK
controls package version 1.0 which are developed
by C-DAC, GIST:
• UEdit
• UListbox
Applications using GIST URDU SDK
• UCombobox Following are some examples of set of applications
• UStatic where the UrduEdit OCX can be useful:
• UButton DatabaseApplication
The UrduEdit Control can be connected to any type
• UCheckbox
of database, and store text as data. You can store (or
• URadiobutton load) text from a database from (or into) the control.
Typical applications that can be created are mail
With the above set of controls, one can create all
merge, report generators, and all types of business
sorts of applications that support Urdu language.
applications where documents are created from
The UrduEdit OCX is multi-line text editor control information stored in a database.
that supports Perso-Arabic Languages. This version Desktop Applications
(1.0) of the control supports URDU as the primary
language. The control supports Naskh as the default With UrduEdit Control you can easily create
script, but can also be used to depict Nastaliq script. applications that need text fields for Perso-Arabic
The control has a number of features to provide languages. The control can be used as single line or
with most of the functionalities needed for a Text multiline control. You can also use Rich Edit feature
editing control. of the control to display text in different fonts, colors,
and sizes.
Features
Web Applications
1. Read & Write in Naskh / Nastaliq script
Create user controls with Visual Basic, use UrduEdit
2. Bilingual text support i.e. English and Naskh Control in web pages.

177
Sample Database application form using Urdu 9. 2 lines of Urdu message display.
Active-X Control
10. Talk key for speech output.
11. English - Urdu dictionary.
12. Displays nearest matching word in English if
the typed word is not available in Dictionary.
13. Translation of the same to Urdu by pressing
Script key.

7. Pocket Translator
Multilingual Pocket Translator was developed Hardware Details
keeping into mind the foreigners traveling to
1. Developed on Motorola 68EC000 (PQFP pack
India. Even it is useful for persons traveling within
age) processor running at 10 MHz.
India (Inter State). Similar products available
worldwide but not for Indian Languages. 2. 2 MB of EPROM for program, fonts, fixed
messages, dictionary & speech o/p.
Features
3. 16 KB of SRAM for Temporary variables.
1. English to Urdu & vice a versa translation.
4. 64 keys Matrix QWERTY type keyboard
2. Fixed messages for 5 different categories such
as Social, Travel, Emergency, shop & restau 5. 100 x 32 Dot matrix LCD display panel.
rant, both for Urdu & English. 500 messages 6. 4-bit ADPCM decoder for speech output.
in each catagory.
Block Diagram of Pocket Translator
3. Messages can be browsed in Urdu & English.
4. Speech output for Fixed messages only.
5. Single Toggle key for selection of various cat
egories.
6. Arrow keys for scrolling through the messages/
dictionary.
7. Script toggle key for changing the script.
8. 4 lines for English message display.

178
8. Multiprompter
An ideal system for news reading and
documentary production. Support for Naskh &
Nastaliq scripts provided.
Read news or your dialogues without tears. You
look into the camera and get to se the text rolling
a a speed that suits your reading pace. And that
too in India, Arabic or European languages. IT
can have multiple mechanical attachments with
optical glass for mounting on camera tripods. On-
line skipping of stories.
Salient Features
Provide support to control the srcolling speed. a
min. to 9 min. max and 0 for pause.
1. Support spellchecker.
2. Support Indian or European languages
3. Upgradeable to News Romm Automation
System.
4. Works under Windows environment.
5. Available in two models : Desktop & portable
based on Laptop.
Following is the screen shot of the GUI (Graphical
User Interface) of the Teleprompter application
running on the windows platform. Used for
creation of the news and documentaries which
are then prompted on a special equipment having
monitor, mirrored glass & camera.
9. LIPS Advanced Creation Station & LIPS for
DVD
LIPS is a revolutionary, multilingual system
meant for Electronic as well as fused subtitling.
It consistes of highly productive software and cost
effective hardware to create subtitles in Indian as
well as foreign languagess along with the time
codes.
LIPS for DVD allows you to create DVDs with
multilingual subtitles in various Indian as well s
foreign languages. It converts the LIPS subtitling
files into a DVD compatible format. This system
supports Diakin and Sonic formats. Generates the
header indicating the background and foreground
colors, edge color, the in-time and out-time as
well as the subtitling image file.
Saient Features
1. Supports all Indian languages inclusive of
URDU, Sindhi & Kashmiri.
2. Naskh & Nastaliq support provided.
3. Support TIF and BMP formats
4. Support different fonts and font sizes
5. Edge color for subtitles can be specified
6. Allows positioning of subtitles on the video
7. Provision for increment and decrement of in
and out time
8. Compatible with PAL as well NTSC formats
Subtitling
179
10. Perso-Arabic Web Site Development Following screen shots shows the main page &
the page containing information related to the
Under the Perso-Arabic resource centre one of
resource centre website.
the major deliverables was to create the web site
in PA scripts & especially in the Nataliq script. Courtesy : Shri. M. D. Kulkarni
Following was the method adopted for design & Chief Investigator - Resource Centre &
development of the website named http:// the Resource Centre Team, GIST
parc.cdacindia.com Centre for Development of Advanced Computing
Designed a special activeX control (UEditWeb) (C-DAC), Pune University Campus,
for web that could display Nastaliq. Ganeshkhind, Pune 411 007
Tel : 00-91-20-5694000/01/02, 5694092
This control supports HTML like tags to set Fax : 91-20-5694059
various attributes. E-mail : mdk@cdacindia.com
Converted USCII data into UEditWeb format.
UEditWeb data was stored in files which were put
in Database.

Server side scripts were written to generate Urdu

content for display.
Ten books will be added to make the site content
rich.

180
 Resource Centre For
Indian Language Technology Solutions – Sanskrit, Japanese, Chinese
Jawaharlal Nehru University, New Delhi
Achievements

Jawaharlal Nehru University, New Delhi-110067

Tel. : 00-91-11-26704772
E-mail : gvs10@hotmail.com
Website : http://202.41.10.205/rciltsjnu
 RCILTS-Sanskrit, Japanese, Chinese Unicode. During this conversion process we
Jawaharlal Nehru University, New Delhi developed Web Content Unicode Converter. This
converter works with ISFOC/ISCII data.
Introduction
RCILTS, JNU is a resource center for Sanskrit
language of DIT, Government of India. At JNU
work started in three languages viz., Sanskrit,
Japanese, and Chinese. It took lot of time and effort
to assemble a cohesive team for the development of
language technology.
In the beginning lot of problems were faced
regarding development tools, software professionals,
as well as language professionals. Due to the socio-
economic nature of the IT industry the attrition of
the IT professionals was very high, so work suffered
a lot. Till today it is becoming very difficult to get
good linguists (computational linguistics) who could
carry forward the task of future development. The RCILTS, JNU website is available at http://
202.41.10.205/rciltsjnu
In the first stage it was decided that we should not
be reinventing the wheel. We shall try to develop The various modules that have been developed at
language technology and resources that is not being RCILTS JNU, is presently integrated for learning
addressed at other RCILTS. India has rich cultural the language. With the development of new modules
heritage and time proven scientific knowledge. All this language learning system shall be used for various
this is largely in the form of Sanskrit literature. So it kinds of language processing (text and speech). The
was decided that we would develop a web based various software modules and language resources that
Sanskrit Language Learning System (as a second are available are
language primarily). It would be of great use to those Learning materials
scholars who look to our heritage knowledge for • Sanskrit Lessons
designing and developing Knowledge based systems. • Sanskrit Exercises
Keeping the above goal in mind at RCILTS, JNU, • Sanskrit Script Learning System
we have designed various modules that teach Sanskrit Lexicons
language in an Asynchronous fashion. In the process • Sanskrit – English Lexicon
of development of these language resources we have • Sanskrit – English Lexicon of Nyaya terms
kept in mind the various standardization aspect of • English – Sanskrit Lexicon
creating these resources. Here we would like to
mention specially the Unicode. All the web content Sanskrit Grammar
that is available on RCILTS JNU site is Unicode • Panini’s Asthadhyayi
compliant. • Laghu Sidhyant Kaumudi
• Dhatu Roop to Dhatu
In the early phases of development we used lot of • Dhatu to Dhatu Roop
tools developed at C-DAC. So lot of content was in • Pratipadik Roop
ISFOC/ISCII based. Earlier all the processing was
• Shabd Roop
done on ISCII data. There was no tool where we
• Upsarg
could create content that were Unicode compliant.
• Sandhi
At a very later stage of development when Unicode
tools/techniques were developed and few of them Tools
bought, then we could convert all the content in • HTML Content Converter

182
• ISCII to UNICODE Database Converter and vice versa
• Devnagari Unicode Keyboard Driver for Browsers
Sanskrit Grammar is primarily based on Panini’s
Ashtyadhyayi and SidhyantKaumudi. We have already
developed modules for Dhatu Roop, Sabd Roop, and
Sandhi. After learning all these modules a learner
knows the basics of Sanskrit language.
We plan to develop Sandhi Vichhed System. With
the development of Sandhi Vichhed system as tool it
shall be very easy for the scholars to read and
understand the age old Sanskrit manuscripts which
are treasures of time proven scientific knowledge.
With the proper understanding of these manuscripts Some extra facilities are given to learn and
it would be easy for them to interpret the scientific remember easily. All the Sanskrit words, used
knowledge in their original context and apply them through out the lessons are available to the user
in the present context for the development of with their roots and meanings. These words are
knowledge based software systems. Knowledge based also connected to the Lexicon and accessible from
Systems would aid in the process of Administration, any page of any lesson. User can also see the new
Business, Management, and Education. words on a particular page that he is reading with
the facilities explained above. The Navigation
Since JNU has a rich language school, in the
Buttons bellow the each page is self-explanatory.
beginning work was also carried out in the area of
Japanese and Chinese. But after some timework on The exercises are based on the lessons. For each
Chinese language discontinued due to paucity of Lesson more than two exercises are available. These
Chinese language professionals. But work in the are objective type questions. Four options are
Japanese language is still on. We have developed lot given for each question. As the option is chosen,
of language learning resources for Japanese too. result is given promptly to the user that it is
correct or not. The Correct answers of a particular
1. Sanskrit Lessons and Exercises
question will be given in the table bellow after
Sanskrit Lessons are useful to learn Sanskrit attempting the questions. So user can see the
Language. There are 21 lessons and the number will answer that he has given as well as the actual answer
increase. Lessons are designed so that, a person who of that question just choosing an alternative.
doesn’t know Sanskrit at all but knows either English Exercises are obviously helpful to practice what a
or Hindi can go through these lessons and learn. user learns on that Lesson.
Sanskrit contents of the Lessons are described both
in English and Hindi.
The basics of the Sanskrit Language are covered
elaborately. And then the grammar part of the
Sanskrit language is explained. Nearly all topics of
grammar are covered with lot of examples and
explanation. Special care is taken for the
pronunciation of the Sanskrit alphabets and the
words. The pronunciation is based on standard
phonetic notations. One can go to the ‘Sanskrit
script learning module’ from the Lesson to learn
the writing process and pronunciation of alphabets.

183
2. Devanagari Script learning and Character
Recognition System
Scripts are the basics of a Language. And writing of
these scripts are very important to learn a Language.
Using this module the user learns Devnagari
alphabets and learns how to write and pronunce
them. Writing strokes for a character are shown with
stroke orders with four different speeds. It takes care
of all characters, conjuncts, matras and numerals
also. And there is audio support to listen the
pronunciation of the alphabet. Recognize a character
after hearing its pronunciation is also over. It is also
hyper-linked with Sanskrit Lessons. User can access
3. Sanskrit-English Lexicon
this module from Sanskrit Lesson.
In this module a user can get the meaning, its
grammatical category and usages of Sanskrit word.
It will also display the phonetic notation of the
word. Grammatical category and usage can be
displayed if available.

The E-R Diagram of Sanskrit English Lexicon is

given below. All the Lexicons follow the same E-
R Diagram

Example- The Sanskrit Vowel “Riri” is written in

four stroke with four different speed. Except Basic
alphabets there are numbers of conjuncts characters
in Sanskrit Language. Their formation and the
writing process in the proper stroke orders are given.
Phonetic notations are given in the left-top corner
so that a user can pronounce the alphabet easily.
Numerals are also important to learn as their
frequent use in the language. Writing process of
Sanskrit numeric numbers are shown with their
stroke order. If there is more than one style of writing
of numeric numbers then the alternatives are The facility for getting input from the user is
provided bellow. provided in both ways. Users can click the
beginning letter of the word. It will display all
Pronunciation of alphabet is important to speak a words, which start with that letter in the right
language. The attention is given for the right and side window. On clicking the particular word the
clear pronunciation of the alphabets and audio meaning with phonetic notation and the
supports are provided. There are some exercises also grammatical category are displayed in the left
for the users in recognizing alphabets according to window.
the pronunciation.

184
 5. Panini Astadhyayi

In this module a user can get the Panini Sutra,

Sutra with Anuvrtti, its Sanskrit Explanation and
English Explanation after giving Astadhyayi Sutra
number or Siddhanta Kaumudi Sutra number.
The user will also get the information that how
many time the Astadhyayi Sutra occur in the
Siddhanta Kaumudi.

The user can select Astadhyayi Sutra or can enter

the Siddhanta Kaumudi Number (If the user gives
both Astadhyayi Sutra and Siddhanta Kaumudi
User can write the input word with the help of Number then the system will accept Astadhyayi
Phonetic Keyboard in the text box. On pressing the Sutra).
submit button the meaning with phonetic notation
and the grammatical category are displayed for that Example- For the inputted Astadhyayi Sutra
inputted word. Number 2.2.23, this Sutra occurs in two places
(Kaumudi Serial Number 529 and 829). If the
4. Dictionary of Nyaya Term user clicks any of the Kaumudi Sutra Number then
This dictionary of Nyaya terms is a part of an all the information of that Sutra will be displayed.
ambitious project of preparing an encyclopedia of If the user clicks Kaumudi Prakaran then all the
Indian logic. The main purpose is to resolve the sutra of that Prakaran will be displayed.
confusion and make uniformity in translating the
Nyaya terms in English. The stress has been given
to the clarity of concepts than on literal translation.
The root and meanings with phonetic notation of
Sanskrit Nyaya term is provided here.
For giving input, it is necessary to click on the
beginning letter of the word in the left window. It
will display all words, which are start with that letter
in the right side window. On clicking the particular
word the root and meanings with phonetic notation
are displayed in the left window.

6. Dhatu to DhatuRoop

Using this module a user can get all the

information of a dhatu, i.e. a user can get dhatu’s
meaning, its gan name, pad name and the roops
after selecting the lakar. For giving input, it is
necessary to click on the beginning letter of a
word in the left window. It will display all words,
which are started with that letter in the right side
window.

185
 Sandhi rules and processes. Sutra number in
Astyadhayi and its description is displayed. User can
learn three type of Svara Sandhi, Vyanjan Sandhi,
Hal Sandhi through this Sandhi module Data is in
Unicode. Sandhi exceptions and options are also
incorporated.
This is the Flowchart for this module. These are the
procedures written to make this module user friendly
and more descriptive.

Example- The inputted dhatu ‘ni’ and the

information of the dhatu ‘ni’ is displayed. The 10
lakar name is displayed the user can select any lakar
and get the roops of that lakar. As this dhatu is an
Ubhyapada Dhatu, the user has to select the pad also
in the next page.
7. Dhatu Roop to Dhatu
This module is the reverse of the Dhatu to DhatuRoop
module. In this module a user can get the dhatu
and its attributes from a dhatu roop.
Example- For the given DhatuRoop “Bhavati”, User
can get all the informations related to that
Dhaturoop like Dhatu Name, Dhatu Arth, Dhatu
Gana, Dhatu Pada, Dhatu Lakar, Purush and
Vachan.

Input Procedure : This procedure is written to take

8. Sandhi
Sandhi means the coalescence of two words coming
in immediate contact with each other. Using this
module the user can get the information about
the input from the user in the form of Unicode
data. Phonetic keyboard is used for this purpose
by which everybody can use this module easily.
CheckSandhi procedure: This procedure is used to
find the Sandhi that appears with the given input.
Panini Ashtadhyayi Sutra Number: This procedure
connects with Ashtadhyayi module. Click on
Sutra number displays the sutras used.
This module takes two words as input. First word
cannot be null but second word can be. A user

186
can input the two words and submit the form to
get the result of the given input.
a. Fill the first word in the given textbox captioned
‘Input First Word’.
b. Fill the second word in the given textbox
captioned ‘Input Second Word’.
c. Click the Submit Button.
Example- For the inputted first word “Dyata” and
second word “ari” It shows all the explanations
related to this Sandhi like its Sutra definition, which
applies in process and the Sutra. It is also displays
the Sutra number in Ashtadhyayi. This hyperlink
leads to the more details of this sutra.
First input word + Second input word = resultant
word
9. Japanese Lessons
The Japanese Lessons are very useful to introduce
the Japanese Language. These are mainly for the
Indians who know Hindi well can learn Japanese as
the contents in the lessons are described in Hindi.
Few Screen shots are given below.
Japanese Language contains a rich set of alphabets.
It is important for the learner to be familiar with
the set of alphabets. Keeping this in mind the most
of fourteen lessons describe the basics of the Japanese
Language with simple examples and explanations.
Grammars of the Japanese Language are discussed
not elaborately but lightly to give idea of verb, tense,
nouns, adjective etc. Care is taken not to make thinks
complicated.
Kanji is another thing that adds a different value to
the Japanese language. The language has a rich set
of Kanji. Kanji’s are explained with their stroke
order.
Lots of facilities are presented to make the learning
process easy and attractive. User can see all the words
of the total lessons from any page. Again the new
words of a section are also provided to the user. From
these words list a user go to the Japanese-Hindi
Lexicon for the details of the word. From the lessons
of alphabet learning a user can go to the Japanese
script learning module that see how the alphabets
can be written and pronounced. These script learning
includes all alphabets with Kanji.
10. Japanese Script learning and Character
Recognition System
Scripts are the basic of a Language. And writing of
these scripts are very important to learn a Language.
Using this module the user learns Japanese alphabets
and learns how to write and pronounce them.
Writing strokes for a character are shown with stroke
orders with four different speeds. It takes care of all
characters with Kanji. Recognize a character after
hearing its pronunciation is also over. It is also hyper-
linked with Japanese Lessons. User can access this
module from Japanese Lesson.
There are different types of alphabets in Japanese
Language. All these are covered here. The writing
process is made user friendly. There are four options
to choose the speed of writing for an Alphabet. And
there is audio support to listen the pronunciation of
the alphabet. Emphasize is given on the stroke order
187
according to which the alphabets are correctly
written.
Except Basic alphabets there are numbers of Kanji
in Japanese Language. Their formation and the
writing process in the proper stroke orders are given.
Pronunciation of alphabet is important to speak a
language. The attention is given for the right and
clear pronunciation of the alphabets and audio
supports are provided.
11. Japanese-Hindi Lexicon
In this module a user can get the attributes of a
Japanese word in Hindi. It is very helpful to the
user who know Hindi and wants to learn Japanese
Language. The facility for getting input from the
user is provided in both ways. Users can click the
beginning letter of the word. It will display all words,
which start with that letter in the right side window.
On clicking particular word the meaning with
phonetic notation and the grammatical category are
displayed in the left window.
188
12. HTML Content Converter
This converter converts ISCII/ISFOC based HTML
content to Unicode based HTML content. It
preserves the structure of the HTML content. It to
a larger extent also preserves the presentation of the
HTML content. It also incorporates the content
encoding directive to UTF-8. So that the resultant
output of the converter is complete in all respect.
Higher level design of the converter:
1. The converter reads the input file
2. Looks for the presence of ISFOC string inside
the content
3. Whenever it finds an ISFOC string it converts
them to ISCII
4. Thereafter it converts the ISCII string to Unicode
data
5. It writes the resultant Unicode data to the output
file.
6. It also updates the content encoding inside the
output data
This is the flow chart of the HTML Content
Converter.
 3. The center will collect and make a repository
of related work being done at national and
international centers. Such as C-DAC, IIT
Kanpur, IISC Banglore, JETRO etc. The
center will establish active contacts with these
national and international bodies for
acquisition of public domain material on the
subject.
4. The format of the Lexicons will be advertised
and discussed with other RCILTS
5. The center will collect and place on web some
more texts such as Hitopdesh etc.
6. Center must enhance existing systems such
as lexicons and developed systems such as
Sandhi and Sandhi Vicched Modules leading
to the development translation aid system for
Sanskrit to other Indian Language and Voice-
Versa. The centers must acquires Hindi/
Other Converters Sanskrit Morphological Analyzers and then
1. ISCII Text to UNICODE Text Converter improve upon them.

2. Unicode Text to ISCII Text Converter 7. Center will also enhance OCR performance
for Sanskrit.
3. ISCII Database to UNICODE Database
8. The Center will also enhance Spell Checkers,
Converter
Basic Word Processors and Text Editors for
4. UNICODE Database to ISCII Database Sanskrit.
Converter
9. The Center will procure a Message Server.
5. Devanagri_Unicode (Phonetic) Keyboard
10. The task on OCR, Fonts, Spell Checker, Text
driver
Editor, Word Processor and Message Server
6. Devanagri_Unicode (Inscript) Keyboard will also be taken up. However, some of these
driver activities may be taken up right away
depending on the currently available funds
7. Japanese_Unicode Keyboard drivers (under
with the resource center.
development)
Following work being done at the center would
13. Current/Future tasks
be available fully to users on Internet ending
1. The utilities developed, being developed and September 30, 2003
to be developed at RCILTS, JNU will be Sanskrit Modules
available on the web.
1. Ashtadhyayi of Panini with Sidhant Kaumudi
2. RCILTS, JNU will participate in the of Nagesh Bhatta (English Tika) and
development of at least two Devanagari Prabhakari Tika (Hindi). In addition
Unicode Fonts (1 true type and 1 open type) Katyayan’s Vartik on Ashtadhyayi will also be
locally, by procuring or by outsourcing. available.

189
2. Dhatu Ratnakar with all forms of all Dhatus. (Sanskrit to other Indian Languages and voice-
versa). A simple Hindi to Japanese and Japanese
3. Pratyahar Module.
to Hindi translators is also planned. The Center
4. Ten Lessons on Sanskrit Sambhasn. also plans to develop contents on Hitopdesh kind
of material both for Sanskrit and Japanese.
5. Samgya word forms (Pratipadic) with 1,000
words. There is non-availability of good expertise in the
field of Chinese language. Centre is therefore of
6. Sarvnam Module with all Pronouns.
the opinion that the center may stop its efforts in
7. Sanskrit-English Lexicon ( Word Meanings developing systems for Chinese language and
only) with 30,000 words. concentrate its energy on Sanskrit and Japanese.
8. English-Sanskrit Lexicon ( Word Meanings Courtesy: Prof. G.V. Singh
only) with 30,000 words. Jawaharlal Nehru University
School of Computer and Systems Sciences
9. 30 Sanskrit Lessons with Exercises. New Mehrauli Road
10. Sandhi Prakaran with exceptions and options. New Delhi – 110 067
(RCILTS for Sanskrit, Chinese & Japanese)
11. Lists of other Sanskrit words, such as Tel: 00-91-11-26107676,26101885
adjectives, conjunctions etc. E-mail: gvs10@hotmail.com
12. Devanagri Script Learning Module with
Matras and Conjuncts.
Japanese Modules
1. 21 lessons with exercises. These lessons shell
constitutes a course which is more than
equivalent to one year diploma course in
Japanese.
2. A simple Hindi to Japanese translation
exercise module.
3. Japanese-Hindi Lexicon ( Word Meanings
only) with 6,000 words.
4. Hindi-Japanese Lexicon ( Word Meanings
only) with 6,000 words.
5. Hiragana Script Learning Module.
6. Katakana Script Learning Module.
7. Kanjhi Script Learning Module.
In addition to adding more lessons the center plans
to augment the lexicons with audio visual support,
usage of words for meanings and add phrasal
grammatical categories of words. Sandhi Vicched
Module will be developed which will aid the
development of simple translation systems

190
 6. TDIL Associates Achieve Honours…Congratulations !
Congrats Dr. Vikas!
On being awarded VIGYAN BHUSHAN by
Hon’ble Prime Minister in a ceremony
organized by UP Hindi Sansthan at Lucknow
on 21 May, 2003.
About Dr. Om Vikas (omvikas@mit.gov.in)
[National Coordinator of TDIL Mission]
B.Tech, M.Tech & Ph.D from I.I.T Kanpur. Fellow of IETE and
member of IEEE. Recipient of Fellow of Russian Academy of
Informatization of Education, Vigyan Bhushan, Atmaram, Vishisht
Padak, Indira Gandhi Rajbhasha awards.
After M.Tech, he served in TCS. Joined NIC in 1977 as Senior
Systems Analysis and rose to the position of Senior Director in 1998.
Served on deputation as visiting professor at IIT Kanpur and
NCERT; and as counselor for Science & Technology in Indian
Embassy at Tokyo, Japan. Under UNDP fellowship studied large
database designs in USA, Canada & Europe (1980).
His research interests include computer architecture, database
design, Language informatics. He had been on program committees
of various international conferences.
He has significantly contributed towards promoting Language
Informatics, Computer Manpower Development, International
Cooperation and Technical Hindi.
Sensitivity to society is evident from his initiatives of publishing
ALOK newsletter in Hindi during his B.Tech; founding People’s
Science Society (Lok Vigyan Parishad) in 1986; launching TDIL
(Technology Development for Indian Languages) in 1990s and
coordinating it in mission mode. Founder editor of
VishwaBharat@tdil
Currently, Senior Director / Scientist ‘G’ and Head, TDIL
(Technology Development for Indian Languages) Mission, Min. of
Communication & IT, New Delhi - 110003.
Congrats Prof. Dhande !
On becoming Director of Indian Institute of
Technology, Kanpur
About Prof. Sanjay Dhande (sgd@iitk.ac.in)
[Chief investigator of RC for ILTS]
Ph.D Mechanical Engineering, IIT Kanpur
and currently he is Director, Indian Institute
of Technology, Kanpur & Professor in Mechanical Engineering
and Computer Science Department, IITK.
He also held the post of Asst. Professor (1979 –85), Professor
(1985 to Till Date), Visiting Professor in Virginia Tech, Virginia,
USA (1992), Head (Mechanical Engg.) (1993 – 95), Dean
(Research and Development) – (1999 -2001), Director (2001
– till date).
He has published numerous papers in National and
International Journals. He has also written books : Kinematics
and Geometry of Planar and Spatial Cam Mechanisms”,
published by Wiley Eastern Ltd., “Computer Aided Design and
Manufacture”, published by the Committee on Science &
Technology in Developing Countries (COSTED), Singapore and
“Computer Aided Engineering Graphics and Design “ is under
publishing by Wiley Eastern Publishers Ltd.
Congrats Prof. Sinha
for Successfully launching on-line Machine
Translation System (http://
anglahindi.iitk.ac.in)
About Prof. R.M.K Sinha (rmk@iitk.ac.in)
[Chief investigator of MT Projects]
Ph.D (CS) IIT Kanpur M.Tech in industrial
Electronics (Electronics & communication Engineering) Indian
Institute of Technology, Kharagpur, 1969, M.Sc.Tech in Electronics
and Communication, University of Allahabad, 1967.
He has taught at many Institutions/Universities in India and has
also been visiting professor/faculty to USA, Canada and Bangkok
Institutions/Universities. Currently he is Professor at Dept. of CS
and Dept. of EE IIT Kanpur. He has published very exhaustively
especially in the area of Indian Language Technologies in various
National and International journals.
He received “Man of the Year Award” from American Biographical
Institute, USA. He is fellow IETE, India, Senior member IEEE,
USA and serves many other organizations in various capacities.
Development of Indian Language Technologies are his main area of
interest which includes AI, NLP, MT, SST, OCR, Document
processing etc. He is credited with development of INSCRIPT
Keyboarding and Coding schemes (ISCII), Integrated Devanagari
Computer Terminal, GIST Terminal, Transliteration among Indian
Languages, Spell-Checker Design, OCR, Anglabharati Machine
Aided Translation System for English to Hindi and English to Urdu
for windows and Solaris/Linux OS. HindiAngla the Hindi to Angla
MAT is in advanced stages of development. He is also working on
Developing MAT system for all Indian Languages, Speech to Speech
Translation System and Lexical Knowledge –Base development named
ShabdKalpTaru..
Congrats Prof. Lehal !
On becoming professor at Punjabi University,
Patiala.
[Chief Investigator of RC-ILTS (Punjabi)]
About Prof. Gurpreet Singh Lehal
(gslehal@lycos.com)
Ph.D (CS) Punjabi University, M.E (CS) TIET
Patiala, M.Sc. Math’s (Honours) from Punjab University.
He has attended, presented and published many technical papers
in the area of Indian Language Technology. He has been working for
more than five years on different projects related to computerization
of Punjabi and guiding many M.Tech. and Ph.D scholars on different
topics related to technological development of Punjabi.
As chief Investigator of the project “Resource Centre for Indian
Language Technology Solutions- Punjabi(April 2000 - June 2003)”
his contributions are Gurmukhi OCR, Punjabi Word Processor,
Gurmukhi to Shahmukhi Transliteration system (in collaboration
with CDAC, Pune), Punjabi Spell checker, Punjabi Sorting Program,
Punjabi font converter, On-line Punjabi-English, English-Punjabi
and Hindi-Punjabi Dictionaries and Web site for On-Line teaching
of Punjabi
The Gurmukhi OCR and Punjabi word processor are ready for
commercialization and already there is a big demand for these
products in India as well as abroad in UK, USA and Canada. MOU
for transfer of technology of the Punjabi spell checker is being finalised
with M/S Modular Systems, Pune.
191
 7. Resource Centres Technology Index

I. Indian Institute of Technology, Kanpur (Hindi, State Government and the Industry in the
Nepali) region.
7.1 Technical Skills
1. Machine Translation
7.2 Products Which can be Launched
2. Speech-to-Speech Translation
7.3 IT Services
3. Lexical Knowledge-Base Development
8. IT Services - Multimedia CDs
4. Optical Character Recognition
8.1 Learn Gujarati Through English
5. Transliteration
8.2 Adolescent Health
6. Spell Checker Design
8.3 Bibliography of books and journals
7. Knowledge Resources
8.4 Classic Knowledge-base
7.1 Gitasupersite
9. The Team Members
7.2 Brahmasutra
7.3 Complete Works of Adi Sankara III. Indian Institute of Technology, Mumbai
7.4 Ramcharitmanas (Marathi, Konkani)
7.5 Upanishads
1. Lexicon And ontology
7.6 Kavi Sammelan
1.1 Lexicon
7.7 Bimari-Jankari
1.2 Ontology
7.8 Paramhans Ram Mangal Das Ji
2. The Hindi Wordnet
7.9 Nepali Texts
3. The Marathi Wordnet
8. Technical Issues
4. Automatic Generation of Concept Dictionary
9. Details of Texts/Commentaries for each of the
& Word Sense Disambiguation
Upanishads
5. Hindi Analysis and Generation
10. The Team Members
6. Marathi Analysis
II. M.S. University of Baroda, Vadodara (Gujarati) 7. Speech Synthesis for Marathi Language
8. Project Tukaram
1. Knowledge Resources
9. Automatic Language Identification of
1.1 Gujarati WordNet
Documents using Devanagari Script
2. Knowledge Tools
10. Object Oriented Parallel & Distributed Web
2.1 Portal
Crawler
2.2 Multilingual Text Editor
11. Designing Devanagari Fonts (Three Types )
2.3 Code Converter
12. Low Level Auto Corrector
2.4 Gujarati Spell Checker
13. Font Converters
3. Translation Support Systems
14. Marathi Spell Checker
3.1 Machine Translation
15. IT Localisation
4. Human Machine Interface Systems
16. Publications
4.1 OCR for Gujarati
17. The Team Members
4.2 Text To Speech (TTS)
5. Localization IV. Thapar Institute of Engineering & Technology,
5.1 Language Technology Human Resource Patiala (Punjabi)
Development
1. Products Developed
6. Standardization
1.1 Spell Checker
6.1 Unicode
1.2 Font Converter
7. Products which can be launched and the service
1.3 Sorting Utility
which the Resource Centre can provide to the
1.4 Bilingual Punjabi/English Word processor

192
 1.5 Gurmukhi OCR 3.3 Information Retrieval System for Bangla
1.6 Gumukhi to Shahmukhi Transliteration Documents
2. Contents Uploaded on Internet 3.4 Script Identification and Separation from
2.1 Punjabi Classic Literature Indian Multi-Script Documents
2.2 Bilingual Dictionaries and Glossary 4. Research & Development
• Punjabi English On-line Dictionary 4.1 Automatic Processing of Hand-Printed
• English Punjabi On-line Dictionary Table-Form Documents
• Hindi Punjabi On-line Dictionary 4.2 Research and Development of Neural
• Glossary of English-Punjabi Network Based Tools for Printed
Administrative Terms Document (in eastern regional scripts)
2.3 On Line Teaching of Punjabi Processing
2.4 On Line Font Conversion Utility 5. The Team Members
2.5 Downloads VI. Utkal University (Oriya)
• Punjabi Spell Checker
• Punjabi Fonts 1. Intelligent Document Processing (OCR for
3. Interaction with Punjab State Govt. Oriya)
2. Natural Language Processing (Oriya)
4. Publications
2.1 Oriya Machine Translation System
5. The Team Members
(OMTrans)
V. Indian Statistical Institute, Kolkata (Bangla) 2.2 Oriya Word Processor (OWP)
1. Core Activities (Multilingual)
2.3 Oriya Morphological Analyser (OMA)
1.1 Website Development and the Language
2.4 Oriya Spell Checker (OSC)
Design Guide
2.5 Oriya Grammar Checker (OGC)
1.2 Training Programmes
2.6 Oriya Semantic Analyzer (OSA)
2. Services
2.7 E-Dictionary (Oriya 1 English)
2.1 Corpus Development
2.8 OriNet (WordNet for Oriya)
• Printed Bangla Document Images &
2.9 SanskritNet (Word Net in Sanskrit)
Ground Truth for OCR and Related
3. Speech Processing System for Indian Languages
Research
3.1 Text to Speech (TTS)
• Development of Bangla Text Corpus
3.2 Speech To Text (STT)
in Electronic Form (including a Bangla
4. The Team Members
Dictionary, and Several Bangla Classics)
• Electronic Corpus of Speech Data Orissa Computer Application Centre (Oriya)
2.2 Font Generation and Associated Tools 1. Products Developed at OCAC
• Public Domain Bangla Font 1.1 Oriya Spell Checker
Generation 1.2 Thesaurus in Oriya
• Converter-Font File to ISCII File and 1.3 Bilingual Electronic Lexicon
Vice-Versa. 1.4 Corpus in Oriya
• Bangla Text Editor 1.5 Bilingual Chat Server
• Bangla Spell Checker 1.6 Net Education
3. Products 1.7 XML Document Creation and
3.1 OCR system for Oriya Manipulation
3.2 Adaptation of Bangla OCR to Assamese 1.8 Oriya OCR

193
 1.9 Oriya E-Mail VIII. Indian Institute of Science, Bangalore,
1.10 Oriya Word Processor with Spell Checker (Kannada)
under Linux
1. Web Sites and Support to Instruction
1.11 Computer Based Training (in Oriya)
1.1 Kannudi
1.12 Oriya Language Based E-Governance
1.2 LT-IISc
Applications
1.3 Bodhana Bharathi : Multimedia Educational
2. ToT done for various projects so far.
CDs for 7th, 8th and 10th Standards
3. Training Programmes run for the officials for 1.4 Bilingual Instructional Aid for Learning
the state Govt. German through Hindi
4. Core Activities 1.5 Information Base in Hindi Pertaining to
4.1 Web Hosting of Oriya Languages Classics German History
4.2 Hosting of Web Sites of Govt. Colleges in 2. Knowledge Bases
Orissa. 2.1 Sudarshana a Web Knowledge base on
5. Products Proposed to be Developed Darshana Shastras
6. The Team Members 2.2 Indian Logic Systems
VII. Indian Institute of Technology, Guwahati 2.3 Indian Aesthetics
(Assamese & Manipuri) 3 Technologies and Language Resources
3.1 Brahmi : Kannada Indic Input Method,
1. Knowledge Resources Word processor
1.1 Corpora 3.2 Open Type Fonts : Sampige, Mallige,
• Assamese Corpora Kedage
• Manipuri Corpora 3.3 Kannada Wordnet
1.2 Dictionaries 3.4 OCR for Tamil
1.3 Design Guides 3.5 OCR of Printed Text Document in
1.4 Phonetic Guides Kannada
2. Knowledge Tools 4. Research
2.1 The RCILTS, IIT Guwahati Website 4.1 Automatic Classification of Languages
2.2 Fonts Using Speech Signals
2.3 Spell Checker 4.2 Algorithms for Kannada Speech Synthesis
2.4 Assamese Language Support for Microsoft 5. Publications
Word
2.5 Morphological Analyzers IX. University of Hyderabad (Telugu)
3. Translation Support System 1. RC for ILTS
4. Human Machine Interface Systems 2. Products
4.1 Optical Character Recognition for 2.1 DRISHTI: OCR
Assamese and Manipuri 2.2 Tel-Spell : Spell Checker
4.2 Speech Recognition System 2.3 AKSHARA : Advanced Multi-Lingual
4.3 Interface for e-Dictionary Text Processor
5. Language Technology Human Resource 2.4 E-mail in Indian Languages
Development 2.5 WILIO : Interactive Web Page in Indian
6. Standardization Languages
7. Publications 2.6 Telugu Corpus
8. Manika Newsletter 2.7 Dictionaries, Thesauri & other Lexical
9. The Team Members Resources

194
 2.8 Morphology 2.2 AKSHARAMAALA TM – Malayalam Font
2.9 Stemmer Package and Script Manager
2.10 Part of Speech Tagging 2.3 Text Editor
2.11 VIDTA : Comprehensive Toolkit for 2.4 Code Converters
Web-Based Education 2.5 ANWESHANAM TM – Malayalam Web
2.12 Grammars & Syntactic Parsers Based Search Engine
2.13 Machine Aided Translation 2.6 Malayalam Portal
2.14 Tools 3. Services
2.14.1 Font Decoding 3.1 SANDESAM TM – Malayalam E-mail Server
2.14.2 Web Crawler for Search Engine 3.2 Malayalam E-Com Application
2.14.3 PSA : A Meta Search Engine 4. Knowledge Resources
2.14.4 Corpus Analysis Tools 4.1 Trilingual Dictionary
2.14.5 Website Development Tools 5. Language Tutors
2.14.6 Character to Font Mapping Tools 5.1 Ezhuthachan – The Malayalam Tutor
2.14.7 Dictionary to Thesaurus Tools 5.2 English Tutor
2.14.8 Dictionary Indexing Tools 6. Other Activities
2.14.9 Text Processing Tools 6.1 Providing Technology Solutions
2.14.10 Finite State Technologies Toolkit 6.2 Interaction with State Government
3. Service & Knowledge Bases 6.3 Training Activities
3.1 On-line Literature 6.4 COWMAC (Consortium of Industries
3.2 History-Society-Culture Portal working in the area of MAlayalam
3.3 On-Line Searchable Directory Computing)
3.4 Character Encoding Standards, Roman 7. Publications
Transliteration Schemes, Tools 8. Expertise gained
3.5 Research Portal 9. Future Plans
3.6 VAANI : A Text to Speech System for 10. The Team Members
Telugu
XI. School of Computer Science & Eng., Anna
3.7 Manpower Development
University Chennai (Tamil)
4. Epilogue
4.1 Strengths and Opportunities 1. Knowledge Resources
4.2 Outreach 1.1 Online Dictionary
5. Publications 1.2 Corpus Collection Tools
6. The Team Members 1.3 Contents Authoring Tools
1.4 Tamil Picture Dictionary
X. Centre for Development of Advanced
1.5 Flash Tutorial
Computing – Thiruvananthapuram (Malayalam)
1.6 e-Handbook of Tamil
1. Human Machine Interface Systems 1.7 Karpanaikkatchi- Scenes from Sangam
1.1 NAYANA TM – Optical Character Literature.
Recognition System for Malayalam 1.8 District Information
1.2 SUBHASHINI TM – Malayalam Text to 1.9 Educational Resources
Speech System (TTS) 2. Knowledge Tools
2. Knowledge Tools 2.1 Language Processing Tools
2.1 NERPADAM TM – Malayalam Spell • Morphological Analyser
Checker • Morphological Generator

195
 2.2Text Editor
2.3Spell Checker
2.4Document Visualization Tool
2.5Utilities
• Code Conversion Utility
• Tamil Typing Utility
3 Translation Support System
3.1 Tamil Parser
3.2 Universal Networking Language (UNL) for
Tamil
3.3 Heuristic Rule Based Automatic Tagger
4. Human Machine Interface System
4.1 Text-to-Speech (Ethiroli)
4.2 Poonguzhali (A Chatterbot)
5. Localization
5.1 Tamil Office Suite (Aluval Pezhai)
• Tamil word Processor (palagai)
• Presentation Tools for Tamil (Arangam)
• Tamil Database (Thaenkoodu)
• Tamil Spread Sheets (chathurangam)
5.2 Tamil Search Engine (Bavani)
6. Language Technology Human Resources
Development
6.1 Creating awareness among the general
public
6.2 Language Technology Training
6.3 Co-ordination with Linguistic & Subject
Exports
6.4 Co-ordination with Govt. & Industry
7. Standardization
8. Research Activities
8.1 Text Summarization
8.2 Latent Semantic Indexing
8.3 Knowledge Representation System based
on Nyaya Shastra
9. Publications
10. The Team Members
XII. Perso-Arabic Resources Centre, CDAC, GIST,
Pune (Urdu, Sindhi, Kashmiri)
1. The PASCII Standard
1.1 Characteristics of Proposed Standard
1.2 Standardization of Urdu Keyboard
1.3 Standardization of Fonts
196
2. Script Core Technology & Rule Engine
2.1 Glyph Property Editor
2.2 Rule Engine
2.3 Testing Version of Urdu Editor
2.4 Font Glyph Editor
2.5 The Font Editor Utility
3. Devanagari/Gurmukhi to Urdu Transliteration
“UTRANS”
3.1 Following is a Sample of Hindi Text
Transliterated into Urdu (UTRANs)
3.2 Following is a Sample of Punjabi Text
Transliterated into Urdu (UTRANs)
4. Dictionary Development Tools
5. Nashir – Word processor
6. The Urdu SDK Controls
7. Pocket Translator
8. Multipromter
9. LIPS Advance Creation Station & LIPS for
DVD
10. Perso-Arabic Web Site Development
XIII. Jawaharlal Nehru University, Delhi (Sanskrit,
Japanese, Chinese)
1. Sanskrit Lessons and Exercises
2. Devanagari Script Learning and Character
Recognition System
3. Sanskrit-English Lexicon
4. Dictionary of Nyaya Term
5. Panini Astadhyayi
6. Dhatu to DhatuRoop
7. DhatuRoop to Dhatu
8. Sandhi
9. Japanese Lessons
10. Japanese Script Learning and Character
Recognition System
11. Japanese-Hindi Lexicon
12. HTML Content Converter
12.1 Other Code Converters
13. Current/Future Tasks
 List of Resource Centers
◗ Indian Institute of Technology
Department of Computer Science & Engineering
Kanpur - 208 016
(RCILTS for Hindi & Nepali)
Prof R M K Sinha
Tel : 00-91-512-2597174,2598254
E-Mail : rmk@iitk.ac.in
Website : http://www.cse.iitk.ac.in/users/langtech
◗ Indian Statistical Institute
Computer Vision and Pattern Recognition Unit
203, Barrackpore Trunk Road, Kolkata –700035
(RCILTS for Bengali)
Prof. B.B Chaudhary
Tel : 00-91-33-25778086 Extn 2852, 25781832, 25311928
E-Mail : bbc@isical.ac.in
Website : http://www.isical.ac.in/~rcbangla/
◗ Indian Institute of Technology
Department of Computer Science & Engineering
Mumbai- 400 076
(RCILTS for Marathi & Konkani)
Prof. Pushpak Bhattacharya
Tel : 00-91-22-25767718,25722545 Extn 5479, 25721955
E-Mail : pb@cse.iitb.ernet.in
Website : www.cfilt.iitb.ac.in
◗ University of Hyderabad
Dept. of CIS
Hyderabad -500046
(RCILTS for Telugu)
Prof. K. Narayan Murthy
Tel : 00-91-40-23100500, 23100518 Extn 4017, 23010374
E-Mail : nmcs@uohyd.ernet.in
Website : http://www.languagetechnologies.ac.in/
◗ Jawaharlal Nehru University
School of Computer and Systems Sciences
New Mehrauli Road, New Delhi – 110 067
(RCILTS for Sanskrit, Chinese & Japanese)
Prof. G.V. Singh
Tel: 00-91-11-26107676, 26101885
E-Mail : gvs10@hotmail.com
Website : http://www.jnu.ac.in
◗ Thapar Institute of Engineering & Technology
Department of Computer Science & Engineering
Patiala 147 001
(RCILTS for Gurmukhi)
Prof. R.K. Sharma
Tel: 00-91-175-2393137/393374, 2283502
E-Mail : rksharma@mail.tiet.ac.in
Website : http://punjabirc.tiet.ac.in
◗ ER & DCI, Vellayambalam
Thiruvananthapuram -695 033
(RCILTS for Malayalam)
Prof. Ravindra Kumar
Tel: 00-91-471-2723333, 2725897, 2726718
E-Mail : ravi@erdcitvm.org
Website : http://www.malayalamresourcecentre.org/
index.jsp/index.jsp
◗ Anna University
School of Computer Science & Engineering
Chennai - 600 025
(RCILTS for Tamil)
Dr. T.V.Geetha/Ms. Ranjani Parthasarthi
Tel : 00-91-44-22351723, 22350397
Extn- 3342/3347, 24422620, 2423557
E-Mail : rp@annauniv.edu
Website : http://ns.annauniv.edu/rctamil/html/eindex.htm
◗ M.S.University of Baroda
Department of Gujarati, Faculty of Arts, Baroda-390 002
(RCILTS for Gujarati)
Shri Sitanshu Y. Mehta
Tel : 00-91-265-2792959
E-Mail : rciltg@satyam.net.in
Website : http://msubaroda.ac.in/rciltg/
◗ Orissa Computer Application Centre
OCAC Building, Plot No. 1/7-D,
Acharya Vihar Square, RP-PO, Bhubaneswar – 751 013
(RCILTS for Oriya)
Shri S.K. Tripathi
Tel : 00-91-674-2582484, 2585851, 2554230(R), 2582490
E-Mail : saroj@ocac.ernet.in
Website : http://www.ilts-utkal.org
◗ Utkal University
Department of Computer Science & Application
Vani Vihar,) Bhubaneswar – 751 004
(RCILTS for Oriya)
Prof. (Ms) Sanghmitra Mohanty
Tel : 00-91-674-2585518, 254086
E-Mail : sangham1@rediffmail.com
Website : http://www.ilts-utkal.org
◗ Indian Institute of Science
Centre for Electronics Design and Technology (CEDT)
Bangalore – 560 012
(RCILTS for Kannada)
Prof. N.J. Rao
Tel. : 00-91-80-3466022, 3942378, 3410764
E-mail : jrao@mgmt.iisc.ernet.in
chairman@mgmt.iisc.ernet.in
◗ Indian Institute of Technology
Department of Computer Science & Engineering
Panbazar, North Guwahati, Guwahati -781 031, Assam
(RCILTS for Assamese & Manipuri)
Prof.Gautam Barua
Tel : 00-91-361-2690401, 2690325-28 Extn 2001, 2452088
E-Mail : gb@iitg.ernet.in, g_barua@yahoo.com
Website : http://www.iitg.ernet.in/rcilts
◗ Centre for Development of Advanced Computing
Pune University Campus, Ganesh Khind Road
Pune - 411 007
(RCILTS for Urdu, Sindhi & Kashmiri)
Shri M.D. Kulkarni
Tel : 00-91-20-25694000, 25694002-09
E-Mail : mdk@cdac.ernet.in
Website : http://parc.cdacindia.com
197
 List of CoIL-Net Centers

◗ Indian Institute of Information Technology ◗ Indian Institute of Technology

Management Kanpur, Uttar Pradesh
Gwalior (Hindi to English Machine-aided Translation System
Madhya Pradesh based on Anubharati Approach)
(IT localization solutions for Madhya Pradesh) Prof. R.M.K. Sinha
Prof. D.P. Agrawal Tel : 00-91-512-2597174
Tel : 00-91-751-2449701 E-Mail : rmk@iitk.ac.in
E-Mail : prof_dpa@hotmail.com
◗ Indira Gandhi National Centre for the Arts
◗ Birla Institute of Technology New Delhi
Mesra, Ranchi (Development of Digital Library for Regional Heritage)
Jharkhand Prof. N.R Setty
(IT localization solutions for Jharkhand) Tel : 00-91-11-23385277
Dr. P.K. Mahanti E-Mail : msignca@yahoo.com
Tel : 00-91-651-2275333
◗ Institution of Electronics and Telecommunication
E-Mail : deptcom@bitsmart.com
Engineering
◗ Banasthali Vidyapith New Delhi
Banasthali, Rajasthan Wg Cdr (Retd) Dr. M.L. Bala
(IT localization solutions for Rajasthan) (IT Learning material in Hindi)
Dr. Aditya Shastri Tel : 00-91-11-4631810
Tel : 00-91-1438-228647/28787 E-Mail : ietend@giasdl01.vsnl.net.in
E-Mail : shastri@bv.ernet.in ◗ Centre for Development of Advanced Computing
◗ Banaras Hindu University Pune
Institute of Technology (Core Technology Development for Hindi)
Uttar Pradesh Shri Mahesh D. Kulkarni
(IT localization solutions for Uttar Pradesh) Tel : 00-91-20-25694000,25694002-09
Dr. K. K. Shukla E-Mail : mdk@cdac.ernet.in
Tel : 00-91-542-2307055/56 Other Centers
E-Mail : shukla@ieee.org
◗ Centre for Development of Advanced Computing
◗ New Government Polytechnic “Anusandhan Bhawan”
Patliputra Colony, Patna C-56/1, Institutional Area, Sector-62
Bihar Noida-201301
(IT localization solutions for Bihar) (Development of Parallel Text Corpora for 12 Indian
Dr. R.S. Singh Languages & Annotated Speech Corpora for Hindi,
Tel : 00-91-612-2262866/700 Marathi &Punjabi)
E-Mail : polypat@bih.nic.in Shri. V.N. Shukla
Tel : 00-91-95120-2402551-6
◗ School of IT
E-Mail : vnshukla@cdacnoida.com
G.G. University
Bilaspur,Chhatisgarh ◗ Centre for Development of Advanced Computing
(IT localization solutions for Chhatisgarh) Plot E2/1,Block GP, Sector V, saltlake
Dr. Anurag Shrivastva Kolkata- 700091
Tel : 00-91-7752-272541 West Bengal
E-Mail : sanurag_cmt@yahoo.co.in (Development of Annotated Speech Corpora for
Bengali, Assamese & Manipuri)
◗ Indian Institute of Technology
Dr. A.B. Saha
Roorkee, Uttranchal
Tel : 00-91-33-23579846, 23575989
(IT localization solutions for Uttaranchal)
E-Mail : amiya.saha@erdcical.org
Dr. R. C. Joshi
Tel : 00-91-1332-285650
E-Mail : joshifcc@iitr.ernet.in

198
 Quick Reference to Previous Issues

Contents Jan. 2001, ek?k ' Contents January 2002, ek?k &

1. Message of Hon’ble Minister Sh. Pramod Mahajan 1-L 1. Calendar of Events-Year 2002 1L-1R
2. TDIL Programme 1-R 2. Reports
3. Overcoming the Language Barrier 2-L 2.1 TDIL Vision 2010 2L-3R
4. Achievements 2-L 2.2 LT-Business Meet & TOT 2001 4L-16L
5. Resource Centres for Language Technology Solutions 3-R 2.3 Intellectual Property Rights (IPR) 16R-16R
6. Potential Products & Services 4-L 2.4 UNESCO-Symposium on Language 17L-19R
7. TDIL Website 4-L in Cyber Space
2.5 SCALLA 2001-Sharing Capability 20L-20R
8. Implementation Strategy 4-L
in Localisation & Human Language
9. International Programmes in 4-R Technologies
Multilingual Computing 2.6 UNESCO - Workshop on Medium 21L-22L
10. MNC Products Supporting Indian Languages 4-R Terms Strategy for Communicatons
11. Tentative List of Indian Language Products 5-L & Information
12. Portals Supporting Indian Languages 7-L 2.7 The Asia Pacific Development 22R-23L
13. Other Efforts 7-L Information Programme (APDIP)
2.8 Workshop on Corpus-based 23R-23R
14. Major Events of Year 2000 7-R
Natural Language Processing
15. Indian Language Technology Vision 2010 9-L 2.9 1st Workshop on Indian Language OCR 24L-24R
2.10 1st International Conference on 25L-25R
Global WordNet
Contents May 2001, T;s’B + 3. Standardization
3.1 Revision of Unicode Standard-3.0 26L-37R
1. Calendar of Events 1-L for Devanagari Script
2. TDIL Website 2-L 3.2 Design Guides (Sanskrit, Hindi, 38L-75R
Marathi, Konkani, Sindhi, Nepali)
3. TDIL Meet 2001 4-L
3.3 Indian Standard Font Code (INSFOC) 76L-77R
4. UNESCO Expert Group on Multilingualism in Cyberspace 9-L 3.4 Indian Standard Lexware Format 78L-86R
5. Lexical Resources for Natural Language Processing 10-L
4. 4.1 Reader’s Feedback 87L-87R
6. Symposium on Translation Support System 10-L
4.2 Frequently Asked Questions 88L-91R
7. Universal Networking Language 10-R
8. Indo-UK workshop on Language Engg. for South 11-R
Asian Languages
9. New Software Testing Facility 12-R
10. Now Domain Name in Regional Languages 12-R
Contents April 2002, pS= )
11. Book Shelf 13-L 1. tdil@Elitex 2002 1L-2L
12. Resource Center(s) for Indian Language 13-L
Technology Solutions 1st Year (2000 - 2001) Progress 1.1 TDIL Programme 2R-3L
13. Impediments in IT Localization & Penetration 15-L 2. Readers Feedback from Abroad 3R-3R
14. Feedback on UNICODE Standard 3.0 15-R
15. Indian Language IT Market 21-L 3. Calendar of Events-Year 2002 4L-4L
16. MS Office XP with Indian Language Support 21-L 4. Standardization 4R-4R
17. Call for Technologies 21-R
4.1 Gujarati Code Chart 6L-24R
4.1.1 Gujarati Code Chart Details
Contents Sept. 2001, +Éζ´ÉxÉ * 4.1.2 Gujarati Script Details
4.1.3 Typical Colloquial Sentences
Special Issue on Language Technology Business Meet
4.2 Malayalam Code Chart 26L-44R
Patrons Message 1-1 4.2.1 Malayalam Code Chart Details
Programme Schedule 1-1 4.2.2 Malayalam Script Details
4.2.3 Typical Colloquial Sentences
I Machine Aided Translation (MAT) 2-3
4.3 Oriya Code Chart 46L-62R
4.3.1 Oriya Code Chart Details
II Operating System (OS) 3-3 4.3.2 Oriya Script Details
4.3.3 Typical Colloquial Sentences
III Human Machine Interface System (HUMIS) 4-16
4.4 Gurmukhi Code Chart 64L-82R
4.4.1 Gurmukhi Code Chart Details
IV Tools 16-29 4.4.2 Gurmukhi Script Details
4.4.3 Typical Colloquial Sentences
V e-Content 30-34 4.5 Telugu Code Chart 84L-101R
4.5.1 Telugu Code Chart Details
VI Other Milestones 37-38 4.5.2 Telugu Script Details
4.5.3 Typical Colloquial Sentences
Quick Reference Guide 39-39 5. Quick Reference to Previous Issues 102L-102R
 Contents July 2002, vk"kk<+ - Contents January 2003, ek?k ,
1. Calendar of Events-Year 2002 1L-1R
1. Calendar of Events-Year 2003 1L-1R
2. TDIL Vision 2L-2R

3. Reader's Feedback 3L-3L

2. Reader’s Feedback 2L-2R
4. Universal Digital Library 3R-15R

5. Indo European IEMCT Conference 16L-17R 3. TDIL Vision 3L-3R

6. Indian Language Spell Checker Design Workshop 18L-24R

7. Indian Standard Font Code 25L-30R 4. Conference Reports 4L-12R

(Devanagari, Gujarati, Punjabi, Malayalam)

8. INSROT Revision 31L-32R

5. Language Technology Papers - Abstracts
(Indian Script to Roman Transliteration)

9. Unicode Standardization
5.1 Human Machine Interface 14L-38R
9.1 Bangla Code Chart 34L-50R
System (HUMIS)
9.1.1 Bangla Code Chart Details
9.1.2 Bangla Script Details a) Integrated I/O Environment
9.1.3 Typical Colloquial Sentences b) Optical Character Recognition
9.2 Assamese Language Details 52L-60R c) Speech Recognition
9.2.1 Typical Colloquial Sentences d) Speech Synthesis
9.3 Manipuri Language Details 62L-67R e) Speech Processing
9.3.1 Typical Colloquial Sentences f ) Typesetting
10. Quick Reference to Previous Issues 68L-68R
5.2 Knowledge Resources (KR) 42L-44R
Contents October 2002, dkfrZd . a) Corpora
b) Dictionary
1. Calendar of Events-Year 2002-03 1L-1L c) Lexical Resources
2. Reader’s Feedback 1R-2R

3. TDIL Vision 3L-3R 5.3 Knowledge Tools (KT) 46L-58R

a) Lexical Tools
4. Technology Watch 4L-4L
b) Utilities
5. MAT Evaluation & Benchmarking 5L-16R

6. OCR Evaluation & Benchmarking 17L-21R 5.4 Language Engineering (LE) 60L-64R
7. Transliteration Tables 22L-26L
a) Grammar
(Punjabi to Urdu & Hindi to Urdu) b) Language & Script Analysis
8. Unicode Standardization
5.5 Localisation (L) 66L-66R
8.1 Kannada Code Chart 28L-40R
8.1.1 Kannada Code Chart Details
a) System Software
8.1.2 Kannada Script Details
8.1.3 Typical Colloquial Sentences 5.6 Translation Support Systems (TSS) 68L-72R
8.2 Tamil Code Chart 42L-56R
8.2.1 Tamil Code Chart Details a) Machine Translation
8.2.2 Tamil Script Details b) Universal Networking Language
8.2.3 Typical Colloquial Sentences c) Wordnet
8.3 Perso-Arabic Standard for 58L-73R
Information Interchange
8.3.1 Urdu Design Guide 5.7 Standardisation 74L-74R
8.3.2 Typical Colloquial
Sentences in Urdu
a) Existing Standards
8.3.3 Sindhi Design Guide 74L-79R b) Draft Standards
8.3.4 Typical Colloquial
Sentences in Sindhi
8.3.5 Kashmiri Design Guide 80L-84R Index 76L-89R
8.3.6 Typical Colloquial a) Paper - abstract
Sentences in Kashmiri b) Addresses
8.4 Vedic Code Set 86L-103R

9. Quick Reference to Previous Issues 104L-104R Quick Reference to Previous Issues 90L-90R

TDIL PROGRAMME
Ministry of Communications & Information Technology
Department of Information Technology
Electronics Niketan, 6, CGO Complex, New Delhi-110003
Telefax : 011-2436 3076 E-mail : tdilinfo@mit.gov.in Website : http://tdil.mit.gov.in