Professional Documents
Culture Documents
Developing of Natural Language Interface To Robot - An Arabic Language Case Study
Developing of Natural Language Interface To Robot - An Arabic Language Case Study
Developing of Natural Language Interface To Robot - An Arabic Language Case Study
7
ISSN 1828-6003 July 2014
Abstract – In this paper, we report the development of an Arabic natural language interface to
robot (ANLI2ROBOT). The imperative sentence in Arabic language is used to command a robot
lifting arm to grasp small metal objects and move them from one place to another. Since this
interface is a natural language processing (NLP) application, lexicon, syntactical, and
morphological subsystems for the Arabic language interrogative sentence were created as
components of this interface. A set of simple familiar Arabic command words, which are related to
the operations of the robot lifting arm, have been defined in the lexicon of the interface, and
different forms of these words have been created by using the morphological subsystem.
ANLI2ROBOT is implemented using Prolog programming language since it gives a
straightforward conversion from Definite Clause Grammar (DCG) to Prolog predicates. This
work, which is classified as human–robot interaction (HRI), contributes the using of Arabic
language to communicate with robot. ANLI2ROBOT is also considered as a written dialogue
system (WDS) that uses Arabic written (not spoken) language. In addition to the success in
handling the different forms of the imperative sentence, the work revealed other significant
findings regarding the use of the Arabic natural language to instruct the robot (and similar
machines). Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved.
Keywords: Human Computer Interaction, Arabic Natural Language Interface, Natural Language
Processing, Artificial Intelligence
1256
Ayad T. Imam, Thamer Al-Rousan, Ashraf Odeh
(syntax) of the imperative sentence and its other accurately master the robot. An example of an
supports, such as the morphology and semantics, among application using this interaction mode is the
others. Hence, the natural language imperative sentence graphical user interface (GUI) used to control a
submitted needs to be analysed using lexical, syntactical, mobile robot in a cluttered laboratory environment as
and semantic knowledge. Once a sentence is developed by Fong et al [6] and Tele-robotic on
syntactically and semantically analysed under Internet to control mobile robot by F. Moutaouakkil
morphology restrictions, the user’s natural language et al. [21]. Another example is the incorporated tele-
imperative sentence is translated to appropriate robotic presence used in performing jobs in space, developed
commands that a robot can execute. The true robot by the Robonaut project of the National Aeronautics
response to the issued imperative sentence relies on and Space Administration (NASA) [5].
accurately syntactically and semantically distinguishing 2. Gesturing and Imitation HRI Approach: Gesturing
this sentence. and imitation are strategies used for knowledge
The contribution of this paper is the development of transfer among human beings. These strategies can be
an Arabic natural language communication instructing a used to teach robots certain tasks that can be
robotic arm to move several small objects from one place generalized to further tasks using demonstrations of
to another in an experiment theatre. Our interest is in performing the task. Actually, this approach is
recording the architecture of the Arabic language robot spontaneous in robots learning human skills, yet a
interface and the impact of the development number of challenges still remain, such as correcting
programming language on such real-time applications. wrong demonstrations and adjusting movements of
This paper is a considerable extension of earlier the robot which are not required [7], [17].
related works, in which no Arabic natural language robot 3. Natural Language Interface HRI (NLIHRI)
interface has been tested before. As far as we know, this Approach: Communication using natural language
is the first paper of its kind focusing on using the Arabic (either spoken or written) is a simple approach that
natural language by non-expert users for instructing a needs no new skill to be learned by the users to
robotic arm. The paper is organized as follows: Section II quickly start using a robot; hence, this approach is the
outlines the previous related works. Section III describes most successful mode of interaction. Developing NLI
the robots and the software architecture of the interface may come either as a standalone interface subsystem
that processes the imperative sentence used for or as a piece of a multimodal interaction system (for
commanding the robot. Section IV reports the example using both speech and vision components).
experiments and results. Section V concludes the We can list a number of examples for multimodal
discussion and outlines future work. interaction such as the one used to teach a robot to
press buttons with a name and an order or the one
used for grasping tasks [9], [11], [12].
II. Related Works
As fully self-ruling robotics are not developed until
III. ANLI2ROBOT System
now, the robot must be controlled by human. Several
modes of human–robot interaction (HRI) have been This paper’s aim is to develop an NLI that allows non-
presented for different service and personal robotic technical users to command, in a laboratory, a robotic
applications. lifting arm in two directions (forwards/backwards and
Each one represents an approach to controlling the left/right). The arm is equipped with a magnetic gripper
robot which varies from the typical computer-based and there are two workbenches (source and destination).
programming approach already applied in industry. The robot is required to reach, grasp, and move an
Actually, there are three standard approaches to object. Fig. 1 illustrates the robotic lifting arm used in
controlling a robot that can be mentioned here as follows: this research. The objects used in the study were made of
1. Tele-operation and Tele-presence HRI Approach: metal, so that they can be attracted by the magnetized
This has been utilized to strongly control the head of the arm, and were positioned on the source
movements of a robot. Using tele-operation and tele- workbench.
presence, HRI has produced a successful robot
control approach. Worth to say that merging of tele-
operation and robotics results tele-robotics, which
refers to robot that can perform only the essential
tasks autonomously; i.e. not fully independent
machine. Since they work under painful and risky
conditions, tele-robotics can be considered as a more
secure and more comfortable alternative of human.
Furthermore, tele-robotics shows the ability to
produce a multipurpose machine and not an
automatic system for certain application [21]. Yet,
this approach needs much time consuming practice to Fig. 1. Robot Diagram [7]
Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved International Review on Computers and Software, Vol. 9, N. 7
1257
Ayad T. Imam, Thamer Al-Rousan, Ashraf Odeh
Moving an object is a two-step process where the first Sentence segmentation into words is a deterministic
step consists of positioning the magnetic head of the arm outcome of the tokenization process to extract a list of
over the object on the source workbench. The second possible words [13]. A predicate for the tokenization of
step begins by activating the magnetic head of the arm to an Arabic sentence, called sting2list, has been developed.
get the object from the source workbench. Generally, in It has the following form: sting2list (S, L), where S is
robotics, a suitable configuration of the grasping task an input sentence and L is an output list containing
must be applied prior to any action. separated words extracted from the sentence.
The functionality of this experimental robot is to The predicate sting2list (S, L), following the flow
grasp, lift upwards and away different small metal diagram shown in Fig. 3, receives, in string form, a
objects from the source workbench to a destination formal Arabic written sentence as the input (that is S),
workbench using a primitive command set consisting of and produces a list of words as the output (that is L).
six commands. The Move Object commands are
subjected to sensors (S4, S5, S6, and S7), with each of
two pairs controlling one of the two motors (M1 and M2)
responsible for moving the arm in the directions
illustrated in Table I.
TABLE I
PRIMITIVE ROBOT COMMANDS [7]
Primitive Command Mechanically Action
Move Right Turn on M1 right wise until S4 is activated
Move Left Turn on M1 left wise until S6 is activated
Move Forwards Turn on M2 right wise until S7 is activated
Move Backwards Turn on M2 left wise until S5 is activated
Lift up Magnetize the head of the arm
Put down Demagnetize the head of the arm
III.2. Lexicon
Fig. 2. Architecture of ANLI2ROBOT System The lexicon is considered to be the backbone for any
NLP based system. Of course, the lexicon holds all the
The developed Arabic natural language robot interface definite types of knowledge, like words, affixes,
is an information system that will receive an Arabic punctuation marks…etc.
imperative sentence in a natural form and carry out the Unlike other natural languages, an Arabic lexicon
processing of it to translate it to the primitive commands mainly contains the roots of Arabic words, affixes, and
that the robot can respond to. As depicted in Fig. 2, the non-root based words. Some researchers have shown that
system comprises the following main components: it is simpler to consider the stem instead of the root as the
basic unit of the Arabic lexicon. Due to the nature of the
III.1. Tokeniser Arabic language, lexicon development poses certain
challenges that lead to considering the organization of
The aim of the tokenization activity is to break text the lexicon of the Arabic language as a set of stems that
into a sequence of small meaningful units, like words, is used by the morphology subsystem to generate
phrases, symbols, or similar, called tokens. These tokens bunches of words[14][16][18].
are the input for the next processing step, which is the As there is no open source tool for developing Arabic
lexical analysis. Often the tokeniser uses a simple dictionary [20], a special purpose Arabic lexicon for
approach in recognizing a token; for example, tokens are ALI2RBOT has been developed. The lexicon of
isolated by whitespace characters (spaces or line breaks). ALI2ROBOT contains the roots of possible Arabic
This approach is used in natural language words that could be used to communicate with the robot.
programming languages. The more complicated and A set of predicates is developed in this research and is
special cases are addressed by other methods to fit the used to check whether the words belong to the Arabic
tokens to a language model. language. Their semantics are:
Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved International Review on Computers and Software, Vol. 9, N. 7
1258
Ayad T. Imam, Thamer Al-Rousan, Ashraf Odeh
Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved International Review on Computers and Software, Vol. 9, N. 7
1259
Ayad T. Imam, Thamer Al-Rousan, Ashraf Odeh
Imperative sentence Imperative Verb + Noun used were simple, direct, and restricted to short-
Imperative sentence Imperative Verb + Phrase contained words. Examples of commonly used little
Phrase Adverbial Phrase | Prepositional Phrase complexity imperative sentences are ‘ ’ارﻓﻊ اﻟﻘﻄﻌﺔand ‘ اﻧﻘﻞ
Adverbial Phrase Adverb of place + Noun ’اﻟﻘﻄﻌﺔ, which mean Move Up and Move Forward,
Prepositional phrase Preposition + Noun respectively. This is not all; the computer handling of
Adverb of place ﺧﻠﻒ| اﻣﺎم |وراء | ﯾﺴﺎر | ﺑﻤﺒﻦ natural language differs in detail from the handling of
Preposition ﻓﻲ|ﻋﻠﻰ|ﻋﻦ|اﻟﻰ|ﻣﻦ artificial language; synonyms for example. For each
Imperative Verb ﺣﺮك | اﻧﻘﻞ | ﺿﻊ | اﺟﻠﺐ word we recorded some possible synonyms. So, we
Noun ﻗﻄﻌﺔ | ﺟﺴﻢ | ﺷﻲء | ذراع | رأس applied a certain degree of complexity to the issued
This set of rules has been implemented as Prolog sentences. The degrees of complexity of the imperative
predicates as follows, where m stands for ‘moving’, a sentence forms used to test ANLI2ROBOT were of little
semantic of a verb, and p stands for ‘place’, another complex.
semantic for a verb: The degree of complexity encompasses the number of
imp_sen([H|T]):- imp_verb(H), noun(T). words used in the sentence, synonyms, and the
imp_sen([H|T]):- imp_verb(H), phrase(T). morphological form criteria used to form the imperative
phrase(S):-adverbal(S);preposition(S). sentence. An example of a complex sentence is ‘ اﻧﻘﻞ
adverbal([H|T]):- adv_place(H),noun(T). ﻗﻄﻌﺘﯿﻦ اﺛﻨﺘﯿﻦ ﻋﻠﻰ ﻣﺮﺗﯿﻦ, which means ‘move two objects in
preposition([H|T]):- pre (H), noun(T). two stages’. The complex forms of the imperative
noun([H|[]]):-morph(H,R,p), is_noun(R). sentence were necessary to check the accuracy of the
imp_verb (H):- morph(H,R,m), is_verb(R). translation of the issued imperative sentences to the
primitive commands. Regarding the complexity of the
imperative sentence, we noted that, as the positions of the
III.5. Semantics objects (certain places of the robot’s work area which are
In linguistics, semantics is defined as the study of the the start and end positions) are a priori known to the
meanings of words and phrases in a certain context of a robot, these locations become the landmarks for
language. In computer science, semantic analysis is the positioning the robot’s gripper. Actually, this criterion
final stage of the analysis task of the compiler, aiming to positively affected the use of the robot in that there was
understand the program well enough to undertake the no need to explicitly position the robot’s gripper to pick
following synthesis [14], [18], [19]. up the metal object or explicitly determine the target
In ANLI2ROBOT, the semantic subsystem works in location to move the objects; this led to using simple
conjunction with the morphology subsystem, based on forms of the imperative sentence more than complex
the principle that the morphology of a word is ones. The relationship between complexity and response
responsible for giving certain meaning to the word. The time is shown in Fig. 4.
Arabic natural language imperative sentence issued by
the user is to be translated to primitive commands, as
previously listed in Table I, depending on the semantic of
the verb given in the sentence, The meanings of other
words are used to check the semantic basis of the
sentence. The predicate used for identifying the semantic
of the issued imperative sentence is semantic (X,Y),
where the input variable, W, is the Arabic verb extracted
from the sentence, and the output Y is the primitive
command of the lifting arm robot. This predicate invokes
the morph (W1,W2, Sem) predicate as follows:
Fig. 4. Accuracy vs. Complexity of Imperative Sentence
semantic (X,Y) :-morph (X,_,Y).
Another important result from ANLI2ROBOT is the
IV. Results effect of the imperative sentence’s complexity on the
execution time. This is particularly relevant to the
Since ANLI2ROBOT is related to a robotic real-time programming language used to develop ANLI2ROBOT.
application, time consumption and accuracy (the total When more than four words are used to construct an
number of correctly executed imperative sentences imperative sentence, as translating the Arabic imperative
issued to the robot) are considered to be performance sentence to the robot’s primitive commands means
indicators. tokenizing, morphing, syntactically parsing the sentence,
During the experiment, common imperative sentence and semantically translating the sentence, the time spent
forms were provided. While issuing various natural on these activities notably increased, as shown in Fig. 5.
language imperative sentences to ANLI2ROBOT, test Overall, Fig. 4 and Fig. 5 suggest that increasing the
users were unsure of the robot’s linguistic knowledge complexity of the imperative sentence leads to increasing
since it is viewed as a machine. Hence, the sentences the response time and failed cases (inaccuracy).
Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved International Review on Computers and Software, Vol. 9, N. 7
1260
Ayad T. Imam, Thamer Al-Rousan, Ashraf Odeh
Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved International Review on Computers and Software, Vol. 9, N. 7
1261
Ayad T. Imam, Thamer Al-Rousan, Ashraf Odeh
Authors’ information
1
Dept. of SE, Faculty of IT, Al-Isra University, Jordan.
2
Dept. of CIS, Faculty of IT, Al-Isra University, Jordan.
Copyright © 2014 Praise Worthy Prize S.r.l. - All rights reserved International Review on Computers and Software, Vol. 9, N. 7
1262