IADIS International Conference e-Learning 2011

TEACHING ASSISTANT TOOLS FOR NL TO FOL

CONVERSION

Isidoros Perikos, Foteini Grivokostopoulou and Ioannis Hatzilygeroudis

University of Patras, School of Engineering
Department of Computer Engineering & Informatics, 26500 Patras, Hellas (Greece)

ABSTRACT
The NLtoFOL system is a web based interactive system that aims at helping the students convert (or translate) natural
language (NL) sentences into first-order logic (FOL) formulas. In this paper, we present two teaching assistant tools to be
embedded into the NLtoFOL system. First, the teaching material manager, which helps the tutor to insert information
about a new sentence’s conversion process, by automatically analyzing the sentence and filling in corresponding data for
all ten conversion process steps. The tutor can also edit them or delete existing ones. Second, the teaching data analyser,
which is used to help the tutor in examining students’ progress. It helps him/her to investigate students’ answers and
errors made by providing useful statistics. It also gives him/her a graphical view of them for a deeper understanding of
difficulties that students face. So, the tutor can reconfigure and adapt the content and its difficulty to the students.

KEYWORDS
Teaching Assistant Tools, Teaching First Order Logic, Natural Language Processing, FOL parsing, Web-based E-
learning system

1. INTRODUCTION
Knowledge Representation & Reasoning (KR&R) is a fundamental topic of Artificial Intelligence (AI). A
basic KR language is First-Order Logic (FOL), which is part of almost any introductory AI course or
textbook. So, teaching FOL as a KR&R language is a vital aspect.
Teaching FOL as a knowledge representation and reasoning language includes many aspects. One of
them is converting (or translating) natural language (NL) sentences into FOL formulas, often called logic
formalization of NL sentences. It is an ad-hoc process; there is no specific algorithm that can be automated
within a computer. This is mainly due to the fact that NL has no clear semantics as FOL does.
Most of existing textbooks do not pay the required attention to the above aspect. They simply provide the
syntax of FOL and definitions of the logical symbols and terms (e.g. Russell and Norvig 2003). Even more
specialized textbooks do the same (e.g. Brachman and Levesque 2004). At best, they provide a kind of more
extended explanations and examples (e.g. Genesereth and Nilsson 1987). They do not provide any systematic
guidance towards it. In a previous work (Hatzilygeroudis and Perikos 2009), a web-based interactive system
for guiding students in translating a NL sentence into a FOL one via a structured process has been
implemented. However, that system does not offer any tools for helping tutors in their tasks.
In this paper, we introduce teaching assistant tools to be embedded into that system that aim at assisting
tutors in their tasks. First, they help tutors in maintaining and updating the teaching material (sentences). In
addition, they help tutors to examine students’ progress and trace difficulties they face. The structure of the
paper is as follows. In Section 2, related work is presented. Section 3 presents the basic syntax of FOL and
the conversion process. Section 4 presents the teaching assistant tools system architecture. Sections 5 and 6
focuse on the two assistant tools. Finally, Section 7 depicts conclusions and provides some directions for
future research.

337
ISBN: 978-972-8939-38-0 © 2011 IADIS

2. RELATED WORK
Logic Tutor (Abraham et al 2001) is an intelligent tutoring system (ITS) for learning formal proofs in
propositional logic (PL) based on natural deduction. As an intelligent system, it adapts to the needs of the
students via keeping user models. Logic-ITA (Lesta and Yacef 2002) is actually a teaching assistant system.
It expands Logic Tutor with new tools which are designed to assist the tutor. The tutor can use these tools to
manage the teaching configuration settings and the teaching material. Also help him/her monitor the
students’ results and their progress. P-Logic Tutor (Lukins et al 2002) is also a kind of intelligent tutoring
system aiming at teaching students fundamental aspects of PL and theorem proving. P-Logic tutor also
provides an environment in which the tutor can track student learning activities. In this way, it helps the tutor
examine how the students solve logic problems and provide appropriate help.
KRRT (Knowledge Representation and Reasoning Tutor) (Alonso etal 2007) is a web-based system that
aims at helping students to learn FOL as a KR&R language. It deals with both knowledge representation and
reasoning with FOL. The translation from NL to FOL takes place in its KR part. However, it does not
provide any help either to students or to tutors. Similarly, in (Lodder etal 2008) a logical exercise solver helps
students to rewrite formulas from propositional logic into disjunctive normal form, but again no teaching
assistant tool is provided.

3. CONVERTING NL TO FOL SENTENCES

3.1 FOL Syntax and Semantics

First-order logic (FOL) is the most widely used logic-based knowledge representation formalism. Higher
order logics are difficult to handle, whereas lower order logics, such as those based on propositional calculus,
are expressively poor. FOL is a KR language used for representing knowledge in a knowledge base, in the
form of logical formulas, to be used for automatically making inferences. Logical formulas or sentences
explicitly represent properties of or relations among entities of the world of a domain, represented by a set D.
The syntax of FOL formulas is based on the following concepts:
• constants: each constant represents a specific entity of D (e.g. Maria, Pluto).
• variables: each variable represents one or more (unknown) entities of D. Each variable can take one
value from a subset of D at a time (e.g. x, y, z).
• function symbols: each function symbol represents an one-to-one relation between two or more entities
of D (e.g. mother-of).
• predicates: each predicate represents a property of an entity or a relation between two or more entities
of D (e.g. tall, loves).
• connectives: ¬ (negation: not), ∨ (disjunction: or), ∧ (conjunction: and), ⇒ (implication: implies).
They connect logical formulas between each other.
• quantifiers: ∀ (universal: for all), ∃ (existential: exists). They quantify universally or existentially
associated variables.
Also, the definition of a term is necessary.
A term is defined as follows:
(i) a constant is a term;
(ii) a variable is a term;
(iii) if f is a function symbol and t1, t2, …, tn are terms
then f (t1 t2 …, tn) is a term;
So, finally
A (syntactically right) FOL formula is defined as follows:
(i) if P is a predicate symbol and t1, t2, …, tn are terms, then P (t1 t2 …, tn) is a FOL formula, which is
called atomic formula or atom;
(ii) if F and G are FOL formulas, then ¬F, (F ∨ G), (F ∧ G) and (F ⇒ G) are FOL formulas too;
(iii) if F is a FOL formula and v is a variable, then (∀v) F and (∃v) F are FOL formulas too.

338
 IADIS International Conference e-Learning 2011

The scope of a quantifier is the subformula at its right hand side. Each quantifier is associated with a
variable. If all occurrences of a variable associated with a quantifier are lying within the scope of the
quantifier, the corresponding FOL formula is called a FOL sentence.
FOL is used for representing knowledge about a domain in the form of logical formulas or sentences.
This is usually made by converting (or translating) natural language (NL) sentences into FOL sentences. For
example, the NL sentence “All men like some cocktail” is converted into the following FOL sentence: (∀x)
(∃y) (man(x) ∧ cocktail(y)) ⇒ likes(x,y). This is not a straight forward process and there is no systematic way
of doing that. Therefore, in a previous work, a structured and interactive process for this conversion was
devised, called the NLtoFOL process, which is presented in the following subsection.

3.2 The NL to FOL Conversion Process

The NLtoFOL process is a structured and interactive process, introduced in (Hatzilygeroudis 2007), that
guides a student in converting a NL sentence into a FOL one and consists of ten steps. We present it here via
an example, the conversion of the NL sentence “All men like some cocktail”.
Step 1: Spot the verbs, the nouns and the adjectives in the sentence and specify the corresponding predicates
or function symbols. There are three such items here:
men→ predicate: man, like→ predicate: likes, cocktails→ predicate: cocktail
Step 2: Specify the number, the types and the symbols of the arguments of the function symbols (first) and the
predicates (next). We do that in the following table:
Predicate Arity Types Symbols
man 1 variable x
cocktail 1 variable y
likes 2 variable, variable x, y
Step 3: Specify the quantifiers of the variables.
x →∀ (because of “All”), y → ∃ (because of “some”)
Step 4: Construct the atomic formulas (or atoms) corresponding to predicates. We construct as man atoms as
the predicates:
Atom 1: man(x), Atom 2: cocktail(y), Atom 3: likes(x,y)
Step 5: Divide produced atoms in groups of the same level atoms. This mainly refers to grouping atoms that
will be connected with each other with some connective:
AtomGroup1: {man(x), cocktail(y)}, AtomGroup2: {likes(x,y)}, AtomGroup2: { man(x)}
Step 6: Specify the connectives between atoms of each group and create corresponding logical formulas.
AtomGroup1
Form1: man(x)∧ cocktail(y) , AtomGroup2
Form2: likes(x,y)
Step 7: Divide produced formulas in groups of the same level formulas. This usually corresponds to
specifying the left and right parts of an implication:
FormGroup1-1: {man(x) ∧ cocktail(y), likes(x,y)}
Step 8: If only one group of formulas is produced, specify the connectives between formulas of the group,
create the next level formula and go to step 10.
FormGroup1-1:
Form1-2: {man(x)∧ cocktail(y)⇒ likes(x,y)}
Step 9: Specify the connectives between formulas of each group, create the next level formulas and go to step
7. Not applicable here.
Step 10: Place quantifiers in the right points in the produced formula to create the final FOL formula.
(∀x) (∃y) (man(x) ∧ cocktail(y)) ⇒ likes(x,y) or alternatively
(∀x) (∃y) (man(x) ⇒ (cocktail(y) ⇒ likes(x,y)))

4. SYSTEM ARCHITECTURE
To support the teaching assistant tools that are introduced here, we extended the architecture of the existing
system with a tutor oriented subsystem, which is presented in Figure 1. It includes four new units: Tutor
Interface (TI), Tutor Interface Configurator (TIC), Teaching Material Manager (TMM) and the Learning Data
Analyzer (LDA).

339
ISBN: 978-972-8939-38-0 © 2011 IADIS

Tutor

Tutor Interface (TI)

Teaching Material Learning Data

Manager Analyzer
(TMM) (LDA)

System Database (SD)

Figure 1. Architecture of the Teaching Assistant System

Through TI, the tutor interacts with the system and gets access to TMM. The tutor with the help of TMM
unit can manage the teaching material. So, the tutor can create new exercises-sentences and also edit or even
delete existing ones. TMM consists of two tools: the Formula Analyzer (FA) and the Equivalent Formula
Producer (EFP). EFP automatically determines the logically equivalent formulas of a given FOL expression
(or formula). FA unit is a tool that analyses a FOL formula and automatically extracts the data needed to fill
in the steps of the sentence’s formalization process according to the NLtoFOL process. Then it stores the data
in the System Database (SD). The System Database (SD) is mainly used to store sentences and related
information. A number of NL sentences analysed in corresponding steps of the NLtoFOL process and related
information are pre-stored in the database.
LDA is mainly used to help the tutor in examining the students’ progress. It presents in a graphical way
useful statistics about students learning progress. It helps the tutor to investigate a student’s answers and
errors. Also, it gives him/her a better understanding of the difficulties that students face.

5. TEACHING MATERIAL MANAGER

As mentioned above the system helps the students learn the conversion of NL sentences into FOL ones by
implementing the NLtoFOL process. This is done by letting the students try as many conversions as they
want, of various difficulty levels, following the 10-step process described in Section 3. During a student
session some kind of help is provided on demand by the system.
Given that, the tutor should maintain a corpus of sentences adequate for the needs of the students.
Therefore, the tutor can add new sentences-examples and edit or delete existing ones, in order to maintain a
corpus of sentences-examples covering all aspects of the formalization process. However, the addition of new
sentences into the NLtoFOL system is a complex and time demanding process for the tutor. The reason is
that, for each NL sentence the tutor should have manually determined the right answers for each step of its
conversion process, according to NLtoFOL process, and then manually insert them into the system.
Furthermore, the tutor should determine logically equivalent FOL formulas that may exist, determine their
conversion process steps and insert them to the system as possible alternative answers too. Given that the
corpus of sentences should be periodically updated, it is obvious how hard is this task for the tutor. In order
to alleviate the tutor’s task, Formula Analyzer (FA) and Equivalent Formula Producer (EFP) tools have been
created.

5.1 Formula Analyzer

Formula Analyzer (FA) operates in two stages. The first deals with the creation of the formation tree of the
FOL expression and the other with the analysis of the atomic formulas.
The creation of the formation tree of a FOL formula shows the complete pedigree of the formula. It
discribes how the atomic formulas (or atoms) are connected and form group of atoms and how those groups
of atoms form formulas. The algorithm for the creation of the FOL tree of a formula is as follows (where by
“assign to” we mean “make a child of ”):

340
 IADIS International Conference e-Learning 2011

1. Insert FOL formula as the root of the tree

2. Read the next first-level element of the FOL formula
2.1. If it is a quantifier, keep its type (universal or existential) and its associated variable’s name.
2.2. If it is an atom, create a leaf for it and assign it to the current connective node (see next step), if
any.
2.3. If it is a disjunction or conjunction connective
2.3.1. If it isn’t of the same type with the current connective, create a new node for it and make it the
current connective node.
2.3.2. Assign possible unconnected leaves to it.
2.4. If it is an implication
2.4.1. Create a new node, assign the current connective node to it and make the new node the
current connective node.
2.4.2. Assign possible unconnected leaves to it.
2.5. If it is a nested expression
2.5.1. Recursively apply steps 2 and 3 to the expression
2.5.2. Connect the resulted subtree to the current node
2.6. If it is a negation, then associate it with its atom.
3. While there are still elements of the FOL formula, go to step 2.
4. Connect the current connective node to the root and return the formation tree.

dog(x)∧loves(x,master-of(x)))⇒loves(maria,x)

∧ loves(x,maria)

dog(x) loves(x,master-of(x))

Figure 2. An example FOL formation tree

So, the structure of the produced formation tree is as follows. The root of the tree is the FOL formula, the
internal nodes are the connectives and the leaves are the atomic formulas of the FOL formula. The data
needed for steps 1-4 of the NLtoFOL conversion process are found in the leafs of the FOL tree and are
calculated in detail in the second stage of the analysis (see below). The data of steps 5-9 is extracted from the
internal nodes of the tree. At step 5, atoms of same level form groups. For example, the formation tree of the
FOL sentence “(∀x) (dog(x) ∧ loves(x, master-of(x))) ⇒ loves(maria, x)” produced after application of
above algorithm is illustrated in Figure 2. Notice that information about quantifiers is not part of the tree. It is
stored in the database as associated knowledge. So, two different sentences may have the same formation
tree, but different (associated info about their) quantifiers. We tested the algorithm with over 50 FOL
formulas and it worked successfully in all cases.

Figure 3. An example atomic formula analysis

341
ISBN: 978-972-8939-38-0 © 2011 IADIS

The second stage of FA concerns analysis of the atomic expressions, i.e. the leaves of the FOL tree. The
tool analyses the atomic expressions to get the needed data. What is needed to be determined are (a) the
number of the arguments that an atom has and (b) the type of each argument. An argument is considered to
be a variable if it is a one-character string, a function if it contains brackets and has argument(s), and a
constant otherwise (see Figure 3).
After the analysis of the FOL formula, the information needed is inserted into the system’s database.
Moreover, EFP is used to determine possible logically equivalent FOL formulas in order to recognize more
than one correct formalization paths.

5.2 Equivalent Formula Producer

The creation of equivalent logic formulas is a fundamental topic of logic. De Morgan’s laws are used to
handle equivalences. In order to determine equivalent formulas a list of logical equivalent formula patterns
has been specified (see Table 1). EFP tries to identify any equivalent formula(s) for the FOL formula(s) given
by the tutor associated with the NL sentence(s), by doing the following:
Table 1. List of FOL Equivalent Formulas
FOL Equivalent Formulas
1. (A∧B)→C ⇔ A→(B→C)
2. (¬A ∨ ¬B) ⇔ ¬(A ∧ B)
3. (A →B) ⇔ ¬A ∨B
4. (A ∨ B) ∧ C ⇔ (A ∧ C) ∨ (B ∧ C)
5. (A ∨ B) ∧ C ⇔ (A ∧ C) ∨ (B ∧ C)
6. (A↔ B) ⇔ (A →B) ∧ (A →B)

1. Replace the atoms with symbols, to create the corresponding FOL pattern.
2. Check if the pattern is found in the equivalent list.
3. If found, get the equivalent patterns(s) and place those to the equivalent stack.
4. For every pattern on the stack, replace symbols with the corresponding atoms, to create FOL formulas.
5. For every FOL formula, call FA to analyze it and insert corresponding info into SD.
For example, the FOL formula “(∀x) (dog(x) ∧ loves(x, master-of(x))) ⇒ loves (maria, x)” can be
converted into the equivalent formula “(∀x) dog(x) ⇒ (loves(x, master-of(x)) ⇒ loves (maria, x))” using the
first equivalence pattern of Table 1. The formation tree of the equivalent formula, produced after application
of EFP algorithm. it is illustrated in Figure 4.

dog(x) ⇒loves(x,master-of(x)))⇒loves(maria,x)

dog(x) ⇒

loves(x,master-of(x)) loves(maria,x)

Figure 4. Formation tree of the example equivalent formula

6. LEARNING DATA ANALYSER

The learning data analyser (LDA) tool aims at helping the tutor in examining a students’ performance. As
mentioned above, while student is converting a NL sentence into FOL using the NLtoFOL process and
interacting with the system, all the necessary interaction information is recorded and stored in the system’s

342
 IADIS International Conference e-Learning 2011

database. This data concerns: sentences that the student has tried, sentences correctly formalized, errors made
during the formulization process of a sentence, hints/help requested from the system, time spent on each step,
and more. The database is constantly extended with new interaction records. The tutor can use the LDA tool
to perform an analysis of the records stored in the database, i.e. track how students are performing and
evaluate their level. The LDA tool, after tutor’s request, automatically calculates a set of predefined features
divided into two major categories: student related and sentence-process related.
Student Related
- Proportion of sentences tried that have correctly been formalized
- Number of errors made during a sentence formalization
- Number of hints taken during a sentence formalization
- Steps of the process where most mistakes were made
Sentence-Process Related
- Proportion of students that get each sentence wrong
- Sentences for which most mistakes were made
- Sentences for which there were most help requests
- Steps of the process for which most mistakes were made
- Steps of the process for which there were most help requests
- Most common mistakes made
Moreover, LDA automatically creates the corresponding graphs to visualise the results obtained from
students’ records (see Figure 5). The LDA tool helps the tutor in tracing a student’s weaknesses and properly
reconfiguring the content of the system.

Figure 5. LDA tool graphical representation

Recall that the aim of the system is not student evaluation but learning. So, the addition of the new
sentences can be based on the students’ weaknesses and errors. Also, the tutor can recalibrate a sentence’s
formalization difficulty based on the students’ performance. The difficulty level of a sentence’s formalization
is empirically determined by the tutor, while inserting it into the system. The data obtained by the LDA tool
can help him/her to re-estimate the difficulty level based on the students’ answers. Also can help the tutor to
deeper understand which factors and at which point(s) affects the formalization difficulties that the students
face. Moreover, he/she can adapt, based on the students performance, the overall sentences difficulty (or
complexity) by adding or deleting easy or difficult ones.

7. CONCLUSION AND FUTURE WORK

The NLtoFOL system is a web based interactive system for helping students in translating (or converting)
natural language (NL) sentences into first-order logic (FOL) formulas. In this paper, two teaching assistant
tools embedded into the NLtoFOL system are presented. The first is TMM tool, which aims at helping the
tutor in managing the content of the system. It consists of two subtools. FOL analyzer tool parses the FOL
formula and extracts the information needed about its conversion process. FOL equivalent tool checks and

343
ISBN: 978-972-8939-38-0 © 2011 IADIS

determines logically equivalent FOL expressions that a given FOL expression may have, thus assuring
inclusion of students alternative answers. The time save due to these tools is quite important.
The second assistant tool is the Learning data analyzer tool, which helps the tutor to monitor the students
performance, trace mistakes that they make and accordingly reconfigure the contents of the system by adding
specific sentences or/and adding/changing hints messages that the system offers.
However, there are some points that the system could be improved. First, LDA could be improved by
adding to it some more features to calculate. For example, based on the students’ records could determine
which features of a sentence affects and at which point its formalization process. Also, based on the data
collected by LDA, we plan to build an intelligent system, which will automatically determine a sentence’s
formalization difficulty.

ACKNOWLEDGEMENT
This work was supported by the Research Committee of the University of Patras, Greece, Program
“Karatheodoris”, project No C901

REFERENCES
Abraham, D., Crawford, L., Lesta, L., Merceron, A., Yacef, K. (2001). The Logic Tutor: A multimedia presentation.
Electronic Journal of Computer-Enhanced Learning .
Alonso, J.A., Aranda, G.A., Martin-Matceos, F.J.: KRRT: Knowledge Representation and Reasoning Tutor. In: Moreno
Diaz, R., Pichler, F., Quesada Arencibia, A. (eds.) EUROCAST 2007. LNCS, vol. 4739, pp. 400–407. Springer,
Heidelberg (2007).
Brachman R.J. and Levesque, H.J. (2004): Knowledge Representation and Reasoning. Elsevier, Amsterdam.
Genesereth, M. R. and Nilsson, N. J. (1987). Logical Foundations of AI. Morgan Kaufmann, Palo Alto.
Hatzilygeroudis.I (2007).:Teaching NL to FOL and FOL to CL Conversions. In: Proceedings of the 20th International
FLAIRS Conference, Key West, FL, May 2007, pp. 309–314. AAAI Press, Menlo Park (2007)
Hatzilygeroudis I. and Perikos I. (2009): A Web-Based Interactive System for Learning NL to FOL Conversion. In:
Ernesto Damiani, Jechang Jeong, Robert J. Howlett, and Lakhmi C. Jain (Eds.): New Directions in Intelligent
Interactive Multimedia Systems and Services – 2, SCI 266, Springer-Verlag, 297-307.
Lesta, L. and Yacef, K. (2002). An intelligent teaching assistant system for Logic. In Cerri, S.A., Gouarderes, G.,
Paraguacu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 421–431. Springer, Heidelberg .
Lodder, J., Passier,H., Stuurman, S. (2008). Using IDEAS in teaching logic, lessons learned. Computer Science and
Software Engineering, International Conference, vol. 5, pp. 553–556 .
Lukins, S., Levicki, A., Burg, J. (2002). A tutorial program for propositional logic with human/computer interactive
learning. In SIGCSE 2002, pp. 381–385. ACM, New York .
Russell, S., Norvig, P. (2003). Artificial Intelligence: a modern approach, 2nd . Prentice Hall, Upper Saddle River
Yacef, K. (2002). Intelligent Teaching Assistant Systems. University of Sydney Technical Report 533.

344