Knowledge Engineering

(IT4362)

Quang Nhat Nguyen

(quang.nguyennhat@hust.edu.vn)

Hanoi University of Science and Technology

School of Information and Communication Technology
Academic year 2021-2022
Content
◼ Introduction
◼ First-order logic
• Syntax and semantics
• Proof
• Model
◼ Knowledge representation
◼ Logic programming
◼ Expert systems
◼ Uncertain reasoning
◼ Knowledge discovery by Machine learning
◼ Knowledge discovery by Data mining

Knowledge Engineering 2
Introduction of logic
◼ Logic is a formal system
◼ Logic is defined by two components: syntax and
semantics
◼ Syntax
❑ Structural properties of the formal system
❑ Symbols that are independent of the meaning

◼ Semantics
❑ Meaning (interpretation) of symbols

Knowledge Engineering 3
Syntax of a logic
◼ Syntax = Language + Proof theory
◼ Language
❑ Proper symbols, expressions, terms, formulas
❑ E.g., one plus one equal two
◼ Proof theory
❑ Set of inference rules that manipulate syntactic expressions
❑ E.g., Inference rule: any plus zero Ⱶ any
◼ For instance, given the rule above and other reference
rules, we can prove the theorem: one plus one equal two
◼ We do all of this (proving) without giving any meaning
(interpretation) to symbols!

Knowledge Engineering 4
Semantics of a logic
◼ Semantics = Meaning (interpretation) of symbols
◼ For example
❑ I(one) is 1  N
❑ I(two) is 2  N
❑ I(plus) is + : N x N → N
❑ I(equal) is = : N x N → {true, false}
❑ I(one plus one equal two) is true

◼ If the interpretation of an expression is true, we say that

the interpretation is a model of the expression
◼ An expression that is true in any interpretations is valid
❑ E.g., The following formula is valid:
(one plus one equal two) OR NOT (one plus one equal two)

Knowledge Engineering 5
Logic: Syntax vs. Semantics (Proof vs. Model)
◼ Logic is a way to formalize and automate reasoning
◼ Reasoning can be done at the level of the syntax – by proofs:
deductive reasoning
◼ Reasoning can be done at the level of the semantics – by
models: model-based reasoning
❑ Semantics reasoning at the level of one interpretation (model):
◼ Given an expression, there exists a model?: satisfiability
◼ Given an expression and an interpretation, the interpretation is a
model of the expression?: model checking
❑ Semantics reasoning at the level of all possible interpretations:
validity checking
❑ Logics that are sound (correct) and complete: provability
corresponds to validity
Knowledge Engineering 6
Limitation of propositional logic (1)
◼ The most basic elements of propositional logic are atomic
propositions
❑ Atomic propositions are treated as single units
❑ Their structure and composition are suppressed

◼ Propositional logic
❑ Advantage: Simplicity
❑ Disadvantage: Lack (less) of expressivity

Knowledge Engineering 7
Limitation of propositional logic (2)
◼ Let’s consider the example
❑ Tuan is a HUST student
❑ Every HUST student studies the course Algebra
❑ Since Tuan is a HUST student, he studies the course Algebra

◼ In propositional logic
❑ Proposition p: “Tuan is a HUST student”
❑ Proposition q: “Every HUST student studies the course Algebra”
❑ Proposition r: “Tuan studies the course Algebra”
❑ But, r CANNOT be deduced from p and q!

Knowledge Engineering 8
First-order logic (FOL) – Introduction
◼ The example mentioned above can be represented in first-order
logic by the following (first-order) formulas
❑ HUST_Student(Tuan): Tuan is a HUST student
❑ x:HUST_Student(x) → Studies_Algebra(x): Every HUST
student studies the course Algebra
❑ Studies_Algebra(Tuan): Tuan studies the course Algebra
◼ We can prove that:
{HUST_Student(Tuan), x:HUST_Student(x) →
Studies_Algebra(x)} Ⱶ Studies_Algebra(Tuan)
◼ In first-order logic
❑ Symbols like Tuan, x are called terms (Tuan is a constant, x is a variable)
❑ Symbols like HUST_Student and Studies_Algebra are called predicates
❑ Symbols like  are called quantifiers
❑ Terms, predicates and quantifiers allows to give structure to formulas

Knowledge Engineering 9
FOL – Language (1)
◼ Four types of symbols
❑ Constants: Names of objects of a given domain (e.g., Tuan)
❑ Variable: Symbols that vary over different objects (e.g., x)
❑ Function symbols: Symbols that map objects of the domain to
objects of the domain (e.g., plus)
❑ Predicates: Relations that might be either true or false (e.g.,
HUST_Student and Studies_Algebra)

◼ Any function or predicate symbol takes a predefined and

fixed number of arguments
❑ E.g., HUST_Student and Studies_Algebra are 1-place predicates
❑ E.g., plus can be a 2-place function symbol

Knowledge Engineering 10
FOL – Language (2)
◼ A term is defined recursively as follows
❑ A constant is a term
❑ A variable is a term
❑ If t1, t2,…,tn are terms and f is n-place function symbol, then
f(t1,t2,…,tn) is a term
❑ Nothing else is a term

◼ Examples of term
❑ Tuan
❑ 2
❑ friend(Tuan)
❑ friend(x)
❑ plus(x,2)

Knowledge Engineering 11
FOL – Language (3)
◼ Atoms
❑ If t1,t2,…,tn are terms and p is a n-place predicate, then
p(t1,t2,…,tn) is an atom
❑ E.g., HUST_Studies(Tuan), HUST_Studies(x),
Studies_Algebra(Tuan), Studies(x)

◼ Formulas is defined as follows

❑ An atom is a formula
❑ If  and  are formulas, then  and  are formulas
❑ If  is a formula and x is a variable, then x: is a formula
❑ Nothing else is a formula

◼ Note that we define x: as x:

Knowledge Engineering 12
FOL – Language (4)
◼ An occurrence of a variable in a formula is bound if and only if the
occurrence is
❑ Within the scope of a quantifier employing the variable, or

❑ In that quantifier

◼ An occurrence of a variable is free if and only if the occurrence of the

variable is not bound
◼ Example
❑ (x: HUST_Student(x))  Studies_Algebra(y)
❑ Both the two occurrences of x are bound, but the occurrence of y is free
◼ A variable is bound in a formula if at least one occurrence of it is
bound in the formula
◼ A variable is free in a formula if at least one occurrence of it is free
in the formula
◼ Formulas without free variables are called closed formulas

Knowledge Engineering 13
FOL – Semantics (1)
◼ An interpretation of a formula  is a pair <D,I>
◼ Domain D is a non-empty set
◼ Interpretation function I is an assignment of value to
each constant, function symbol, and predicate symbol –
such that:
❑ For each constant c: I(c)  D
❑ For each n-place function symbol f: I(f): Dn → D
❑ For each n-place predicate P: I(P): Dn → {true, false}

Knowledge Engineering 14
FOL – Semantics (2)
◼ Interpretation of a closed formula. Let ,  and  are
first-order formulas
❑ If  is , then I()=false if I()=true, and
I()=true if I()=false
❑ If  is (), then I()=false if one of I() or I()
is false, and I()=true if both of I() and I() are
true
❑ Let x:(x) be a closed formula. Then,
I(x:(x))=true if I()(d)=true for every d  D

Knowledge Engineering 15
FOL – Semantics (3)
◼ A formula  is satisfiable (consistent) if and only if there
exists an interpretation <D, I> such that I() – We denote
as ╞I 
❑ If ╞I , then we say that I is a model of , and I
satisfies 
◼ A formula  is unsatisfiable (inconsistent) if and only if
no interpretations satisfy 
◼ A formula  is valid if and only if every interpretation
satisfies  – We denote as ╞ 

Knowledge Engineering 16
FOL – Proof theory
◼ Proof theory is based on a Deductive machinery
◼ A Deductive machinery is a pair <A,D>
❑ A is a set of axioms
❑ D is a set of inference rules
◼ The set of axioms A is a subset of the formulas 
◼ Inference rules are relations among formulas in 
◼ Let  () be a set of formulas and  () a formula. A
deduction of  from  is obtained by applying inference rules
either to formulas in  or to axioms in A and by obtaining 
◼ If there exists a deduction of  from , then we write  Ⱶ . A
proof is a deduction where = 
◼ If there exists a proof of , then we write Ⱶ , and we say that 
is a theorem

Knowledge Engineering 17
Hilbert deduction system
◼ Let’s call <A,D> a deductive machine,  () a set of formulas,
 () a formula.
◼ We say that  is derivable from , and we write  Ⱶ , if and
only if there exists a sequence 1,…, n of formulas (each
i) such that:
1. Each i is either
• an axiom (iA), or
• a formula in  (i), or
• the result of the application of an inference rule of D to some
formula j, with j<i
2. n = 
◼ The sequence 1,…, n is called a deduction of  from 
◼ The formulas in  are called assumptions
◼ If =, then we write Ⱶ  and say that  is provable (it is a theorem)
Knowledge Engineering 18
Hilbert deduction system – Axioms
◼ Let’s call ,  and  are formulas in 
◼ Axiom 1.  → ( → )
◼ Axiom 2. ( → ( → )) → (( → ) → ( → ))
◼ Axiom 3. ( → ) → ( → )
◼ Axiom 4. x:(x) → (t), if t is a term that is free for x in (x)
❑ A term t is free for x in  if and only if no variable in t has bound
occurrences in (x/t) – where (x/t) is the formula obtained by
replacing every free occurrence of x in  with t

◼ Axiom 5. (x:( → )) → ( → x:), if x is not free in 

Knowledge Engineering 19
Hilbert deduction system – Inference rules
◼ Modus Ponens. If we have a deduction of  from  and a
deduction ( → ) from , then we have a deduction of  from
()
 Ⱶ ,  Ⱶ ( → )
() Ⱶ 
E.g., If {IsKid(Tuan), IsKid(Tuan) → Likes(Tuan, Ice-cream)}, then
Likes(Tuan, Ice-cream)

◼ Universal Elimination. To substitute a variable with a constant

symbol
 Ⱶ x:(x)
 Ⱶ (a)
E.g., If x:Likes(x, Ice-cream), then Likes(Tuan, Ice-cream)
Knowledge Engineering 20
Hilbert deduction system – Example
Prove that: {(p→q), (q→r)} Ⱶ (p→r)

◼ [1]. (q→r) → (p → (q→r)), by the Axiom 1 (replacing  with

(q→r) and  with p)
◼ [2]. (q→r) is an assumption
◼ [3]. (p → (q→r), by the inference rule Modus Ponens
◼ [4]. (p → (q→r) → ((p→q) → (p→r)), by the Axiom 2
◼ [5]. (p→q) → (p→r), from [3] and [4] and by Modus Ponens
◼ [6]. (p→q) is an assumption
◼ [7]. (p→r), from [5] and [6] and by Modus Ponens

Knowledge Engineering 21
Hilbert deduction system – Axiom 4
◼ Axiom 4. x:(x) → (t), if t is a term that is free for x in (x)
❑ A term t is free for x in  if and only if no variable in t has bound
occurrences in (x/t) – where (x/t) is the formula obtained by
replacing every free occurrence of x in  with t

◼ What happens if the restriction is removed?

◼ Let’s consider the following example:

◼ x:(y:Equal(y,x)) → y:Equal(y,y)
◼ x:(y:y=x) → y:y=y
◼ x:(y:yx) → y:yy

Knowledge Engineering 22
Hilbert deduction system – Axiom 5
◼ Axiom 5. (x:( → )) → ( → x:), if x is not free in 

◼ What happens if the restriction is removed?

◼ Let’s consider the following example:

◼ (x:(Prime(x)→Prime(x))) →
(Prime(x)→x:Prime(x))

Knowledge Engineering 23
Important properties of a logical system
◼ Consistency
❑ No theorem of the system contradicts another

◼ Soundness
❑ The system's inference rules never allow a false inference from a
true premise
❑ If a system is sound and its axioms are true, then its theorems are
also guaranteed to be true

◼ Completeness
❑ If a theorem is true, it can be proven

◼ The Hilbert deduction system is sound and complete

Knowledge Engineering 24