Formal Concept Analysis

Isabelle Bloch

LTCI, Télécom Paris

isabelle.bloch@telecom-paris.fr

I. Bloch (Telecom Paris) Symbolic AI 1 / 18

Objectives

Symbolic learning
Data mining.
Knowledge discovery
...
Input
data expressed as a table objects × attributes
Output
concept lattice:
clusters = formal concepts (nodes of the Hasse diagram)
sub-concept / super-concept hierarchy (partial order)
attribute implications:
representative set of dependencies among data

I. Bloch (Telecom Paris) Symbolic AI 2 / 18

A small example

I y1 y2 y3
x1 X X X
x2 X X
x3 X X

xi = objects
yi = attributes
example of concept: ({x1 , x2 }, {y1 , y3 })
example of attribute implication: {y1 } ⇒ {y3 }

I. Bloch (Telecom Paris) Symbolic AI 3 / 18

Historical notion of concept

Port-Royal logic (traditional logic): formal notion of concept

A. Arnauld, P. Nicole : La logique ou l’art de penser, 1662.

concept = extent (objects) + intent (attributes)

Later:
G. Birkhoff in the 1940’s: Lattice theory
M. Barbut, B. Monjardet in the 1970’s: partial order, classification
R. Wille in the 1980’s: hierarchy of concepts
B. Ganter and R. Wille in the 1990’s : formal concept analysis

I. Bloch (Telecom Paris) Symbolic AI 4 / 18

Formal Concept Analysis (FCA) (Ganter et al. 1997)

Set of objects G .
Set of attributes M.
Relation I ⊆ G × M: (g , m) ∈ I = object g has attribute m.
Formal context: K = (G , M, I ).
Derivation operators α : P(G ) → P(M) and β : P(M) → P(G ):

∀X ⊆ G , α(X ) = {m ∈ M | ∀g ∈ X , (g , m) ∈ I }

∀Y ⊆ M, β(Y ) = {g ∈ G | ∀m ∈ Y , (g , m) ∈ I }
(also denoted by 0 or ↑, ↓)
Example: α({x1 , x2 }) = {y1 , y3 }, β({y2 }) = {x1 , x3 }

I. Bloch (Telecom Paris) Symbolic AI 5 / 18

Formal concept and concept lattice

(X , Y ), X ⊆ G , Y ⊆ M is a formal concept if
α(X ) = Y and β(Y ) = X
Formal concept a = (e(a), i(a)), extent e(a) ⊆ G , intent i(a) ⊆ M.
(X , Y ) formal concept iff it is maximal for the property X × Y ⊆ I .
Partial ordering:

(X1 , Y1 )  (X2 , Y2 ) ⇔ X1 ⊆ X2 (⇔ Y2 ⊆ Y1 )

reflects the sub-concept / super-concept relation.

C: set of concepts of the context K = (G , M, I ).
(C, ) is a complete lattice, called concept lattice.

I. Bloch (Telecom Paris) Symbolic AI 6 / 18

Example of a context and its concept lattice from Wikipedia

K composite even odd prime square

1 × ×
2 × ×
3 × ×
4 × × ×
5 × ×
6 × ×
7 × ×
8 × ×
9 × × ×
10 × ×

Objects are integers from 1 to 10.

Attributes are composite (c) (i.e. non prime integer strictly greater
than 1), even (e), odd (o), prime (p) and square (s).
K = (G = {1, 2...10}, M = {c, e, o, p, s}, I ).

I. Bloch (Telecom Paris) Symbolic AI 7 / 18

K composite even odd prime square
1 × ×
2 × ×
3 × ×
4 × × ×
5 × ×
6 × ×
7 × ×
8 × ×
9 × × ×
10 × ×

The pair ({3, 5, 7}, {o, p}) is a formal concept.

I. Bloch (Telecom Paris) Symbolic AI 8 / 18

Galois connection
(α, β) is a Galois connection between the posets (P(G ), ⊆) and
(P(M), ⊆):
∀X ∈ P(G ), ∀Y ∈ P(M), Y ⊆ α(X ) ⇔ X ⊆ β(Y )
Equivalently:
1 ∀X1 , X2 ⊆ G , X1 ⊆ X2 ⇒ α(X2 ) ⊆ α(X1 )
2 ∀Y1 , Y2 ⊆ M, Y1 ⊆ Y2 ⇒ β(Y2 ) ⊆ β(Y1 )
3 ∀X ⊆ G , X ⊆ β(α(X ))
4 ∀Y ⊆ M, Y ⊆ α(β(Y ))
Consequently:
∀X ⊆ G , α(X ) = α(β(α(X )))
∀Y ⊆ M, β(Y ) = β(α(β(Y )))
αβ and βα are closure operators, i.e. increasing, extensive and
idempotent.
α(∪i Xi ) = ∩i α(Xi )
β(∪j Yj ) = ∩j β(Yj )
I. Bloch (Telecom Paris) Symbolic AI 9 / 18
Conversely
A : P(G ) → P(M)
B : P(M) → P(G )
(A, B) Galois connection
⇒ (A, B) is induced by a binary relation I
(g , m) ∈ I ⇔ m ∈ A({g }) ⇔ g ∈ B({m})
the derivation operators are then exactly α = A and β = B

I. Bloch (Telecom Paris) Symbolic AI 10 / 18

 Infimum and supremum of a family (Xt , Yt )t∈T of formal concepts:
!
^ \ [

(Xt , Yt ) = Xt , α β( Yt )
t∈T t∈T t∈T
!
_ [
\
(Xt , Yt ) = β α( Xt ) , Yt
t∈T t∈T t∈T

They are formal concepts.

I. Bloch (Telecom Paris) Symbolic AI 11 / 18

Clarified context
= without redundant columns or rows
The concept lattices before and after clarification are isomorphic.

Example

y1 y2 y3 y4
y1 y2 y3
x1 X X X X
x1 X X X
x2 X X X
– clarification → x2 X X
x3 X X X
x3 X X
x4 X X X
x5 X
x5 X

Another possible context reduction: based on reducible elements

I. Bloch (Telecom Paris) Symbolic AI 12 / 18

Concept lattice construction

Remark:
∀X ⊆ G , (β(α(X )), α(X )) is a formal concept
∀Y ⊆ M, (β(Y ), α(β(Y ))) is a formal concept
smallest concept: (β(α(∅)), α(∅))
⇒ starting from the smallest concept, iteratively add objects and compute
closure.

Main issue: complexity ⇒ several algorithms to reduce it.

|β(Y )|
Extension: add support information (frequent intents) Supp(Y ) = |G |

I. Bloch (Telecom Paris) Symbolic AI 13 / 18

Attribute implication

= description of some dependencies between data

Validity:
K = (G , M, I ), A ⊆ M, B ⊆ M
Subset M 0 of attributes (M 0 ⊆ M)
Attribute implication: A ⇒ B
A ⇒ B valid (true) in M 0 iff A ⊆ M 0 implies B ⊆ M 0 .
A ⇒ B valid (true) in K iff A ⇒ B valid in M 0 = {α({g }) | g ∈ G }.
Example
y1 y2 y3 y4
x1 X X X X
x2 X X X {y1 } ⇒ {y3 } valid
x3 X X X {y2 , y3 } ⇒ {y4 } valid
x4 X X X {y1 , y3 } ⇒ {y4 } not valid
x5 X X
I. Bloch (Telecom Paris) Symbolic AI 14 / 18
Reasoning with attribute implications
Theories
Models
Inference
Definition of non-redundant bases of implications
Guigues-Duquenne basis:
T = {A ⇒ α(β(A)) | A pseudo-intent of K}
Pseudo-intent: A ⊆ M such that
A 6= α(β(A))
α(β(B)) ⊆ A for each pseudo-intent B ⊂ A

I. Bloch (Telecom Paris) Symbolic AI 15 / 18

Extensions

Many-valued context: (G , M, W , I ), I ⊆ G × M × W (W = set of

values for attributes).
Fuzzy context: I : G × M → [0, 1], I (g , m) = degree to which object
g satisfies property m.
Links with possibility theory.
...

I. Bloch (Telecom Paris) Symbolic AI 16 / 18

A few applications

Classification and clustering.

Recognition.
Reinforcement learning (states / actions).
Information retrieval, knowledge extraction.
Social networks
Spatial reasoning.
Completing knowledge bases in description logics.
Inference, abduction...
...
Several softwares available – See e.g.
http://www.upriss.org.uk/fca/fcasoftware.html

I. Bloch (Telecom Paris) Symbolic AI 17 / 18

A few references

Marc Barbut and Bernard Monjardet, Ordre et classification,

Hachette, 1970.
Radim Belohlavek, Introduction to Formal Concept Analysis,
http://belohlavek.inf.upol.cz/vyuka/IntroFCA.pdf
Claudio Carpineto and Giovanni Romano, Concept Data Analysis:
Theory and Applications, John Wiley & Sons, 2004.
Nathalie Caspard, Bruno Leclerc and Bernard Monjardet, Finite
Ordered Sets: Concepts, Results and Uses, Cambridge University
Press, 2012.
Bernhard Ganter and Rudolf Wille, Formal Concept Analysis,
Springer, 1999.
Bernhard Ganter, Gerd Stumme, Rudolf Wille (Eds.): Formal Concept
Analysis Foundations and Applications. Springer, LNCS 3626, 2005.

I. Bloch (Telecom Paris) Symbolic AI 18 / 18