Weighted and Multi-Valued Automata:

Determinization Methods

Miroslav Ćirić
Department of Computer Science
Faculty of Sciences
University of Niš, Serbia
miroslav.ciric@pmf.edu.rs

Erasmus+ Compact Lecture on Weighted Automata and Networks

Institut für Informatik, Universität Leipzig, Germany
May 27 – June 23, 2018

1 Miroslav Ćirić Weighted and Multi-Valued Automata

 Deterministic WFA

⋆ (S, +, ·, 0, 1) – semiring
⋆ A = (A, X, σ, τ, {δx}x∈X ) – WFA over S

⋆ complete deterministic WFA – CDWFA

(∃1 a ∈ A) σ(a) , 0
(∀x ∈ X)(∀a ∈ A)(∃1 b ∈ A) δx(a, b) , 0

⋆ crisp-deterministic WFA – c-DWFA – CDWFA with

(∃1 a0 ∈ A) σ(a0 ) = 1
(∀x ∈ X)(∀a ∈ A)(∃1 b ∈ A) δx(a, b) = 1
δx : A → A is a function – we write δx (a) = b
⋆ behaviour of a c-DWFA – [[A]](u) = τ(δu (a0 ))

2 Miroslav Ćirić Weighted and Multi-Valued Automata

 Multi-valued languages

⋆ S – arbitrary set, 0, 1 ∈ S – distinguished elements

⋆ f : X∗ → S – multi-valued language over S

⋆ derivative of f w.r.t. u ∈ X∗ – language fu : X∗ → S given by

fu (v) = f (uv)

f
⋆ derivative automaton of f – Af = (Af , X, σf , τf , {δx}x∈X )
◦ Af = {fu | u ∈ X∗ }
◦ σf = {f } i.e., σf (f ) = 1, σf (g) = 0 for g , f
◦ τf = {g} = g(ε)
f f
◦ δx (g) = gx , x ∈ X, δ x : Af → Af
⋆ c-DWA, possibly infinite

3 Miroslav Ćirić Weighted and Multi-Valued Automata

 Derivative automaton

Theorem (Minimal c-DWFA of a multi-valued language)

The derivative automaton Af is a unique (up to an isomorphism) mini-
mal c-DWA recognizing f .

⋆ kernel languages of f – f [i] = {u ∈ X∗ | f (u) = i}, i ∈ Im(f )

Theorem (Derivative automaton)

⋆ Af is a subdirect product of derivative automata of kernel

languages of f .
⋆ Af is isomorphic to the accessible part of the direct product of
derivative automata of kernel languages of f .

4 Miroslav Ćirić Weighted and Multi-Valued Automata

 c-DWFA recognizable languages

⋆ f is c-DWFA recognizable ⇔ Af is finite

Theorem (c-DWFA recognizable language)

f is c-DWFA recognizable if and only if it has a finite range and all its
kernel languages are recognizable.

Theorem (c-DWFA recognizable language)

f is c-DWFA recognizable if and only if it can be represented as

f = i1 K1 + i2 K2 + . . . + in Kn

where i1 , i2, . . . , in ∈ S and K1 , K2 , . . . , Kn are recognizable crisp-lang-

uages.

5 Miroslav Ćirić Weighted and Multi-Valued Automata

 Factor c-DWFA

⋆ congruence on a c-DWFA A – equivalence π on A satisfying

(a, b) ∈ π ⇒ (δx (a), δx (b)) ∈ π

⋆ additional condition: (a, b) ∈ π ⇒ τ(a) = τ(b) (π ⊆ ker(τ))

⋆ factor automaton w.r.t. π – Aπ = (Aπ , X, σπ , τπ , {δπx }x∈X )

◦ Aπ = {πa | a ∈ A} – set of equivalence classes
◦ σπ = {πa0 }
◦ τπ (πa ) = τ(a)
◦ δπx (πa ) = πδx (a) , x ∈ X, δπx : Aπ → Aπ

⋆ Aπ is a c-DWFA equivalent to A

6 Miroslav Ćirić Weighted and Multi-Valued Automata

 Minimization of a c-DWFA

Theorem (The greatest congruence)

The relation πτ on A given by
(a, b) ∈ πτ ⇔ (∀u ∈ X∗ ) τ(δu (a)) = τ(δu (b))
is the greatest congruence on A contained in ker(τ)
The factor automaton Aπ is a minimal c-DWFA equivalent to A.

Theorem (Computation of the greatest congruence)

Let {̺n }n∈N be a decreasing sequence of equivalences on A given by
̺1 = ker(τ), ̺n+1 = {(a, b) ∈ ̺ | (∀x ∈ X) (δx (a), δx (b)) ∈ ̺n }.
Then there is n ∈ N such that ̺n+1 = ̺n and ̺n = πτ .

7 Miroslav Ćirić Weighted and Multi-Valued Automata

 Nerode automaton

⋆ S – semiring
⋆ A = (A, X, σ, τ, {δx}x∈X ) – WFA over S

⋆ Nerode automaton of A – AN = (AN , X, σN , τN , {δNx }x∈X )

◦ AN = {σu | u ∈ X } – σε = σ, σux = σu · δx , u ∈ X∗ , x ∈ X,
∗

◦ σN = {σε }
◦ τN (σu ) = σu · τ, u ∈ X∗
◦ δN
x (σu ) = σux , u ∈ X∗ , x ∈ X x ∈ X, δN
x : AN → AN

Theorem (Nerode automaton)

The Nerode automaton AN is a c-DWA equivalent to A.

8 Miroslav Ćirić Weighted and Multi-Valued Automata

 Construction of the Nerode automaton

⋆ transition tree TN of AN – directly from A

⋆ pointer s(·) – points nodes to integer values
⋆ T1 = {σε } (root), set s(σε ) = 1, compute τN (σε ) = σε · τ
⋆ when Ti is constructed, nodes are labelled ’closed’ or ’non-closed’
⋆ compute Ti+1 : for any non-closed leaf σu ∈ Ti and x ∈ X
◦ add a node σux = σu · δx
◦ add an edge between σu and σux labelled by x
◦ if σux = σv for an already computed σv
mark σux ’closed’, set s(σux ) = s(σv )
◦ otherwise: compute τN (σux ) = σux · τ
set s(σux ) – the next unassigned integer
⋆ termination: when all leaves are marked ’closed’
⋆ transition graph of AN – erase closure marks, glue leaves to
interior nodes with the same pointer value
9 Miroslav Ćirić Weighted and Multi-Valued Automata
 Example (original automaton)

a3
x

y
a1 y x
x, y

x
a2 y

Transition graph of an NFA A

10 Miroslav Ćirić Weighted and Multi-Valued Automata

 Example (Transition tree)

   
σx2 yx σx2 y2 σxyx2 σ(xy)2

 x y x y 
σx3 σx2 y σxyx σxy2

x y x y
 
σx2 σxy σyx σy2

x y x y
σx σy
x y
σε

Transition tree of AN

11 Miroslav Ćirić Weighted and Multi-Valued Automata

 Example (Transition graph)

x, y

σx2 y σxyx

y x x, y
y

σx2 σxy
x

x y
y
σx σy
x
x y
σε

Transition graph of AN (7 states)

12 Miroslav Ćirić Weighted and Multi-Valued Automata

 Finiteness, number of states

⋆ generalization of the Accessible Subset Construction

⋆ difference: – AN can be infinite

⋆ if S is locally finite then AN is finite for any WFA A over S

⋆ even if AN is finite, its number of states can be enormously large

⋆ Problem 1: – Are there any determinization methods that

provide smaller automata?

⋆ Problem 2: – Are there any determinization methods that

can provide finite automata even if AN is infinite?

13 Miroslav Ćirić Weighted and Multi-Valued Automata

 Children automaton of AN

⋆ Children automaton – AcN = (AcN , X, σN,c , τN,c, {δN,c

x }x∈X )

◦ X = {x1 , . . . , xn }
◦ AcN = {σcu | u ∈ X∗ }
σcu = (σux1 , . . . , σuxn , σu · τ) = (σu · δx1 , . . . , σu · δxn , σu · τ)
◦ σN,c = {σcε }, τN,c (σcu ) = σu · τ
◦ δN,c
x (σu ) = σux
c c

Theorem (Children automaton)

The children automaton AcN is a c-DWA equivalent to A.
Besides, AcN is finite if and only if AN is finite and |AcN | 6 |AN |.

14 Miroslav Ćirić Weighted and Multi-Valued Automata

 Construction of the children automaton (example)

   
c c
σx2 yx σx2 y σx2 y2 σxyx2 σxyx σ(xy)2

 x y x y 
σx3 σ c2 σx2 y σxyx
c
σxy σxy2
x

x y x y
 
σxc σyc
σx2 σxy σyx σy2
y x
x y
σεc
σx σy
x y
σε

Simultaneous construction of the transition tree of AN

and the transition graph of AcN

15 Miroslav Ćirić Weighted and Multi-Valued Automata

 Construction of the children automaton (cont.)

x, y

x
σc2 c
σxy
x
y
x y
y
σcx σcy
x
x y
σcε

Transition graph of AcN (5 states)

16 Miroslav Ćirić Weighted and Multi-Valued Automata

 Use of weakly right invariant quasi-order matrices

⋆ S – additively idempotent semiring or unital quantale

⋆ A = (A, X, σ, τ, {δx}x∈X ) – WFA over S
⋆ π – weakly right invariant quasi-order matrix
⋆ Aπ = (Aπ , X, σπ , τπ , {δπx }x∈X )
◦ Aπ = {πu | u ∈ X∗ }, πε = σ · π, πux = πu · δx · π
π π
◦ σ = {πε }, τ (πu ) = πu · τ, δπx (πu ) = πux

Theorem (Automaton Aπ )
The automaton Aπ is a c-DWA equivalent to A.
Besides, |Aπ | 6 |AN | and Aπ can be finite even if AN is infinite.

⋆ similar construction – transition tree and graph of Aπ

⋆ children automaton of Aπ – even smaller

17 Miroslav Ćirić Weighted and Multi-Valued Automata

 Use of weakly left invariant quasi-order matrices

⋆ θ – weakly left invariant quasi-order matrix

⋆ Aθ is isomorphic to AN (anything new)
⋆ Aθ = (Aθ , X, σθ , τθ , {δθx }x∈X )
◦ Aθ = {θu | u ∈ X∗ }, θε = θ · τ, θxu = θ · δx · θu
◦ σθ = {θε }, τθ (θu ) = σ · θu
◦ δθx (θu ) = θxu

Theorem (Automaton Aθ )
Aθ is a c-DWA equivalent to the reverse automaton of A.
|Aθ | 6 |AN | and Aθ may be finite even if AN is infinite.

⋆ AN – reverse Nerode automaton of A – Nerode automaton of

the reverse automaton of A – AN = Aθ , θ identity matrix.

18 Miroslav Ćirić Weighted and Multi-Valued Automata

 Brzozowski type determinization

⋆ S – arbitrary semiring

Theorem (Brzozowski type determinization)

(AN )N is a minimal c-DWA equivalent to A.

⋆ Problem: In the first step AN could be infinite

⋆ there are examples that AN can be infinite even if the minimal
c-DWA equivalent to A is finite

Theorem (Brzozowski type determinization)

Let θ be the greatest weakly left invariant quasi-order matrix on a WFA
A over a unital quantale.
Then (Aθ )N is a minimal c-DWA equivalent to A.

19 Miroslav Ćirić Weighted and Multi-Valued Automata

 Use of degrees of language inclusion

⋆ S – complete residuated lattice

∗
⋆ τu ∈ SA , τu = δu · τ, τa ∈ SX , τa (u) = τu (a), f = [[A]]
^ ^
⋆ du (a) = τw (a) → σu · τw = τa (w) → fu (w) = I(τa , fu )
w∈X∗ w∈X∗
⋆ du (a) = I(τa , fu ) – degree of inclusion of τa in fu
⋆ Ad = (Ad , X, σd, τd , {δdx }x∈X ), Ad = {du | u ∈ X∗ }
◦ σd = {dε }, τd (du ) = du · τ, δdx (du ) = dux

Theorem (Automaton Ad )
Ad is a minimal c-DWA equivalent to A.

⋆ theoretically faster than Brzozowski type determinization

⋆ τw can be replaced by θw , where θ is a weakly left invariant q-o.m.
20 Miroslav Ćirić Weighted and Multi-Valued Automata
 CDWFA vs. cDWFA

⋆ crisp-deterministic WFA – recognize languages of finite range

(series of finite support)
⋆ complete deterministic WFA – can recognize languages of
infinite range (series of infinite support)

⋆ Problem: Determinization methods that provide complete

deterministic WFA

⋆ Mohri (1997), Kirsten and Mäurer (2005), de Mendı́vil and

Garitagoitia (2014, 2016, 2018)

⋆ Stanimirović, Ćirić, Ignjatović (2018)

21 Miroslav Ćirić Weighted and Multi-Valued Automata

 Factorization of vectors

⋆ Factorization: D = (f , g), f : SA → SA , g : SA → S
α = g(α) · f (α), for all α ∈ SA
g(0) = 1
⋆ Kirsten and Mäurer (2005)

⋆ Maximal factorization:
g(a · α) = a · g(α), for all a ∈ S, α ∈ SA , a · α , 0
f (a · α) = f (α)

⋆ Mohri’s factorization: S – unital quantale + divisibility

y 6 x ⇔ x · (x\y) = y = (y/x) · x

 1
 if α = 0

_
gM (α) = 
 α(a) otherwise , (fM (α))(a) = gM (α)\α(a)


a∈A

22 Miroslav Ćirić Weighted and Multi-Valued Automata

 Determinization by means of factorization

⋆ D = (f , g) – factorization in SA
⋆ AD = (AD , X, σD , τD , {δDx }x∈X )
D
◦ AD = {σu | u ∈ X }, ∗
σDε = f (σ), σD D
ux = f (σu · δx )
(
g(σ) if α = σD ε
◦ σD (α) = , τD (α) = α · τ, if α = σDu
0 otherwise
(
D g(σD
u · δx ) if α = σD D
u , β = σux , u ∈ X ,
∗
◦ δx (α, β) =
0 otherwise

Theorem (de Mendivil, Garitagoitia (2014))

If it is finite, AD is a CDWFA equivalent to A.

⋆ de Mendivil, Garitagoitia (2018) – Brzozowski procedure

⋆ maximal factorization – smaller AD
23 Miroslav Ćirić Weighted and Multi-Valued Automata
 Finiteness

⋆ Question: Under what conditions AD is finite?

⋆ Mohri (1997), Kirsten and Mäurer (2005)

sufficient condition – twins property

⋆ de Mendivil, Garitagoitia (2014)

weaker sufficient condition – representable-cycles property

⋆ our contribution – necessary and sufficient condition

weak representable-cycles property

24 Miroslav Ćirić Weighted and Multi-Valued Automata

 Our improvements

⋆ use of weakly right and left invariant quasi-order matrices

D,π
⋆ AD D D D
π = (Aπ , X, σπ , τπ , {δx }x∈X ), π – weakly right invariant
D D
◦ Aπ = {πu | u ∈ X }, ∗
πε = f (σ · π), πD
D D
ux = f (πu · δx · π)
(
D g(σ) if α = σD ε
◦ σ (α) = , τD (α) = α · τ, if α = πD u
0 otherwise
(
D,π g(πD u · δx · π) if α = πD D
u , β = πux , u ∈ X ,
∗
◦ δx (α, β) =
0 otherwise

Theorem (Automaton AD
π)

The automaton AD π is a CDWA equivalent to A.

Besides, |Aπ | 6 |AD | and AD
D
π can be finite even if AD is infinite.

⋆ children automaton of AD
π – even smaller

25 Miroslav Ćirić Weighted and Multi-Valued Automata

 WFAs over strong bimonoids

⋆ Strong bimonoid – (K, +, ·, 0, 1) : (K, +, 0), (K, ·, 1) – monoids

(K, +, 0) – commutative, 0 – multiplicative zero
⋆ ”semirings which might lack distributivity”
⋆ product of matrices over K need not be associative
⋆ A = (A, X, σ, τ, {δx}x∈X ) – WFA over K, u = x1 x2 . . . xn ∈ X∗
⋆ Initial algebra semantics – [[A]]i : X∗ → K
[[A]]i (u) = σu · τ, σu = (. . . ((σ · δx1 ) · δx2 ) · · · ) · δxn
⋆ Transition semantics – [[A]]t : X∗ → K
[[A]]t (u) = (σ · δu ) · τ, δu = (. . . ((δx1 · δx2 ) · δx3 ) · · · ) · δxn
⋆ Run semantics – [[A]]r : X∗ → K
X
[[A]]r (u) = σ(a0 )·δx1 (a0 , a1 )·δx2 (a1 , a2 )·. . .·δxn (an−1 , an )·τ(an )
(a0 ,a1 ...,an )∈An+1

26 Miroslav Ćirić Weighted and Multi-Valued Automata

 WFAs over strong bimonoids (cont.)

Theorem (Droste, Stüber, Vogler (2010))

[[A]]i = [[A]]i for each WFA A over K ⇐⇒ K is right distributive.

⋆ Crisp-determinizaton – construct a c-DWFA over K equivalent

to A w.r.t. a given semantics

⋆ Nerode automaton AN
equivalent to A w.r.t. the initial algebra semantics

⋆ Myhill automaton AM – similarly constructed

equivalent to A w.r.t. the transition semantics

⋆ necessary and sufficient conditions for finiteness of AN and AM

27 Miroslav Ćirić Weighted and Multi-Valued Automata

 The multiset construction
S
⋆ T – submonoid of (K, ·, 1) generated by x∈X Im(δx )
⋆ Assumptions: ⋄ R = (Im(σ) · T) \ {0} – finite
⋄ each k ∈ R · Im(τ) has a finite additive order
⋆ then A × R is finite
⋆ i = max{i(k) | k ∈ R · Im(τ)} i(k) – index of k in (K, +)
⋆ p = lcm{p(k) | k ∈ R · Im(τ)} p(k) – period of k in (K, +)
⋆ Pu (a, k) ⊆ A|u|+1 – all paths labelled by u, ending in a, of weight k
⋆ Pε (a, k) = {a} for k = σ(a), Pε (a, k) = ∅ for k , σ(a)
⋆ multisets: pu : A × R → N, πu : A × R → {0, 1, . . . , i + p − 1}
pu (a, k) = |Pu (a, k)| – number of paths
(
pu (a, k) if pu (a, k) < i
πu (a, k) =
i + [(pu (a, k) − i) mod p] if pu (a, k) > i

28 Miroslav Ćirić Weighted and Multi-Valued Automata

 The multiset construction (cont.)

⋆ Aπ = (Aπ , X, σπ , τπ , {δπx }x∈X )

◦ Aπ = {πu | u ∈ X∗ }
X
◦ σπ = {πε }, τπ (πu ) = πu (a, k)(k · τ(a))
(a,k)∈A×R

◦ δπx (π) = πux

Theorem (Ćirić, Droste, Ignjatović, Vogler (2010))

Under the above assumptions, Aπ is a c-DWFA equivalent to A w.r.t.
run semantics.

⋆ the starting assumptions hold if K is bi-locally finite

29 Miroslav Ćirić Weighted and Multi-Valued Automata