S101/S503

物理情報系のSMTモデル検査
SMT-based Model Checking of
Cyber-Physical Systems
2023年10月，October, 2023
石井 大輔 / Daisuke ISHII
次世代デジタル社会基盤研究領域，Next-gen digital infrastructure area
dsksh@jaist.ac.jp, https://www.dsksh.com
 概要 / Summary
• Topics
- 物理情報系 / Cyber-physical systems (CPS)
- モデルベース開発 / Model-based development
- モデル検査 / Model checking
- etc.
• Goal: Get an idea of how to improve the dependability
of CPS
SMT
Encoding solving Decoding
Logic Proof of
CPS model Satis ability
formula dependability

2
fi
物理情報系とモデルベース開発
Cyber-Physical Systems and Model-based
Development
物理情報系 / Cyber-physical systems (CPS)
• Computer systems that is tightly integrated with physical
environment
- Input: sensors; Output: actuators
- Hybrid system of discrete and continuous behaviors
- Example: automobiles, airplanes, robots, houses, medical
devices, power plants, etc.

• Inter-disciplinary area
- Programming languages, software engineering, numerical
simulation, control theory, optimization, etc. Image courtesy of NSF. 4
物理情報系 / Cyber-physical systems (CPS)
• Original denition: Integration of computation with
physical processes [H. Gill@NSF 2006]
- Composed of heterogeneous, potentially distributed,
components and systems, CPS bridges the digital and physical
worlds [NIST, 2017]
- A keyword of a science and technology policy in the U.S.
✴ Cf. Industrie 4.0 [Germany 2011]
✴ Cf. Society 5.0 [Japan 2016]

5
fi
物理情報系 / Cyber-physical systems (CPS)
• Typical structure:
Computation

Sensing Data

Action / Actuation

Physical Component
E. Lee and S. Seshia: Introduction to Embedded Systems (2nd Ed.), MIT Press, 2017. 6
Example CPS
• (Autonomous, electric) computer-controlled cars
• Smart watches with biosensors

https://waymo.com/waymo-driver/ (c) Apple

7
Example CPS
• Cyber-physical gaming
• ROS-based robots

(c) Nintendo
(c) Niryo 8
Development process of CPS
• V model with early model-based V&V (veri cation and validation)

Requirement
Model-based Model V&V System V&V
analysis
design
Conceptual Testing w/ System
modeling simulation integration

Simulation w/
Detailed modeling
hardware

Hardware
implementation

fi
Model-based design
• Development approach that exploits a "model" of the
target system
- Create a virtual product on a computer and simulate/analyze its
behaviors
- Parameter calibration, performance evaluation, V&V, etc.
- Models may be iteratively re ned into real products

• Before: Paper speci cation → Costly prototyping

- Cf. NASA's Apollo project

10
fi
fi
Modeling CPS
• Describe two components x'(t) = u(t) cos θ(t)

- Physical component (plant): Continuous system y'(t) = u(t) sin θ(t)

✴ E.g. mathematical equations θ'(t) = w(t)

- Cyber component (controller): Discrete system right straight

✴ E.g. state transition systems

stop left

• CPS modeling languages

- Provide syntax for describing both continuous and discrete
behaviors
- Example
✴ MATLAB/Simulink/State ow (graphical)
✴ Acumen [Taha+], HydLa [Ueda, Ishii+], Lustre [Caspi+] (textual) 11
fl
 Example CPS model
• Line tracer

https://automaticaddison.com/how-to-make-a-
line-following-robot-using-raspberry-pi/

[Lee&Seshia, 2017]
12
 Example CPS model (line tracer)
• Physical component (plant) Output signal x'(t) = u(t) cos θ(t)
- Di erential equation y'(t) = u(t) sin θ(t)
θ'(t) = w(t)

Input signal

• Cyber component (controller) right straight

- State transition system

✴ Each state feeds input to the plant stop left

13
ff
Example CPS model (line tracer)
• Model described with Simulink/State ow

14

fl
Example CPS model (line tracer)
model Vehicle(eps1, eps2) =
initially
• Model described m = "straight",
x = 0, x' = 0, y = 0, y' = 0, th = pi/2, th' = 0,
with Acumen u = 0, w = 0, e = 0
always
[Taha+ 2015] x' = u*cos(th), y' = u*sin(th), th' = w,
match m with
[ "stop" -> u = 0, w = 0, m+ = "straight"
| "straight" -> u = 10, w = 0,
if e > eps2 then m+ = "right"
elseif e < -eps2 then m+ = "left" noelse
| "right" -> u = 10, w = -pi,
if abs(e) < eps1 then m+ = "straight"
elseif e < -eps2 then m+ = "left" noelse
| "left" -> u = 10, w = pi,
if abs(e) < eps1 then m+ = "straight"
elseif e > eps2 then m+ = "right" noelse
],
... 15
Example CPS
• Example name
- P (physical component)
- D (sensing data from the physical component)
- C (what to compute w/ the cyber component)
- A (actuation data from the cyber component)
• Automobile
- P: motor, steering system, tires
- D: surrounding image, past trajectory of the vehicle
- C: safe driving control
- A: driving input (steering, acceleration and braking)
16
Example CPS
• Robot arm
- P: robot arm hardware
- D: current joint angles, end-e ector position
- C: smart control (e.g. model-predictive control)
- A: input voltage to the joint motors
• Smartwatch
- P: human user
- D: body temperature, heart rate, blood pressure
- C: human state estimation, anomaly detection
- A: sounds and images
17
ff
Benefits of the model-based
approach
• Simulation
- Execute a model on computers 表2 サーモスタットの統計的モデル検査結果. 124 CHAPTER 4. TAYLOR MODEL FLOWPIPES FOR HYBRID

- Understanding by imitation θ 0.1 0.3 0.5 0.7 0.9 x y= 3

n 59 167 930 96 50 `1 `2
x 2 [0.9, 1.1]

• Testing
Sat 27 81 428 36 24 y 2 [ 1.1, 0.9]
ẋ = y ẋ = y
実行時間 (s) 477 1150 6070 700 390 ẏ = x2 ẏ = x3

採用した仮説 H0 H0 H1 H1 H1 x y 3 2  y 

- Check the behavior, performance and

別途入力する. 検査では 3 つのステップを繰り返し実施し, シ
y= 2

Figure 4.16: Hybrid automaton of the 2-dimensional stable system

ミュレーション結果およびモニタリング成否のデータを蓄積す
other properties based on simulations
る. 各シミュレーションは, フロントエンドプロセスがシミュ
図5 サーモスタットの検査に関する可視化 (軌道集合) の例.
レーション用のプロセスを fork することで行う. 各変数の軌道 横軸は時間, 縦軸は temp の値を表す.

• Model checking
データやモニタリング結果が JSON ファイルとして出力され
る. フロントエンドプロセスは JSON ファイルを介してモニタ
リング結果を取得し, 集計結果について SPRT 法による仮説検 P1: req(y)

-
P2: req(y)

Check the possible states of a model

定を実施する.
6. 5 検査結果の可視化機能の実装
y: 1

統計的モデル検査の結果, シミュレーション・モニタリング
exhaustively and check a property
結果を保存した JSON ファイル群が生成される. 検査結果を直
P1: CS
P2: req(y)
y: 0
P1: req(y)
P2: CS
y: 0

感的に把握できるようにするため, これらのファイルの内容を
[Chen, 2015]
rigorously 要約し, グラフ描画ライブラリ Breeze-viz を用いて可視化す
（注 9）
P1: rel(y) P1: req(y)
る機能を実装した. 本機能に可視化したい model インスタン P2: req(y) P2: rel(y)

ス識別子および変数名を与え, 無作為シミュレーションされた
y: 1
図 6 サーモスタットの検査に関する可視化
Figure 4.17: Flowpipe over-approximations (of乱数集合
y: 1
) の例.
the 2-dimensional
18
stable
横軸は k1, 縦軸は k2 の値を表す.
SMTソルバーを用いたCPSモデル検査
CPS Model Checking using SMT Solvers
Overview
• Issues in the modeling/simulation/veri cation of
cyber-physical systems (CPS)
- Autonomy and scaling (in size and complexity) of systems
- Approximations made in modeling (abstraction) and
simulation (numerical errors)
- Computational di culty in veri cation
- etc.

• Objective: Enable to model CPS appropriately and to

verify their useful properties
20
ffi
fi
fi
モデル検査の対象 / Model checking target
• We aim at the veri cation of CPS models

Requirement
Model checking System V&V
analysis

Conceptual Testing w/ System

modeling simulation integration

Detailed Simulation w/
modeling hardware

Hardware
implementation

21
fi
 Model checking via logic encoding
• Example: veri cation of a Simulink model
in@1,sum@1,dl@1,... : real

-1 <= in@1 <= 1 /\

1 sum <= 6 >0 1
sum@1 = in@1 + 0.9*dl@1 /\
vi_20
in vo_28 dl@1 = 0.0 /\
_vr_165 out out@1 = if sum@1<=0.6 then 1 else 2
Z-1
/\ -1 <= in@2 <= 1 /\
sum@2 = in@2 + 0.9*dl@2 /\
dl
vd_25
vs_27
dl@2 = sum@1 /\
out@2 = if sum@2<=0.6 then 1 else 2
2.0 7 ...
6
1.5 5 out@10 = 2
1.0 4
3
0.5 2 Predicate logic formula encoding
1
2 4 6 8 10
the model and a property
2 4 6 8 10

Input signal Output satisfying the property 22

fi
 (Discrete-time) Simulink models
• Diagrams describing computation on signals
158 D. Ishii et al.
1, 1, 1, ... 1, 1.9, 2.71, ...

1
1 8 Const1● ●
○
●
○ ●
○ ○
● ○
● ○
In1 ● ● ○
○ >5 ● ○
○ s
Saturate 61 1 1● ● ○ ● 1
Out1 ● ○
○ ● o
Add In1
4 ●
●
○
○
Out1
● ○
● ○
2 ● ○
●
■ ○
■ ■ Subsystem1
■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■
0○ ■ ■ ■ ■ i

Gain Delay 5 Const2

10 15
Switch

(a) Model S1 : An integrator. (b) Model S2 : A switch after S1 .

23
I
O
n
u
1
t
1
Invariance properties
• Properties stating that "always ϕ holds" where ϕ is a
property on a state
Out < 10
- Example: "always Out < 10"
Example: "always Out ≠ 2"
8 ● ○

-
● ●
○ ○
● ●
○ ○ ● ○
● ●
○ ○ ● ○
6 ● ●
○ ○ ● ○ s
● ○ ● o
● ○
4 ●
●
○
○
● ○
2 ● ○
●
■ ○
■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ ■
0○ ■ ■ ■ ■ i

5 10 15

24
 SMT solvers
• Tool for checking satis ability modulo theories
- Input: predicate logic formulas
- Based on e cient search algorithms
- E.g. Z3, https://github.com/Z3Prover/z3
- E.g. CVC5, https://cvc5.github.io/

• Example theories
- Integer/real arithmetic x = 2y + 1 ∧ ( x > y z=y/x)
- Equality and functions x, y: integer variables
- Bit vectors z: real variables
- Di erential equations Solution: Satis able
x = 3, y = 1, z = 0.333...
25
ff
ffi
fi
fi
SMT solving example
Description in the SMT-LIB format:
;; Declaration of constants.
(declare-const x Int)
(declare-const y Int)
(declare-const z Real)
x = 2y + 1 ∧ ( x > y z=y/x)
;; Assertions of constraints.
x, y: integer variables (assert (= x (+ (* 2 y) 1)))
(assert (=> (> x y) (= z (/ y x))))
z: real variables
;; Check the satisfiability.
(check-sat)

;; Additional assertion.
(assert (and (> x y) (< x 0)))

;; Check again.
(check-sat)
26
Solving process of SMT solver
-1 <= in <= 1 /\ P1 /\ P1: T
sum = in + 0.9*dl /\ P2 /\ P2: T
dl = 5.0 /\ P3 /\ P3: T
(not sum <= 0.6 \/ out = 1) /\ (not P4 \/ P5) /\ P4: F
(sum <= 0.6 \/ out = 2) /\ (P4 \/ P6) /\ P5: F
out = 2 P6 P6: T
Extraction of
Predicate logic formula SAT checking
propositional formula
9

8
-1 <= in <= 1 /\
sum = in + 0.9*dl /\
SAT 7
dl = 5.0 /\
(assignment
example)
* 6

5 not sum <= 0.6 /\

out = 2 /\
/ UNSAT 4

-1.0 -0.5 0.0 0.5 1.0 1.5

out = 2
Analysis w/ arithmetic solvers Put back the predicates 27
 Example: SMT-based model checking of
Simulink models
SMT
Encoding solving Decoding
160 D. Ishii et al. Veri cation
CPS model Logic formula Solving result
result
Simulink model, Encoder
Begin
invariance, and (for system definition)
Model checker
other parameters
IR
generation For each subsystem/block b in the IR Prove
No
Generate block Printing invariance Yes
table Encode Yes ?
exactly ?
Generate No
subsystem table
Approx. Exact K-
BMC
printer printer induction
Slicing /
Rate separation
b

Invariance proof or
End
a counterexample

D. Ishii,
Fig. 2. The process T. Tomita, model
of SMT-based T. Aoki, T.Q. Ngô, T.B.N. Do, H. Takai:
checking.
SMT-Based Model Checking of Industrial Simulink Models,
https://www.gaio.co.jp/products/prompt-2/ ICFEM, LNCS 13478, pp. 156-172, 2022.
28
where s ∈ D(V ), i ∈ D(V ), o ∈ D(V ), I(s ), and T (s ,s ,i ,o )
fi
 Simulink encoding example
• Models can be regarded as relations on input, output, and
internal state signals
• Description of the relations in SMT-LIB
I(s0) ⟺ s0 = 0

158 D. Ishii et al.

T(s, s′, i, o) ⟺ ∃lv, . . .
∧ o = lv + 0.9s0 ∧ s1 = o

1
1 Const1
In1 1 1 1 >5 1
Saturate Out1
Add In1 Out1

Subsystem1

Gain Delay Const2 Switch

29

;
(
(
)
.
;
.
d
d
.
(
(
B
(
)
S
e
e
=
s
l
o
e
p
)
f
f
@
(
i
i
o
t
e
s
a
n
n
0
l
c
@
(
n
e
e
(
i
d
-
-
f
0
R
l
f
f
i
v
c
e
u
u
(
(
0
a
=
(
=
a
n
n
)
s
t
l
)
i
t
i
o
a
s
n
o
r
@
t
(
i
a
n
(
t
u
s
+
n
1
@
r
o
(
s
a
(
l
f
o
1
v
t
s
(
I
)
e
@
(
(
R
a
c
)
*
1
e
0
n
I
a
0
d
n
(
R
l
-
.
t
)
T
9
e
)
1
(
.
a
s
i
)
l
)
@
R
i
)
)
e
0
)
B
a
)
)
o
l
)
)
o
)
(
l
o
R
e
a
l
)
)
(
.
.
d
;
(
.
(
B
(
)
;
<
e
s
l
o
I
e
)
f
=
n
@
(
i
o
t
a
n
v
0
o
l
(
n
e
a
(
d
r
-
1
R
l
i
f
v
a
e
0
u
(
(
n
=
(
=
a
n
)
t
s
l
)
t
o
a
s
158
i
r
n
@
1
t
(
a
In1
(
u
s
s
+
n
1
t
@
r
s
r
a
l
o
u
1
1
v
t
m
)
e
(
R
(
)
*
(
1
e
e
c
n
a
0
(
t
l
I
-
.
)
n
a
9
Saturate
1
t
t
(
)
i
s
a
)
o
@
i
n
R
)
0
.
e
)
)
a
)
)
l
)
)
Add

Gain
(
D. Ishii et al.
o
R
e
a
l
)
)
Delay
o ≤ 10

1
Out1
• Invariant instrumentation

1
In1
1
Simulink encoding example

Subsystem1
Const2
Const1
>5

Switch
1
Out1

30
Simulink encoding example
• Encoding of execution paths
...

;; Encoding of the execution path.

(declare-const s@i Real)
(assert (init s@i))

(declare-const s@0 Real) (declare-const i@0 Real)

158 D. Ishii et al. (declare-const o@0 Int)
(assert (trans 0 s@i s@0 i@0 o@0))

(declare-const s@1 Real) (declare-const i@1 Real)

(declare-const o@1 Int)
1
1 (assert (trans 1 s@0 Const1
s@1 i@1 o@1))
In1 1 1 1 >5 1
Saturate Out1
Add
;; Check
In1 the reachability at step 1. Out1
(check-sat)
Subsystem1

Gain Delay Const2 Switch

31
Model checking applications
• Test generation for the boundary test objectives
- Search for an input signal that can enable a Switch block
• Coverage testing
- Check whether every block is activated at lease once in a
test execution

• Safety veri cation

- Instrument a failure detection circuit and check whether it is
activated

32
fi
 properties.

Model # blocks # s’s P1 P2 Q1 Q2

Experimental result (1)th

# b’s k
12 1 11 7 —
# b’s k # b’s
11
k d # b’s
1 1 —
k d

(2)th 16 1 14 2 14 7 14 10 2 14 29 2

• Model checking using the encoding

(3)
(4)
24
290
method
1
32
22 15 —
289 12 —
and
th

th
the
289
Z3
22 ≥ 34 2 —
1 1 —

SMT solver (5)

(6)
33
479
8
39
26 21 —
321 2 327 1
25
259
1 1 —
1 1 321 1 2

• Invariance veri cation on arti

(8) cial712and
(7)
188 industrial
27 1 327 1 Simulink
291
27
18
1 3 327
116 1 103 1 58 1 2 9 1 3
1 3

models (9) 574 30 19 1 407 ≥ 1 12 1 1 407 1 3

PM A

• Comparison with

TO

TO

TO

TO

TO

TO

Error
Error
PM E
1000 SLDV

A
100

Error
Error
an o cial veri cation 10

tool SLDV (a) Falsification.

A TO

A TO
TO

TO
TO
TO
TO
TO

TO

TO
TO

TO
Error
1000
Outperformed on 17 100
out of 28 properties 10

(b) Invariance verification.

33
ffi
fi
fi
fi
 研究プロジェクト / Research project
• Application
development

Robot arm controller

[M2 Chen] Driver-assistance system [D1 中釜]

• Development of an SMT-solver-based MC method

[M2 Tien Dat] [Ishii+ 2022]
SMT
Encoding solving Decoding
Logic Proof of
CPS model Satis ability
formula dependability

34
fi
 Example master's research
• Compositional checking of models
- Issue: Scalability issue of the model checking process
- Approach: Composition of the target and module-wise processing
- Research items: 部品A ¦=（□In ) ⇒ (□Out )
A A
✴ Compositional MC method 部品B ¦= (□InB) ⇒ (□OutB)
✴ Automation of compositional MC
Compositional 部品A ¦¦ 部品B ¦¦ 外部環境 ¦= 安全性
system
SMT
Encoding solving
Logic Veri cation
CPS module Solving result
formula result
Decoding
…

…
Logic
CPS module Solving result Compositional
formula
reasoning 35
fi
Example master's research
• Test method for a robot arm
- Issue: Gap between the model and the
implementation in the robot development
✴ E.g. gap w.r.t time and poses

- Approach:
Application of the model-based testing
(MBT) method
- Research items
✴ Speci cation of the robot model
✴ Formalization of the conformance between
the robot movements
✴ Conformance checking method
36
fi
まとめ
Conclusion
 まとめ / Conclusion
• 物理情報系 / Cyber-physical systems
• Model-based design of CPS
- Description of continuous and right straight
x'(t) = u(t) cos θ(t)
discrete behaviors y'(t) = u(t) sin θ(t)
- Simulation, testing, veri cation stop left
θ'(t) = w(t)

• SMT-based model checking

SMT
Encoding solving Decoding
Logic Veri cation
CPS model Solving result
formula result
38
fi
fi
 課題 / Assignment (S101)
物理情報系 (CPS) の例 (講義中の例とは異なるもの) を考えなさい．
講義内容を参考にして，例について項目P, D, C, Aを書いてください．
合計100字以内で答えよ．

Consider an example of cyber-physical systems that is

di erent from the examples in the lecture.
By referring to the lecture content, describe the items
P, D, C and A for the example.
Answer within 40 words.

39
ff
課題 / Assignment (S101)
• 解答例 (76文字) 自動運転の電気自動車．
P：モーター，ステアリング系，タイヤ．
D：周囲の画像，車の位置・速度．
C：安全・快適な運転．
A：操舵入力，アクセル量，ブレーキ量．

• Example (29 words)

Autonomous electric vehicle.
P: Motor, steering system, and tires.
D: Surrounding images, and vehicle position/velocity.
C: Safe and confortable driving.
A: Steering input, acceleration amount, and braking amount. 40
課題 / Assignment (S503)
物理情報系 (CPS) の例と，当該系の安全性を脅かす事故の例を考え，簡潔に書
きなさい．講義中の例とは異なる系・事故を考えよ．
例について以下の項目C，P，Sを書きなさい．
• C: 原因となりうる情報部品の欠陥．
• P: 原因となりうる物理部品の欠陥．
• S: 当該事故の対策方法．
Describe brie y an example cyber-physical system and an example of
an accident that threatens the safety of the system. Consider a CPS/
accident that is di erent from the examples in the lecture.
Describe the following items C, P and S for the example.
• C: Defect(s) in the cyber component that could cause the accident.
• P: Defect(s) in the physical component that could cause the accident.
• S: Method(s) to prevent the accident. 41
fl
ff
課題 / Assignment (S503)
合計125字以内で答えよ．/ Answer within 50 words.
• 解答例 (86文字)
例：自動運転の自動車が路傍に衝突する事故．
C：敵対的な標識イメージの誤認識．
P：カメラの故障による取得イメージ欠損．
S：多数の・多様な生成イメージを使った自動テストを実施．

• Example (36 words)

Example: Accident in which an automated vehicle crashes into a roadside.
C: Misidenti cation of adversarial sign images.
P: Image loss due to camera malfunction.
S: Automated testing with generating a large number and variety of images. 42
fi
 学生募集 / Call for new lab members
• 現在の研究プロジェクト
/ Current projects

Robot arm controller

Driver-assistance system
SMT-based model checking method
SMT
Encoding solving Decoding
Logic Proof of
CPS model Satis ability
formula dependability

• その他のテーマでも / Or, other themes

- 例: 機械学習系の検証 / E.g. veri cation of ML systems 43
fi
fi