Computer Aided Verification 32nd

International Conference CAV 2020 Los

Angeles CA USA July 21 24 2020
Proceedings Part I Shuvendu K. Lahiri
Visit to download the full and correct content document:
https://textbookfull.com/product/computer-aided-verification-32nd-international-confer
ence-cav-2020-los-angeles-ca-usa-july-21-24-2020-proceedings-part-i-shuvendu-k-la
hiri/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Computer Aided Verification 32nd International

Conference CAV 2020 Los Angeles CA USA July 21 24 2020
Proceedings Part II Shuvendu K. Lahiri

https://textbookfull.com/product/computer-aided-
verification-32nd-international-conference-cav-2020-los-angeles-
ca-usa-july-21-24-2020-proceedings-part-ii-shuvendu-k-lahiri/

Runtime Verification 20th International Conference RV

2020 Los Angeles CA USA October 6 9 2020 Proceedings
Jyotirmoy Deshmukh

https://textbookfull.com/product/runtime-verification-20th-
international-conference-rv-2020-los-angeles-ca-usa-
october-6-9-2020-proceedings-jyotirmoy-deshmukh/

Automated Technology for Verification and Analysis 16th

International Symposium ATVA 2018 Los Angeles CA USA
October 7 10 2018 Proceedings Shuvendu K. Lahiri

https://textbookfull.com/product/automated-technology-for-
verification-and-analysis-16th-international-symposium-
atva-2018-los-angeles-ca-usa-october-7-10-2018-proceedings-
shuvendu-k-lahiri/

HCI International 2020 Posters 22nd International

Conference HCII 2020 Copenhagen Denmark July 19 24 2020
Proceedings Part I Constantine Stephanidis

https://textbookfull.com/product/hci-
international-2020-posters-22nd-international-conference-
hcii-2020-copenhagen-denmark-july-19-24-2020-proceedings-part-i-
Computer Aided Verification 28th International
Conference CAV 2016 Toronto ON Canada July 17 23 2016
Proceedings Part II 1st Edition Swarat Chaudhuri

https://textbookfull.com/product/computer-aided-
verification-28th-international-conference-cav-2016-toronto-on-
canada-july-17-23-2016-proceedings-part-ii-1st-edition-swarat-
chaudhuri/

HCI International 2020 Late Breaking Posters 22nd

International Conference HCII 2020 Copenhagen Denmark
July 19 24 2020 Proceedings Part I Constantine
Stephanidis
https://textbookfull.com/product/hci-international-2020-late-
breaking-posters-22nd-international-conference-
hcii-2020-copenhagen-denmark-july-19-24-2020-proceedings-part-i-
constantine-stephanidis/

Computer Aided Verification 30th International

Conference CAV 2018 Held as Part of the Federated Logic
Conference FloC 2018 Oxford UK July 14 17 2018
Proceedings Part II Hana Chockler
https://textbookfull.com/product/computer-aided-
verification-30th-international-conference-cav-2018-held-as-part-
of-the-federated-logic-conference-floc-2018-oxford-uk-
july-14-17-2018-proceedings-part-ii-hana-chockler/

HCI International 2015 Posters Extended Abstracts

International Conference HCI International 2015 Los
Angeles CA USA August 2 7 2015 Proceedings Part I 1st
Edition Constantine Stephanidis (Eds.)
https://textbookfull.com/product/hci-international-2015-posters-
extended-abstracts-international-conference-hci-
international-2015-los-angeles-ca-usa-
august-2-7-2015-proceedings-part-i-1st-edition-constantine-
stephanidis-eds/
HCI International 2020 Posters 22nd International
Conference HCII 2020 Copenhagen Denmark July 19 24 2020
Proceedings Part II Constantine Stephanidis

https://textbookfull.com/product/hci-
international-2020-posters-22nd-international-conference-
hcii-2020-copenhagen-denmark-july-19-24-2020-proceedings-part-ii-
 Shuvendu K. Lahiri
Chao Wang (Eds.)
LNCS 12224

Computer Aided
Verification
32nd International Conference, CAV 2020
Los Angeles, CA, USA, July 21–24, 2020
Proceedings, Part I
Lecture Notes in Computer Science 12224

Founding Editors
Gerhard Goos
Karlsruhe Institute of Technology, Karlsruhe, Germany
Juris Hartmanis
Cornell University, Ithaca, NY, USA

Editorial Board Members

Elisa Bertino
Purdue University, West Lafayette, IN, USA
Wen Gao
Peking University, Beijing, China
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Gerhard Woeginger
RWTH Aachen, Aachen, Germany
Moti Yung
Columbia University, New York, NY, USA
More information about this series at http://www.springer.com/series/7407
Shuvendu K. Lahiri Chao Wang (Eds.)
•

Computer Aided
Verification
32nd International Conference, CAV 2020
Los Angeles, CA, USA, July 21–24, 2020
Proceedings, Part I

123
Editors
Shuvendu K. Lahiri Chao Wang
Microsoft Research Lab University of Southern California
Redmond, WA, USA Los Angeles, CA, USA

ISSN 0302-9743 ISSN 1611-3349 (electronic)

Lecture Notes in Computer Science
ISBN 978-3-030-53287-1 ISBN 978-3-030-53288-8 (eBook)
https://doi.org/10.1007/978-3-030-53288-8

LNCS Sublibrary: SL1 – Theoretical Computer Science and General Issues

© The Editor(s) (if applicable) and The Author(s) 2020. This book is an open access publication.
Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 International
License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution
and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and
the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this book are included in the book’s Creative Commons license,
unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative
Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use,
you will need to obtain permission directly from the copyright holder.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, expressed or implied, with respect to the material contained herein or for any errors or
omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
 Preface

It was our privilege to serve as the program chairs for CAV 2020, the 32nd
International Conference on Computer-Aided Verification. CAV 2020 was held as a
virtual conference during July 21–24, 2020. The tutorial day was on July 20, 2020, and
the pre-conference workshops were held during July 19–20, 2020. Due to the
coronavirus disease (COVID-19) outbreak, all events took place online.
CAV is an annual conference dedicated to the advancement of the theory and
practice of computer-aided formal analysis methods for hardware and software sys-
tems. The primary focus of CAV is to extend the frontiers of verification techniques by
expanding to new domains such as security, quantum computing, and machine
learning. This puts CAV at the cutting edge of formal methods research, and this year’s
program is a reflection of this commitment.
CAV 2020 received a very high number of submissions (240). We accepted 18 tool
papers, 4 case studies, and 43 regular papers, which amounts to an acceptance rate of
roughly 27%. The accepted papers cover a wide spectrum of topics, from theoretical
results to applications of formal methods. These papers apply or extend formal methods
to a wide range of domains such as concurrency, machine learning, and industrially
deployed systems. The program featured invited talks by David Dill (Calibra) and
Pushmeet Kohli (Google DeepMind) as well as invited tutorials by Tevfik Bultan
(University of California, Santa Barbara) and Sriram Sankaranarayanan (University of
Colorado at Boulder). Furthermore, we continued the tradition of Logic Lounge, a
series of discussions on computer science topics targeting a general audience.
In addition to the main conference, CAV 2020 hosted the following workshops:
Numerical Software Verification (NSV), Verified Software: Theories, Tools, and
Experiments (VSTTE), Verification of Neural Networks (VNN), Democratizing Soft-
ware Verification, Synthesis (SYNT), Program Equivalence and Relational Reasoning
(PERR), Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), Formal
Methods for Blockchains (FMBC), and Verification Mentoring Workshop (VMW).
Organizing a flagship conference like CAV requires a great deal of effort from the
community. The Program Committee (PC) for CAV 2020 consisted of 85 members – a
committee of this size ensures that each member has to review a reasonable number of
papers in the allotted time. In all, the committee members wrote over 960 reviews while
investing significant effort to maintain and ensure the high quality of the conference
program. We are grateful to the CAV 2020 PC for their outstanding efforts in evalu-
ating the submissions and making sure that each paper got a fair chance. Like last
year’s CAV, we made the artifact evaluation mandatory for tool paper submissions and
optional but encouraged for the rest of the accepted papers. The Artifact Evaluation
Committee consisted of 40 reviewers who put in significant effort to evaluate each
artifact. The goal of this process was to provide constructive feedback to tool devel-
opers and help make the research published in CAV more reproducible. The Artifact
vi Preface

Evaluation Committee was generally quite impressed by the quality of the artifacts,
and, in fact, all accepted tools passed the artifact evaluation. Among the accepted
regular papers, 67% of the authors submitted an artifact, and 76% of these artifacts
passed the evaluation. We are also very grateful to the Artifact Evaluation Committee
for their hard work and dedication in evaluating the submitted artifacts. The evaluation
and selection process involved thorough online PC discussions using the EasyChair
conference management system, resulting in more than 2,000 comments.
CAV 2020 would not have been possible without the tremendous help we received
from several individuals, and we would like to thank everyone who helped make CAV
2020 a success. First, we would like to thank Xinyu Wang and He Zhu for chairing the
Artifact Evaluation Committee and Jyotirmoy Deshmukh for local arrangements. We
also thank Zvonimir Rakamaric for chairing the workshop organization, Clark Barrett
for managing sponsorship, Thomas Wies for arranging student fellowships, and Yakir
Vizel for handling publicity. We also thank Roopsha Samanta for chairing the Men-
toring Committee. Last but not least, we would like to thank members of the CAV
Steering Committee (Kenneth McMillan, Aarti Gupta, Orna Grumberg, and Daniel
Kroening) for helping us with several important aspects of organizing CAV 2020.
We hope that you will find the proceedings of CAV 2020 scientifically interesting
and thought-provoking!

June 2020 Shuvendu K. Lahiri

Chao Wang
 Organization

Program Chairs
Shuvendu K. Lahiri Microsoft Research, USA
Chao Wang University of Southern California, USA

Workshop Chair
Zvonimir Rakamaric University of Utah, USA

Sponsorship Chair
Clark Barrett Stanford University, USA

Publicity Chair
Yakir Vizel Technion - Israel Institute of Technology, Israel

Fellowship Chair
Thomas Wies New York University, USA

Local Arrangements Chair

Jyotirmoy Deshmukh University of Southern California, USA

Program Committee
Aws Albarghouthi University of Wisconsin-Madison, USA
Jade Alglave University College London, UK
Christel Baier Technical University of Dresden, Germany
Gogul Balakrishnan Google, USA
Sorav Bansal India Institute of Technology, Delhi, India
Gilles Barthe Max Planck Institute, Germany
Josh Berdine Facebook, UK
Per Bjesse Synopsys, USA
Sam Blackshear Calibra, USA
Roderick Bloem Graz University of Technology, Austria
Borzoo Bonakdarpour Iowa State University, USA
Ahmed Bouajjani Paris Diderot University, France
Tevfik Bultan University of California, Santa Barbara, USA
Pavol Cerny Vienna University of Technology, Austria
viii Organization

Sagar Chaki Mentor Graphics, USA

Swarat Chaudhuri University of Texas, Austin, USA
Hana Chockler King’s College London, UK
Maria Christakis Max Planck Institute, Germany
Eva Darulova Max Planck Institute, Germany
Cristina David University of Cambridge, UK
Ankush Desai Amazon, USA
Jyotirmoy Deshmukh University of Southern California, USA
Cezara Dragoi Inria, France
Kerstin Eder University of Bristol, UK
Michael Emmi Amazon, USA
Constantin Enea Université de Paris, France
Lu Feng University of Virginia, USA
Yu Feng University of California, Santa Barbara, USA
Bernd Finkbeiner Saarland University, Germany
Dana Fisman Ben-Gurion University, Israel
Daniel J. Fremont University of California, Santa Cruz, USA
Malay Ganai Synopsys, USA
Ganesh Gopalakrishnan University of Utah, USA
Orna Grumberg Technion - Israel Institute of Technology, Israel
Arie Gurfinkel University of Waterloo, Canada
Alan J. Hu The University of British Columbia, Canada
Laura Humphrey Air Force Research Laboratory, USA
Franjo Ivancic Google, USA
Joxan Jaffar National University of Singapore, Singapore
Dejan Jovanović SRI International, USA
Zachary Kincaid Princeton University, USA
Laura Kovacs Vienna University of Technology, Austria
Daniel Kroening University of Oxford, UK
Ori Lahav Tel Aviv University, Israel
Akash Lal Microsoft, India
Anthony Lin TU Kaiserslautern, Germany
Yang Liu Nanyang Technological University, Singapore
Francesco Logozzo Facebook, USA
Ruben Martins Carnegie Mellon University, USA
Anastasia Mavridou NASA Ames Research Center, USA
Jedidiah McClurg Colorado School of Mines, USA
Kenneth McMillan Microsoft, USA
Kuldeep S. Meel National University of Singapore, Singapore
Sayan Mitra University of Illinois at Urbana-Champaign, USA
Ruzica Piskac Yale University, USA
Xiaokang Qiu Purdue University, USA
Mukund Raghothaman University of Southern California, USA
Jan Reineke Saarland University, Germany
Kristin Yvonne Rozier Iowa State University, USA
Philipp Ruemmer Uppsala University, Sweden
 Organization ix

Krishna S India Institute of Technology, Bombay, India

Sriram Sankaranarayanan University of Colorado at Boulder, USA
Natarajan Shankar SRI International, USA
Natasha Sharygina University of Lugano, Switzerland
Sharon Shoham Tel Aviv University, Israel
Alexandra Silva University College London, UK
Anna Slobodova Centaur Technology, USA
Fabio Somenzi University of Colorado at Boulder, USA
Fu Song ShanghaiTech University, China
Aditya Thakur University of California, Davis, USA
Ashish Tiwari Microsoft, USA
Aaron Tomb Galois, Inc., USA
Ashutosh Trivedi University of Colorado at Boulder, USA
Caterina Urban Inria, France
Niki Vazou IMDEA, Spain
Margus Veanes Microsoft, USA
Yakir Vizel Technion - Israel Institute of Technology, Israel
Xinyu Wang University of Michigan, USA
Georg Weissenbacher Vienna University of Technology, Austria
Fei Xie Portland State University, USA
Jin Yang Intel, USA
Naijun Zhan Chinese Academy of Sciences, China
He Zhu Rutgers University, USA

Artifact Evaluation Committee

Xinyu Wang (Co-chair) University of Michigan, USA
He Zhu (Co-chair) Rutgers University, USA
Angello Astorga University of Illinois at Urbana-Champaign, USA
Subarno Banerjee University of Michigan, USA
Martin Blicha University of Lugano, Switzerland
Brandon Bohrer Carnegie Mellon University, USA
Jose Cambronero Massachusetts Institute of Technology, USA
Joonwon Choi Massachusetts Institute of Technology, USA
Norine Coenen Saarland University, Germany
Katherine Cordwell Carnegie Mellon University, USA
Chuchu Fan Massachusetts Institute of Technology, USA
Yotam Feldman Tel Aviv University, Israel
Timon Gehr ETH Zurich, Switzerland
Aman Goel University of Michigan, USA
Chih-Duo Hong University of Oxford, UK
Bo-Yuan Huang Princeton University, USA
Jeevana Priya Inala Massachusetts Institute of Technology, USA
Samuel Kaufman University of Washington, USA
Ratan Lal Kansas State University, USA
Stella Lau Massachusetts Institute of Technology, USA
x Organization

Juneyoung Lee Seoul National University, South Korea

Enrico Magnago Fondazione Bruno Kessler, Italy
Umang Mathur University of Illinois at Urbana-Champaign, USA
Jedidiah McClurg Colorado School of Mines, USA
Sam Merten Ohio University, USA
Luan Nguyen University of Pennsylvania, USA
Aina Niemetz Stanford University, USA
Shankara Pailoor The University of Texas at Austin, USA
Brandon Paulsen University of Southern California, USA
Mouhammad Sakr Saarland University, Germany
Daniel Selsam Microsoft Research, USA
Jiasi Shen Massachusetts Institute of Technology, USA
Xujie Si University of Pennsylvania, USA
Gagandeep Singh ETH Zurich, Switzerland
Abhinav Verma Rice University, USA
Di Wang Carnegie Mellon University, USA
Yuepeng Wang The University of Texas at Austin, USA
Guannan Wei Purdue University, USA
Zikang Xiong Purdue University, USA
Klaus von Gleissenthall University of California, San Diego, USA

Mentoring Workshop Chair

Roopsha Samanta Purdue University, USA

Steering Committee
Kenneth McMillan Microsoft Research, USA
Aarti Gupta Princeton University, USA
Orna Grumberg Technion - Israel Institute of Technology, Israel
Daniel Kroening University of Oxford, UK

Additional Reviewers

Shaull Almagor Antti Hyvarinen

Sepideh Asadi Matteo Marescotti
Angello Astorga Rodrigo Ottoni
Brandon Bohrer Junkil Park
Vincent Cheval Sean Regisford
Javier Esparza David Sanan
Marie Farrell Aritra Sengupta
Grigory Fedyukovich Sadegh Soudjani
Jerome Feret Tim Zakian
James Hamil
 Contents – Part I

AI Verification

NNV: The Neural Network Verification Tool for Deep Neural Networks
and Learning-Enabled Cyber-Physical Systems . . . . . . . . . . . . . . . . . . . . . . 3
Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez,
Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak,
and Taylor T. Johnson

Verification of Deep Convolutional Neural Networks Using ImageStars. . . . . 18

Hoang-Dung Tran, Stanley Bak, Weiming Xiang, and Taylor T. Johnson

An Abstraction-Based Framework for Neural Network Verification . . . . . . . . 43

Yizhak Yisrael Elboher, Justin Gottschlich, and Guy Katz

Improved Geometric Path Enumeration for Verifying ReLU

Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Stanley Bak, Hoang-Dung Tran, Kerianne Hobbs,
and Taylor T. Johnson

Systematic Generation of Diverse Benchmarks for DNN Verification . . . . . . 97

Dong Xu, David Shriver, Matthew B. Dwyer, and Sebastian Elbaum

Formal Analysis and Redesign of a Neural Network-Based Aircraft

Taxiing System with VERIFAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Daniel J. Fremont, Johnathan Chiu, Dragos D. Margineantu,
Denis Osipychev, and Sanjit A. Seshia

Blockchain and Security

The Move Prover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Jingyi Emma Zhong, Kevin Cheang, Shaz Qadeer, Wolfgang Grieskamp,
Sam Blackshear, Junkil Park, Yoni Zohar, Clark Barrett,
and David L. Dill

End-to-End Formal Verification of Ethereum 2.0 Deposit Smart Contract. . . . 151

Daejun Park, Yi Zhang, and Grigore Rosu
xii Contents – Part I

Stratified Abstraction of Access Control Policies. . . . . . . . . . . . . . . . . . . . . 165

John Backes, Ulises Berrueco, Tyler Bray, Daniel Brim, Byron Cook,
Andrew Gacek, Ranjit Jhala, Kasper Luckow, Sean McLaughlin,
Madhav Menon, Daniel Peebles, Ujjwal Pugalia, Neha Rungta,
Cole Schlesinger, Adam Schodde, Anvesh Tanuku, Carsten Varming,
and Deepa Viswanathan

Synthesis of Super-Optimized Smart Contracts Using Max-SMT. . . . . . . . . . 177

Elvira Albert, Pablo Gordillo, Albert Rubio, and Maria A. Schett

Verification of Quantitative Hyperproperties Using Trace

Enumeration Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Shubham Sahai, Pramod Subramanyan, and Rohit Sinha

Validation of Abstract Side-Channel Models for Computer Architectures . . . . 225

Hamed Nemati, Pablo Buiras, Andreas Lindner, Roberto Guanciale,
and Swen Jacobs

Concurrency

Semantics, Specification, and Bounded Verification of Concurrent Libraries

in Replicated Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Kartik Nagar, Prasita Mukherjee, and Suresh Jagannathan

Refinement for Structured Concurrent Programs . . . . . . . . . . . . . . . . . . . . . 275

Bernhard Kragl, Shaz Qadeer, and Thomas A. Henzinger

Parameterized Verification of Systems with Global

Synchronization and Guards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Nouraldin Jaber, Swen Jacobs, Christopher Wagner, Milind Kulkarni,
and Roopsha Samanta

HAMPA: Solver-Aided Recency-Aware Replication . . . . . . . . . . . . . . . . . . . . 324

Xiao Li, Farzin Houshmand, and Mohsen Lesani

Root Causing Linearizability Violations . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

Berk Çirisci, Constantin Enea, Azadeh Farzan,
and Suha Orhun Mutluergil

Symbolic Partial-Order Execution for Testing Multi-Threaded Programs . . . . 376

Daniel Schemmel, Julian Büning, César Rodríguez, David Laprell,
and Klaus Wehrle
 Contents – Part I xiii

Hardware Verification and Decision Procedures

fault: A Python Embedded Domain-Specific Language

for Metaprogramming Portable Hardware Verification Components . . . . . . . . 403
Lenny Truong, Steven Herbst, Rajsekhar Setaluri, Makai Mann,
Ross Daly, Keyi Zhang, Caleb Donovick, Daniel Stanley,
Mark Horowitz, Clark Barrett, and Pat Hanrahan

Nonlinear Craig Interpolant Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

Ting Gan, Bican Xia, Bai Xue, Naijun Zhan, and Liyun Dai

Approximate Counting of Minimal Unsatisfiable Subsets . . . . . . . . . . . . . . . 439

Jaroslav Bendík and Kuldeep S. Meel

Tinted, Detached, and Lazy CNF-XOR Solving and Its Applications

to Counting and Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Mate Soos, Stephan Gocht, and Kuldeep S. Meel

Automated and Scalable Verification of Integer Multipliers . . . . . . . . . . . . . 485

Mertcan Temel, Anna Slobodova, and Warren A. Hunt Jr.

Interpolation-Based Semantic Gate Extraction and Its Applications

to QBF Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508
Friedrich Slivovsky

TARTAR: A Timed Automata Repair Tool . . . . . . . . . . . . . . . . . . . . . . . . . . 529

Martin Kölbl, Stefan Leue, and Thomas Wies

Hybrid and Dynamic Systems

SAW: A Tool for Safety Analysis of Weakly-Hard Systems. . . . . . . . . . . . . 543

Chao Huang, Kai-Chieh Chang, Chung-Wei Lin, and Qi Zhu

PIRK: Scalable Interval Reachability Analysis for High-Dimensional

Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
Alex Devonport, Mahmoud Khaled, Murat Arcak, and Majid Zamani

AEON: Attractor Bifurcation Analysis of Parametrised Boolean Networks . . . 569

Nikola Beneš, Luboš Brim, Jakub Kadlecaj, Samuel Pastva,
and David Šafránek

A Novel Approach for Solving the BMI Problem in Barrier

Certificates Generation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582
Xin Chen, Chao Peng, Wang Lin, Zhengfeng Yang, Yifang Zhang,
and Xuandong Li
xiv Contents – Part I

Reachability Analysis Using Message Passing over Tree Decompositions. . . . 604

Sriram Sankaranarayanan

Fast and Guaranteed Safe Controller Synthesis for Nonlinear

Vehicle Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 629
Chuchu Fan, Kristina Miller, and Sayan Mitra

SeQuaiA: A Scalable Tool for Semi-Quantitative Analysis of Chemical

Reaction Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653
Milan Češka, Calvin Chau, and Jan Křetínský

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667

 Contents – Part II

Model Checking

Automata Tutor v3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Loris D’Antoni, Martin Helfrich, Jan Kretinsky, Emanuel Ramneantu,
and Maximilian Weininger

Seminator 2 Can Complement Generalized Büchi Automata via Improved

Semi-determinization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
František Blahoudek, Alexandre Duret-Lutz, and Jan Strejček

RTLola Cleared for Take-Off: Monitoring Autonomous Aircraft . . . . . . . . . . 28

Jan Baumeister, Bernd Finkbeiner, Sebastian Schirmer,
Maximilian Schwenger, and Christoph Torens

Realizing x-regular Hyperproperties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Bernd Finkbeiner, Christopher Hahn, Jana Hofmann,
and Leander Tentrup

ADAMMC: A Model Checker for Petri Nets with Transits against Flow-LTL . . . 64
Bernd Finkbeiner, Manuel Gieseking, Jesko Hecking-Harbusch,
and Ernst-Rüdiger Olderog

Action-Based Model Checking: Logic, Automata, and Reduction . . . . . . . . . 77

Stephen F. Siegel and Yihao Yan

Global Guidance for Local Generalization in Model Checking . . . . . . . . . . . 101

Hari Govind Vediramana Krishnan, YuTing Chen, Sharon Shoham,
and Arie Gurfinkel

Towards Model Checking Real-World Software-Defined Networks . . . . . . . . 126

Vasileios Klimis, George Parisis, and Bernhard Reus

Software Verification

Code2Inv: A Deep Learning Framework for Program Verification. . . . . . . . . 151

Xujie Si, Aaditya Naik, Hanjun Dai, Mayur Naik, and Le Song

MetaVal: Witness Validation via Verification . . . . . . . . . . . . . . . . . . . . . . . 165

Dirk Beyer and Martin Spiessl

Recursive Data Structures in SPARK. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Claire Dross and Johannes Kanig
xvi Contents – Part II

Ivy: A Multi-modal Verification Tool for Distributed Algorithms . . . . . . . . . 190

Kenneth L. McMillan and Oded Padon

Reasoning over Permissions Regions in Concurrent Separation Logic . . . . . . 203

James Brotherston, Diana Costa, Aquinas Hobor, and John Wickerson

Local Reasoning About the Presence of Bugs: Incorrectness

Separation Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Azalea Raad, Josh Berdine, Hoang-Hai Dang, Derek Dreyer,
Peter O’Hearn, and Jules Villard

Stochastic Systems

Maximum Causal Entropy Specification Inference from Demonstrations . . . . 255

Marcell Vazquez-Chanlatte and Sanjit A. Seshia

Certifying Certainty and Uncertainty in Approximate Membership

Query Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Kiran Gopinathan and Ilya Sergey

Global PAC Bounds for Learning Discrete Time Markov Chains . . . . . . . . . 304
Hugo Bazille, Blaise Genest, Cyrille Jegourel, and Jun Sun

Unbounded-Time Safety Verification of Stochastic Differential Dynamics . . . 327

Shenghua Feng, Mingshuai Chen, Bai Xue, Sriram Sankaranarayanan,
and Naijun Zhan

Widest Paths and Global Propagation in Bounded Value Iteration

for Stochastic Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Kittiphon Phalakarn, Toru Takisaka, Thomas Haas, and Ichiro Hasuo

Checking Qualitative Liveness Properties of Replicated Systems

with Stochastic Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
Michael Blondin, Javier Esparza, Martin Helfrich, Antonín Kučera,
and Philipp J. Meyer

Stochastic Games with Lexicographic Reachability-Safety Objectives . . . . . . 398

Krishnendu Chatterjee, Joost-Pieter Katoen, Maximilian Weininger,
and Tobias Winkler

Qualitative Controller Synthesis for Consumption Markov

Decision Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421
František Blahoudek, Tomáš Brázdil, Petr Novotný, Melkior Ornik,
Pranay Thangeda, and Ufuk Topcu

STMC: Statistical Model Checker with Stratified and Antithetic Sampling . . . 448
Nima Roohi, Yu Wang, Matthew West, Geir E. Dullerud,
and Mahesh Viswanathan
 Contents – Part II xvii

AMYTISS: Parallelized Automated Controller Synthesis for Large-Scale

Stochastic Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
Abolfazl Lavaei, Mahmoud Khaled, Sadegh Soudjani, and Majid Zamani

PRISM-games 3.0: Stochastic Game Verification with Concurrency,

Equilibria and Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
Marta Kwiatkowska, Gethin Norman, David Parker, and Gabriel Santos

Optimistic Value Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488

Arnd Hartmanns and Benjamin Lucien Kaminski

PrIC3: Property Directed Reachability for MDPs. . . . . . . . . . . . . . . . . . . . . 512

Kevin Batz, Sebastian Junges, Benjamin Lucien Kaminski,
Joost-Pieter Katoen, Christoph Matheja, and Philipp Schröer

Synthesis

Good-Enough Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541

Shaull Almagor and Orna Kupferman

Synthesizing JIT Compilers for In-Kernel DSLs . . . . . . . . . . . . . . . . . . . . . 564

Jacob Van Geffen, Luke Nelson, Isil Dillig, Xi Wang, and Emina Torlak

Program Synthesis Using Deduction-Guided Reinforcement Learning . . . . . . 587

Yanju Chen, Chenglong Wang, Osbert Bastani, Isil Dillig, and Yu Feng

Manthan: A Data-Driven Approach for Boolean Function Synthesis . . . . . . . 611

Priyanka Golia, Subhajit Roy, and Kuldeep S. Meel

Decidable Synthesis of Programs with Uninterpreted Functions. . . . . . . . . . . 634

Paul Krogmeier, Umang Mathur, Adithya Murali, P. Madhusudan,
and Mahesh Viswanathan

Must Fault Localization for Program Repair . . . . . . . . . . . . . . . . . . . . . . . . 658

Bat-Chen Rothenberg and Orna Grumberg

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681

AI Verification
 NNV: The Neural Network Verification
Tool for Deep Neural Networks
and Learning-Enabled Cyber-Physical
Systems

Hoang-Dung Tran1,2 , Xiaodong Yang1 , Diego Manzanas Lopez1 ,

Patrick Musau1 , Luan Viet Nguyen3 , Weiming Xiang5 , Stanley Bak4 ,
and Taylor T. Johnson1(B)
1
University of Nebraska, Lincoln, USA
taylor.johnson@vanderbilt.edu
2
Vanderbilt University, Nashville, USA
3
University of Dayton, Dayton, USA
4
Stony Brook University, Stony Brook, USA
5
Augusta University, Augusta, USA

Abstract. This paper presents the Neural Network Verification (NNV)

software tool, a set-based verification framework for deep neural networks
(DNNs) and learning-enabled cyber-physical systems (CPS). The crux
of NNV is a collection of reachability algorithms that make use of a vari-
ety of set representations, such as polyhedra, star sets, zonotopes, and
abstract-domain representations. NNV supports both exact (sound and
complete) and over-approximate (sound) reachability algorithms for ver-
ifying safety and robustness properties of feed-forward neural networks
(FFNNs) with various activation functions. For learning-enabled CPS,
such as closed-loop control systems incorporating neural networks, NNV
provides exact and over-approximate reachability analysis schemes for
linear plant models and FFNN controllers with piecewise-linear activa-
tion functions, such as ReLUs. For similar neural network control systems
(NNCS) that instead have nonlinear plant models, NNV supports over-
approximate analysis by combining the star set analysis used for FFNN
controllers with zonotope-based analysis for nonlinear plant dynamics
building on CORA. We evaluate NNV using two real-world case stud-
ies: the first is safety verification of ACAS Xu networks, and the second
deals with the safety verification of a deep learning-based adaptive cruise
control system.

The material presented in this paper is based upon work supported by the Defense
Advanced Research Projects Agency (DARPA) through contract number FA8750-18-
C-0089, the National Science Foundation (NSF) under grant numbers SHF 1910017 and
FMitF 1918450, and the Air Force Office of Scientific Research (AFOSR) through award
numbers FA9550-18-1-0122 and FA9550-19-1-0288. The U.S. Government is authorized
to reproduce and distribute reprints for Government purposes notwithstanding any
copyright notation thereon. Any opinions, finding, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily reflect the
views of AFOSR, DARPA, or NSF.
c The Author(s) 2020
S. K. Lahiri and C. Wang (Eds.): CAV 2020, LNCS 12224, pp. 3–17, 2020.
https://doi.org/10.1007/978-3-030-53288-8_1
4 H.-D. Tran et al.

Keywords: Neural networks · Machine learning · Cyber-physical

systems · Verification · Autonomy

1 Introduction

Deep neural networks (DNNs) have quickly become one of the most widely used
tools for dealing with complex and challenging problems in numerous domains,
such as image classification [10,16,25], function approximation, and natural lan-
guage translation [11,18]. Recently, DNNs have been used in safety-critical cyber-
physical systems (CPS), such as autonomous vehicles [8,9,52] and air traffic col-
lision avoidance systems [21]. Although utilizing DNNs in safety-critical applica-
tions can demonstrate considerable performance benefits, assuring the safety and
robustness of these systems is challenging because DNNs possess complex non-
linear characteristics. Moreover, it has been demonstrated that their behavior
can be unpredictable due to slight perturbations in their inputs (i.e., adversarial
perturbations) [36].

Fig. 1. An overview of NNV and its major modules and components.

In this paper, we introduce the NNV (Neural Network Verification) tool,

which is a software framework that performs set-based verification for DNNs
and learning-enabled CPS, known colloquially as neural network control systems
(NNCS) as shown in Fig. 21 . NNV provides a set of reachability algorithms that
can compute both the exact and over-approximate reachable sets of DNNs and
NNCSs using a variety of set representations such as polyhedra [40,53–56], star
sets [29,38,39,41], zonotopes [32], and abstract domain representations [33]. The
reachable set obtained from NNV contains all possible states of a DNN from
bounded input sets or of a NNCS from sets of initial states of a plant model.
NNV declares a DNN or a NNCS to be safe if, and only if, their reachable sets do
not violate safety properties (i.e., have a non-empty intersection with any state
satisfying the negation of the safety property). If a safety property is violated,
1
The source code for NNV is publicly available: https://github.com/verivital/nnv/.
A CodeOcean capsule [43] is also available: https://doi.org/10.24433/CO.0221760.
v1.
 NNV: The Neural Network Verification Tool 5

Table 1. Overview of major features available in NNV. Links refer to relevant files/-
classes in the NNV codebase. BN refers to batch normalization layers, FC to fully-
connected layers, AvgPool to average pooling layers, Conv to convolutional layers, and
MaxPool to max pooling layers.

Feature Exact analysis Over-approximate analysis

Components FFNN, CNN, NNCS FFNN, CNN, NNCS
Plant dynamics (for Linear ODE Linear ODE, Nonlinear ODE
NNCS)
Discrete/Continuous Discrete Time Discrete Time, Continuous Time
(for NNCS)
Activation functions ReLU, Satlin ReLU, Satlin, Sigmoid, Tanh
CNN Layers MaxPool, Conv, BN, AvgPool, FC MaxPool, Conv, BN, AvgPool, FC
Reachability methods Star, Polyhedron, ImageStar Star, Zonotope, Abstract-domain, ImageStar
Reachable Yes Yes
set/Flow-pipe
Visualization
Parallel computing Yes Partially supported
Safety verification Yes Yes
Falsification Yes Yes
Robustness Yes Yes
verification (for
FFNN/CNN)
Counterexample Yes Yes
generation

NNV can construct a complete set of counter-examples demonstrating the set

of all possible unsafe initial inputs and states by using the star-based exact
reachability algorithm [38,41]. To speed up computation, NNV uses parallel
computing, as the majority of the reachability algorithms in NNV are more
efficient when executed on multi-core platforms and clusters.
NNV has been successfully applied to safety verification and robustness anal-
ysis of several real-world DNNs, primarily feedforward neural networks (FFNNs)
and convolutional neural networks (CNNs), as well as learning-enabled CPS. To
highlight NNV’s capabilities, we present brief experimental results from two
case studies. The first compares methods for safety verification of the ACAS
Xu networks [21], and the second presents safety verification of a learning-based
adaptive cruise control (ACC) system.

2 Overview and Features

NNV is an object-oriented toolbox written in Matlab, which was chosen in part

due to the prevalence of Matlab/Simulink in the design of CPS. NNV uses the
MPT toolbox [26] for polytope-based reachability analysis and visualization [40],
and makes use of CORA [3] for zonotope-based reachability analysis of nonlinear
plant models [38]. NNV also utilizes the Neural Network Model Transformation
Tool (NNMT) for transforming neural network models from Keras and Tensor-
flow into Matlab using the Open Neural Network Exchange (ONNX) format,
and the Hybrid Systems Model Transformation and Translation tool (HyST) [5]
6 H.-D. Tran et al.

Fig. 2. Architecture of a typical neural network control system (NNCS).

for plant configuration. NNV makes use of YALMIP [27] for some optimization
problems and MatConvNet [46] for some CNN operations.
The NNV toolbox contains two main modules: a computation engine and an
analyzer, shown in Fig. 1. The computation engine module consists of four sub-
components: 1) the FFNN constructor, 2) the NNCS constructor, 3) the reach-
ability solvers, and 4) the evaluator. The FFNN constructor takes a network
configuration file as an input and generates a FFNN object. The NNCS con-
structor takes the FFNN object and the plant configuration, which describes
the dynamics of a system, as inputs and then creates an NNCS object. Depend-
ing on the application, either the FFNN (or NNCS) object will be fed into a
reachability solver to compute the reachable set of the FFNN (or NNCS) from
a given initial set of states. Then, the obtained reachable set will be passed to
the analyzer module. The analyzer module consists of three subcomponents: 1)
a visualizer, 2) a safety checker, and 3) a falsifier. The visualizer can be called to
plot the obtained reachable set. Given a safety specification, the safety checker
can reason about the safety of the FFNN or NNCS with respect to the specifica-
tion. When an exact (sound and complete) reachability solver is used, such as the
star-based solver, the safety checker can return either “safe,” or “unsafe” along
with a set of counterexamples. When an over-approximate (sound) reachability
solver is used, such as the zonotope-based scheme or the approximate star-based
solvers, the safety checker can return either “safe” or “uncertain” (unknown).
In this case, the falsifier automatically calls the evaluator to generate simulation
traces to find a counterexample. If the falsifier can find a counterexample, then
NNV returns unsafe. Otherwise, it returns unknown. Table 1 shows a summary
of the major features of NNV.

3 Set Representations and Reachability Algorithms

NNV implements a set of reachability algorithms for sequential FFNNs and
CNNs, as well as NNCS with FFNN controllers as shown in Fig. 2. The reachable
set of a sequential FFNN is computed layer-by-layer. The output reachable set
of a layer is the input set of the next layer in the network.

3.1 Polyhedron [40]

The polyhedron reachability algorithm computes the exact polyhedron reach-
able set of a FFNN with ReLU activation functions. The exact reachability
 NNV: The Neural Network Verification Tool 7

computation of layer L in a FFNN is done as follows. First, we construct the

affine mapping I¯ of the input polyhedron set I, using the weight matrix W and
the bias vector b, i.e., I¯ = W × I + b. Then, the exact reachable set of the
layer RL is constructed by executing a sequence of stepReLU operations, i.e.,
¯
RL = stepReLUn (stepReLUn−1 (· · · (stepReLU1 (I)))). Since a stepReLU oper-
ation can split a polyhedron into two new polyhedra, the exact reachable set
of a layer in a FFNN is usually a union of polyhedra. The polyhedron reach-
ability algorithm is computationally expensive because computing affine map-
pings with polyhedra is costly. Additionally, when computing the reachable set,
the polyhedron approach extensively uses the expensive conversion between the
H-representation and the V-representation. These are the main drawbacks that
limit the scalability of the polyhedron approach. Despite that, we extend the
polyhedron reachability algorithm for NNCSs with FFNN controllers. However,
the propagation of polyhedra in NNCS may lead to a large degree of conserva-
tiveness in the computed reachable set [38].

3.2 Star Set [38, 41] (code)

The star set is an efficient set representation for simulation-based verification of
large linear systems [6,7,42] where the superposition property of a linear system
can be exploited in the analysis. It has been shown in [41] that the star set is
also suitable for reachability analysis of FFNNs. In contrast to polyhedra, the
affine mapping and intersection with a half space of a star set is more easily com-
puted. NNV implements an enhanced version of the exact and over-approximate
reachability algorithms for FFNNs proposed in [41] by minimizing the number
of LP optimization problems that need to be solved in the computation. The
exact algorithm that makes use of star sets is similar to the polyhedron method
that makes use of stepReLU operations. However, it is much faster and more
scalable than the polyhedron method because of the advantage that star sets
have in affine mapping and intersection. The approximate algorithm obtains an
over-approximation of the exact reachable set by approximating the exact reach-
able set after applying an activation function, e.g., ReLU, Tanh, Sigmoid. We
refer readers to [41] for a detailed discussion of star-set reachability algorithms
for FFNNs.
We note that NNV implements enhanced versions of earlier star-based reach-
ability algorithms [41]. Particularly, we minimize the number of linear program-
ming (LP) optimization problems that must be solved in order to construct the
reachable set of a FFNN by quickly estimating the ranges of all of the states in
the star set using only the ranges of the predicate variables. Additionally, the
extensions of the star reachability algorithms to NNCS with linear plant mod-
els can eliminate the explosion of conservativeness in the polyhedron method
[38,39]. The reason behind this is that in star sets, the relationship between
the plant state variables and the control inputs is preserved in the computation
since they are defined by a unique set of predicate variables. We refer readers to
[38,39] for a detailed discussion of the extensions of the star-based reachability
algorithms for NNCSs with linear/nonlinear plant models.
8 H.-D. Tran et al.

3.3 Zonotope [32] (code)

NNV implements the zonotope reachability algorithms proposed in [32] for

FFNNs. Similar to the over-approximate algorithm using star sets, the zono-
tope algorithm computes an over-approximation of the exact reachable set of a
FFNN. Although the zonotope reachability algorithm is very fast and scalable, it
produces a very conservative reachable set in comparison to the star set method
as shown in [41]. Consequently, zonotope-based reachability algorithms are usu-
ally only more efficient for very small input sets. As an example it can be more
suitable for robustness certification.

3.4 Abstract Domain [33]

NNV implements the abstract domain reachability algorithm proposed in [33]

for FFNNs. NNV’s abstract domain reachability algorithm specifies an abstract
domain as a star set and estimates the over-approximate ranges of the states
based on the ranges of the new introduced predicate variables. We note that
better ranges of the states can be computed by solving LP optimization. How-
ever, better ranges come with more computation time.

3.5 ImageStar Set [37] (code)

NNV recently introduced a new set representation called the ImageStar for use
in the verification of deep convolutional neural networks (CNNs). Briefly, the
ImageStar is a generalization of the star set where the anchor and generator
vectors are replaced by multi-channel images. The ImageStar is efficient in the
analysis of convolutional layers, average pooling layers, and fully connected lay-
ers, whereas max pooling layers and ReLU layers consume most of the com-
putation time. NNV implements exact and over-approximate reachability algo-
rithms using the ImageStar for serial CNNs. In short, using the ImageStar, we
can analyze the robustness under adversarial attacks of the real-world VGG16
and VGG19 deep perception networks [31] that consist of >100 million param-
eters [37].

4 Evaluation

The experiments presented in this section were performed on a desktop with

the following configuration: Intel Core i7-6700 CPU @ 3.4 GHz 8 core Processor,
64 GB Memory, and 64-bit Ubuntu 16.04.3 LTS OS.

4.1 Safety Verification of ACAS Xu Networks

We evaluate NNV in comparison to Reluplex [22], Marabou [23], and ReluVal

[49], by considering the verification of safety property φ3 and φ4 of the ACAS Xu
 NNV: The Neural Network Verification Tool 9

neural networks [21] for all 45 networks.2 All the experiments were done using
4 cores for computation. The results are summarized in Table 2 where (SAT)
denotes the networks are safe, (UNSAT) is unsafe, and (UNK) is unknown.
We note that (UNK) may occur due to the conservativeness of the reachability
analysis scheme. Detailed verification results are presented in the appendix of
the extended version of this paper [44]. For a fast comparison with other tools,
we also tested a subset of the inputs for Property 1–4 on all the 45 networks. We
note that the polyhedron method [40] achieves a timeout on most of networks,
and therefore, we neglect this method in the comparison.
Verification Time. For property φ3 , NNV’s exact-star method is about 20.7×
faster than Reluplex, 14.2× faster than Marabou, 81.6× faster than Marabou-
DnC (i.e., divide and conquer method). The approximate star method is 547×
faster than Reluplex, 374× faster than Marabou, 2151× faster than Marabou-
DnC, and 8× faster than ReluVal. For property φ4 , NNV’s exact-star method
is 25.3× faster than Reluplex, 18.0× faster than Marabou, 53.4× faster than
Marabou-DnC, while the approximate star method is 625× faster than Reluplex,
445× faster than Marabou, 1321× faster than Marabou-DnC.

Table 2. Verification results of ACAS Xu networks.

ACAS XU φ3 SAT UNSAT UNK TIMEOUT TIME(s)

1 h 2 h 10 h
Reluplex 3 42 0 2 0 0 28454
Marabou 3 42 0 1 0 0 19466
Marabou DnC 3 42 0 3 3 1 111880
ReluVal 3 42 0 0 0 0 416
Zonotope 0 2 43 0 0 0 3
Abstract Domain 0 0 45 0 0 0 8
NNV Exact Star 3 42 0 0 0 0 1371
NNV Appr. Star 0 29 16 0 0 0 52
ACAS XU φ4
Reluplex 3 42 0 0 0 0 11880
Marabou 3 42 0 0 0 0 8470
Marabou DnC 3 42 0 2 2 0 25110
ReluVal 3 42 0 0 0 0 27
Zonotope 0 1 44 0 0 0 5
Abstract Domain 0 0 45 0 0 0 7
NNV Exact Star 3 42 0 0 0 0 470
NNV Appr. Star 0 32 13 0 0 0 19

2
We omit properties φ1 and φ2 for space and due to their long runtimes, but they
can be reproduced in the artifact.
10 H.-D. Tran et al.

Conservativeness. The approximate star method is much less conservative

than the zonotope and abstract domain methods. This is illustrated since it can
verify more networks than the zonotope and abstract domain methods, and is
because it obtains a tighter over-approximate reachable set. For property φ3 ,
the zonotope and abstract domain methods can prove safety of 2/45 networks,
(4.44%) and 0/45 networks, (0%) respectively, while NNV’s approximate star
method can prove safety of 29/45 networks, (64.4%). For property φ4 , the zono-
tope and abstract domain method can prove safety of 1/45 networks, (2.22%)
and 0/45 networks, (0.00%) respectively while the approximate star method can
prove safety of 32/45, (71.11%).

4.2 Safety Verification of Adaptive Cruise Control System

To illustrate how NNV can be used to verify/falsify safety properties of learning-
enabled CPS, we analyze a learning-based ACC system [1,38], in which the ego
(following) vehicle has a radar sensor to measure the distance to the lead vehicle
in the same lane, Drel , as well as the relative velocity of the lead vehicle, Vrel .
The ego vehicle has two control modes. In speed control mode, it travels at a
driver-specified set speed Vset = 30, and in spacing control mode, it maintains
a safe distance from the lead vehicle, Dsaf e . We train a neural network with 5
layers of 20 neurons per layer with ReLU activation functions to control the ego
vehicle using a control period of 0.1 s.
We investigate safety of the learning-based ACC system with two types of
plant dynamics: 1) a discrete linear plant, and 2) a nonlinear continuous plant
governed by the following differential equations:
ẋlead (t) = vlead (t), v̇lead (t) = γlead , γ̇lead (t) = −2γlead (t) + 2alead − μvlead
2
(t),
ẋego (t) = vego (t), v̇ego (t) = γego , γ̇ego (t) = −2γego (t) + 2aego − μvego
2
(t),
where xlead (xego ), vlead (vego ) and γlead (γego ) are the position, velocity and accel-
eration of the lead (ego) vehicle respectively. alead (aego ) is the acceleration con-
trol input applied to the lead (ego) vehicle, and μ = 0.0001 is a friction param-
eter. To obtain a discrete linear model of the plant, we let μ = 0 and discretize
the corresponding linear continuous model using a zero-order hold on the inputs
with a sample time of 0.1 s (i.e., the control period).
Verification Problem. The scenario we are interested in is when the two vehi-
cles are operating at a safe distance between them and the ego vehicle is in
speed control mode. In this state the lead vehicle driver suddenly decelerates
with alead = −5 to reduce the speed. We want to verify if the neural network
controller on the ego vehicle will decelerate to maintain a safe distance between
the two vehicles. To guarantee safety, we require that Drel = xlead − xego ≥
Dsaf e = Ddef ault + Tgap × vego where Tgap = 1.4 s and Ddef ault = 10. Our anal-
ysis investigates whether the safety requirement holds during the 5 s after the
lead vehicle decelerates. We consider safety of the system under the following
initial conditions: xlead (0) ∈ [90, 92], vlead (0) ∈ [20, 30], γlead (0) = γego (0) = 0,
vego (0) ∈ [30, 30.5], and xego ∈ [30, 31].
 NNV: The Neural Network Verification Tool 11

Table 3. Verification results for ACC system with different plant models, where V T
is the verification time (in seconds).

v lead(0) Linear plant Nonlinear plant

Saf ety V T (s) Saf ety V T (s)
[29, 30] SAFE 9.60 UNSAFE 346.62
[28, 29] SAFE 9.45 UNSAFE 277.50
[27, 28] SAFE 9.82 UNSAFE 289.70
[26, 27] UNSAFE 17.80 UNSAFE 315.60
[25, 26] UNSAFE 19.24 UNSAFE 305.56
[24, 25] UNSAFE 18.12 UNSAFE 372.00

Verification Results. For linear dynamics, NNV can compute both the exact
and over-approximate reachable sets of the ACC system in bounded time steps,
while for nonlinear dynamics, NNV constructs an over-approximation of the
reachable sets. The verification results for linear and nonlinear models using the
over-approximate star method are presented in Table 3, which shows that safety
of the ACC system depends on the initial velocity of the lead vehicle. When
the initial velocity of the lead vehicle is smaller than 27 (m/s), the ACC system
with the discrete plant model is unsafe. Using the exact star method, NNV can
construct a complete set of counter-example inputs. When the over-approximate
star method is used, if there is a potential safety violation, NNV simulates the
system with 1000 random inputs from the input set to find counter examples. If
a counterexample is found, the system is UNSAFE, otherwise, NNV returns a
safety result of UNKNOWN. Figure 3 visualizes the reachable sets of the relative
distance Drel between two vehicles versus the required safe distance Dsaf e over
time for two cases of initial velocities of the lead vehicle: vlead (0) ∈ [29, 30] and
vlead (0) ∈ [24, 25]. We can see that in the first case, Dref ≥ Dsaf e for all 50
time steps stating that the system is safe. In the second case, Dref < Dsaf e in
some control steps, so the system is unsafe. NNV supports a reachLive method
to perform analysis and reachable set visualization on-the-fly to help the user
observe the behavior of the system during verification.
The verification results for the ACC system with the nonlinear model are
all UNSAFE, which is surprising. Since the neural network controller of the
ACC system was trained with the linear model, it works quite well for the linear
model. However, when a small friction term is added to the linear model to form a
nonlinear model, the neural network controller’s performance, in terms of safety,
is significantly reduced. This problem raises an important issue in training neural
network controllers using simulation data, and these schemes may not work in
real systems since there is always a mismatch between the plant model in the
simulation engine and the real system.
Verification Times. As shown in Table 3, the approximate analysis of the ACC
system with discrete linear plant model is fast and can be done in 84 s. NNV
12 H.-D. Tran et al.

Actual Distance (blue) vs. Safe Distance (red)

80

60

Distance
40

20
0 5 10 15 20 25 30 35 40 45 50
Control Time Steps
Actual Distance (blue) vs. Safe Distance (red)
60
Distance

50

40

30
0 5 10 15 20 25 30 35 40 45 50
Control Time Steps

Fig. 3. Two scenarios of the ACC system. In the first (top) scenario (vlead (0) ∈
[29, 30] m/s), safety is guaranteed, Drel ≥ Dsaf e . In the second scenario (bottom)
(vlead (0) ∈ [24, 25] m/s), safety is violated since Dref < Dsaf e in some control steps.

also supports exact analysis, but is computationally expensive as it constructs

all reachable states. Because there are splits in the reachable sets of the neu-
ral network controller, the number of star sets in the reachable set of the plant
increases quickly over time [38]. In contrast, the over-approximate method com-
putes the interval hull of all reachable sets at each time step, and maintains a
single reachable set of the plant throughout the computation. This makes the
over-approximate method faster than the exact method. In terms of plant mod-
els, the nonlinear model requires more computation time than the linear one. As
shown in Table 3, the verification for the linear model using the over-approximate
method is 22.7× faster on average than of the nonlinear model.

5 Related Work
NNV was inspired by recent work in the emerging fields of neural network and
machine learning verification. For the “open-loop” verification problem (verifica-
tion of DNNs), many efficient techniques have been proposed, such as SMT-based
methods [22,23,30], mixed-integer linear programming methods [14,24,28], set-
based methods [4,17,32,33,48,50,53,57], and optimization methods [51,58]. For
the “closed-loop” verification problem (NCCS verification), we note that the
Verisig approach [20] is efficient for NNCS with nonlinear plants and with Sig-
moid and Tanh activation functions. Additionally, the recent regressive polyno-
mial rule inference approach [34] is efficient for safety verification of NNCS with
nonlinear plant models and ReLU activation functions. The satisfiability mod-
ulo convex (SMC) approach [35] is also promising for NNCS with discrete linear
Another random document with
no related content on Scribd:
Or look at the painting of another vault on the opposite page. This is
more stiff than the former, because it was executed nearly a century
later; still, there is nothing to declare its Christian character until the
eye rests on the Good Shepherd, who appears below the principal
part of the decoration.

Painting on Vault of an Arcosolium in Cemetery of Prætextatus.

We are not saying that the artists who executed these paintings
had no Christian meaning in them; on the contrary, we believe that
they had, and that the paintings really suggested that meaning to
those who first saw them. For we know, on the authority of Tertullian,
that “the whole revolving order of the seasons” (which are
represented in the second painting) was considered by Christians to
be “a witness of the resurrection of the dead.” This, therefore, was
probably the reason why they were painted here; and no Christian
needs to be reminded that our Lord spoke of Himself under the
image of a vine, which sufficiently explains the first painting. Still the
fact remains that the representations themselves are such as might
have been used by Christian and by Pagan artists indifferently. If any
of our readers feel disappointed that the first essays of the Christian
painter should not have had a more distinctly Christian character,
they must remember that a new art cannot be created in a moment.
If the Christian religion in its infancy was to make use of art at all, it
had no choice but to appropriate to its own purposes the forms of
ancient art, so far as they were pure and innocent; by degrees it
would proceed to eliminate what was unmeaning, and substitute
something Christian.
Some writers have supposed that Christians used at first Pagan
subjects as well as Pagan forms of ornamentation; and they point to
the figure of Orpheus, which appears in three or four places of the
Catacombs, and to that of Psyche also, which may be seen about as
often. So insignificant a number of exceptions, however, would
scarcely suffice to establish the general proposition, even if they
were in themselves inexplicable. But, in truth, the figure of Orpheus
has no right to be considered an exception at all, for he was taken by
some of the early Fathers as a type of our Lord; and it was even
believed by some of them, that, like the sybil, he had prophesied
about Him. Clement of Alexandria calls our Lord the Divine
enchanter of souls, with evident reference to the tale of Orpheus;
and the same idea will have occurred to every classical scholar, as
often as he has heard those words of the Psalmist which speak of
the wicked as “refusing to hear the voice of the charmer, charm he
never so wisely.” When, then, we find Orpheus and his lyre, and the
beasts enchanted by his song, figured on the walls or roofs of the
Catacombs, we have a right to conclude that the artist intended a
Christian interpretation to be given to his work; and a similar
explanation may be given of any other subjects of heathen
mythology which have gained admittance there.
If we were asked to name the subject which seems to have been
used most frequently in the early decorations of the Catacombs, we
should give the palm to the Good Shepherd; nor is this preference to
be wondered at. Any one who has meditated upon the words in
which our Blessed Lord took this title to Himself, will easily
understand why the first Christians, living in the midst of heathen
persecutors, should have delighted to keep so touching an image
always before them. They scratched it, therefore, roughly on the
tombstone as they laid some dear one in the grave; they carved it on
their cups, especially on the sacred chalice; they engraved it on
signet rings and wore it on their fingers; they placed it in the centre of
the paintings with which they covered the ceiling of their
subterranean chapels, or they gave it the chief place immediately
over the altar. We meet with it everywhere, and everybody can
recognise it.
There are, however, one or two peculiarities in its mode of
treatment which require a word of explanation. The shepherd is
generally represented as a young man lightly clad, with his tunic girt
high about his loins, denoting thereby his unwearied activity; he is
surrounded by sheep, or he carries one on his shoulders, bearing it
home to the fold,—the most tender act of his office. And there is
nothing in this but what we might naturally have expected. But he is
also sometimes represented with a goat instead of a sheep upon his
shoulders; and, in later paintings, he has the pastoral reed or tuneful
pipe either hanging on the tree by his side or he is playing on it. Now
this last particular has no place in the gospel parable, and the former
seems directly opposed to it, since the goat is the accepted symbol
of the wicked, the sheep only of the good. Hence these points have
been taken up by some critics, either as tokens of thoughtless
carelessness on the part of the Christian artists, or as proofs that
their work, whether consciously or unconsciously, was merely copied
from some Pagan original. Neither of these remarks appears to be
just. The images of a shepherd in Pagan art, with scarcely a single
exception, are of a very different kind; and the particular details
objected to are not only capable of receiving a Christian
interpretation, they even express consoling Christian truths. St.
Gregory Nazianzen speaks of the anxious care of the shepherd as
he sits on the hillside, filling the air with the soft notes of his pipe,
calling together his scattered flock; and he observes that in like
manner the spiritual pastor, desirous to recall souls to God, should
follow the example of his Divine Master, and use his pipe more
frequently than his staff. Then, as to the substitution of the goat for
the sheep, it was probably intended as a distinct protest against the
un-Christian severity of those heretics, who in very early times
refused reconciliation to certain classes of penitent sinners.
Not many, however, of the most ancient Christian paintings are of
the same simple and obvious character as the Good Shepherd. The
leading feature which characterises most of them is this, that they
suggest religious ideas or doctrines under the guise of artistic
symbols or historic types. One doctrine specially prominent in them,
and most appropriately taught in cemeteries, is that of the
resurrection and the everlasting life of happiness which awaits the
souls of the just after death. It is in this sense that we must
understand not only the frequent repetitions of the stories of Jonas
and of Lazarus—the type and the example of a resurrection—but
also of Daniel in the lions’ den, and the three children in the fiery
furnace. These last, indeed, very probably had reference also to the
persecution which the Christians were then suffering, and were
intended to inspire courage and a confident expectation that God
would deliver them, even as He had delivered His chosen servants
of old; but, as they are spoken of in very ancient Christian
documents (e.g., in the hymns of St. Ephrem and in the Apostolic
Constitutions) as foreshadowing the future triumph of the body over
death, whence these too had been in a manner delivered, we prefer,
in obedience to these ancient guides, to assign this interpretation to
them; at any rate, it is certain that this interpretation cannot be
excluded. Figures also of the deceased, with arms outstretched in
prayer, sometimes accompanied by their names, or standing in the
midst of a garden, or, again, figures of birds pecking at fruits and
flowers, we understand as images of the soul still living after death,
received into the garden of Paradise, and fed by immortal fruits.
Sometimes there may be a difference of opinion perhaps as to the
correctness of this or that interpretation suggested for any particular
symbolical painting; but the soundness of the principle of
interpretation in itself cannot be called in question, nor will there
often be any serious difficulty in its application, among those who
study the subject with diligence and candour. The language, both of
Holy Scripture and of the earliest Fathers, abounds in symbols, and
it was only natural that the earliest specimens of Christian art should
exhibit the same characteristic. More was meant by them than that
which met at first the outward senses; without this clue to their
meaning, the paintings are scarcely intelligible,—with it, all is plain
and easy.

Tombstone from the very ancient Crypt of St. Lucina, now united with the
Catacomb of St. Callixtus.

Take, for example, the figure of an anchor, so repeatedly

represented on gravestones and other monuments of the
Catacombs; so rarely, if indeed ever, to be found on Pagan
monuments. What influenced the early Christians in the selection of
such a figure? what meaning did they attach to it? This enquiry
forces itself upon our minds, if we are intelligent students of Christian
archæology, anxious to understand what we see: and if we are also
prudent and on our guard against being led astray by mere fancy, we
shall conduct the enquiry by the same laws and principles as we
should apply to the interpretation of some perplexing riddle in
heathen art. We should first examine the literature of the age and
people to whom it was supposed to belong, and see if any light could
be thrown upon it from that source. In the present instance,
therefore, we turn to the sacred literature of the Christians, and we
find there a passage which speaks of the duty of “holding fast the
hope that is set before us, which hope we have as an anchor of the
soul, sure and firm.” We assume, then, provisionally, as a basis of
further enquiry, that an anchor may perhaps have been used as an
emblem of Christian hope. Continuing our search in the same sacred
books, we find that there was a special connection in the Christian
creed between hope and the condition of the dead. It is written that
Christians are not sorrowful about those who die, “as others who
have no hope.” The conclusion is obvious, that a reference to hope
is just one of those things which might not unreasonably be looked
for on a Christian’s grave-stone, since it was something on which
they prided themselves as a point of difference between themselves
and others. This greatly confirms our conjectural interpretation of the
symbol, and we proceed with some confidence to apply it to every
example of its use that we can meet with; for if it is the right key, it
cannot fail to unlock all the problems that will come before us. In
doing this, we are first struck by the fact that in several instances the
very names of the deceased persons on whose epitaphs the anchor
is engraved, themselves also meant the same thing. They were
called Spes, Elpis, Elpidius, Elpizusa; all names coming from the
Latin or Greek word for hope. Next, we observe that many of these
anchors are so fashioned as to contain a hidden yet unmistakable
representation of a cross; and, reflecting that the one only ground of
a Christian’s hope is the cross of Christ, we hail this also as lending
further support to our theory. Yet once more, we find many of the
epitaphs contain the same idea, expressed in distinct words written
in the ordinary alphabet and not in these hieroglyphics, so to call
them,—we find Spes in Deo, Spes in Deo Christo &c. Finally, we
often find the anchor united with one or more of several other
symbols, to which, by a similar but independent process, we can
assign a certain signification. We try, then, whether our rendering of
the anchor as equivalent to “hope” will make sense, as a schoolboy
would say who was trying to translate a piece of Greek or Latin into
English, in all these other places; and if it does, we are satisfied that
our interpretation can be no longer disputed. A false reading of a
single symbol might chance to fit one monument, or two, or three;
but to say that any false reading will fit hundreds of separate
monuments, fit all equally well, and succeed in extracting a
consistent meaning from each, is to assert what no sane man can
believe.
Those who know the way in which the interpretation of the
Egyptian hieroglyphics was first guessed at, and then triumphantly
established against all gainsayers, by a similar process of reasoning,
will not dispute the soundness of the argument by which the
meaning of the anchor has been arrived at. We cannot attempt to
vindicate our interpretation of all the other symbols used by Christian
artists with the same minuteness of detail, neither is it necessary. All
will accept the dove as a fitting symbol of the simplicity, the
gentleness, purity, and innocence of a Christian soul gone to its rest,
and a sheep as fitly representing a disciple of Christ.
Another emblem, the fish, requires more words of explanation,
because it is capable of receiving a double meaning. At first sight,
our thoughts at once recur to the words of our Blessed Lord to St.
Peter and his brother, “Follow me, and I will make you fishers of
men,” and no doubt this will sufficiently explain many old Christian
paintings or sculptures in which the fish appears. Taking this idea for
our guide, we can understand why a man angling and catching a fish
should find a place on the walls of a church, whether above ground
or below. Such a representation in these sacred places was inspired
by the same doctrinal teaching, and suggested the same ideas, as
were present to the old Christian preachers when they spoke of men
being caught by the bait of charity and the hook of preaching, and
being drawn out of the bitter waters of this world, not to have their life
taken from them, which is the fate that awaits the natural fish when it
is caught, but that they may be made partakers of a new and
heavenly life. This, however, will not enable us to decypher other
symbolical paintings into which the fish enters, and which are found
with equal frequency among the decorations of the Christian
cemeteries. It is necessary that we should learn another, and, as it
would seem, a still more common use of the fish. Just as the dove
might stand for the Holy Ghost, and also for a soul sanctified by the
Holy Ghost—just as the lamb or sheep might stand either for the
Lamb of God, or for those who are “the people of His pasture and
the sheep of His hand”—so the fish, too, was used not only to
represent a Christian, but also, still more frequently perhaps, Christ
Himself. To understand how this could be, we must study a little
Greek, which may be easily apprehended, however, even by those
who are not scholars, if they will fix their attention for a few moments
on the accompanying plan:—

Ι ΗϹΟΥϹ = Jesus
Χ ΡΙϹΤΟϹ = Christ
Θ ΕΟΥ = of God
Υ ΙΟϹ = Son
Ϲ ΩΤΗΡ = Saviour

The Greek for fish is here written perpendicularly, one letter above
another, ΙΧΘΥϹ; and it is seen that these five letters are the initial
letters of five words, which, together, contain a tolerably complete
account of what Christ is. He is Jesus Christ, Son of God,
Saviour. Thus, this one word, ιχθυς, or fish, read in this way, tells a
great deal about our Lord’s name and titles; it is almost a miniature
creed, or, as one of the Fathers expresses it, “it contains in one
name a whole multitude of holy names.” It would take us too long to
enquire into the origin of this device for expressing our Lord’s name
and titles in so compendious and secret a form. Clearly, whoever
may have invented it, it was very ingenious, and specially convenient
at those times and places where men dared not speak of Him freely
and openly. We cannot say when it began, but it was in universal use
throughout the Church during the first three hundred years of her life,
and then, when she was in the enjoyment of peace and liberty, it
gradually dropped, first out of sight in Christian monuments, and
then out of mind also in Christian literature. But, during the ages of
persecution, it had sunk deep into the habits of Christian thought and
language; it became, as it were, a part of the very Catechism,—
every baptized Christian seems to have been familiar with it, whether
he lived on the banks of the Tiber or of the Po, of the Loire, of the
Euphrates, or of the Nile. In all these parts of the world, writers in
books, poets in hymns, preachers in sermons, artists in painting, the
very masons themselves on gravestones, made use of it without a
word of explanation, in a way that would utterly mystify any modern
Christian community. Who would now dream of carving or painting a
fish upon a gravestone in a Christian churchyard? yet scores of
graves in the Catacombs were so marked, and some of them with
hardly a word or an emblem upon them besides. Or what meaning
could we attach to the picture of a dove or a lamb standing on a
fish’s back, if we did not understand that the fish represented Christ,
and the dove or the lamb a Christian, so that the whole symbol stood
for a Christian soul supported by Christ through the waves and
storms of life? Or again, only imagine a Christian in these days
having buried with him, or wearing round his neck during life, a little
figure of a fish cut in ivory, or crystal, or mother of pearl, or some still
more costly material? Yet a number of those who were buried in the
Catacombs did this; and some of these fish even bear an inscription,
calling upon the fish to be a Saviour!
It was necessary to give this explanation of certain symbols, and
to justify it by sufficient examples, before we proceed to study any of
the more complex paintings in the Catacombs. But now, with these
thoughts in our minds, let us enter the Cemetery of St. Callixtus, and
look on a figure represented two or three times on a wall of one of its
most ancient chambers: a fish swimming and carrying on its back a
basket of bread, and in the midst of the loaves of bread, a glass
vessel containing a red liquid. What is this but bread and wine, the
elements of the Sacrament of Love, and Jesus Christ Its reality? St.
Jerome, when speaking of a holy bishop of Toulouse who had sold
the gold and silver vessels of his church to relieve the poor, uses
these words, “What can be more rich than a man who carries the
body of Christ in a basket of wicker-work, and the blood of Christ in a
vessel of glass?” Here are undeniably the basket of wicker-work and
the vessel of glass; and who can doubt that we have the other also,
veiled under the figure of the fish?
 Consecration of the Holy Eucharist.

Let us go to another part of the same cemetery, and consider a

painting which with some variations is repeated in three or four
successive chambers, all opening out of one of the primitive
galleries. Bread and fish lie on a three-legged table, and several
baskets of bread are arranged along the floor in front of it, or a man
and woman stand by the side of the table. The woman has her arms
outstretched in the form of a cross, the ancient attitude of Christian
prayer; the man, too, is stretching forth his hands, but in another
way: he holds them forward, and especially his right hand, over the
bread and fish, in such a way as to press upon every Catholic
intelligence the idea that he is blessing or consecrating what is
before him. To modern eyes, indeed, his vestment does not look
worthy of one engaged in the highest act of Christian worship;
perhaps, at first sight, it almost strikes us as hardly decent.
Nevertheless, to the Christian archæologist, this very vestment is a
strong confirmation of the view we are taking of the real sense of the
painting. For it is the Greek pallium, or philosophers’ cloak; and we
know that at the time to which this painting belongs (the end of the
second or beginning of the third century) it was a common practice
to preach the Word of God in this particular costume. Tertullian, who
was living at the same time, wrote a treatise De Pallio, in which, in
his own peculiar style, he defended its use, and congratulated the
pallium on its promotion to be a Christian vestment. It was not until
fifty years later that St. Cyprian objected to it, both as not sufficiently
modest in itself and as vainglorious in its signification.
If there were any lingering uncertainty as to whether these figures
were really intended to have reference to the Holy Eucharist, or
whether our interpretation of them may not have been fanciful and
arbitrary, an examination of the other decorations of the same
chambers will suffice to remove it. For it will be seen that, whilst in
closest connection with them are other suitable emblems or figures
of the same Divine Sacrament, they are also uniformly preceded by
representations of the initiatory Sacrament of the Christian covenant,
without which no man can be admitted to partake of the Eucharist;
and they are followed by a figure of the Resurrection, which our Lord
Himself most emphatically connected with the eating of His flesh and
the drinking of His blood, saying, “He that eateth my flesh and
drinketh my blood, hath everlasting life, and I will raise him up at the
last day.” These three subjects, Baptism, the Holy Eucharist, and the
Resurrection occupy the three perfect sides of the chamber, the
fourth side being, of course, broken by the entrance; and, taken in
their right order, they faithfully depict the new life of a Christian; the
life of divine grace, first imparted by baptism, then fed by the Holy
Eucharist, and finally exchanged for an everlasting life of glory.

The Smitten Rock.

 Let us look at the figures of these subjects in detail, and see how
they are represented here. First, we have Moses striking the rock, a
scene which occurs over and over again in the Catacombs, and
which in these chambers commences the series of paintings we are
examining; it is to be seen on the left-hand wall as we enter. St. Paul
tells us that “the rock was Christ;” the water, then, which flowed from
it must be those streams of Divine grace whereby His disciples are
refreshed and sustained during their pilgrimage through the
wilderness of this world, and this grace is first given in the waters of
baptism. Next we have a man fishing, which has been already
explained; and (in one instance at least) this is followed by another
man performing the very act of baptism on a youth who stands
before him; the youth stands in the water, and the man is pouring
water over his head. Lastly, on the same wall, is the paralytic
carrying his bed on his shoulders—the same, doubtless, who was
miraculously cured at the pool of Bethsaida, which pool the fathers of
the Church uniformly interpret as typical of the healing waters of the
Christian sacrament.

The Sacrament of Baptism.

 Eucharistic Feast.

On the wall opposite the doorway, the central scene is a feast

wherein seven men are seated at a table, partaking of fish and
bread; and there is a history in the last chapter of St. John’s Gospel,
of which it may be taken as a literal representation. It was when our
Lord “showed Himself to His disciples at the Sea of Tiberias, and He
showed Himself, after this manner. There were together Simon Peter
and Thomas who is called Didymus, and Nathanael who was of
Cana of Galilee, and the sons of Zebedee, and two others of His
disciples”—seven in all. “And they went a-fishing, but caught nothing.
Jesus appeared to them on the shore.” Then there follows the
miraculous draught of fishes; and as soon as they came to land, they
saw “hot coals lying, and a fish laid thereon, and bread. Jesus saith
to them, Bring hither of the fishes which you have now caught. And
Jesus cometh and taketh bread and giveth them, and fish in like
manner.” Such is the letter of the gospel narrative; but this narrative
is in fact a mystical and prophetic representation of the Church
gathered together out of the waters of the world, and fed by the Holy
Eucharist. The hundred and fifty-three great fishes that were caught
represent the large numbers of the faithful that were drawn into the
Church by apostolic preaching; the fish laid on the hot coals is Jesus
Christ in His Passion, His Body “delivered for us” on Mount Calvary,
given to us also to be our food in the Blessed Sacrament whereby
“we show the death of the Lord until He come.” The faithful caught in
the net of the Church must be brought to that broiled fish (Piscis
assus, Christus passus, says St. Augustine), that crucified Lord, and
they must be incorporated with Him by partaking of the living Bread
which came down from Heaven.

Sacrifice of Isaac.

Resurrection of Lazarus.
 Such is the full meaning of the scene at the Sea of Tiberias, as
interpreted according to the unanimous consent of the Fathers; and
the adjuncts of this picture show that it was intended to be so
understood here also; for on one side is the figure of the
consecration already described; and on the other, the sacrifice of
Isaac by his father, which was surely a most lively type of the
sacrifice of Christ upon the altar; wherein blood is not really shed,
but the Lamb is only “as it were slain,” just as Isaac was not really
slain, but was received back from the dead, “for a parable.” Lastly, as
has been mentioned before, there follows on the third wall of the
same chamber the natural complement of the rest; the doctrine of
the Resurrection, as contained in the fact of the rising again of
Lazarus. Thus, this whole series of paintings, executed at the end of
the second century, or within the first twenty or thirty years of the
third, and repeated (as has been said) in several successive
chambers, was a continual homily, as it were, set before the eyes of
the faithful, in which they were reminded of the beginning, progress,
and consummation of their new and supernatural life.
We do not say that every modern Christian who looks at these
paintings will thus read their meaning at once; but we believe that all
ancient Christians did so, because it is clear from the writings of the
Apostles themselves and their successors, that nothing was more
familiar to the Christian mind of those days than the symbolical and
prophetical meaning of the facts both of the Old and of the New
Testaments. They believed the facts themselves to have taken place
just as they are recorded, but they believed also that they had a
mysterious signification, whereby the truths of the Christian faith
were insinuated or expressed, and that this was their highest and
truest meaning. “Perhaps there is no one recorded miracle of our
Lord,” says St. Gregory, “which is not therefore selected for
recording because it was the type of something to happen in the
Church;” and precisely the same was felt to be true also of the
histories of the patriarchal and Jewish dispensations. “All these
things had happened to them in figure, and they were written for our
correction, upon whom the ends of the world are come.”
 It may not be often possible to trace as clearly as we have just
done in a single instance, the logical order and dependence of the
several subjects that were selected for representation in each
chamber of the Catacombs; they may not always have been so
admirably arranged as to be in fact equivalent, as these were, to a
well-ordered dogmatic discourse. Nevertheless it is only when read
in this way, that the decoration of the Catacombs can be made
thoroughly intelligible; and it is certain that some such meaning must
have been intended from the first. The extremely limited number of
Biblical subjects selected for representation, while such an immense
variety is really contained in the Bible (and so many of those that are
neglected might have seemed equally suitable for the purpose), and
then again, the thoroughly unhistorical way in which these few
subjects are dealt with, shows clearly that the principle of selection
was theological rather than artistic. The artists were not left to
indulge their own unfettered fancy, but worked under ecclesiastical
supervision; and the Bible stories which they depicted were not
represented according to their historical verity, because they were
not intended to be a souvenir of past facts, but to symbolise and
suggest something beyond themselves. In order, therefore, to
understand them, it is necessary to bring them face to face with the
Christian doctrines which they foreshadow.
 Noe in the Ark.

Look, for example, at the numerous pictures of Noe in the ark

which appear in the Catacombs, all resembling one another, but
none resembling the reality. Instead of a vessel, three stories high,
containing eight human beings and specimens of every kind of
animal, we see only a narrow box, barely large enough to hold one
person, and that person sometimes a lady, whose name is also
inscribed upon it perhaps, being the same lady (as we learn from the
inscription) who lies buried in the adjacent tomb. If all ancient
Christian literature had perished, we should have been at a loss to
comprehend this enigma; but as soon as we know that the Fathers
of the Church speak of it as an acknowledged fact, which “nobody
doubts” (to use St. Augustine’s words), that the Church was typified
by the ark, a ray of light begins to dawn upon us; and when we call
to mind that St. Peter himself speaks of the waters of baptism as
saving men’s souls, “even as Noe and his family were saved by the
waters of the flood,” all is at once made clear. We see plainly that the
friends of the deceased have intended to signify that he had been
received into the ark of the Church and made a Christian by baptism.
And if they had added to the composition, as they often did, the
figure of a dove bringing an olive branch to the person standing in
the ark, this also enters into the same interpretation; it was
symbolical of that Divine peace which comes to the soul in this world
by faith, and which is a pledge of the peace given by everlasting
happiness in the next.

Scenes from the History of Jonas.

The frequent repetition of the story of Jonas in a Christian

cemetery needs no explanation, our Lord himself having put it
forward as a type of His own resurrection, and so a pledge of ours
also. The particular form, however, under which this story appears,
was not suggested, as Noe’s ark was, by the place which it held in
the cycle of Christian doctrine, but rather by a certain Pagan model
with which the Romans of that day were very familiar. The
mythological tale of Andromeda, and the sea-monster to which she
was exposed on the coast near Joppa (for so the story ran), was a
favourite subject for the decoration of the walls in Roman villas,
temples, and other public buildings. It may be seen in Pompeii, and,
much nearer to the Catacombs, in Rome itself—e.g., in the barracks
of one of the cohorts of the imperial police, discovered a few years
ago in Trastevere; and in both places the monster is the precise
counterpart of that which is always represented as swallowing or
casting up Jonas; a kind of dragon, with large head and ears, a long
slender neck, and a very tortuous body. Of course, in the infancy of
Christian art, it was convenient to have a model at hand to represent
an unknown monster, and, as we have said, we do not doubt that
this is the true history of its origin. Still this was not the only reason
which recommended the adoption of so grotesque a form; it offered
the further advantage of creating as strong a contrast as possible
between this “great fish,” which was a type of death, and the ordinary
fish, which, as we have seen, was the recognised symbol of the
Author of life.
Another incident in the life of Jonas, which was often painted in the
Catacombs, was his resting on the east side of the city of Nineve,
under the shade of a certain plant which God caused to grow up for
his protection, and which He again caused as suddenly to wither
away. In the days of St. Jerome and St. Augustine there was a
dispute between those learned doctors as to the precise nature of
this plant; and in the course of it St. Jerome appealed to these
paintings as bearing testimony in favour of his own rendering of the
Hebrew word. We need not enter into the merits of the dispute, but it
is important to note the fact of the appeal, as it peremptorily refutes
the ridiculous assertions of certain authors of the present day, who
would assign very recent dates to these and similar paintings in the
Catacombs. We know that St. Jerome was very fond, when a boy, of
visiting these places, and it is interesting to hear him appealing to the
paintings he had seen in them as to “ancient witnesses.” It would be
still more interesting, if we could say with certainty what were the
motives which led the ancient Christians to choose this subject for
such frequent contemplation; whether they read in it only a very
striking lesson as to the watchfulness of Divine Providence, or
whether it had a more subtle meaning, as a type of the mercy of God
which overshadows the souls of the faithful in the long sleep of death
which goes before the Sun of the Resurrection. But where no clue is
supplied by the writings of cotemporary, or nearly cotemporary
authors, we prefer to keep silence rather than to insist on any
doubtful interpretation. All that need be said is that such a painting
was certainly not out of place in a Christian Church or cemetery, any
more than the story of Adam and Eve, or any other Biblical narrative
which has reference to the doctrines or promises announced by
Christianity to the world.
We do not pretend to enumerate here all the subjects from the Old
and New Testaments that were painted in the Catacombs. We are
but naming those that were used most frequently, that seem most
interesting, or whose signification can be most precisely determined.
Those who have seen the Catacombs themselves will call to mind
others of which we have not spoken, but we think their meaning is
generally obvious so as to need no explanation. We will name one
class only of these paintings; those in which our Lord and His
Blessed Mother appear. Our readers will hardly expect to find
anything that pretends to be a portrait of either one or the other. We
have seen that the disposition in primitive Christian art was to
represent facts rather than persons, and the mystery which the facts
signified rather than the facts themselves. Christ, therefore, appears
most commonly in the typical character of the Good Shepherd, and
as such is represented in appropriate form and with suitable
accessories, or He sits in the midst of His Apostles, with a chest of
volumes at His feet, as the Great Teacher of the world. Once,
indeed, His head and bust form a medallion occupying the centre of
a roof in a chamber of the Cemetery of St. Domitilla, the same in
which appear Orpheus and his lyre. It is a work of the third century;
there is more evidence of an intention to give a definite individual
type of countenance, neither is the type altogether unlike that which
the practice of later ages has consecrated by traditional usage.
Nevertheless others of the fourth century are evidently not copies of
the same model, so that it is clear that in those early days there was
no uniform agreement upon the subject.