44.
Strategies for Mainstream Usage of Formal Verification
Raj S. Mitra
Texas Instruments, Bangalore
rsm@ti.com
Abstract
Formal verification technology has advanced significantly in
recent years, yet it seems to have no noticeable acceptance as a
mainstream verification methodology within the industry. This
paper discusses the issues involved with deploying formal
verification on a production mode, and the strategies that may
need to be adopted to make this deployment successful. It
analyses the real benefits and risks of using formal verification in
the overall verification process, and how to integrate this new
technology with traditional technologies like simulation. The
lessons described in this paper have been learnt from several years
of experience with using commercial formal verification tools in
industrial projects.
Categories and Subject Descriptors: B.6.3
[Hardware]: Logic Design - Design Aids – Verification.
General Terms: Verification, Experimentation.
Keywords: Formal verification, Emerging technologies.
1. Introduction
At the last Design Automation Conference, Jan Rabaey made a
comment on formal verification [1], mentioning the possibility
that someday IEEE will publish a comic strip on adventures in
formal verification. His reasoning was that the word “adventure”
implies an image of “pretty much ad-hoc and in the exploratory
space” and there is no a production process yet, and hence the
analogy with the state of affairs of formal verification (FV) today.
This comment is not isolated; most users of FV tools would
testify to the “adventurous” nature of FV usage in industrial
projects (not on toy designs), and the frustrations it causes. This is
in spite of the recent advances seen in this technology and
numerous research papers published in conferences every year.
There have been pockets of acceptance – some components of
some products (processors and controllers) have utilized FV – but
by and large, a general acceptance has been lacking.
What is first needed today is an unbiased root-cause analysis of
this systemic failure – why are engineers reluctant to use this
promising technology, or whether this technology is usable at all.
Secondly, becoming aware of the limitations, in technology and
management, the requirement is to define a feasible strategy for
mainstream adoption of formal verification.
This paper addresses these two topics. It does not propose any
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
DAC 2008, June 9-13, 2008, Anaheim, California, USA
Copyright 2008 ACM 978-1-60558-115-6/08/0006 …5.00
800
new algorithms or abstraction techniques for FV. Instead, the
main question asked (and answered to some extent) here is: What
does it take to move FV from a fringe technology to a mainstream
usage (i.e. non-experimental planned production usage, with
predictable outcomes)? Throughout this paper, we are referring to
the usage of commercial production quality tools (not university
prototypes) – today these are typically in the areas of Model
Checking and Sequential Equivalence Checking. We are also not
referring to Combinational Equivalence Checking, which is
widely used in the industry now. In this paper, we have used the
acronym FV to generically refer to both “Formal Assertion
Verification” as well as to “Sequential Equivalence Checking”,
FAV to specifically refer to the former, SEC to refer to the latter,
and ABS to refer to “Assertion based Simulation”. Although,
strictly speaking, ABS is not a FV technique, it is the precursor to
FV and hence needs to be treated in the same context.
We start this paper with a brief summary of the current
verification process, and then enumerate the main challenges in
adopting FV in this context. Then we analyze the advantages
(returns on investment) we can expect from FV, and subsequently
also discuss that technical advantages are not the sole criteria that
determine the success of adoption. Finally, based on the previous
discussions, we suggest strategies for making the usage of FV
more widespread in the industry than what it is today.
The strategies suggested in this article are based on lessons
learnt during the application of FV in our organization, over the
last five years and over multiple projects, with several best-in-
class commercial FV tools. We have cited several lessons taken
from research in diffusion of innovation, but these have been
actually put into practice in the context of FV deployments in our
organization.
2. Current Verification Process
To set the context for this paper, we begin with a short
introduction to the verification process – as it is practiced today.
Most of this discussion will not be new to the readers, but it will
help in appreciating the problems with and the value of adopting
FV in a mainstream mode. We will deal only with functional
verification, since only that is relevant for formal verification.
A SOC is created by hooking up several IPs. The functionality
of each IP is verified separately, and then the SOC / subsystem
hookup is verified too (its reset conditions, performance
characteristics, etc). The individual IPs are verified partly by the
designers, who run a few directed smoke tests and filter out the
most obvious bugs, and mostly by the verification engineers, who
do an elaborate job of developing testbenches and eliminating all
kinds of bugs from the DUT (design under test) through directed,
random, and directed-random tests. Although a fine balance is
usually worked out between the two teams for different parts of
the verification, finally the responsibility of finding the bugs in
the implemented system is shouldered by the verification engineer
alone.
 Stop simulation now?
Items found faulty
Detected Bugs
at final audit
Coverage or
Simulation Time
(a)
Fig 1: (a) Quality-Time Continuum of Verification by Simulation; (b) A Typical Path of Frustration
The simulation verification process is a traversal of the Quality-
Time Continuum (Fig 1-a), whose principal value is a monotonic
increase in quality, and not the achievement of an absolute level
of quality. The final result is usually a trade-off between the two
parameters – quality (measured by coverage data, bug trends, etc)
and the project schedule. Verification is suspended when the
curve reaches the desired levels of coverage metrics, or when the
clock runs out and it is time to tape out. We say suspended, and
not stopped, because there is no such thing as a target of achieving
“complete” verification through random and directed-random
simulation, due to the very nature of simulations, and hence there
is really no end to the simulation process.
This simulation process, essentially random and incomplete, has
a strong theoretical foundation in the statistical testing principles
expounded by Deming [2]. A testcase is nothing but a sample of
the different paths in the module, which is inspected for
conformance with the specification. Hence, valuable lessons from
past work on product quality measurement and improvement may
be applied to the field of verification also. We discuss some of
them below.
The plateau / saturating nature of the curve in Figure 1-a can be
seen as an extension of Figure 1-b (which is a variation of Figure
33 of [2]) from the context of product manufacturing – the first set
of results comes from the removal of “special” (i.e. specific)
causes, and then the system reaches “statistical stability” and the
curve plateaus to a level. Subsequent improvements can come
only from addressing the “common” (i.e. process) causes and
reducing the system’s variation – i.e. (in this context) by
improving the verification process through root-cause analysis of
the gaps in verification. Without getting into the details of
Deming’s theory, the important analogies to be drawn from his
work are the following:
1. Definition of quality depends on the agent (user or worker),
and may change with time. However, quality can be defined
operationally in terms of some indices that can be measured
(here, several coverage metrics – functional as well as
structural). An operational definition puts communicable
meaning into a concept, and has to be agreed upon in advance.
The indices should measure the quality of the product (DUT)
as well as of the procedure of measurement (its verification).
2. Inspection, automatic (e.g. simulation) or manual (code
reviews), simply does not find all the defectives, especially
Quality improves by removal of special causes
(detecting and fixing individual causes of defects)
Stabilized (i.e., within
Control Limits)
Continued improvement is
expected but will not happen
Time
(b)
when they are rare. Hence, failure in execution must be
defined in advance, in terms of measurable metrics.
3. Distribution and trends in the measurement indices indicate
the health of the system. Control limits on the indices indicate
whether the system is in stable statistical control or not, and
once it is observed to be stable, no further improvement is
possible by quick fixes; improvements to the processes are
now necessary to reduce the variation of the system.
4. The indices are intended to show whether the system is stable,
they should never be used to indicate specific defects. In
verification, code coverage data should never be used to make
specific code fixes; they should only be used to indicate that
some broad areas of the code are yet unverified.
5. Statistical stability of the process of measurement is also vital
– e.g. will we get the same results if we change the simulator
or the engineers? It is also important to give the measurement
instrument a good context (environment) to do its work, e.g.
measuring an ageing fluid is better done at its source. An
example of this in the simulation context is that assertions
inserted by designers help improve the quality of verification.
In summary, the simulation process, random and inherently
incomplete, is very similar to the statistical inspection techniques.
The latter recommends inspection, not to remove all the product
defectives, but its metrics to be used as indicators for identifying
areas of process improvement – in recruitment, training, ensuring
input quality, design, and supervision. But this is not the purpose
for which simulation is used today; its main purpose is to detect
bugs, and it is not a surprise that it falls far short of expectation,
especially in the context of today’s large and complex SOCs.
This recognition of the gap in verification for large SOCs,
which was further triggered by the attention attracted by the
Pentium bug – in both the academia and industry, has urged the
demand for FV in recent years. But it quickly became apparent
that FV tools are a discontinuity from current simulation practices
– they cannot be used in the same context and by the same users
as of simulation, and hence their acceptance did not really take off
in the verification industry.
3. Problems with FV Usage
FV introduces a shift (discontinuities) from the current
verification practices in the following areas:
801
1. While assertions can also be used with simulations, the styles
of coding differ significantly between the assertions written
for FAV as compared to the assertions written for ABS. It is
necessary to write FAV-friendly assertions in monitor style [3]
that are optimized for minimizing the introduction of extra
state elements. In contrast, assertions and code written for
ABS typically do not pay any heed for extra state elements
being added, and this is the coding practice that the
verification engineers are currently accustomed to.
2. FV is applicable to small modules only, due to capacity
limitations of current technology. However, breaking up the
DUT into smaller parts is increasingly being done in the
simulation context also, because it is getting increasingly
difficult to feed in relevant inputs, to activate selected parts of
the code, from a far away module boundary. This requires that
we break up the DUT in smaller logical parts (structurally or
temporally), add constraints to manage this separation, and
verify the parts separately – and the same process applies for
both simulation and FV. But the problem gets dramatically
acute for FV, where the sizes of the individual modules are
much smaller than what can be handled by simulation.
3. The capacity problem, compounded with the many innovative
(and tool-dependent) abstraction techniques available to solve
it, manifests in a more serious problem – the lack of
predictability of the verification process. Given a DUT, there
is no metric (based on number of gates, number of state
elements, or structure of the design, etc) which can predict
before starting verification, whether we will finally get some
answer or end up with a capacity bottleneck. As a result,
unplanned manual abstractions are usually added during the
project cycle, thereby breaking the projected verification
schedule. This is quite unlike the simulation context, where
we know that we will get “some” results on “any” DUT, and
can make a reasonable experience-based schedule prediction.
4. The partitioning of modules (to handle capacity limitations)
usually introduces artificial boundaries, and lead to the writing
of constraints on these boundaries. Until these constraints
have been refined very perfectly, FV catches any and every
hole in these constraints and reports them as failures. These
are called “false failures” because they are not design bugs but
environment modeling errors, and they have to be fixed before
the real design bugs can be caught. In the initial stages of
applying FV, the number of reported false failures can be
significant, and a lot of time is spent in refining the constraints
for eliminating them. (This item is not critical by itself, but
manifests in the next two items.)
5. FV requires a very close interaction with the design team, to
(a) write assertions for elements inside the module (to
overcome the capacity problem), and (b) help in writing and
refining constraints (to eliminate false failures). The latter part
is very critical, because the very close interaction it demands
with the design team makes it a show stopper occasionally.
6. The writing of efficient properties and constraints, and
intensively micro-managing their iterations in a very
structured way [4], is the only way to “squeeze” results from
FV tools on medium sized modules. But this practice is quite
alien to the current simulation engineers, who are more used
to writing large testbench software. Some organizations have
solved this problem by maintaining a central team of FV
experts, whose services are available for any project that
802
requires the application of FV. These FV experts are
intensively trained in the writing of properties efficiently, and
in the usage of FV tools, and they are able to share FV
expertise and insights (and also code – reusable assertions and
glue logic) across multiple projects. Most of the reported
success stories in FV, i.e. the application of FV to solve
complex verification problems which have defied a simulation
solution, have been brought about through such central teams
of FV experts. But this model of application is not scalable,
and also brings management problems with it – the central
team is often considered alien by the project team, culturally
and by organizational hierarchy. Forming teamwork among
these two teams is a management challenge, and requires extra
initiatives (e.g. appreciations, incentives, and interventions).
7. There is no metric for FV to show its progress – it is either
completely done (pass or fail) or not done (run out of capacity,
or bounded proofs). (In some restricted situations bounded
proofs are useful, e.g. a 5 cycle bounded proof for a processor
whose pipeline latency is 4, can be construed as a full proof.)
Hence, other metrics are used for FV, to indicate under-
specification (not enough properties written) or over-
constraints, but these do not correlate with simulation metrics.
Some of the above discontinuities in verification practice are
related to the usage of the new technology (e.g. writing efficient
assertions and constraints, using abstractions to overcome
capacity limitations, etc), and these can be overcome, to a large
extent, by trainings – in the theory of FV (to increase general
awareness of FV technology), the methodology to be followed,
and the usage of commercial tools and their specific engines.
The other category of discontinuities, which disrupt the flow of
work, are considered more severe by the design and verification
engineers, and are the real hurdles in the path to mainstream usage
of FV. These are the items: (3) low predictability of the FV
process, (5) the requirement for having a very close integration
with the design team, (6) difficulty of managing a central team,
and (7) lack of metrics to show progress of verification. These
process discontinuities, resulting from technology discontinuities,
are the real barriers to the mainstream usage of FV.
4. Expected Benefits from FV
After the analysis of the problems with adopting FV, let us also
analyze the benefits that we can expect to get from applying FV.
Then we can compare the gain against the pain, and decide if
there is a ROI (return on investment) on adopting FV on a larger
scale. Many frustrations occur on the usage of FV because most
people are not quite clear on what to expect from FV.
To the enthusiast and the un-experienced, FV promises the goal
of “complete” verification. But, because of the capacity
limitations of current tools and the subsequent heavy usage of
constraints to narrow down the scope of FV runs, it is effectively
rendered as incomplete as simulation. Some would even advocate
FV to be more incomplete than simulation, because in simulation
we at least have a set way to make progress when coverage goals
are not achieved. Hence, completeness of verification is certainly
not the goal of FV, for the verification of medium to large sized
circuits. (However, for very small circuits, i.e. with less than 100
state elements, this can still be considered a realizable goal.)
But before answering the question of FV’s ROI, we need to first
define the verification targets we are seeking to achieve.

ABS
Normal
FAV Simulations
Detected Bugs
Verification Time
Fig 2: Bug Detection Rates
Understanding that both simulation and FV are incomplete (one
inherently, the other effectively so), the main question applicable
to both technologies is – when should we stop the verification?
Today’s verification signoff criteria are set by simulation cover-
age metrics that we are well accustomed to, and our verification
goal is to meet (or converge towards) them as fast as possible.
This leads to what we think is the key to solving the ROI
question of FV – what we need is a quicker detection of bugs, and
not the detection of all bugs. And this is where FV does help,
because in the initial stages of the design cycle it (and also ABS)
catches bugs much faster than traditional simulations do. Fig 2
shows this relative advantage. It is a variation of the curve shown
in [5], but with a curve for FAV also. Simulations start slightly
later than FAV, because there is a time associated with creating
testbenches for simulations, whereas the writing and validation of
assertions can start along with the RTL development. The initial
bug detection rate is also higher for FAV, because it is not
dependent on creating testcases to hit the relevant situations.
However, the FV curve terminates sooner than the simulations
due to capacity limitations. Further progress, to catch deep corner
case bugs with FV, comes with extreme efforts (as discussed
earlier). A similar benefit is also observed while using SEC [6] –
designers can quickly apply SEC to validate recent changes
against their previous golden model, and thus reduce costly
iterations between the design and verification teams.
This raises a related question – who should be the FV user: the
designer or the verification engineer? So far, we have been talking
only about the verification engineer, but if we take a holistic view
of system process improvement (as in [2]), then it is evident that
the system’s variation can be reduced only by creating high
quality designs in the first place, and that indicates that the
designers should be the primary users of FV. Hence, we have to
consider three very different use scenarios for FV (in sequence):
Designer: The designer writes white-box assertions at the IP
level, and proves them (or, applies SEC to prove iterative
correctness) before handing over the IP to the verification team.
The automated usage (described below) may be applied by the
designers too.
Automated: Here, the design or verification team mostly relies
on the usage of (a) plug-in assertion IPs (plug-and-play pre-built
assertion packages of standard protocols), and (b) pre-defined
checks within the FV tool (e.g. for state machine reachability
checks, dead code checks, etc), to catch as many bugs as early as
possible. It is quite likely that some of these checks will force the
FV tool into its capacity limitation, but these checks will just need
to be run later through simulations again.
803
This automated usage is also applicable at the SOC level, for
the verification of chip-level connectivities [7], e.g. IP-
connectivity, pin-muxing, boundary pads, DFT connectivity, etc.
Today, the common practice is to use simulations for this purpose,
but to activate the different paths requires a huge amount of
testcases and their setup, where all that is required is to validate
that the connections of the IP blocks have been done properly. At
first glance, FV is not applicable at the chip level. But if all the
IPs of the chip are black-boxed, then all that remains are the
connections to be verified (which have a minimal number of state
elements), and FV can be easily applied there. The generation of
the assertions from the specification of the chip connectivity can
be automated, thereby reducing human effort further, and the
whole process can be mostly reduced to a push-button exercise.
Another automated application is a bug-hunting mode, with the
use of a semi-formal technology [8]. After completing ABS or
FAV to the extent feasible, the DUT can be thrown open to the
semi-formal tool, and the tool be run for days or weeks to find
more bugs. Of course, there has to be a measure of progress of
this verification, giving a sense of confidence that the tool is
increasing its coverage and is not looping in a local region.
Deep: The verification team writes end-to-end assertions, and
attempts to catch deep corner case bugs (e.g. the detection of
hard-to-locate silicon bugs, or performance bugs in a subsystem).
The skills required for this effort are exceptional and have to be
nurtured with care within the organization, through the
development of small central teams.
Of these different application scenarios, the automated
techniques have the highest chance of becoming mainstream
usage, followed by the designer usage (because that requires a
change in mindset), and finally the deep and difficult usage. It is
likely that the many verification engineers are not going to
become FV experts to detect corner case bugs.
5. It is not only ROI
On the hard and long path to converting simulation users to FV,
a typical response that one faces, after FV has caught a bug that
simulations have not caught, is “So what? Simulations would have
caught it in a few more days”. Are there really any bugs that FV
can catch but simulations cannot? Technically, no; it is just a
matter of time before the right set of vectors is simulated. The
trick is to find all the right set of vectors as quickly as possible,
and that is what ROI of FV is all about. But in practice, it is
extremely hard to convince a simulation team with this argument.
Most discussions on why FV adoption is not picking up in the
industry focus on the ROI of FV. However, a comparison with the
adoption rates of other new technologies [9] shows that most
innovations diffuse at a disappointingly slow rate, at least in the
eyes of the inventors and technologists who have created them.
Reasons for slow adoption vary: availability of other competing
remedies, the change agent not being a prominent figure,
compatibility with existing solution, and vested interests.
In general, the problem of FV adoption is very similar to the
problem with adoption of preventive innovations – a class of
innovations that have demonstrated a particularly slow rate of
adoption. A preventive innovation is a new idea that an individual
adopts now in order to lower the probability of some unwanted
future event, and examples in this category are the adoption of
seat belts in cars, the usage of contraceptives, etc. Not only are the
rewards of adoption delayed in time, but it cannot be proven
(beyond some statistical data, for which there are as many
proponents as opponents) as to whether it is actually essential. FV
seems to fall in this category – it excels in finding corner bugs, but
then they are rare anyway and chances are that they will not be
detected in the silicon if they are not caught by simulation.
The knowledge-attitude-practice gap (KAP-gap, [9] page 176)
in the diffusion of preventive innovations can sometimes be
closed by a cue-to-action – an event occurring at a time that
crystallizes a knowledge into an action. Some cues-to-action
occur naturally – the above article reports that many people begin
to use contraceptives after they experience a pregnancy scare. In
other cases, a cue-to-action may be created by a change agency –
some national family planning programs pay incentives to create
potential adopters. Similarly, we have witnessed that in the
projects where a late corner case bug caused a delay in a chip’s
schedule or caused a respin, FV’s advantages are acknowledged
and the team is more willing to adopt FV in future projects. But
the same “shock” may have to be witnessed by other teams before
they themselves start using FV as aggressively.
6. Strategies for FV Deployment
The above discussions on ROI of FV and the non-ROI problems
of adoption lead to the following strategies that can be considered
for effective mainstream usage of FV. They are presented in
approximately the sequence in which they should be introduced to
the project teams, so that resistance faced is kept to the minimum.
1. Encourage the integration of assertions into simulation,
through ABS. This will enable engineers (both verification
engineers and designers) to start the habit of writing assertions
in their code, and using them without changing the current
work flows. These assertions may not be coded efficiently
enough for the application of FAV, but at the least this will
remove the first level of resistance to the paradigm shift.
2. Use automatically generated assertions to integrate FAV into
current simulation flows, to get quick coverage reports on
dead code, state machine reachability, etc. This will introduce
the teams to the usage of FAV tools in their simplest use
models, and will encourage more usage and experimentation.
3. Besides writing assertions for simulation, also make available
pre-packaged plug-and-play assertion packages for standard
interfaces and protocols. This will not involve any significant
departure from the simulation flows and, will also be the
introduction of running the FAV tools to verify assertions.
4. Automatically generate assertions on chip-level connectivities,
through flow automations. This will also not require the
verification teams to write assertions, but to use them to
reduce verification cycle time.
5. Along with the adoption of the easy steps, it would be wise to
create and sustain a central team of FV experts in the
organization, to help with critical verification problems. This
is not mainstream usage in the sense that the scope of
application is limited to a small set of people, but the impact
to the organization is significant. But, as mentioned earlier,
this requires a different management practice.
6. Recognizing that simulation engineers will probably never
adopt the rigors of writing efficient assertions and applying
FV in full detail, at this time consider shifting the target base
from verification to design. Ask designers to write white box
804
assertions, and run the FAV and SEC tools themselves.
Besides creating a set of white box assertions which can be
used in ABS later on, this will also create a new paradigm
shift in the work flow – shift the onus of module level
verification to the designers. In turn, this paradigm shift will
make designers aware of the verification problem, thus
enabling the development of design for verification techniques
and other new methodologies for verification. This process
improvement may have the highest impact on silicon quality
(a la Deming [2]) – which is the real goal of verification (not
coverage metrics).
7. The last step is a catch-all condition – after trying other
approaches, also try the semi-formal techniques. There are no
guarantees (at least with today’s tools) for finding all
remaining bugs or for proving the absence of bugs, but it is at
least worth a try in critical situations.
The above strategies have been summarized in Table 1, where
effort and impact (on project schedule) are marked qualitatively as
Low, Medium and High. The steps with Low effort can be easily
made mainstream activities; the step with High effort is less likely
to see mainstream usage. Note that these strategies do not require
the advancement of technology beyond what is currently available
today (e.g. the integration of FV and simulation through unified
coverage metrics may create a far more mainstream adoption of
FV, but that technology is not available today).
To accelerate the above steps for adoption, additional
management practices may be used, taken from the lessons learnt
from the diffusion of other innovations [9]. We have witnessed
the validity and usefulness of these lessons in our experience of
FV adoption. These include (but are not limited to) the following:
8. Opinion leaders of a social system (and a semiconductor
organization is such a system) are influential people who
have earned social accessibility based on their technical
competence (not necessarily in the same domain as that of
the new innovation) and who are recognized to conform to
the system’s norms. They can lead in the spread of new
ideas, or they can head an active opposition. Hence, it is very
vital to identify such organizational opinion leaders, and
garner their support for the spread of FV usage.
9. Diffusion is essentially a process of communication, and
channels are necessary for communication. Sharing of
knowledge (e.g. what works well and what does not work
well for FV) is very critical for educating the people on the
latest developments and advances in this area, and these have
to be customized for the specific organization. Sharing of
best practices in a common website (as in [10], and ensuring
that it is not biased towards any specific tool vendor) will
help to spread the knowledge across multiple teams.
10. Generally, the fastest rate of adoption of innovations stems
from authority decisions (depending, of course, on how
innovative the authorities are). If management is convinced
of the need for attempting FV and making progress through
experimentation, it can plan to reduce the number of
simulation engineers in a project and replace them with FV
engineers in the same project. Thus the overall verification
resource bandwidth remains the same but a part of it gets
allocated for FV. This can help in getting early results in the
context of specific projects and thereby make a case for more
proactive usage of FV in the subsequent projects. But this is
 to be handled delicately – early victories through
management fiat sometimes turn into subsequent failure.
11. An innovation diffuses more rapidly if it can be customized
to specific needs of the social system. Integration into the
flows used in the organization, such that some segments of
usage become mostly push-button (as in the checks for SOC
connectivity) or the identification of clear rules for applying
certain classes of checks (as for protocol checking for the
dominant protocols used in the organization), will help in
getting more users quickly starting to use the FV tool.
12. Technology clusters, consisting of one or more
distinguishable elements of technology that are perceived as
being closely interrelated, are usually adopted more rapidly
together. FV is a collection of technologies (FAV, SEC,
semi-formal technologies, formal coverage analysis, etc) and
together they have a higher chance of adoption than by
treating each as a separate entity. We have witnessed that
those who are interested in FAV are also the ones who are
willing to experiment with SEC – which is a relatively new
technology. Hence, a collective approach should be taken for
encouraging their adoption.
7. Conclusion
In this paper, we have presented lessons learnt, based on our
experience, as a dozen strategies for FV deployment. We have
deliberately avoided citing any specific project example, because
data from one example can be quite misleading without
understanding its full context (completeness of the specification,
IP delivery procedure, verification expertise of team, FV expertise
& training undertaken, etc). But we have attempted to keep this
paper enumerative and objective. (See [4], for detailed
methodologies and examples of using FV.)
Taking a long view of the matter, the core problems with the
adoption of FV are: (1) the naïve understanding that it yields a
complete verification, and (2) the word “verification” in the name.
FV seems like an adventure sport (difficult and unpredictable),
and many people give up on it due to this, only because the
expectations of ROI are wrong. Understanding the real ROI of FV
allows proper usage of FV, and also opens up opportunities for the
development of advanced tools in this segment. It is also
necessary to understand the scope of simulations and know what
to expect from it. Hence we have spent considerable effort in this
paper to draw analogies with statistical testing principles, which
have a strong theoretical foundation and a successful history.
Secondly, we must realize that FV is not really an ideal task for
verification engineers, given the expertise and scope of today’s
verification engineers. Also, applying FV in isolation (i.e. post-
design) creates many of the capacity barriers artificially. What is
needed is more usage of FV by the designers, and at a higher level
of abstraction than RTL. This paradigm shift has been happening
(albeit slowly) in several industries during recent years.
But that is just the beginning of process improvements. The
growing complexity of SOCs (including hardware-software
interactions) may soon overcome the technical advances in
805
hardware verification. What we really need is a dramatic
improvement of our processes, to be able to create correct-by-
construction designs in the first place, to verify the specification
vs the intent (not just implementation vs written specification),
and reduce inspection only to be used for indicating that we are on
the right track. Formal methods of specification, modeling,
analysis and design are necessary for this – not just “formal
verification”. What this paper really establishes is that FV is a set
of technologies for quality improvement and, understanding the
true meaning of Quality, process improvement is the way towards
that goal – not just through inspections. These processes include
tools, flows, and people.
References
[1] J Rabaey, “Design without borders: A tribute to the legacy of
A Richard Newton”, Keynote at DAC, June 2007.
[2] W E Deming, “Out of the crisis”, MIT Press, 1982.
[3] K Shimizu, et al, “A specification methodology by a collection
of compact properties as applied to the Intel Itanium processor
bus protocol”, CHARME, 2001, Scotland.
[4] A Jain, et al, “Formal Assertion based Verification in
Industrial Setting”, Tutorial at DAC 2007, Foils available at
http://www.facweb.iitkgp.ernet.in/~pallab/formalpub.html
[5] H Foster, “Unifying traditional and formal verification
through property specification”, Designing Correct Circuits
(DCC), April 2002, Grenoble.
[6] A Mathur, V Krishnaswamy, “Design for verification in
system level models and RTL”, DAC, June 2007.
[7] S Roy, “Top Level SOC Interconnectivity Verification using
Formal Techniques”, Microprocessor Test and Verification
Workshop, Austin, December 2007
[8] Ho Pei-Hsin, et.al, “Smart simulation using collaborative
formal and simulation engines” , ICCAD, November 2000.
[9] E M Rogers, “Diffusion of Innovations”, 5th edition, Free
Press, New York, 2003.
[10] “Formal verification patterns”,
http://www.oskitech.com/wiki/index.php?title=Main_Page
Description of Step Effort Impact
1 Assertions in simulations (ABS) M M
2 Auto-generated assertions to augment L M
simulations (deadcode, reachability)
3 Pre-packaged assertion IPs L H
4 Auto-generated assertions for SOC L H
connectivity
5 Deep FV, with central team H H
6 Designers using FV (FAV, SEC) M H
7 Semi-formal bug-hunting* L L
L = Low, M = Medium, H = High
* Low effort is after applying ABS and / or FV. Low impact is
because there is no metric of progress.
Table 1: Strategies for FV Usage