WHITE PAPER

Accelerating SoC Verification Closure with

Unified Verification Management Solution

Authors Introduction
Kirankumar Karanam Functional verification of system-on-chip (SoC) designs requires best-in-class tools linked
Manager, Applications together in a unified solution in order to address exponential complexity challenges. There is
Engineering, Synopsys
no one-size-fits-all method for verification. Complex designs require a combination of virtual
prototyping, static checks, formal analysis, simulation, emulation and FPGA prototyping. The
Anika Malhotra
execution of all the tools in the solution must be managed to minimize project time, engineering
Senior Product Marketing
Manager, Synopsys effort and compute resources. Metrics must be gathered and aggregated in a unified way to
assess verification status and determine when to tape out.

There are four major phases in SoC verification: planning, execution, analysis and closure. As
shown in Figure 1, each phase is the primary focus at a different stage of the project. However,
the process is iterative since analysis and closure may lead to additional execution runs or
revisions to the verification plan. Closing the coverage and plan gaps consumes the most
resources throughout the process. To aid in this, tracking and integration with other tools are
two additional processes that span all four phases. This white paper discusses the challenges
and requirements for the phases and presents the Synopsys unified solution.

Missing Coverage Points Plan

(In spec, but not in verification plan) Gap
Coverage Goal
Coverage
Gap

Closure
Coverage %

Analysis

Execution

Planning Project Time 25% 100%

Tracking

Integration

Figure 1: The four phases of SoC verification

synopsys.com
Planning Phase
The entire SoC verification process begins with the creation of the verification plan. At one time this was a paper document listing
the design features and the simulation tests to be written for each feature to verify its proper functionality. Today, engineers list the
coverage targets for each feature in the design specification and link them to the tests intended to hit the targets. These targets
may exist in the testbench or in the design itself, and may include code coverage, functional coverage and assertions. Some
coverage targets might be hit by formal analysis, emulation or other verification engines rather than simulation. There are several key
requirements for this approach to be effective:

• The features in the plan must be linked directly to content (text, figure, table, etc.) in the design specification
• Specification reviews must be used to reveal any gaps in the verification plan
• The plan must be kept in in sync with the specification
• Coverage results must be aggregated intelligently and back annotated into the verification plan for a unified view of achieved
verification progress

The Synopsys verification management solution meets these requirements, and more. This unified solution includes Synopsys
Verification Planner, Verdi® Automated Debug System and VC Execution Manager. As shown in Figure 2, Verification Planner links
coverage metrics, the executable verification plan and the design specification together, keeping everything in sync as the design
evolves over the course of the project. Users can easily create the plan from the specification and link the plan to coverage, ensuring
that the entire specification is mapped to measurable verification results. Simulation, emulation, static and formal coverage results
from the Synopsys Verification Continuum are back annotated to the plan, thereby linking to the specification as well.

Coverage Verification plan Specification

Figure 2: Verification plan association with coverage targets and design specification

Execution Phase
A modern SoC requires tens of thousands of simulation and formal runs to achieve the coverage goals. Executing these tests must
be highly automated to minimize turnaround time (TAT) and to use cloud or grid compute resources, storage space and tool licenses
efficiently. This requires an execution management tool that:

• Launches jobs for all verification engines, managing jobs and resources
• Monitors job progress and tracks results
• Analyzes test results and mines data to guide the next steps

2
VC Execution Manager drives verification efficiency with automated and integrated regression testing. It manages tests for all
Synopsys verification tools, automatically collecting pass/fail and coverage results and saving them in a persistent regression
database. The runs are fully automated, while allowing users many options for control and customization. VC Execution Manager
reruns failing tests with debug options turned on, and fully integrates with Verification Planner and the Verdi graphical user interface
(GUI) for viewing and analyzing results and debugging failures. It helps users move into the analysis phase with failure bucketing and
root cause analysis. Figure 3 shows the combined flow for the solution.

Verification Planner Browser GUI

VC Execution Manager

Regression
Verdi database Project tools

SQL views

Custom reports
Regression engine Verification files

vdb
vdb vdb Excl
Tests vdb Plan Spec
vdb file

Figure 3: Synopsys verification management flow

Since the primary requirement in this phase is efficient resource usage and low TAT, ordering of tests is critical. Similar regression
test suites are run many times over the course of the project, and the information from each run can be used to improve the next run.
Among its many other capabilities, the solution shortens regression runs and reduces TAT by:

• Optimizing tests for regression run time

• Skipping any tests that don’t hit any new coverage targets
• Scheduling tests across compute resources for maximum efficiency
• Proactively managing cloud and grid storage and compute resources
• Deploying unique resiliency features to prevent regression interruptions due to unreliable hardware or unavailable tool licenses

Analysis Phase
Once the execution phase has run all available tests, the verification team must analyze the results to determine what to do next. The
first step is to debug failing tests in a user-friendly GUI. Users must be able to move easily among source code, schematic, waveform
and hierarchy views, with cross-probing and hyperlinked navigation. Once the source of each test failure is determined, the required
changes are made in the design or testbench code and the test is included in the next regression run to validate the fix. The second
step is aggregating the coverage results from all passing tests, including those run with different engines, for a unified coverage view.

During the analysis phase, the Synopsys verification management solution:

• Presents users a unified verification dashboard for Verification Continuum engines

• Provides customizable charts and reports to guide the analysis
• Segregates test failures due to system events from functional failures that require review and debug
• Provides many industry-leading features to debug test failures
• Displays failure buckets from the execution phase for easier debug

3
 • Analyzes test pass/fail results against the verification plan and tracks status of open bugs against the plan
• Merges coverage metrics from passing tests, compares coverage results from different regression runs to aid in root cause
analysis, back-annotates results into the verification plan and ranks tests by their contribution to coverage, as shown in Figure 4.

Figure 4: Ranking tests by coverage

Closure Phase
Once all tests are passing, verification closure requires achieving the coverage goals, as seen in Figure 1. Missing coverage targets
can be addressed in one of three ways:

• Formal analysis proves the target unreachable, so it is not counted in the metrics
• Users determine that the target is not relevant and add it to an exclusion list
• Users modify the regression tests to hit the reachable coverage target

The Synopsys verification management solution has several key features to help users achieve verification closure. Unreachability
(dead code) results from VC Formal™ are imported and reflected in coverage metrics. Users can easily edit and manage exclusions,
which adapt as the design evolves. Hitting missing coverage targets may involve extending test runs, tweaking constraints for current
tests, writing directed tests or running more formal analysis. The “what-if” analysis capabilities, as shown in Figure 5, allow users to
add guard expressions to their coverage targets and see a real time update of coverage metrics.

Active covergroup Real time

instances coverage data

Real time instance options

and values

Annotated covergroup bins

Users can add guard expressions and values

Figure 5: Coverage closure with what-if analysis

4
Tracking Process
One critical aspect of an effective SoC verification solution is the ability to observe results at any of the four phases and to track
these results over time. The unified dashboard shows both the latest status and historical trends. As previously noted, it aggregates
metrics from multiple engines, performs coverage analysis and grades tests. It has many advanced charting, reporting and analytics
capabilities. Using available Structured Query Language (SQL) views, users can generate browser-based custom dashboards, charts
and reports to serve the needs of different project teams.

Integration Process
Beyond the verification solution, there are other tools used in requirements planning and project management that are part of the
overall SoC development flow. Synopsys provides many links to integrate seamlessly with tools from other vendors and open-
source initiatives. These include out of the box integrations with engineering requirements management systems, including Jama
and Dynamic Object-Oriented Requirements System (DOORS). The features in the specification and verification plan are driven by
the requirements, so establishing a link between them aids in project management and satisfies industrial standards that require
requirements tracking. The solution also integrates with:

• Project management tools, including Jenkins and Redmine

• Product lifecycle tools, including Jira
• Cloud and grid management software, including IBM Spectrum LSF and Univa Grid Engine
• User-customized solutions via an open database and application programming interface (API)

Summary
SoC verification is an extraordinarily complex and expensive undertaking, addressing an essentially infinite state space. Extensive
automation is required to navigate through the planning, execution, analysis and closure phases in an efficient manner while using the
most advanced cloud and grid resources. Effective verification requires specifying the proper coverage targets, establishing closure
goals, and reaching those goals through regression testing and careful exclusions. Synopsys Verification Continuum provides the
industry’s broadest and deepest solution for every one of these aspects of SoC verification. Unified Verification Planner, VC Execution
Manager and Verdi capabilities are natively integrated within the Verification Continuum to automate verification management.

Looking forward, Synopsys’ unified verification management capabilities are growing more advanced every day, including powerful
artificial intelligence (AI) and machine learning (ML) techniques being integrated now to deliver breakthrough gains in verification
efficiency. The Synopsys verification management solution has been used by countless SoC teams developing processors, graphics
engines, AI and ML applications, Internet of Things (IoT) devices, and many other cutting-edge products to deliver verification closure
on schedule and on time.

©2021 Synopsys, Inc. All rights reserved. Synopsys is a trademark of Synopsys, Inc. in the United States and other countries. A list of Synopsys trademarks is available
at synopsys.com/copyright.html . All other names mentioned herein are trademarks or registered trademarks of their respective owners.
01/26/21.CS624494971 ExecMan wp.
Pub: January 2021