PyVSC: SystemVerilog-Style Constraints, and

Coverage in Python
M. Ballance
Milwaukie, OR, USA
matt.ballance@gmail.com

Abstract—Constrained-randomization and functional Coverage is captured as a set of individual coverpoint

coverage are key elements in the widely-used SystemVerilog-based instances, and is not organized into covergroup types and
verification flow. The use of Python in functional verification is instances in the same way that SystemVerilog does.
growing in popularity, but Python has historically lacked support
for the constraint and coverage features provided by B. SystemC
SystemVerilog. This paper describes PyVSC, a library that SystemC has been one of the longer-lasting languages used
provides these features. for modeling and verifying hardware that is built as an
embedded domain-specific language within a general-purpose
Keywords—functional verification, Python, constrained programming language (C++). While libraries for constraints
random, functional coverage and coverage targeting SystemC are not directly applicable in
I. INTRODUCTION Python, it’s worthwhile considering approaches taken.
Verification flows based on constrained-random stimulus The SystemC Verification Library (SCV) provides basic
generation and functional coverage collection have become randomizations and constraints, using a Binary Decision
dominant where SystemVerilog is used for the testbench Diagram (BDD) solver. BDD-based constraint solvers are
environment. Consequently, verification engineers have known to have performance and capacity challenges, especially
significant experience with the constructs and patterns provided with sizable constraint spaces. No support for capturing
by SystemVerilog. functional coverage is provided.
There is growing interest in making use of Python in The CRAVE [4] library sought to bring higher performance
verification environments – either as a complete verification and capacity to randomization in SystemC using Satifiability
environment or as an adjunct to a SystemVerilog verification Modulo Theories (SMT) constraint solvers. While there is some
environment. It is desirable in both of these cases to be able to evidence that some support for functional coverage was
make use of constrained random stimulus generation and to developed [5], this is not part of the published CRAVE library.
capture functional coverage. The PyVSC[1] library brings these
features to Python, with the goal of providing as similar a user III. MODELING CONSTRAINED-RANDOM STIMULUS
experience as possible to what SystemVerilog-knowledgeable PyVSC is considered an embedded domain-specific
verification engineers are familiar with. The PyVSC library also language (eDSL) because it effectively extends the Python
takes advantage of advances in solver technology to support language with new semantics for constraint solving and
constraint complexity on par with what is in use in coverage capture. There are several key Python language
SystemVerilog-based environments. features that are used to layer these new semantics on top of the
Python language. Library classes and methods are used to
II. RELATED WORK capture some constraints and coverage constructs. Python
Other projects have, of course, sought to bring constrained decorators are used to identify Python classes and methods as
randomization and/or functional coverage collection to having specific significance. Python introspection is used to
languages used for functional verification that lack built-in obtain source locations for declarations, to automatically-assign
support. names to elements, and to dynamically modify user-declared
classes. Python ‘with’ statements are used to identify statement
A. cocotb-coverage blocks. PyVSC uses the Boolector SMT solver[6] to solve the
Possibly the most relevant of these packages, given the target user-specified constraints.
of Python, is the cocotb-coverage library [2][3]. This library
A. Data Fields and Classes
supports basic randomization and functional coverage
collection. The constraints used by the cocotb-coverage library Both SystemVerilog and PyVSC use data fields with a
are captured as executable Python methods and are evaluated specified bit width. This is important to ensure that both
using a pure-Python SAT solver. This approach to capturing and constraints and coverage constructs are interpreted correctly.
solving constraints restricts each field to having no more than PyVSC supports creating data elements individually, but data
two constraints, and can result in poor performance and elements are typically grouped with constraints in randomizable
capacity. classes.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

 @vsc.randobj @vsc.randobj
class my_s(object): class my_base_s(object):

def __init__(self): def __init__(self):

self.a = vsc.rand_bit_t(8) self.a = vsc.rand_bit_t(8)
self.b = vsc.rand_uint8_t() self.b = vsc.rand_bit_t(8)

@vsc.constraint @vsc.constraint
def ab_c(self): def ab_c(self):
self.a < self.b self.a < self.b

Figure 1 - Randomizable Class Declaration @vsc.randobj

class my_ext_s(my_base_s):
As shown in Figure 1, a randomizable class is identified with
def __init__(self):
the @vsc.randobj decorator. This builds infrastructure into the super().__init__()
class to support randomization, and ensures that constraint
elaboration occurs after a class instance is created. @vsc.constraint
Class members that can be constrained are created using def ab_c(self):
self.a > self.b
methods provided by the library. The width of data fields can
either be specified directly, or via convenience methods such Figure 3 - Overriding Constraints
as vsc.rand_uint8_t that create data fields with commonly-
used widths. In the example in Figure 3, the relationship a > b will hold
for all instances of class my_ext_s because the ab_c constraint
B. Class Constraints in the my_ext_s type overrides the ab_c constraint in the
Constraints are specified as Python statements within Python my_base_s type.
methods decorated with @vsc.constraint. Constraint methods
are evaluated once to construct a constraint data model from the Figure 4 summarizes the constraint statements currently
statements within the method. supported by PyVSC and those supported by SystemVerilog.

@vsc.constraint
def ab_c(self): Feature SystemVerilog PyVSC
with vsc.if_then(self.a == 1): Algebraic constraints Y Y
self.b == 1 Integer fields Y Y
with vsc.else_if(self.a == 2): Enum fields Y Y
self.b == 2
with vsc.else_if(self.a == 3): Fixed-size arrays Y Y
self.b == 4 Variable-size arrays Y Y
with vsc.else_if(self.a == 4): dist constraint Y Y
self.b == 8
with vsc.else_if(self.a == 5):
soft constraint Y Y
self.b == 16 inside constraint Y Y
solve ordering Y Y
Figure 2 - If/else Constraints unique constraint Y Y
foreach constraint Y Y
The difference between Python statements and PyVSC
constraint_mode Y Y
constraints is most visible when control-flow constraints are
used, as shown in Figure 2. In these cases, Python statements rand_mode Y Y
cannot be used directly. It is necessary to use the PyVSC- if/else constraint Y Y
provided construct to capture the constraint intent. Figure 4 - Supported Constraint Statements

Just as with SystemVerilog, constraint blocks are considered C. Randomizing a Data Field
virtual, in that a same-named constraint in a sub-class overrides PyVSC randomziable classes have randomize and
the constraint in the super-class. randomize_with methods that are used for randomizing fields
within that class.
 @vsc.randobj @vsc.covergroup
class my_base_s(object): class my_covergroup(object):

def __init__(self): def __init__(self):

self.a = vsc.rand_bit_t(8) self.with_sample(
self.b = vsc.rand_bit_t(8) a=vsc.bit_t(4)
)
@vsc.constraint self.cp1 = vsc.coverpoint(
def ab_c(self): self.a, bins={
self.a < self.b "a" : vsc.bin(1, 2, 4),
"b" : vsc.bin(8, [12,15])
item = my_base_s() })
for i in range(10):
with item.randomize_with() as it: Figure 7 - Declaring a covergroup type
it.a == i
PyVSC covergroups support several mechanisms to provide
Figure 5 - Randomizing a Class with Additional Constraints coverage data for sampling. Figure 7 shows using the
with_sample method, which creates a method named sample on
PyVSC supports randomization with additional constraints, the covergroup class. Coverage data is provided via method
as well as randomization considering just the constraints parameters each time the sample function is called.
declared within the class. Figure 5 shows an example of adding
additional constraints. The randomize_with class method is @covergroup
called using a Python with clause. Additional constraints are class my_covergroup(object):
specified within the with clause. All legal constraints supported
def __init__(self, a):
by PyVSC may be used with an inline constraint.
super().__init__()
@vsc.randobj
self.cp1 = coverpoint(a,
class my_item(object):
bins=dict(
a = bin_array([], [1,15])
def __init__(self):
))
self.a = vsc.rand_bit_t(8)
a = 0;
self.b = vsc.rand_bit_t(8)
cg = my_covergroup(lambda:a)
a=1
@vsc.constraint
cg.sample() # Hit the first bin of cp1
def valid_ab_c(self):
self.a < self.b
Figure 8 - Binding sampling data at instantiation
item = my_item()
Another approach is shown in Figure 8. In this case,
# Always generate valid values sampling data is provided via a lambda function that is specified
for i in range(10): when the covergroup is instanced. Calling ‘sample’ on the
item.randomize() covergroup causes that lambda to be called and read the current
value of the data to be sampled.
item.valid_ab_c.constraint_mode(False)
B. Specifying Coverpoints and Bins
# Allow invalid values
for i in range(10): Coverpoints and bins are specified in the covergroup class
item.randomize() constructor. User-specified single bins and arrays of bins are
supported. If no bins are specified, bins are automatically
Figure 6 - Using constraint_mode to disable constraints partitioned according to the auto-bin max.
PyVSC also supports constraint-control methods, such as @vsc.covergroup
rand_mode for variables and constraint_mode for constraint class my_covergroup(object):
blocks. Figure 6 shows using constraint_mode to temporarily
def __init__(self):
disable a constraint block.
self.with_sample(
a=vsc.bit_t(4),
b=vsc.bit_t(4))
IV. MODELING FUNCTIONAL COVERAGE self.cp1 = vsc.coverpoint(self.a, bins={
"a" : vsc.bin_array([], [1,15])
Modeling functional coverage constructs with PyVSC })
follows the same patterns used for modeling randomizable self.cp2 = vsc.coverpoint(self.b, bins={
classes and constraints. "b" : vsc.bin_array([], [1,15])
})
A. Declaring a Covergroup
self.cp1X2=vsc.cross([self.cp1, self.cp2])
A covergroup type is declared with PyVSC as a regular
Python class decorated with the @vsc.covergroup decorator. Figure 9 - Coverpoint Cross
 Coverpoints crosses are also supported, as shown in Figure Properly retrieving and saving coverage data is the other key
9. aspect to integrating PyVSC into an environment.
A summary of coverage features supported by PyVSC and
SystemVerilog is shown below.
import vsc
Feature SystemVerilog PyVSC …
covergroup type Y Y vsc.report_coverage(details=True)
coverpoint bins Y Y Figure 11 - Reporting Functional Coverage Textually
coverpoint ignore_bins Y N
coverpoint illegal_bins Y N
coverpoint single bin Y Y The report_coverage method, shown in Figure 11, creates a
coverpoint array bin Y Y textual coverage report with information on all covergroup types
coverpoint auto bins Y Y and instances. By default, the coverage report is written to the
coverpoint transition bin Y N console (stdout), as shown in Figure 12. The output of
cross auto bins Y Y report_coverage can also be written to a file.
cross bin expressions Y N
cross explicit bins Y N
cross ignore_bins Y N
cross illegal_bins Y N

Figure 12 - Example Textual Coverage Report

V. ENVIRONMENT INTEGRATION
The PyVSC library is environment independent and, for PyVSC also provides the write_coverage_db method to save
example, does not require the use of cocotb[7]. This collected functional coverage data in Accellera Unified
independence allows PyVSC to integrate into any Python-based Coverage Interoperability Standard (UCIS) [8] format, as shown
environment, but also imposes some overhead on the end user. in Figure 13.
Aside from the modeling layer used to express randomizable import vsc
data, constraints, and coverage, PyVSC has two integration …
points with the environment: the random seed, and saving vsc.write_coverage_db("cov.xml")
coverage data. Users of PyVSC must ensure that the random
seed is properly configured, and that coverage data is properly Figure 13 - Saving Coverage Data in UCIS Format
saved before the environment exits.
PyVSC also provides access to a coverage report as a Python
A. Random Seed Management object hierarchy, which can be used to produce a coverage report
Random generation is controlled by a seed, in order to ensure in a custom format. Showing how to use this mechanism for
that results can be reproduced. PyVSC derives its randomization manipulating coverage data is beyond the scope of this paper.
from the Python random package, so configuring the random- C. Example Integration
package seed also sets the PyVSC seed.
One example of PyVSC usage is the Google RISC-V DV
Some environments, such as cocotb [7], configure the global project [9]. The RISC-V DV project provides a random
random seed based on user-specified environment variables instruction-stream generator for the RISC-V ISA. The RISC-V
and/or command-line options. In other cases, the random seed DV project also defines functional coverage metrics to be
must explicitly be configured as part of the PyVSC environment collected on the result of running these randomly-generated
integration. In these cases, the seed should be set directly, as programs.
shown in Figure 10.
The project currently provides two primary implementations
import random of randomization and functional coverage collection: a
… SystemVerilog model that uses the random solver and coverage
random.seed(0) collection mechanisms within a SystemVerilog-compliant
simulator, and a pure-Python implementation using PyVSC.
Figure 10 - Configuring the Random Seed

B. Saving Coverage Data

In most cases, coverage data must be converted to some
textual or binary persistent form in order to make use of it Figure 14 - RISC-V DV Flow
outside the environment run in which it was collected. PyVSC
provides several ways to access the collected coverage data. Figure 14 shows the RISC-V DV flow. The random
instruction-stream generator creates assembly-level test
programs. These programs are compiled and executed by either more complex constraints. Because PyVSC constraint and
an ISS or RTL model of a RISC-V processor. Test-program coverage constructs are implemented in terms of Python
execution results in a series of execution-trace files which are language features, supporting programmatic enhancement of the
processed and sampled by the functional coverage model to existing constructs is natural.
produce coverage data about the actual instructions that
executed. C. Performance Improvements
PyVSC is a pure-Python library built on top of the Boolector
The RISC-V DV Python flow creates approximately 50 SMT solver. Since the Boolector solver does most of the
covergroups, and around 300 coverpoints and crosses. Coverage compute-bound work of constraint solving, PyVSC’s
data is saved as a UCIS XML file. performance is quite good. That said, there are cases –
VI. WORKING WITH COVERAGE DATA specifically with arrays and iterative constraints – where
Python’s interpreted nature leads to slower than desirable
Once coverage data has been collected, it can be displayed performance.
as a simple text report that shows the coverage achieved by each
covergroup instance as well as the total coverage collected by all A natural next step is to re-implement the core constraint and
instances of a covergroup type. Coverage data can also be saved coverage data mode in C++ and expose it as a Python extension.
to a Unified Coverage Interchange Standard (UCIS) XML file Early experiments with this approach have shown significant
for processing by other tools. performance improvements are possible in cases where the
number of variables is significant.
D. Integration Improvements
As described in section V, PyVSC currently provides little
or no environment-specific integration for specific
environments. This keeps PyVSC free of dependencies on
specific environments, but at the cost of requiring users to
determine how to integrate PyVSC’s features with the services
provided by the target environment.
One area of future work is to enhance the integration
between PyVSC and various existing verification environments,
such as cocotb.
Figure 15 - Coverage Data Visualized
VIII. CONCLUSION
Figure 15 shows a graphical view of coverage data captured Randomization and functional coverage are key to industrial
in UCIS XML format. functional verification practice. PyVSC brings these features to
Python in a form that will be familiar to SystemVerilog users.
VII. FUTURE WORK This boosts the capabilities of Python-based verification by
While PyVSC is currently in a usable state, there are features making it easier for SystemVerilog practitioners to reuse their
that are still on the near-term roadmap, as well as extensions to knowledge of constraints and coverage in Python.
be considered. The major areas of effort are listed below.
REFERENCES
A. Complete supporting SystemVerilog Constructs [1] M. Ballance, PyVSC, GitHub, https://github.com/fvutils/pyvsc,
The highest priority for future work with PyVSC is to [2] M. Cieplucha, W. Pleskacz, “New Constrained Random and Metric-
complete implementation of missing features that Driven Verification Methodology Using Python”, DVCon Europe 2017
SystemVerilog supports. Currently, most of the constraint http://events.dvcon.org/2017/proceedings/papers/02_3.pdf
constructs are supported. More work exists to complete support [3] M. Cieplucha, W. Pleskacz, cocotb-coverage, GitHub,
of all functional coverage constructs. https://github.com/mciepluc/cocotb-coverage
[4] F. Haedicke, H. M. Le, D. Große, and R. Drechsler, “CRAVE: An
B. Beyond SystemVerilog advanced constrained random verification environment for SystemC,”. In
International Symposium on System-on-Chip, pp. 1-7, 2012
Thus far, PyVSC has focused on achieving feature parity
[5] H. M. Le, R. Dreschler, “Boosting SystemC-based Testbenches with
with the constraint and functional coverage constructs supported Modern C++ and Coverage-Driven Generation”, Conference: DVCon
by SystemVerilog. But, supporting these features using a library Europe 2015
in the context of a general-purpose programming language [6] R. Brummayer, A. Biere, “Boolector: An Efficient SMT Solver for Bit-
leaves open the possibility to go much further. Vectors and Arrays”, Conference: Tools and Algorithms for the
Constructions and Analysis of Systems (TACAS) 2009
Coverage and constraint constructs are built into the [7] cocotb, GitHub, https://github.com/cocotb/cocotb
SystemVerilog language. While this makes them easily
[8] Accellera, Unified Coverage Interoperability Standard,
accessible as first-class language features, it also makes them https://www.accellera.org/downloads/standards/ucis
difficult or impossible to use programmatically. One interesting [9] Google, RISC-V DV, GitHub, https://github.com/google/risc
direction to explore with PyVSC is the hybrid space where user-
specified constraints are programmatically processed to build