Covered Info

Covered - The Verilog Code Coverage Analyzer

Introduction

Covered is a Verilog code coverage analysis tool that can be useful for determining how well a diagnostic test suite is covering the design under test. Typically in the design verification work flow, a design verification engineer will develop a self-checking test suite to verify design elements/functions specified by a design's specification document. When the test suite contains all of the tests required by the design specification, the test writer may be asking him/herself, "How much logic in the design is actually being exercised?", "Does my test suite cover all of the logic under test?", and "Am I done writing tests for the logic?". When the design verification gets to this point, it is often useful to get some metrics for determining logic coverage. This is where a code coverage utility, such as Covered, is very useful.

Covered reads in the Verilog design files and a VCD or LXT formatted dumpfile from a diagnostic run and generates a database file called a Coverage Description Database (CDD) file, using the score command. Covered's score command can alternatively be used to generate a CDD file and a Verilog module for using Covered as a VPI module in a testbench which can obtain coverage information in parallel with simulation. The resulting CDD file can be merged with other CDD files from the same design to create accummulated coverage, using the merge command. Once a CDD file is created, the user can use Covered to generate various human-readable coverage reports in an ASCII format or use Covered's GUI to interactively look at coverage results, using the report command. Additionally, as part of Covered's score command, race condition possibilities are found in the design files and can be either ignored, flagged as warnings or flagged as errors. By specifying race conditions as errors, Covered can also be used as a race condition checker.

Covered currently supports Verilog-1995, Verilog-2001 (with the exception of config blocks currently), and some SystemVerilog constructs. Metrics that are generated include the following:

Line coverage
Toggle coverage
Memory coverage
Combinational logic coverage
FSM state and state-transition coverage
Assertion (functional) coverage

Latest News

11/26/2009

Development release covered-20091126 made. This is a bug fix release only.

10/24/2009

Stable release covered-0.7.7 made. This is a bug fix release only.

Fixed compilation warnings when compiling on 64-bit Mac OS X and Debian-based platforms.
Updates to build scripts to help downstream Debian releases builds.
Fixed bug 2880705. $Id: keywords containing newlines are now handled properly. Additionally, fixing issues with multiply instantiated modules within a generate block.
Fixed bug 2881869. Fixed a stack overflow issue in the gen_item_resolve function that would cause segmentation faults when too many items were being generated within a single generate block.
Fixed bug 2882433. Fixed the "ERROR! Parameter used in expression but not defined in current module" error when a generated module instance has a parameter override of a parameter with the same name as the parameter within the module that contains the generate block.

08/24/2009

Stable release covered-0.7.6 made. This is a bug fix release only.

Fixed misspelling in report generator code (misspelling showed up in text reports)
Fixed issues with performing module merging with modules containing generate blocks configured differently for different instantiations of the same module.

08/02/2009

Development release covered-20090802 made. This development release adds several performance enhancements and bug fixes to the new inlined code coverage flow, including the following:

Adding support for $random and $urandom system calls to inlined coverage.
Includes all fixes made to the stable 0.7.5 release.
Adding support for $value$plusargs system calls to inlined coverage.
Fixing issue with generated IF statements.
Added user documentation for inlined coverage flow and score options.
Fixing issue with generated code interrupting comma-separated assign statements.
Performed code simplification and performance improvement with the way statements were handled internally.
Removed unnecessary calls to simulation functions when using inlined code coverage (this added a performance penalty).
Improved performance of inlined code generator for sizing generated signals.
Fixed memory indexing issues related to memory coverage.
Added support for static function and static ternary operators for inlined code coverage.
Added code to differentiate functions used statically and not to do the right thing for inlined code coverage accumulation.
Added vcd_diff script which checks the dumpfile output from non-inlined and inlined design files to verify that the inlined code generator does not change the result. This check is now a part of all inlined regression runs.
Made several performance improvements to the VCD file reader. The reader is now 10-20% faster.
Added support for Verilator regressions runs and ported a couple of diagnostics to Verilator format.
Adding check to make sure that a CDD file without inlined mode set that reads a VCD file containing inlined coverage data emits an error to the user and exits gracefully.
Added -inline-comb-depth score option to allow the user to specify a shallower combinational coverage depth to be generated -- improving inlined simulation and coverage performance.
For Verilator runs, inserted pragmas around intermediate combinational logic expression signals to exclude them from being output to VCD files. This improves simulation and coverage performance for Verilator runs (other simulators that have a VPI that automatically remove these signals from generating change callbacks).
Performing code replace of some actual code with pre-calculated intermediate expression values for further simulation performance improvements.
Added "e" option to -inline-metrics which allows event coverage to be turned on/off independently of other combinational logic coverage. This allows further simulation and coverage performance improvements (especially for Verilator runs).
Added optimization that causes code generation to be skipped for assertion files when assertion coverage is not required.
Full regressions now runs cleanly with all code changes.

08/02/2009

Stable release covered-0.7.5 made. This is a bug fix release only.

Fixed bug 2808818. If a generate variable name collided with a reg/wire name, Covered was not emitting an error.
Fixed bug 2808820. If no signal was used from the dumpfile and at least one signal needs information from the dumpfile, Covered needed to signal a user error.
Fixed bug 2812321. Parameterized/generated modules could get incorrect coverage calculated for them.
Fixed bug 2812495. Fixed a crash issue. There is another part to this bug report that is not fixed, however.
Fixed bug 2813405. A design run with the -g score option caused the GUI to freeze when viewed.
Fixed bug 2813948. Fixed assertion issue with merging scored and unscored CDD files.

06/17/2009

Stable release covered-0.7.4 made. This is a bug fix release only.

Updated regression files for the new 2.4 version of the OVL.
Fixed bug 2804585. Memory reads in LHS part selects were not being marked for memory coverage.
Fixed issue with VPI usage in a VCS simulation with generate statements.
Fixed bug 2805191. Automatic tasks/functions that manipulate variables outside of the task/function can cause incorrect toggle coverage for those signals.
Fixed bug 2806855. Generate blocks generating module instantiations could lead to score command errors (segfaults, internal assertion errors, etc.)

06/04/2009

Stable release covered-0.7.3 made. This primarily fixes a few bugs in the compile of Covered "out of the box". It seems that even with the regression testbench, things can still slip through the cracks :( Anyhow, please use this release instead of the 0.7.2 release.

05/29/2009

Stable release covered-0.7.2 made. This is primarily a bug fix release with a few new features added to the CLI. Here are the details of the changes.

Fixed bug 2791651. Memory deallocation errors occurred when syntax errors were being reported by the parser.
Fixed bug 2791599. Whitespace prior to a `line or #line directive were not being handled properly.
Fixed bug 2794588. If a module was specified in a -v option after its directory was specified by the -y option to the score command, the module was not found for parsing.
Fixed bug 2794684. If a normal (not generate) case statement within a generate block will output the case expression to be output to the CDD more than once, leading to internal assertion errors when the CDD file is read.
Fixed bug 2795088. When a CDD file is opened from the wizard GUI window, the open file window can be placed behind the wizard window. Instead the wizard window should disappear once a selection button has been clicked.
Fixed bug 2795086. If the user clicked on the global exclusion reason listbox when it is empty, a Tcl/Tk error message box was raised.
Fixed bug 2795089. If the GUI detailed combinational logic window is used to view several expressions one after the other, Covered can segfault.
Fixed bug 2795583. Score command segfaults when a module is instantiated within a generate block and overrides a parameter value within the module.
Fixed bug 2795640. Variables instantiated within a generate block caused issues with Covered when simulated with VCS.
Fixed bug where memory elements being assigned via non-blocking assignments were not being evaluated, leading to incorrect coverage output.
CLI updates/fixes:
- When the 'debug on' command is specified, a line specifying that the debug mode is now on is output (previously nothing was output (because the debug mode was off).
- Changed the 'debug on' command to 'debug less' and 'debug more' where the prior only outputs the executed statements and timestep information during simulation while the latter outputs what 'debug on' used to output (extremely verbose).
- Fixed bug 2795209. When an unknown CLI command was specified, a memory error occurred.
- Fixed bug 2795215. Status bar was attempting to be output during simulation when debug mode was turned on. This created some unreadable/messy output.
- Changed the 'goto <num>' command to 'goto time <num>'.
- Added 'goto line [<filename>:]<num>' command which simulates until the specified line number is about to be simulated.
- Added 'goto expr <signal> <bool_op> <value>' command which simulates until the given expression evaluates to a value of true.
- Added support for handling the Ctrl-C interrupt when the score command is simulating with the -cli option specified. In this case, simulation will immediately stop and return a CLI prompt which will allow the user to continue interacting with the simulation.
Updated user guide documentation to include the changes made to the CLI.

05/07/2009

Stable release covered-0.7.1 made. This is a bug fix release only. Here are the details:

Fixed bug 2782473. CDD files being merged from different testbenches but with similar leading hierarchy (but different top-level modules) which would lead to internal assertion errors.
Fixed bug 2785453. Wires declared in generated named scopes were not handled correctly by Covered in VPI mode of operation, leading to inaccurate coverage information.
Fixed bug 2786986. An always block with a part select in the sensitivity list was triggering on the entire signal change rather than the specific part select, leading to a potential degradation in performance and inaccuracy in coverage information.
Allow time variable types to be included for coverage.
Fixing permission issue with the install-sh script that some people would get after first downloading and installing.
Updated README and INSTALL files to be more accurate.
Fixed coverage accuracy issue for code that uses variable part selects in LHS of expressions.

04/26/2009

Stable release covered-0.7 made. This is a significant improvement over the 0.6 release, providing Verilog language enhancements, significant score optimizations, new rank and exclude commands, an enhanced merging capability, a multitude of GUI enhancements, a complete overhaul of the user documentation, many bug fixes, and much more.

Click here for a history of news notes.

Downloads

Stable Download Versions

Development Download Versions

Documentation

For on-line user documentation for the latest release, please go to the User's Guide page. A copy of the user's manual will also be available in the download. Documentation for the GUI report utility is available in the Help menu of the GUI.

Bug Reporting

If you encounter any problems with the Covered tool, please send these problems along with the version of Covered that you are using to the Covered bug reporting utility on SourceForge.net . Please describe the problem as accurately as possible or send an example that uncovers the problem. I will try to look at these as soon as possible.

ToDo List

The following are items that are on the todo list. Some of them are long term features that I want as part of this tool.

Enhance Verilog code parser to allow coverage on more of the language.
Add inlined code coverage generation capability for scoring.