Chapter 8. Using Covered

Table of Contents

8.1. Work Flow
Scoring the Design
Merging CDDs
Creating Coverage Reports
Ranking CDDs
Excluding CDD Coverage Points
8.2. Covered usage
8.3. Covered global options

8.1. Work Flow

Covered has four basic commands that it uses for gaining coverage information about a design and displaying that information to the user.

  • score

  • merge

  • report

  • rank

Scoring the Design

To begin using Covered, you will need to create a special database file called a coverage description database (CDD). Covered uses this file to store coverage-specific information about the DUT. This file is a text-based file that is written in a specific format (the format of this file is not discussed in this document but can be found in the developer's document) that Covered understands.

To create the CDD and populate it with the simulation dumpfile results, you must use the score (see Chapter 9, The score Command for details) command. This command parses in the DUT files and generates an initial CDD for the design. This CDD does not contain any coverage details but only the design elements that Covered needs to gain coverage results.

Once the initial CDD has been created, the score command reads in the specified VCD/LXT2 dumpfile from the given DUT. If Covered recognizes that the VCD/LXT2 dumpfile does not correspond to the DUT that it parses (dumpfile was not generated from the DUT specified), an error message is supplied to the user telling them this. Once the dumpfile has been parsed, Covered generates a new version of the CDD containing a populated database. This CDD may be merged with another CDD from the same DUT and/or reports may be generated from this CDD.

Merging CDDs

When two or more CDDs have been generated from the same DUT. It is often desirable to merge their contents to see what the total coverage is for both simulation runs. The reasons why merging is desirable is that often several tests are written to exercise the entire design (not just one). This makes tests easier to write and maintain. Since, typically, only one test is run per simulation run and that test does not fully test the DUT, it is necessary to combine the results of the multiple simulations to the see the combined effect of the tests on the design. This is where merging comes into play.

Merging multiple CDDs from the same design is easy with Covered, and is accomplished with the merge (see Chapter 10, The merge Command for details) command. This command reads in two or more CDDs, merges their results and outputs the merged CDD to a different file or replaces the first CDD with the results of the merge.

Creating Coverage Reports

After a CDD is created or merged, the contents of the CDD can be extracted and transformed into human-readable coverage reports with the report (see Chapter 11, The report Command for details) command. A summary report containing the percentage of metrics covered is generated for each module. Additionally, to aid in understanding what and why something was not covered, a verbose reporting mechanism exists. Verbose reporting will explicitly point to and describe why something was not fully covered for a specific metric.

Reports are text-based output that by default are directed toward standard output but may, with the use of an option flag, be output to a specific file for storage purposes.

Ranking CDDs

After a collection of CDDs have been generated from a single design (most likely due to a regression run), it is desirable to know which CDDs have an effect on coverage and which CDDs do not add any new coverage information. The CDDs that do not provide additional coverage may be unnecessary for regression and removing them may significantly improve a regression runs time (which can help in a projects schedule).

Additionally, it also good to know an order that simulations should be run in to produce the maximum amount of coverage the quickest. This information can be useful for creating a quick (or "sanity check" or "confidence") regression that runs in a relatively short period of time and, when run successfully, provides the engineer a level of confidence that recent changes have not severely broke the functionality of the design.

This is the purpose of the rank (see Chapter 12, The rank Command for details) command. It produces a report that shows the diagnostics that add to the overall coverage of the suite, the order that those diagnostics should run, and each diagnostic's overall effect on coverage when it is run.

Excluding CDD Coverage Points

Not all unhit coverage points reported by Covered will be valid for your design. Some coverage points may be extraneous, some may be due to obsolete logic that is not pulled from the design for various reasons, and some coverage points may be hit in other verification environments but not in the current environment.

Whatever the reason may be (and some reasons may be much more specific that these suggestions), Covered allows the user to exclude those coverage points from coverage reporting (they will still be reported but they will be considered "covered" points instead of "uncovered" points). Additionally, each excluded coverage point can have an option message attached to it that explains the reason for exclusion. This message is stored with the coverage information in the CDD file and can be displayed in a standard coverage report or in a stand-alone exclusion report. This information helps to document the coverage analysis process.

In addition to being able to exclude coverage points, the user can also include previously excluded coverage points (in the event that a mistake was made or if the design has changed). Coverage exclusion can be accomplished via the command-line exclude command or within the GUI coverage analyzer.

8.2. Covered usage

Covered is initiated with the following command:

covered (-h | -v | [-P [filename]] [-B] [-D|-T|-Q]) (score|merge|report|rank) command_options)

The -v command (described below) is mutually exclusive from the rest of the commands and has the highest priority. If Covered is called with this option and any other options, the version will only be displayed.

The -h command (described below) is mutually exclusive from the rest of the commands and has the second highest priority. If Covered is called with this option and the -v option is not specified, the usage information will be displayed to the screen.

The -Q option suppresses all normal user output from standard output. The only output that will not be suppressed are messages to standard error. This option may be used with any of the Covered commands to achieve this effect. By default, normal user output is sent to standard output.

The -T option suppresses all normal user output from standard output with the exception of the Covered header banner, warning messages. This type of output is a bit more informative than output generated with the -Q option but not as much as running without any output limiters.

The -D option outputs debug information to standard output. Its effect overrides the -Q option. This option may be used in conjunction with any other Covered command for the purposes of debugging tool failures. By default, debug mode is turned off and must be enabled in the build of Covered (using the --enable-debug option to configure). This command should only be used for debugging as it generates an immense amount of output which may hinder performance.

The -B option obfuscates all design-sensitive signal, module, task, function, instance, file and parameter names when outputting them to either stdout, stderr or a file (with the exception of CDD file output). This mode is intended to be used for sharing output information with others (including the developer's of Covered) which is sensitive information for the purposes of debugging.

The -P option causes Covered to enable internal code profiling and generates a profiling report that is output to either the given filename or the default filename of "". This option is only available if the --enable-profiling configuration option is specified and will cause performance degradation. This option should only be used by Covered developers or those interested in finding performance bottlenecks in Covered's code. It is not recommended that the --enable-profiling configuration option and the -P global option be specified for normal users of Covered. See Section 15.3, “Source Code Profiling” for more information on profiling and understanding the profiling report file.

8.3. Covered global options

The following table describes options that Covered will accept apart from its normal commands which are described in the next several chapters.

Table 8.1. Global Options to Covered

Option Description
-Q Quiet mode. Suppresses user output to screen. Use in conjunction with a command.
-T Terse mode. Causes all output to be suppressed with the exception of warning messages and the Covered header information. Use in conjunction with a command.
-D Debug mode. Outputs code debug information to screen. Use in conjunction with commands.
-B Obfuscate. Obfuscates all design sensitive names before outputting them.
-P [filename] Profiling mode. Outputs profiling report to either the given filename or "" if no filename was specified.
-h Generates usage output to standard output.
-v Outputs current version of Covered.