Table of Contents
The score command is initiated with the following call:
covered [global_options] score -t top_module [options]
The top_module parameter is required for the scoring purposes and represents the top-most module that will be covered. It is important to note that Covered does not need to parse the entire DUT. It only parses that part of the design which will receive coverage results. More on this will be described Section 9.3, “Specifying What to Cover” below.
The following table describes the additional options that can be passed to the score command. Note that it is optional in how you tell Covered where to find the Verilog design files; however, it is not optional to exclude the Verilog files. If no Verilog can be found that contains at least the top instance, Covered will generate an error message to the user specifying that it was unable to find the required Verilog modules.
Table 9.1. Options to score Command
|-A ovl||Causes OVL assertions to be used for assertion coverage. This flag must be given to the score command if assertion coverage metrics are needed in the report command. See Section 2.6, “Assertion Coverage”.|
|-cli [filename]||Causes the command-line debugger to be used during VCD/LXT2/FST dumpfile scoring. If filename is specified, this file contains information saved in a previous call to savehist on the CLI and causes the history contained in this file to be replayed prior to the CLI command prompt. If filename is not specified, the CLI prompt will be immediately available at the start of simulation. This option is only available when Covered is configured with the --enable-debug option. See Section 15.2, “Command-Line Interface (CLI)”.|
|-conservative||If this option is specified, any logic blocks that contain code that would cause coverage discrepancies leading to potentially inaccurate coverage results are removed from coverage consideration. See Section 9.7, “The reason for the -conservative feature” for more information.s|
|-D name[=value]||Defines the specified name to 1 or the specified value.|
|-dumpvars [filename]||If this option is specified without the -vcd, -lxt or -fst options, the design is parsed, a CDD file is created and a top-level Verilog module named filename (if this value is specified) or "covered_dump.v" (if filename is not specified) is created. If compiled as a top-level module along with your design and no other $dumpfile/$dumpvars calls are made, this dump module will provide additional performance gains. If either the -vcd, -lxt or -fst options are specified, this option has no effect. See the section called “Selecting What to Dump”.|
|-e module||Name of module, task, function or named begin/end block to not score. You may specify this option more than once on a command-line; one for each block to remove from coverage. See Section 9.4, “Specifying What Not to Cover (Coverage Exclusions)”.|
|-ea||Specifies that all always blocks in the design should be excluded from coverage consideration. This option may help to increase performance of the score command if all always blocks in the design do not need to be checked for coverage. By default, always blocks are considered for coverage. See Section 9.4, “Specifying What Not to Cover (Coverage Exclusions)”.|
|-ec||Specifies that all continuous assignments in the design should be excluded from coverage consideration. This option may help to increase performance of the score command if all assign blocks in the design do not need to be checked for coverage. By default, continuous assignments are considered for coverage. See Section 9.4, “Specifying What Not to Cover (Coverage Exclusions)”.|
|-ef||Specifies that all final blocks in the design should be excluded from coverage consideration. This option may help to increase performance of the score command if all final blocks in the design do not need to be checked for coverage. By default, final blocks are considered for coverage. See Section 9.4, “Specifying What Not to Cover (Coverage Exclusions)”.|
|-ei||Specifies that all initial blocks in the design should be excluded from coverage consideration. This option may help to increase performance of the score command if all initial blocks in the design do not need to be checked for coverage. By default, initial blocks are considered for coverage. See Section 9.4, “Specifying What Not to Cover (Coverage Exclusions)”.|
|-ep [name]||Causes any code that is inbetween inline coverage pragmas (eg. '// coverage off' ... '//coverage on') to be excluded from coverage consideration. The code specified between these pragmas is not removed from resimulation but is marked as being excluded from coverage for reporting purposes. These exclusions may be undone via the GUI, if necessary, without requiring the design to be resimulated by Covered. If name is specified, its value will be used by Covered as the pragma keyword to search for in the design (instead of the default value of 'coverage'). This feature can be useful in the event that other tools being used on the design use a similar pragma name and there are tool conflicts. See Section 9.4, “Specifying What Not to Cover (Coverage Exclusions)”.|
|-F module=[in_expr,]out_expr||Specifies the location to find an FSM to score where module is the Verilog module containing the FSM, in_expr is the input state expression and out_expr is the output state expression. If in_expr is not specified, out_expr is used as both the input and output state expression.|
|-f filename||Name of file containing additional arguments to parse. You may specify this option more than once on a command-line.|
|-fst dumpfile||Name of FST dumpfile to score design with. If -vcd, -lxt or this option is not used, Covered will only create an initial CDD file from the design and will not attempt to score the design.|
|-g [module=](1|2|3)||Specifies what generation of Verilog to use for parsing the design. If module is specified, uses the given generation value for that module only. If module is not specified, applies that generation to the entire design. This option may be specified multiple times on the command-line, where options to the left take precedence over the values to the right. 1=Verilog-1995, 2=Verilog-2001, 3=SystemVerilog.|
|-h||Outputs usage information for the score command.|
|-I directory||Directory to find included Verilog files. You may use this option as many times as necessary on the command-line, one for each directory.|
|-i instance||Instance name of top-level module. Necessary if module to verify coverage is not the top-level module in the design. If not specified, -t value is used. See Section 9.3, “Specifying What to Cover”.|
|-lxt dumpfile||Name of LXT2 dumpfile to score design with. If -vcd, -fst or this option is not used, Covered will only create an initial CDD file from the design and will not attempt to score the design.|
|-m message||Allows the user to specify information about this CDD file. This information can be anything (messages with whitespace should be surrounded by double-quotation marks), but may include something about the simulation arguments to more easily link the CDD file to its simulation for purposes of recreating the CDD file.|
|-o filename||Name of CDD to write coverage information to.|
|-p filename||Overrides filename used for outputting intermediate preprocessor output. See Section 9.9, “Other Notes”.|
|-P parameter_scope=value||Performs a defparam on the specified parameter with value. See Section 9.5, “Overriding Parameters (-P option)”.|
|-rS||Performs race condition checking and removes any offending logic blocks from coverage consideration without reporting the information. See Section 4.2, “Race Condition Handling”.|
|-rW||Performs race condition checking, removes any offending logic blocks from coverage consideration, and displays this information to standard output. This is the default race condition checking behavior. See Section 4.2, “Race Condition Handling”.|
|-rE||Performs race condition checking, reports any offending logic and immediately stops scoring. This option allows Covered to be used as a static race condition checking tool. See Section 4.2, “Race Condition Handling”.|
|-rI[=module name]||If =module name is not specified, race condition checking is not performed for the entire design. If =module name is specified, race condition checking is not performed on the specified module only. This option may be specified more than once. See Section 4.3, “Avoiding Race Condition Checking”.|
|-rP[=name]||Skip race condition checking for all code surrounded by "// racecheck off/on" embedded pragmas. The "racecheck" keyword can be changed by specifying =name where name is the new name for the race condition pragma keyword. See Section 4.3, “Avoiding Race Condition Checking”.|
|-S||Outputs simulation statistics after simulation has completed. This information is currently only useful for the developers of Covered.|
|-top_ts (1|10|100)(s|ms|us|ns|ps|fs)/(1|10|100)(s|ms|us|ns|ps|fs)||This option is only valid when the -vpi or -dumpvars options have been specified. This option allows the user to specify a timescale for the generated Verilog module created with the -vpi or -dumpvars options. If this option is not specified, no timescale will be created for the generated module. If whitespace is needed between the various values, place the entire contents of timescale in double quotes.|
|-ts number||When scoring occurs, this option allows the user to see how far the simulator has progressed by outputting the current timestep to standard output. The value of number specifies how many timesteps are allowed to be simulated before outputting the current timestep (results in less calls to output stream).|
|-T (min|typ|max)||Specifies which value to use in min:typ:max delay expressions. Default is typ.|
|-v filename||Name of specific Verilog file to score. You may specify this option more than once on a command-line; one for each filename to parse.|
|-vcd dumpfile||Name of VCD dumpfile to score design with. If -lxt, -fst or this option is not used, Covered will only create an initial CDD file from the design and will not attempt to score the design.|
|-vpi [filename]||If this option is specified without the -vcd, -lxt or -fst options, the design is parsed, a CDD file is created and a top-level Verilog module named filename (if this value is specified) or "covered_vpi.v" (if filename is not specified) is created along with a PLI table file called filename.tab or "covered_vpi.v.tab". Both of these files are used in the compilation of the simulator to use Covered as a VPI module. If either the -vcd, -lxt or -fst options are specified, this option has no effect. See the section called “Creating an LXT2 dumpfile”.|
|-Wignore||Suppresses any warning output during code parsing and/or simulation.|
|-y directory||Directory to find unspecified Verilog files. You may specify this option more than once on a command-line; one for each directory required.|
|+libext+.extension[+.extension]*+||Extensions of Verilog files to allow in scoring.|
Covered is not a logic simulator and, therefore, is not required to parse the entire DUT. True simulation is performed by the Verilog simulator used to generate the VCD/LXT2/FST dumpfile. Covered, therefore, is given the ability to parse only the part of the DUT that is to be scored for coverage. Having this ability to ignore parts of the DUT has several advantages in the way of performance. Less memory is necessary for coverage simulation/ calculation. Additionally, less processing is also required which leads to faster turnaround. On the other hand, since Covered ignores the parts of the design not being scored, some information is not automatically extracted by Covered that is required for scoring the specified part of the DUT. The defparam statement is one such statement. More on this is mentioned in the following section.
The following example describes a complete DUT and how to cause Covered to score various parts of the DUT.
Assume that the following Verilog modules are used to create this tree.
module "test" instantiated as "test" in the above figure
module "foomod" instantiated as "foo"
module "barmod" instantiated as "bar"
module "amod" instantiated as "a"
module "bmod" instantiated as "b"
To get coverage for the entire design, the covered command line would be:
covered score -t test -i test (rest of command-line)
Because both the -t and -i options are the same value (the module name is the same as the instance name), only the -t option needs to be specified. If the -i option is not specified, Covered will assume that the instance name of the top module is the same name as the top module itself. Note that the -t option always needs to be specified for the score command.
To get coverage for the instance foo and all modules below it, the covered command line would be:
covered score -t foomod -i test.foo (rest of command-line)
In this case, we do not wish to score the module test (the top-level file for the design) but rather wish to gain coverage information for module foomod and all instantiated modules underneath it. Since foomod is not the top-level design file, the -i option must be specified to tell Covered what the hierarchical scope of the top-level score file is in relation to the entire design. The main reason why this is necessary is so that Covered will be able to locate the dumpfile information for the modules/instances that need to be scored.
To get coverage for only the instance foo (and none of the modules instantiated below it in the hierarchy), Covered command-line would look like the following:
covered score -t foomod -i test.foo -e amod -e bmod
Here we see the use of the -e option, it takes a module name as its value. When a module with the matching name is found in the hierarchy, it and all of its submodules are removed from coverage. You may specify as many -e arguments as necessary to select the appropriate modules to cover.
To get coverage for the instance foo and all modules below it, in addition to the instance bar, you would have to run Covered twice (once for each top instance).
The -vcd vcd_dumpfile, -lxt lxt_dumpfile or -fst fst_dumpfile parameter is simply the name of the VCD/LXT2/FST dumpfile that contains all of the module instances selected by the -t parameter (in combination with the -e option -- see below).
Besides the -e option (scope exclusion), Covered comes equipped with several other means of excluding codes from coverage consideration. The -ea, - ec, -ei and -ef exclusion options cause Covered to not simulate always blocks, continuous assignment blocks, initial blocks and final blocks, respectively, allowing for additional score command performance enhancement opportunities. These options affect all code that is under coverage consideration for the current score command, and cannot be turned on/off on an individual basis.
The -ep exclusion option allows Covered to exclude all code that has user-specified coverage pragmas embedded inline in the code being considered for coverage. By default, these pragmas look like the following:
Example 9.1. Inline Pragmas to Turn Coverage Exclusion On/Off
module foo; // coverage off ... // coverage on endmodule;
Any registers, nets, variables, blocks, expressions, FSMs or assertions found within these pragmas within the design being parsed by Covered will be excluded from coverage consideration when the -ep option is specified. If the -ep option is not specified, these pragmas are ignored by Covered.
The code that is found between these pragmas is not removed from resimulation, however. Therefore, if the user decides that any code found within these pragmas needs to be considered, they may change the exclusion properties via the GUI without needing to rerun Covered's score command. Coverage numbers in generated reports will treat excluded code as though it has been hit (i.e., the coverage points will be a part of the total number and the hit counts will be increased as though these coverage points were hit during simulation).
The coverage pragmas may use either the single-line C++ comment style (//) or the multi-line C comment style (/* */). By default, the pragma is
identified with the keyword 'coverage' following the comment characters. This value can be changed by the user, if needed, by specifying a string
value to the right of the -ep option in the score command. This can be useful if there are other programs that use the same pragma keyword and
causes conflicts with Covered's use of this pragma name. The value specified after the keyword is either 'off' (meaning that code should be
excluded from coverage consideration after this comment) or 'on' (meaning that code should be considered for coverage after this comment). The
values of '
off' and '
on' have the same meanings regardless of the pragma keyword so be sure to rename the pragma keyword
to something that is appropriate.
Coverage pragmas may be embedded within each other, if desired. That is the following code example is allowed and handled properly by Covered' parser:
Example 9.2. Example of Embedded Coverage On/Off Pragmas
module foobar; reg b, c, d, e; // coverage off wire a; // coverage off assign a = b & c; // coverage on assign e = ~d; // coverage on endmodule
The net effect of this example is that all code between the outer '
coverage off' and '
coverage on' pragmas are
excluded from coverage.
One last note about the -ep option is that coverage exclusion is restricted on a per file basis. That is, if the user specifies a
// coverage off' pragma within a file and does not specify a '
// coverage on' pragma later in the file, the next file
to be parsed by Covered will be handled as if coverage is on. Coverage pragmas can, however, span across multiple modules or other entities within
the same file.
The defparam statement can override any parameter specified in the DUT from any module in the DUT. Since parameter information is not typically specified in VCD/LXT2/FST dumpfiles, Covered needs to know about all parameter values at score time. Since all defparam statements are ignored by the Covered parser (even those in the DUT being scored), the user must inform Covered of any parameters that need to be overridden. This is accomplished with the -P option to the score command.
The first value in the -P option is the hierarchical reference to the parameter that needs to be overridden. The second value (following the '=' sign) specifies the value to assign the parameter to. This will override any default value assigned to that parameter. The user may not override the same parameter value twice on the command-line. Additionally, if the parameter is already being overridden in the design via an inline parameter override (parameter overrides specified in the '#(...)' part of the module instantiation).
As an example of how to override a parameter value on the score command-line, let's assume that there is a parameter called 'google' in the module called 'amod' used in the example in Section 9.3. The parameter definition in this module looks like the following:
parameter google = 4'hf;
To override this value to a new value of 18, use the following option in the score command:
In this example, notice that we have not only changed the value of google, but we have also changed the bit-width of google from 4 bits to 32 bits (the size of an integer). This is considered legal behavior by Covered.
It is also possible to specify non-integer values to the parameter override option. Decimal, binary, octal and hexidecimal values may also be specified, using the same syntax as they are specified in Verilog. The following examples would also be other ways to override the value to 18.
-P test.foo.a.google=5'b10010 -P test.foo.a.google=4'o22 -P test.foo.a.google=12'h12 -P test.foo.a.google='d18
In fact, you may specify any value (with the exception of real numbers) to override a parameter, including the following value types: 'x', 'X', 'z', 'Z', '_', '?'. This means that unknowns may be specified in a parameter. Note that this may generate user reported errors if parameters overridden to unknown values are used in mathematical equations or are used to specify the width of wires, regs, etc.
If a defparam statement is encountered in the DUT being scored, a warning message will be output to the user specifying that it has encountered a defparam statement. The defparam statement is ignored by Covered; however, if the parameter that it is overridding is used in the DUT being scored, the results of Covered may differ from actual simulation results. As such, please review all of these warning types to determine if the parameter should be overridden or not. If defparams are not parsed (seen by Covered), no such warning will be output. Please review the DUT for defparams which affect the DUT being scored.
At this time Covered does not have the ability to automatically extract FSM information from the design. As a result, it is up to the user to specify FSM information for Covered to use in scoring FSMs. The ability to automatically extract FSM information is planned to be supported in the future; however, it is always a good idea to allow the user to specify and potentially override Covered's automatic extraction in the case where Covered either misses an FSM or incorrectly determines logic to be an FSM in cases where none exists. The information specified in this section describes how to tell Covered where an FSM exists within the design as of the current development version of Covered.
First, we need to make a few assumptions about the type of FSMs that Covered will be able to handle and, in the future, identify.
The FSM is a synchronous design with edge-triggered flip-flops.
There are no FSMs embedded in another FSM.
The FSM is not a pipe-lined system.
The entire FSM must exist within the same module.
All state registers/variables must be less than 2^16 bits in width.
There is currently only one way, via the command-line, to specify to Covered where to find FSMs within the design. In the future, the inline Verilog-2001 "$attribute" function will be used to also allow the user to specify to Covered where an FSM is located. Using the command-line method, no changes need to be to source Verilog code for Covered's purposes.
To specify where an FSM exists within the design, the -F option must be specified to Covered's score command. Each -F option specifies one FSM within the design. Multiple -F options to the score command are allowed, allowing more than one FSM to be scored simultaneously. The -F option takes one argument that is subdivided into two or three parts:
The module specifies the Verilog module where the FSM state variables exist. The input_state_expression specifies the name of the variable(s) that represent the input state of the state machine. The output_state_expression specifies the name of the variable(s) that represent the output state of the state machine. Most FSMs have one variable as the input state (in most cases a registered version of the output state) such that the input variable is different than the output variable of the state machine. Specifying both variables in the -F option provides the user to more accurately specify to Covered the FSM. In the case that an FSM has the same variable for the input and output state, the user must specify only the output state variable (omitting the input_state_expression and comma character in the argument).
The input and output expression to the -F command can represent any signals, parts of signals, or combinations of signals. Constant values or parameters in the specification of the input/output expression is currently not supported (in fact, there are no future plans to support this behavior unless user-feedback demands it). The following examples show various legal ways to specify input/output expressions. The syntax is limited to what it shown in these examples.
Example 1 specifies that the entire vector called "a" should be considered both the input and output state variable.
Example 2 specifies that bit 2 of the vector called "a" should be considered both the input and output state variable.
Example 3 specifies that bits 3:1 of the vector called "a" should be considered both the input and output state variable.
Example 4 specifies that vectors "a" and "b" should be concatenated to form the input/output state variable. Only one set of curly brackets may be used to specify a concatenated state expression; however, 2 or more signals may be specified within those brackets.
Any combination of the above four examples can be used to formalate the input state expression or the output state expression.
If the user specifies an FSM that Covered is not able to find (either the module was not included in the design that Covered is scoring or the FSM uses Verilog code that Covered is not able to handle), an error message is displayed to the user. It is then up to the user to determine if the FSM was incorrectly specified to Covered or otherwise. This warning is not considered to be a bug in Covered (unless of course, it can be proven that the FSM should have been found).
At the current time, Covered does not have the ability to extract the states and state transitions of the specified FSM from the design. Currently, it also doesn't have the ability to specify this information in other ways. To provide meaningful coverage information for the FSM, Covered outputs all hit states and state transitions in the report when all achievable states and state transitions are not known. This information will be able to be specified by the user in future versions of Covered, with the ability to automatically extract this information also in the works.
Though Covered has support for many Verilog-1995, Verilog-2001 and SystemVerilog constructs there are handful of supported constructs that are dependent on correct simulation ordering to provide accurate coverage information. In certain cases, Covered will be able to accurately determine the order of simulation, but in some cases, it may not. In these cases, it is best for Covered to run in a "conservative" mode of operation whereby logic blocks that contain these code issues are removed from coverage consideration. Removing logic blocks does not have a negative impact (in terms of coverage accuracy) on the rest of the design. The only negative consequence is that no coverage results can be obtained for a given logic block.
For example, consider the following code:
module main; foo a(); bar b(); endmodule module foo; integer a; initial begin #10; a = $random; end endmodule module bar integer a; initial begin #5; a = $random; end endmodule
If only module "foo" is chosen to gather coverage information, the call to the $random function will result in a value that will not match the actual simulation if the -conservative option is not specified. This is due to the ordering importance of $random system task calls. The first call to $random will return a value of X while the second call will return a value of Y. If we remove the code that calls $random the first time, the code that still calls X will receive the incorrect value. This discrepancy in simulation results will result in incorrect coverage results.
If the -conservative option is specified to the score command, Covered will automatically remove any coverage blocks that make a call to the $random system function from coverage consideration and any assigned values within these removed blocks will get their value from the dumpfile instead of being calculated by Covered directly. This will result in correct coverage information for the module "foo".
The constructs that can cause issues with Covered simulation correctness (in some situations) are the following:
It is currently up to the user to determine whether liberal or conservative coverage accumulation should occur and use the -conservative option to the score command as needed.
Covered supports the $test$plusargs and $value$plusargs system function calls and, as such, runtime command-line options are allowed to be specified when scoring the design. To specify plusargs to the score command, simply specify them anywhere after the score command keyword just as you would specify them to a simulator. Any plusargs specified to the score command which are not recognized for compilation purposes are used for $test$plusargs and $value$plusargs comparisons,
Note that specifying plusargs to the score command is only necessary when the score command is given either the -vcd, -lxt or -fst options. If the VPI method of coverage scoring is used, specify the plusargs to the simulation runtime as required. Covered will automatically extract the plusargs from the simulator runtime for its purposes.
When the Verilog files are parsed, a preprocessor is run on each file that is to be read in by Covered. The preprocessor generates an intermediate form of each file, resolving defined values and included files. The intermediate form is output to a file which is called "tmpXXXXXX" where XXXXXX represents some alphanumeric sequence chosen by the OS via the mkstemp() function. The file is guaranteed to be unique to all other files in the directory and it is automatically removed by Covered when parsing is completed.
If the user does not want Covered to use this naming convention for specifying this file, the user may use the -p option (see Options section above) to override the default name. Covered will then use this filename instead of generating a unique name. Make sure that the file specified does not contain any useful data since Covered will overwrite this file when the score command is run.