The rank command provides several mechanisms for specifying CDD files to rank. Feel free to use any combination of methods for providing this information to Covered as it makes sense to do in your environment. The following is the list of input methods:

The rank command has several options that allow the user to weight any and all of the coverage metrics that Covered supplies. Weights are used to ascribe the importance of its metric in ranking the value of a CDD file.

Weight values are non-negative numbers whose values are relative to each other. A value of 0 effectively disables the associated weight from consideration in ranking. All other values are considered relative to each other. In other words, if the line weight is set to a value of 5 while the toggle weight is set to 1, line coverage is considered 5 times more important than toggle coverage in ranking.

By default, each metric is given an equal weight of 1. Adjust each of the metrics as necessary to potentially modify which CDD files are chosen and what order those CDD files are given in the ranked list of diagnostics to run.

After the rank command has been executed, the output will either be displayed to standard output (if no -o option was specified) or it will be output to a file. If the -names_only option was specified, the output will be a list of CDD filenames (one per line) which are output in the order in which they should be run to achieve full coverage in the shortest amount of time. If the -names_only option was not specified, the output will be in the form of a table (similar to reports generated with the report command) which describes not only the order that CDD files should be run but also various statistical data and the names of CDD files which are considered redundant in terms of coverage information. The following example shows an example of this report output.

                                           ::::::::::::::::::::::::::::::::::::::::::::::::::::
                                           ::                                                ::
                                           ::     Covered -- Simulation Ranked Run Order     ::
                                           ::                                                ::
                                           ::::::::::::::::::::::::::::::::::::::::::::::::::::



* Reduced 3 CDD files down to 2 needed to maintain coverage ( 33% reduction,   1.5x improvement)
* Reduced 9 timesteps down to 6 needed to maintain coverage ( 33% reduction,   1.5x improvement)

-----------+-------------------------------------------+----------------------------------------------------------
           |                ACCUMULATIVE               |                             CDD
Simulation |-------------------------------------------+----------------------------------------------------------
Order      |        Hit /      Total     %   Timesteps |  R  Name               Hit /      Total     %   Timesteps
-----------+-------------------------------------------+----------------------------------------------------------
         
         1            9           17   53%           3       rank1c.cdd           9           17   53%           3 
         2           13           17   76%           6       rank1b.cdd           8           17   47%           3 

---------------------------------------  The following CDD files add no additional coverage  ----------------------------------------------

         3           13           17   76%           9       rank1a.cdd           8           17   47%           3

        

After the header information, there is a summary of the ranking reduction (if one occurred) specified in both number of files and number of simulation timesteps. Example 12.2, “Rank Output With Required Files” shows what this information looks like when no file reduction has occurred.

After the summary information, there is the rank output table which displays both the individual CDD information and the accumulative coverage information. Each row of the table contains one CDD file where the first row contains the first CDD that should be run in regression for maximum coverage impact. Each successive row contains CDD files that add less and less coverage to the overall coverage gained. If there were CDD files that were found to not add any coverage information to the above CDD files, a line is drawn and these CDD files displayed below that line. These CDD files can be excluded from regression because they don't add any useful coverage information over-and-above the CDD files that are selected to be run in regression.

The first major column is the simulation order. This value increments by one for each successive row. If you would like to create a shorter running regression that is limited to a certain number of simulations, this row information can be useful for figuring out which simulations to include in these type of regressions.

The second major column is the accumulative coverage statistics which include the following:

The third major column is the individual coverage information for the CDD file listed in the row which includes the following information:

                                           ::::::::::::::::::::::::::::::::::::::::::::::::::::
                                           ::                                                ::
                                           ::     Covered -- Simulation Ranked Run Order     ::
                                           ::                                                ::
                                           ::::::::::::::::::::::::::::::::::::::::::::::::::::



No reduction occurred 

-----------+-------------------------------------------+------------------------------------------------------------
           |                ACCUMULATIVE               |                              CDD
Simulation |-------------------------------------------+------------------------------------------------------------
Order      |        Hit /      Total     %   Timesteps |  R  Name                 Hit /      Total     %   Timesteps
-----------+-------------------------------------------+------------------------------------------------------------

         1            9           17   53%           3       rank1.1c.cdd           9           17   53%           3
         2           13           17   76%           6    *  rank1.1b.cdd           8           17   47%           3
         3           13           17   76%           9    *  rank1.1a.cdd           8           17   47%           3

        

This section describes the overall algorithm that is used to rank the specified CDD files. The algorithm is being presented in this document for the purposes of providing an explanation for how diagnostics are selected for ranking and how they are ranked. Knowing this information is not necessary for using the rank command or understanding its results.

The algorithm makes use of three different types of structures. The first structure is a bit-compressed structure where each bit corresponds to one coverage point in the associated CDD file. The purpose of this structure is to store a large amount of information as concisely and immediately available as possible. In addition to coverage point information, each structure also contains the following information:

Each of these structures are maintained in an array that is iterated on during the ranking process.

The second structure is an array of 16-bit values where each value corresponds to a single coverage point. Its value is a sum of the number of unranked CDD files that have hit that coverage point. Each value is decremented by one when a CDD file is moved from the unranked list to the ranked list and that CDD file hits the given coverage point. If a coverage point value is a value of 1 and a CDD file has its corresponding coverage bit set, we know that the CDD is the only one that hits the coverage point.

The third structure is similar to the second and works similarly, except that it stores the number of CDD files in the ranked list that hit a particular coverage point.

The specified CDD files are ranked in a series of steps, some of which are reiterated for each CDD that is selected to be ranked. Here is the basic algorithm (in pseudo-code):

      
  calculate the size of the compressed coverage point structure
  
  foreach CDD in CDD_list {
    convert CDD to compressed coverage point structure
  }
  
  foreach CDD in CDD_list {
    if this CDD contains uniquely hit coverage points {
      move the CDD file into the ranked list
    }
  }
  
  foreach CDD in unranked list {
    foreach CDD in unranked list {
      calculate score (((total_num_cov_points_hit / num_timesteps) * 100) * metric_weight_value)
      if any unranked elements are less than the -depth option value and are hit by this CDD {
        set unique_found to true
      }
      if our score > highest score and a "unique" coverage point was found {
        set highest_score to our score
      }
    }
    move the CDD file indexed by highest_score from the unranked list to the ranked list
  }
  
  resort ranked list from most unique to least unique (if there is a tie, select the CDD that hit the most coverage
    points in the least amount of time)