SimCon - Fortran Analysis, Engineering & Migration About UsDownloadsPricing Home | SimCom logo
$keywords!
WinFPT - Fortran Program Metrics and Code Quality Indices
Consultancy
Migration & Software QA
 
WinFPT & FPT
WinFPT
FPT Summary
 
Error Correction
Hosts
Metrics
Migration
Optimization/Optimisation
QA
Reports
Security
Standardisation  
Updates & Release notes
$keywords!
WinFPT & FPT - Special Issues
Checking Expressions
Continuation Lines
Datapool
File Handling
File Names
Hex Numbers
Program Control
Record/Replay
Tab Format
Tracing Execution
 
Gould-SEL
VMS & Unix
 
Associated Companies
IQS Solutions
 
 

How do you measure the quality of a Fortran code?

How do you assess the workload in maintaining a code - the cost of ownership?

How do you assess the risk in using and maintaining a code?

 

Metrics of Fortran Code Quality - Quality from Counting Errors

FPT and WinFPT generate a simple measure of the quality of code based on the number of errors and warnings generated per thousand lines of code. Some errors and warnings are always generated when code is analysed. Some are generated only in response to specific tests such as the argument check and name check. It is best to write a script of standard tests which will be used at a site to test the code quality. For example:


! millibug.fsp			13-Sep-02		John Christopherson
! *****************************************************************************
!
% check alignment
!
! Ignore distinction between e.g. REAL X and REAL*4 X
% argument check to ignore default sizes
% check arguments
!
% check data in COMMON blocks
% check data in structures
!
% check expressions
!
% equivalence check to ignore default sizes
% check equivalence
!
% check names
!
% check program flow
!
% check usage
!
! End of millibug.fsp ! *******************************************************

FPT and WinFPT report the index of quality in the list file at the end of processing. For example, the report for GNX, an old VAX/VMS program, is:


Diagnostic Summary

          Total diagnostics 590        Highest severity    2

          Notes                          168
          Warnings                       273
          Errors                         149

Code Quality Index

          Error weighting                 10
          Total lines                  10017
          Index per 1000 lines       176.001

The index per 1000 lines, the "millibug rating" is computed here as

1000 * (10 * Number_of_Errors + Number_of_Warnings) / Number_of_lines

The weighting factor for errors may be changed by the user. It is probably best written in the configuration file, config.fsp in the main installation directory, for example:


Diagnostic Summary

          Total diagnostics 590        Highest severity    2

          Notes                          168
          Warnings                       273
          Errors                         149

Code Quality Index

          Error weighting                 10
          Total lines                  10017
          Index per 1000 lines       176.001

The listing also shows the contributions made by different types of disgnostic to the overall quality index. The table for GNX, for example, shows:


Contributions to Code Quality Index

 No.  Description                                                 Count Contrib

1269 FORTRAN keyword used as local identifier name.                     2     2
1273 FORTRAN auxiliary keyword used as identifier name.                 4     0
1281 FORTRAN intrinsic used as an array name                            1     0
1459 Fortran keyword EXIT used, presumably to call system service       1     1
1585 Arguments inconsistent with sub-program declaration.              85   850
1887 Arguments inconsistent with other calls.                           3    30
1967 Unable to determine value of a PARAMETER.                          5     0
2241 References are made to sub-programs which have not been read.      1     0
2315 The file has been referenced elsewhere by a different name        24     0
2517 ANSI FORTRAN 77 intrinsic used as a local identifier               2     2
2847 Multiple initialisation of COMMON block                            1     1
2849 DATA for COMMON outside BLOCK DATA                                 1     1
3047 Fortran 90 intrinsic name used as an identifier                    2     2
3073 Danger of overflow due to different integer sizes                229   229
3235 Symbol is read but never written to                               61   610
3237 Symbol is written to but never read                                2     2
3239 Symbol is declared but never used                                128     0
3241 Name is used inconsistently in different contexts                  1     1
3243 Object is mis-aligned in memory                                   14    14
3247 Mixed EQUIVALENCE - Objects of different type or size             18    18
3291 String shorter than parameter in parameter assignment              5     0

Our experience is that any program with a millibug rating greater than 10 is unlikely to run without problems. The rating of 176 for GNX is alarming. The table of contributions shows that most of the problems come from mis-matched arguments and unassigned variables. This was a VAX program, where arguments with size mis-matches were still likely to be handled correctly and all unassigned variables were set to zero. It would not port well to a PC!

 

Metrics of Fortran Code Quality - Quality of Comments

FPT and WinFPT generate a measure of the number of comments in the code. This forms part of the metrics report, for example:


                                  Main Code      System       Total
Code and comments
  Declaration lines                    2817           0        2817
  Executable lines                     3629           0        3629
  Total code lines                     6446           0        6446

  Comment text lines                   2019           0        2019
  Comment separator lines               221           0         221
  Blank lines                          1331           0        1331
  Total comment lines                  3571           0        3571

  Total lines                         10017           0       10017

  Trailing comments                     540           0         540
  Words in comments                   13001           0       13001

Comment lines are classified as:
  • Blank lines, containing nothing, or nothing but the comment delimiter
  • Separator lines, containing a bar of "-", "=", "*" or similar characters across a line
  • Text lines, with separate words
A count is also made of the number of trailing comments, delimited by "!", "(*" or "//" symbols.

Probably the best measure of the level of commentary is the count of the number of separate words in the comment text lines and the trailing comments.

 

Metrics of Fortran Code Quality - Quality of Variable Names
Code written with short and meaningless variable names can be almost unmaintainable. It is difficult to write a tool to measure the quality of variable names, but two issues have been identified:
  • Names visible in multiple sub-programs should be unique. They should not also be used locally for different objects.
  • Names should be sufficiently long to be meaningful.
WinFPT and FPT count the number of names visible in multiple routines which are also used locally, and measure the mean lengths of the names visible in multiple routines. These are, for example, module, function, subroutine and COMMON block names, and the names of objects which are public in modules or in COMMON blocks.

The measures are written in the metrics report, for example:


Usage of Names for Objects Other than Local Variables
  Names used for non-locals             864
  % used inconsistently               0.694
  Mean length (Characters)            5.231

In the code analysed, only 0.694% of the names visible in multiple routines (exatly one name) was also used for a local variable. This is very good. The non-local names were on average only 5.231 characters long. This is not really long enough for a reliable naming convention or for the names to be fully meaningful. It is still very much better than some of the commercial codes which we have seen!

 

Measuring the Workload of Maintaining Fortran Codes

There are three issues:

  • The code quality - described above
     
  • The size of the code
     
  • The complexity of the code
WinFPT and FPT report the size of the code as part of the code metrics report, for example:


Program Metrics
===============

                                  Main Code      System       Total
Files
  Primary files                           8           0           8
  Include files                           7           0           7

Code and comments
  Declaration lines                    2817           0        2817
  Executable lines                     3629           0        3629
  Total code lines                     6446           0        6446


The cyclomatic complexity is a measure of the number of branches in the code. It is reported for the entire code, for example:


                                  Main Code      System       Total
Program flow and cyclomatic complexity
  Normal path code nodes               1785           0        1785
  Exception code nodes                   30           0          30
  Total nodes                          1815           0        1815

  Normal path code edges               2562           0        2562
  Exception code edges                  218           0         218
  Total code edges                     2780           0        2780

  Executable sub-programs                10           0          10

  Normal path code complexity           777           0         777
  Total code complexity                 965           0         965

  Complexity per sub-program         96.500                  96.500

Code which handles exceptions, and code in system routines (if included in the WinFPT analysis) are identified separately in the reports.

The program analysed is a pre-compiler which contains several switches (Computed GOTO statements) and many handlers for specialised constructs. It is therefore very complex. A mean cyclomatic complexity of 10 to 20 per sub-program is more usual.

The cyclomatic complexities of the individual sub-programs are also reported, for example:


Cyclomatic Complexities
=======================
                                           Nodes       Edges    Complexity

   ERRPAR                                     29          43          14
   ERRSCL                                     21          33          12
   GNX                                         9          13           4
   GNXEND                                     76         124          48
   GNXINI                                     70          91          21
   GNXPAR                                   1096        1688         592
   GNXREA                                     35          53          18
   GNXREL                                    335         521         186
   GNXSDA                                    124         189          65
   GNXTXT                                     20          25           5

This is usefule in identifying hot-spots in maintenance. In the code analysed the routine GNXPAR is the parser. This has several 128-way switch lists on input character used to recognise and process variable names and numbers. The high complexity is a cause for concern!

 

Risk Assessment of Fortran Codes

You inherit a code from another company, another department or from a legacy environment.

What is the risk in using and maintaining it?

There are two issues:

  • The risk of using the code without changing it depends on its quality. Use the quality index above, but note that any changes you make to the code incur maintenance risks
     
  • The risk in maintaining the code depends on its size and complexity, but above all else on its readability and the clarity of the comments. The authors believe that the measures of meaningfulness of the variable names and the word count of the comments are probably the best available measures.

     

Copyright ©1995 to 2008 Software Validation Ltd. All rights reserved.