Software Metrics
(SENG 421)
Course Outline

• Nariman Mani
  (nmani@ucalgary.ca)

1. Sept/2/2008: The midterm exam is scheduled for: October 24, 2008 (Fri), 11:00 AM-12:00 NOON, ICT 116.
2. Oct/23/2008: The final exam is scheduled for: December 12, 2008 (Fri), 12:00-15:00, Room TBA.

1. Course Description and Outline

This course is a step by step description of the software metrics. It includes introduction to foundations of measurement theory, models of software engineering measurement, software products metrics, software process metrics and measuring management. The course is composed of the following basic modules:

• Measurement theory (overview of software metrics, basics of measurement theory, goal-based framework for software measurement, empirical investigation in software engineering)
• Software product and process measurements (measuring internal product attributes: size and structure, measuring external product attributes: quality, measuring cost and effort, measuring software reliability, software test metrics, object-oriented metrics)
• Measurement management

2. Course Web Site

The SENG 421 course home page contains links to up-to-date course information, problem assignments, announcements, as well as lab and examination scheduling. The SENG 421 course home page is available through the B.H. Far's home page at the URL:
(http://www.enel.ucalgary.ca/People/far/Lectures/SENG421/)

3. Allocation of Marks

1. Chapter 1, 2, 3, 4
2. Chapter 7, 8, 9
3. Chapter 10 (10.1 only)
4. Chapter 11 (11.1 to 11.3 only)
5. Chapter 12 (12.1 to 12.4 only)
6. All handouts distributed in the class

4. Problem Assignments and Labs

Regular problem assignments will be given out of the course textbook, but are not required to be handed in for marking.

A list of projects will be posted on the course WWW page. The lab reports should be handed in for check and marking. Some of the reports may need online submission and marking using the WWW interface pages specially designed for this course. The reports are reviewed and a group discussion will be held in the lab hours.
Please drop your printed and electronic version of report in the designated dropbox in front of the lab room (SENG 421 dropbox, 2nd floor ICT building).

5. Details of The Assignments

A good starting point article is: Stan Rifkin, What makes measuring software so hard? (also in PDF form) IEEE Software, May/June 2001, Vol. 18, No. 3, pp. 41-45.

The following archive is quite useful:

• Thomas Fetcke's Software Metrics Sites

For the second part of the assignment, you must come up with a crisp suggestion that the manager will base her/his decision on it. Your suggestion may be such that you cannot recommend one because ....
Using models, graphs and/or tables to back up the argument are considered better than reporting in narrative style. Some useful hints are:

• Defining the scales for the attributes that are going to be compared, mapping the measured (given) values to the scales and discuss the properties of the scales.
• Defining a model for the requirement assessment from both the manager and user viewpoints.
• Issues with goal, quality and compatibility of the parameters.
• Defining a quality model based on the attributes and trying to map the measured (given) values to the model.
• Using Reliability Demonstration Chart to assess the acceptance/rejection of the system.

Assignment no. 2 and 3 (combined)

One of the difficulties that you may face is that each of the GQ(I)M steps (specially steps 6-10) may require details that may only be available in a real situation. It may give the students too much freedom to assume the details and proceed. You may want to discuss some of the assumptions with the instructor or TAs in the review lab sessions.

You should start with a business goal (Step 1), which is the title of the project you've selected. In Step 2 (identify what we want to know) try to ask 5-6 questions for the entities of interest (i.e., inputs and resources; internal artefacts; activities and flows; products and by-products) and then group the questions that address a common entity to convert them to a few (about 3-4) subgoals (Step 3). In order to convert the subgoals to measurement goals you need to define entities and attributes for each question listed for each subgoal. This may be a long and repetitive task. In Step 5 (formalizing measurement goals) you must define the Purpose-Perspective-Environment and Constraints trio for each object of interest. There may be 10-20 or more of the objects of interest, depending on the project. In Step 6 (Identifying Quantifiable Questions and Indicators) you may derive questions for all measurement goals but proceeding to the indicators for only one of the measurement goals will be sufficient. In Step 7 you should prepare a cross-reference checklist for the data elements to be collected and the indicators identified in step 6. In Step 8 you formalize the measures by defining scale, range and precision for all the entities and prepare the definition checklist for them. Step 9 is about analysis, diagnosis and actions. You may need to consider the actions only for new projects. Finally, in Step 10 you must fill in the measurement plan template that will serve as your final report for this project together with the output documents for each step of the GQIM process.
Note that the average expansion rate for goal to subgoals and subgoals to measurement goals is around 4-5. This means that an initial business goal may be associated with 16-20 measurement goals. Expansions more than the avearge may make the project hard to manage.

There are a few sample project reports available for the students to review and find out what a project report may include:

1. The Tardiness Factor: A Plan for Improving Project Timelines and Product Delivery

Assignment 2-3 self assessment page (60K, MS word format). Please use it when you submit the report in electronic form.

Assignment no. 4

The overall objective is to determine adjusted function point count for a software system. There are several steps necessary to accomplish this. The actual sequence or order of steps is not necessary.

1. Determine type of function point count
2. Determine the application boundary
3. Identify and rate transactional function types to determine their contribution to the unadjusted function point count.
4. Identify and rate data function types to determine their contribution to the unadjusted function point count.
5. Determine the value adjustment factor (VAF)
6. Calculate the adjusted function point count.

The final function point count (adjusted function point count) is a combination of both unadjusted function point count (UFP) and the general system characteristics (GSC’s).

You may proceed as follows:

1. Start with defining the requirements for your project. Note that you should proceed to the extent that the detailed requirements are sufficient to measure the function points.
2. Read relevant sections of the Function Points Analysis Training Manual (110 Pages) (by David Longstreet, David@SoftwareMetrics.com). A copy of this document is also downloadable from course web page. The document is a step-by-step guide to measure FP. Follow the steps mentioned there. Many of the concepts can only be comprehended and decided through the discussion among the team members.

Assignment no. 5

• Original binary for COCOMO II, manuals and documets for COCOMO Project
  
• USC COCOMO II.1999.0 Software: Windows platforms (3.26mb) (the Windows version is self-extracting program that installs USC COCOMO II.1999.0 and Excel Analyzer Tool which imports into Excel a CSV data file exported from COCOMO II.)
  
• COCOMO Manual
  
• COCOMO II Model Manual
  
• COCOMO II User Manual
  

1. Application Composition model is for the early phases. At this stage usually you only have the requirements for the system yet to be built. The Application Composition model will let you calculate the effort using object-points by estimating the number of screens and reports from the specification.
2. Early Design model is for exploring architectural alternatives or incremental development strategies. The Early Design model is used to evaluate alternative software/system architectures and concepts of operation. An unadjusted function point count (UFC) is typically used for sizing. With the choice of programming language, this value is converted automatically to KLOC. However, it is possible to directly input the LOC or effects of adapting an already existing code. An important step here is to define the specification of the inputs, ouputs and internal files to derive the function point (UFC) for each program module. Another important task is to identify the values of 7 effort adjustment factors (EAF) for each module.
3. Post-Architecture model is for more accurate cost and effort estimates. The Post-Architecture model can be used during the actual development and maintenance of a product. The Post-Architecture model includes a set of 17 cost drivers and a set of 5 scale factors determining the projects scaling component. The cost drivers and scale factors can be calculated using the COCOMO II tool in the same way as in the Early Design Model.

1. Project Estimation: Online Bookshop Detailed report.

6. The Mid and Final Examinations

The midterm examination is a one hour exam and will cover the material up to the date of the examination. It covers approximately the chapters 1-4 and 7-8 of the course recommended textbook plus any other handouts. The exact contents will be announced in the class and on the course Web page. The exact time and location will be announced in the class and on the course's WWW page well in advance.

Calculators are permitted in all examinations and quizzes. The examinations and quizzes are all closed-book and closed-notes. The students do not need to memorize the relationships or formulae. They will be given whenever required.

7. Detail Contents

• Introducing the course.
• What is software measurement?
• What are software metrics?

• Metrology
• Property-oriented measurement
• Meaningfulness in measurement
• Measurement quality
• Measurement process
• Scale
• Measurement validation
• Object-oriented measurement
• Subject-domain-oriented measurement

• Software measure classification
• Goal-based paradigms: Goal-Question-Metrics (GQM) and Goal-Question-Indicator-Metrics (GQIM)
• Applications of GQM and GQIM
• Case studies

• Software engineering investigation
• Investigation principles
• Investigation techniques
• Formal experiments: Planning
• Formal experiments: Principles
• Formal experiments: Types
• Formal experiments: Selection
• Guidelines for empirical research

• Software size
• Software Size: Length (code, specification, design)
• Software Size: Reuse
• Software Size: Functionality (function point, feature point, object point, use-case point)
• Software Size: Complexity

• Software structural measurement
• Control-flow structure
• Cyclomatic complexity
• Data flow and data structure attributes
• Architectural measurement

• Midterm review session: (Wed, October 22, 2008)
  
• Midterm examination (Fall 2008) (Fri, October 24, 2008)
  

• Software cost model
• COCOMO and COCOMO II
• Constraint model
• Software Lifecycle Management (SLIM)
• Cost models: advantages and drawbacks

• Software quality
• Software quality models: Boehm's model, McCall's model, ISO 9126 model, etc.
• Basic software quality metrics
• Quality management models
• Measuring customer satisfaction
• Software Quality Assurance (SQA)

• Reliability concepts and definitions
• Software reliability models and metrics
• Fundamentals of software reliability engineering (SRE)
• Reliability management models

• Test concepts, definitions and techniques
• Estimating number of test case
• Allocating test times
• Decisions based on testing
• Test coverage measurement
• Software testability measurement
• Remaining defects measurement

• Object-Oriented measurement concepts
• Basic metrics for OO systems
• OO analysis and design metrics
• Metrics for productivity measurement
• Metrics for OO software quality
• Experience-based guidelines

• Final review session: (Dec 5th, 2008)
  
• Final examination (Fall 2008): December 12, 2008 (Fri), 12:00-15:00
  

8. Textbooks and Suggested References

• Software Metrics: A Rigorous and Practical Approach, (2nd ed.) (638p.), N.E. Fenton and S.L. Pfleeger, PWS Publishing, 1998.
  ISBN 0-534-95425-1.

• Metrics and Models in Software Quality Engineering, Stephen H. Kan, 2nd ed. (560 p.), Addison-Wesley Professional (2002).
  ISBN: 0201729156.

• Software Engineering Measurement, John C. Munson, Auerbach Publications, 2003 (443 pages)
  ISBN:0849315034

• Software Metrics: Measurement for Software Process Improvement, BA Kitchenham, Blackwell Pub, 1996.
  ISBN: 1855548208.

• Applied Software Measurement: Assuring Productivity and Quality, C. Jones, McGraw-Hill, 1996.

• Software Metrics: A Guide to Planning, Analysis, and Application, C. Ravindranath Pandian, Auerbach Publications, CRC Press Company, 2004.

• Software Engineer's Reference Book, J. McDermid (Edt.), Butterworth Heinemann, 1993.
  ISBN 0-7506-0813-7.

9. Other Information

• International Function Point (FP) user group
• The IIT Software Engineering Group, Institute for Information Technology, National Research Council, Canada
• Software Engineering Management Research Laboratory (University of Quebec, Montreal).
  
• Thomas Fetcke's Software Metrics Sites.
  
• Norman Fenton's WWW page.
  
• Tools and Solutions for Software Development.
  
• Cost and Effort Estimation (by M. Shaw, U of C).
• COCOMO II Model Definition Manual (Boehm, et al,1997).
• The TQM Journal (ISSN: 0954-478X)
• TQM and ISO 9000 Web-based Information
• Christopher Lott's collection of metric tools for C and C++
• Quantitative Software Management (SLIM Products)

1. Albrecht, A.J.,
   Measuring application Development Productivity proc. IBM Application Development Joint SHARE/GUIDE Symposium, pp. 83-92, (1979).
2. Albrecht, A.J. and Gaffney, J.F.,
   Software Function, Source Lines of Code and Development Effort Prediction: A Software Science Validation, IEEE Trans. Software Engineering, vol. 9, no.6, pp.639-648, (1983).
3. The ami Handbook: A Quantitative Approach to Software Management
   London, England: The ami Consortium, South Bank Polytechnic, 1992.
4. Armitage, J.W. and Kellner, M.I.,
   A Conceptual Schema for Process Definitions and Models, 53-165. Proceedings of the 3rd International Conference on the Software Process, Oct. 10-11, 1994. IEEE Computer Society Press, 1994.
5. Basili, V.R. and Weiss D.,
   A Methodology for Collecting Valid Software Engineering Data, IEEE Transactions on Software Engineering, vol. 10, pp.728-738, 1984.
6. Basili, V.R. and Rombach, H.D.,
   The TAME Project: Towards Improvement-Oriented Software Environments, IEEE Transactions on Software Engineering, vol. 14, no. 6, 758-773, 1988.
7. Briand, L., Morasca, S., Basili, V.,
   Property-Based Software Engineering Measurement, IEEE Transactions on Software Engineering, vol. 22, no. 1, 1996.
8. Chidamber, S.R., Darcy, D.P., Kemerer, C.F.,
   Managerial Use of Metrics for Object-Oriented Software: An Exploratory Analysis, IEEE Transactions on Software Engineering, vol. 24, no. 8, August 1998.
9. Chidamber, S.R., Kemerer, C.F.,
   A Metrics Suite for Object Oriented Design, IEEE Transactions on Software Engineering, vol. 20, pp. 476-493, 1994.
10. Caws, P.,
    Definition and Measurement in Physics, Measurement: Definitions and Theories, C. West Churchman and Philburn Ratoosh, ed., pp. 3-17, John Wiley and Sons, Inc., 1959.
11. Churchman, C.W.,
    Why Measure?, Measurement: Definitions and Theories, C. West Churchman and Philburn Ratoosh, ed., pp. 83-94, John Wiley and Sons, Inc., 1959.
12. Fenton, N.E.,
    Software Metrics: A Rigorous Approach, Chapman and Hall, 1991.
13. Fenton, N.E. and Whitty, R.,
    Introduction, pp. 1-19. Software Quality Assurance and Measurement, A Worldwide Perspective, Norman Fenton, Robin Whitty, and Yoshinori Iizuka, ed., pp. 1-19, International Thomson Computer Press, 1995.
14. Ghiselli, E.E.; Campbell, J.P.; and Zedeck, S.,
    Measurement Theory for the Behavioral Sciences, W. H. Freeman and Company, 1981.
15. Humphrey, W.S.,
    Managing the Software Process, Addison-Wesley, 1989.
16. Jones, C.,
    Applied Software Measurement, McGraw-Hill, (1996).
17. Kan, S.H.,
    Metrics and Models in Software Quality Engineering, Addison-Wesley, 1995.
18. Kirchner, P.,
    Measurements and Management Decisions, Measurement: Definitions and Theories, C. West Churchman and Philburn Ratoosh, ed., pp.64-89, John Wiley and Sons, Inc., 1959.
19. McCabe, T.J.,
    A Complexity Measure, IEEE Transactions on Software Engineering, vol.2, no.4, pp. 308-320, 1976.
20. Rombach, H.D. and Ulery, B.T.,
    Improving Software Maintenance Through Measurement, Proceedings of the IEEE, vol. 77, no. 4, 581-595, 1989.
21. Shepperd, M. and Ince, D.,
    Derivation and Validation of Software Metrics, Clarendon Press, 1993.
22. Stevens, S.S.,
    On the Theory of Scales of Measurement, Science, vol. 103, no. 2684, 677-680, 1946.
23. Stevens, S.S.,
    Mathematics, Measurement, and Psychophysics, Handbook of Experimental Psychology, S. S. Stevens, ed., pp. 1-49, John Wiley and Sons, Inc., 1951.
24. Weinberg, G.M.,
    Quality Software Management, Vol. 2: First-Order Measurement, Dorset House Publishing, 1993.
25. Wiener, N.,
    A New Theory of Measurement: A Study in the Logic of Mathematics, Proceedings of London Mathematical Society, vol. 2, no. 19, pp.181-205, 1920.
26. Weyuker, E.J.,
    Evaluating Software Complexity Measures, IEEE Transactions on Software Engineering, vol. 14, no. 9, 1988.
27. Zuse, H.,
    Software Complexity: Measures and Methods, Walter de Gruyter, 1991.

1. IEEE Software.
   
2. IEEE Transactions on Software Engineering.
3. IEE Proceedings - Software.
4. Transactions on Software Engineering and Methodology (TOSEM), ACM.
   
5. Information and Software Technology, Elsevier Science.
6. Annals of Software Engineering, Kluwer.
7. Automated Software Engineering, Kluwer.
8. Empircal Software Engineering Journal, Kluwer.
   
9. Software Practice and Experience, Wiley.
10. Journal of Software Maintenance, Wiley.
11. International Journal of Software Engineering and Knowledge Engineering, World Scientific.