University of Calgary Logo
Department of Electrical and Computer Engineering
 

Software Reliability and Software Quality
(SENG 521)
Course Outline

()
Course Brochure
This course is also available as an industrial course of the Lecture Series on Software Systems for The Future. Please click on the above image for the 2 pages course brochure (PDF format) and contact the series editor for reservation. You can purchase the course CDROM online. The CDROM includes the most recent version of all the slides, handouts and other deliverables.

Instructor: Dr. Behrouz H. Far (Associate Professor, Schulich School of Engineering)
   
Course/Year: Software Engineering Program (SENG)/ 4th Year
   
Department: Electrical and Computer Engineering
   
School Year: 2012 (Fall)
   
Timetable: Lecture: Mon., Wed., Fri., (11:00-11:50)
Lab: Mon. 16:00-17:50
   
Room: Lectures: EEEL 349 (3rd Floor Environment Building)
Labs: ICT 217 (2nd floor, ICT Building)
   
Background Courses: SENG 437 or 421
   
Lecture Format: Lecture (3x1 hour sessions per week, total 36 hours) +
Lab (1x2 hours session per week, total 24 hours)
   
Instructor's Contact: Office: ICT 543
Tel. (403) 210-5411
email. _far@ucalgary.ca
   
Teaching Assistants:
Emad Mohammad (eamohamm@ucalgary.ca)
Note: For discussion related to the lab assignments, TAs and the course instructor can be reached either in the Lab (ICT 217) or at their desks during the official lab hours only. For contacts other than lab hours please use email or arrange an appointment in advance.
   
Course Outline: Course Outline (PDF format)


Announcements
  1. Nothing yet!


1. Course Description and Outline

SENG521 is a step by step description of software quality and software reliability engineering process. It includes introduction to software quality, prediction and measurement of software size and cost, software reliability engineering process, defining necessary reliability, developing operational profiles, decision making based on the test results, techniques to improve and predict software reliability, application of quality concept to agile and incremental software development processes. The focus is on the reliability of object-oriented software systems. A workshop (project) is designed to reinforce the presented material. In the workshop, the students will actually go through the estimation and evaluation of quality of a realistic software project.

2. Course Web Site

The SENG 521 course home page contains links to up-to-date course information, problem assignments, announcements, as well as laboratory and examination scheduling. The SENG 521 course home page is available through the B.H. Farfs home page at the URL:
(http://www.enel.ucalgary.ca/People/far/Lectures/SENG521/)

3. Allocation of Marks

Criteria Total mark Other info
Midterm examination 15%
Laboratory Reports and Quizes 50% See Assignments schedule below.
Final examination 35% The final examination will be scheduled by the Registrar's Office at a time in the two-week period following the end of classes in December. Note: The timetable for Registrar Scheduled exams can be found at the University's Enrolment Services website, http://www.ucalgary.ca/registrar/.
The final examination covers all the distributed materials and takes about 3 hours to complete.
Note: It is necessary to earn a passing grade of at least 50% on the final exam in order to pass the course. It is also necessary to submit all Laboratory Reports to pass the course.

4. Problem Assignments and Laboratories

Regular problem assignments will be given out of the course textbook, but are not required to be handed in for marking.

During the course the students are grouped into teams composed of 2-3 members. Each team is asked to design a moderately realistic software project and test it. A list of projects that the students can select among them will be posted on the course web page. The laboratories reports (design and test documents) of various phases of the assigned project should be handed in for check and marking. The reports are reviewed and a group discussion will be held in the tutorial periods. Unlike many undergraduate courses, and similar to what is actually performed in industrial practice, the laboratories hours are for group discussion and review of the projects. Additional discussion and review hours can be arranged by the students, TAs and the instructor.

5. Details of The Assignments

There are 5 assignments throughout the SENG 521 course each counting for %20 of the total marks for assignments.
The goal of the group assignment is to walk through software estimation and the software reliability engineering process. As for the group assignments, the students are divided into teams composed of 2-3 members. Each team selects a project (see list of projects and deadlines below) and estimates, designs and tests it following the SRE guidelines. The concentration will be on software estimation and interpretation of the test data rather than coding and testing of the project.
Each team should prepare a journal that the estimations, analysis, design and test documents will be added to it gradually. This helps communication among team members as well as reviews by the Instructor and TAs.
  1. Unlike many other undergraduate courses and similar to industrial practices, the laboratory hours are not for delivering tutorials or solving problems. The designated lab hours are for group discussion and review of the projects. The meetings for review and discussion can be held either in the designated lab space or outside.
  2. The designated lab hours are the core time that the TAs and course Instructor are available to review the projects. It is the students' responsibility to arrange meetings and seek advice from them during these hours. Meeting other than the core time is not guaranteed and requires prior arrangement with the TAs or Instructor and is subject to their availability. Attending the review sessions and discussions during the lab hours is necessary and failing to attend will cost each team/member about 10% deduction of the total mark for the corresponding assignment.
  3. Assignments are to be delivered by their respective due dates. The due date for each assignment is explicitly mentioned on the assignments handouts. Unexcused delayed submission or failing to submit the required deliverables will be treated as zero. Students must submit ALL assignments to pass the course.
  4. For group assignment one copy for the group will be sufficient.
  5. The language for the deliverables must be English with ASCII or ISO-8859-1 character set.
  6. The deliverable pages should be numbered and each assignment should have a cover page bearing the assignment number, student(s) name and student(s) ID number(s), and submission date. The cover page is excluded from the minimum and maximum required pages.
  7. Assignments must be delivered in both electronic and hardcopy form. In case of electronic submission only formatted documents (MS Word, HTML, Postscript or PDF) are acceptable. Using other word processor software is encouraged but the output should be saved or converted to one of the above mentioned formats. Unformatted text or LaTeX files should be compiled or converted to Postscript or PDF before submission. HTML formatted files are acceptable only if accompanied by all the links and inline graphics, etc., in a single file (i.e., web archive).
  8. Electronic version of assignments must be uploaded to the course Blackboard page at the due date together with the printed ones. Notice that electronic assignments WILL NOT be accepted if delivered via email.
  9. For electronic submissions, the student is responsible to have the electronic media made readable. Defective material will be rejected and marked zero.
  10. For electronic submissions, the student is responsible to have the files and attachments scanned for viruses before submission. Virus infected deliverables will be deleted automatically and marked zero.
  11. Inquiries about the assignments can be forwarded to either TA or the course instructor. Email inquiries are acceptable, provided that the keyword SENG521 and the assignment number (ASSIGNMENT no. 1 to 5), e.g.,
    SENG 521: ASSIGNMENT no. 1
    be mentioned in the Subject line of the message. Inquiries not complying with this will be discarded with no action.
Metrics for Student Assessment The students will be evaluated along with 7 metrics:
  • Motivation
  • Learning
  • Interest and progress
  • Retention
  • Ability degree
  • Attention
  • Effort

Assignment no. Submission Deadline Other info
1 05/Oct./2012
(Friday)
4:30 PM
Software Project Size Workshop
(Group assignment)
2 22/Oct./2012
(Monday)
4:30 PM
Software Project Time and Effort Workshop
(Group assignment)
3 05/Nov./2012
(Monday)
4:30 PM
Defining Necessary Reliability Workshop
(Group assignment)
4 19/Nov./2012
(Monday)
4:30 PM
Developing Operational Profiles Workshop
(Group assignment)
5 3/Dec./2012
(Monday)
4:30 PM
Prepare, Execute Test and Release Workshop
(Group assignment)
Documents Assignments outline (PDF format)
Assignments (PDF format)


6. Detail Contents

Regular Sessions
Chapter 1 Overview of software reliability engineering (2 sessions)
  • Introducing the course.
  • What is software reliability?
  • What factors affect software quality?
  • What is software reliability engineering?
Chapter 2 Software Size (3 sessions)
  • Software size
  • Size: Length (code, specification, design)
  • Size: Reuse
  • Size: Functionality (function point, feature point, object point, use-case point)
  • Size: Complexity
Chapter 3 Software Cost and Effort (3 sessions)
  • Software cost model
  • COCOMO and COCOMO II
  • Constraint model
  • Software lifecycle management (SLIM)
  • Cost models: advantages and drawbacks
Chapter 4 Software Quality (2 sessions)
  • Software quality
  • Software quality models: Boehmfs model, McCallfs model, ISO 9126 model, etc.
  • Software quality standards and metrics
  • Measuring customer satisfaction
  • Software quality assurance (SQA)
  • Software quality management
Chapter 5 Overview of Software Reliability Engineering (1 session)
  • Hardware and software reliability
  • The SRE process
Chapter 6 Review of software Reliability Models (2 sessions)
  • Basic Features of the Software Reliability Models.
  • Single Failure Model.
  • Reliability Growth Model.
  • Exponential Failure Class Models.
  • Weibull and Gamma Failure Class Models.
  • Infinite Failure Category Models.
  • Bayesian Models.
  • Early Life-Cycle Prediction Models.
Chapter 7 Defining necessary reliability (3 sessions)
  • Introducing failure severity class and failure intensity objective concepts.
  • Steps in defining necessary reliability.
  • Computing failure intensity objective for developed software.
  • Case studies.
Chapter 8 System reliability (2 sessions)
  • System reliability
  • Reliability Block Diagram (RBD)
  • Serial and parallel configuration
  • Active redundancy
  • Hazard analysis: FMEA, FTA
Review
week
Review and midterm exam (2 sessions)
Chapter 9 Strategies to meet reliability objective (2 sessions)
  • Fault prevention strategy: software product and process improvement using ISO 9000-3.
  • Introducing fault tolerance concepts and definitions. Coincident, correlated and dependent faults.
  • Fault tolerance phases.
  • Recovery block mechanism and Acceptance testing.
  • Exception handling, expected and unexpected events.
  • Construction of robust software systems.
  • Defensive programming, dual software technique.
  • Adjudication by voting.
  • Recovery blocks, N-version programming, Consensus recovery block, Acceptance voting, N self-checking programming.
Chapter 10 Developing operational profiles (2 sessions)
  • Defining function, operation, run, run type, operational mode, operational profile.
  • Representation of operational profile.
  • Procedure to define operational profile.
  • Create functions/operations list.
  • Determine occurrence rate of individual operations.
  • Determine occurrence probabilities.
Chapter 11 Preparing and executing test (3 sessions)
  • Direct and indirect input variables.
  • Operation, load and regression test.
  • What is a test case?
  • How to manage test cases?
  • Test procedure.
  • Equivalence classes and boundary conditions.
  • How to allocate test time among system components based on test type (feature test, regression test, load test) and operation modes?
  • In what order the test should be carried on?
  • Running a successful test
  • How to document the execution results?
Chapter 12 Software reliability tools (2 sessions)
  • Introducing SRE simulation and reliability growth tools such as: SMERFS, SRMP, SoftRel, CASRE.
  • Case study using CASRE.
Chapter 13 Applying failure data to guide decisions (3 sessions)
  • Guiding decisions related to certification test.
  • Guiding decisions related to reliability growth test.
  • Guiding decisions related to adequacy of tests.
  • Handling program evolution, ureported failures and variations of operational profiles.
Chapter 14 Deploying software reliability engineering (2 sessions)
  • Software Quality System (SQS); Software Quality Assurance (SQA) and Software Reliability Engineering (SRE)
  • Quality, test and data plans
  • Roles and responsibilities
  • Sample quality and test plan
  • Defect reporting procedure
Chapter 15 Advanced topics (3 sessions)
  • SRE for Agile development
  • Cleanroom software development
Course
Review
Review and final exam (1 session)
  • Final review of the course (5/Dec./2012)
  • Final examination TBA.

7. Textbooks and Suggested References

Reference Materials: The course handouts are the major source of information. The handouts, slides and extra reading materials will be distributed on each session. All the handouts and other materials are also downloadable from the course web page. For those with limited access to the internet, a CDROM containing all the documents will be distributed on demand.
   
Recommended Text and Reference Books:
   
 
   
 
   
 
  • Software Reliability: Measurement, Prediction and Application, J.D. Musa, A. Iannini, K. Okumoto, (621 p.), McGraw-Hill (1987).
    ISBN 0-07-044093-X.
   
Here is a list of other textbooks that may be found useful:
  1. Addison-Wesley - A Practical Guide to Testing Object Oriented Software (2001).
  2. Addison-Wesley - Critical Testing Processes Plan Prepare Perform Perfect (2003).
  3. Addison-Wesley - Effective Software Testing (2002).
  4. Addison-Wesley - Software Test Automation (1999).
  5. Addison-Wesley - Test Driven Development By Example (2002).
  6. Addison-Wesley - Testing Embedded Software (2003).
  7. Addison-Wesley - Testing Extreme Programming (2002).
  8. Apress - Software Exorcism A Handbook for Debugging and Optimizing Legacy Code (2003).
  9. Apress - Test Driven Development A J2EE Example (2005).
  10. Artech House - Building Reliable Component based Software Systems (2002).
  11. Artech House - ISO 9001 2000 Quality Management System Design (2003).
  12. Artech House - Practical Guide To Software Quality Management 2Ed (2003).
  13. Artech House - Software Fault Tolerance Techniques and Implementation (2001).
  14. Artech House - Testing and Quality Assurance for Component Based Software (2003).
  15. CMU TR022-96 - Cleanroom Software Engineering Reference Model.
  16. CRC Press - Design for Reliability (2001).
  17. Elsevier Science - Reliability Maintainability and Risk (2001).
  18. IEEE - Preventive Risk Management For Software Projects.
  19. IEEE - SWEBOK Software Engineering.
  20. IEEE 1008-1987 - Standard for Software Unit Testing (1986).
  21. Kluwer - Computing System Reliability Models and Analysis (2004).
  22. Manning - JUnit Recipes Practical Methods for Programmer Testing (2005).
  23. Microsoft Press - Testing Software Patterns (2003).
  24. Morgan Kaufmann - Unit Testing in Java How Tests Drive the Code (2003).
  25. NIST - The Economic Impacts of Inadequate Infrastructure for Software Testing (2002).
  26. OReilly - Perl Testing A Developers Notebook (2005).
  27. Pragmatic Programmers - Pragmatic Unit Testing in C# with Nunit (2004).
  28. Pragmatic Programmers - Pragmatic Unit Testing in Java With Junit (2003).
  29. Pragmatic Programmers - Pragmatic Unit Testing in Java with JUnit (2003).
  30. Prentice-Hall - Java Testing And Design From Unit Testing To Automated Web Tests (2004).
  31. Prentice-Hall - Java Testing Design and Automation (2004).
  32. Prentice-Hall - Software Development Building Reliable Systems (1999).
  33. Springer Verlog - Practical Software Testing (2003).
  34. Springer Verlog - Reliability Engineering Theory and Practice 3Ed (1999).
  35. Thomson - Game Testing All in One (2005).
  36. Wiley - Automated Web Testing Toolkit - Expert Methods For Testing and Managing Web Applications (2001).
  37. Wiley - Case Studies in Reliability and Maintenance (2003).
  38. Wiley - Optimal Reliability Modeling Principles and Applications (2003).
  39. Wiley - Reliability Survivability and Quality of Large Scale Telecommunication Systems (2003).
  40. Wiley - Reliability of Computer Systems and Networks - Fault Tolerance Analysis and Design (2002).
  41. Wiley - Software Quality Engineering - Testing, Quality Assurance and Quantifiable Improvement (2005).
  42. Wiley - Software Testing Fundamentals Methods and Metrics (2003).
  43. Wiley - Statistical Design and Analysis of Experiments 2Ed (2003).
  44. Wiley - Statistical Methods for Survival Data Analysis 3Ed (2003).
  45. Wiley - The Art of Software Testing 2Ed (2004).
  46. Wiley - The Web Testing Companion (2003).


8. Other Information

Related Links:


All the slides and notes can be viewed on-line using Netscape Navigator or MSIE (version 3.x or later) browsers. Copies of the slides in Portable Document Format (PDF) is available for on-line download. Please note that the course downloadable materials are provided solely for the internal use for the registered students of this course. External and industrial participants may contact the author for availability of the materials.
All Unix, PC and MAC users can download, view and print the PDF version of the documents using Adobe's Acrobat Reader.


This page was created by Dr. B.H. Far.If you may find omissions, glitches, have suggestions for improvement of the material presented here, please contact me.
  Copyright Terms. THIS DOCUMENT AND ITS ENTIRE CONTENTS ARE COPYRIGHT 2001 BY DR. B.H. FAR. COPYING, REPUBLISHING AND DISTRIBUTING THIS DOCUMENT IN WHOLE OR IN PART IS PROHIBITED BY LAW. IF YOU DESIRE TO REPUBLISH PARTS OF THIS DOCUMENT IN ELECTRONIC FORM, PLEASE CONTACT THE AUTHOR.