Organizations demonstrate that they meet the goals for each level by producing evidence of work processes performed within key process areas (KPAs) of the individual projects and within the company. KPAs can be thought of as functional areas or offices, such as quality assurance, configuration management, or the office of system design and development. It is within the KPAs that specific guidelines, in the form of questions, are provided. When questions within each KPA at a given level can be answered in the positive, the answers validated with some form of physical output, and the personnel who produced the output can explain how the output is produced, how it is used, and what happens to it after it is produced, ratings are awarded.
The formal CMM evaluation process is conducted by auditors from outside the organization who want confirmed levels of capability in order to conduct business with the federal government. The audits are performed by people trained in assessing software development efforts that are based on the criteria spelled out in the model. Specific pieces of information, referred to as evidence, are validated for all functional areas of a project. The assessment training is provided by the SEI, which is associated with Carnegie Mellon University, in Pittsburgh, Pennsylvania.
Representatives fro m the SEI are actively promoting the concepts and methods presented in the CMM, both nationally and internationally. What was originally developed as a tool for the Department of Defense is now being used by other federal government agencies and is beginning to reach into the commercial marketplace as well.
PARALLEL POINTS BETWEEN ISO 9000 AND SEI CMM
The strongest areas in which a parallel effort may be drawn between ISO 9000-3, 9001, and CMM appear to be: peer reviews, software product engineering, software configuration management, software quality assurance, and requirements management. Practices that are more strongly addressed by the ISO quality standards than by CMM include: process change management, technology change management, defect prevention, quantitative process management, integrated software management, organization process definition, and organization process focus. It is important to note that both the ISO standards and the CMM address many additional areas wherein the relationship may be moderate to weak.
An international organization dedicated to the quality assessment process has undertaken an initiative called Software Process Improvement and Capability dEtermination (SPICE). This international organization is committed to the development of a standard for software process assessment or through the
implementation of some other means in order to support companies doing business across borders.
DOES ISO 9000 CERTIFICATION OR SEI CMM RATING
CONSTITUTE QUALITY MANAGEMENT?
An important element to keep in mind is that both ISO standards and SEI CMM are tools that an organization can use to achieve a true quality program. If either of the methodologies has been institutionalized and developed until a formal certification or rating has been achieved, the organization has been recognized by external sources as having a viable quality management program at the time of the audit. However, as previously stated, maintaining a quality management program goes beyond the institutionalization of processes. Because a good quality management program will have defined and repeatable processes, what must really be defined are the quality goals as they pertain to a company’
s specific business.
Since this is not a static environment, a company must continue to perform necessary analyses to determine what characteristics and attributes of quality are right for its products and customers as the business environment continues to evolve.
This means that the quality management program must sustain activity in all areas affected by ongoing and new development projects. Personnel at all levels should be encouraged to contribute and participate in the analysis and evaluation. The culture of the company must continue to be aware of the investment and the benefits of the program to them. The quality management program should undergo continuous improvement by updating the goals as well as the processes used to achieve the goals as the environment changes. In this way, a company is assured of having a successful, ongoing quality management program.
The inherent danger in relying on tools to accomplish this, rather than culture and commitment, lies in producing the procedures and the paperwork that allow ISO
certification or CMM rating to be granted without providing the education and integrating the processes into the corporate culture to ensure real quality. It is possible that neither of these methodologies is the right tool for a particular organization to follow to develop a quality management program, especially if industry standard practices and the customer base does not require the formal certification of the organization’
s quality by an external agency.
Chapter 12: An Almost Perfect Software Project: Using SEI Core Measurements
J.W.E. Greene
OVERVIEW
Is it possible to successfully plan and manage software development with minimal data? The Carnegie Mellon Software Engineering Institute (SEI) recommends that four core measures be made on software developments, namely, software size, time, effort, and defects.[1] Thus, the interesting question becomes whether or not software development can be done with these core measures.
The only way to prove the practicality and the benefits is to use the core measures and show the results. The background to the development set out here involves the purchasing department of a telecommunications operator (telco), which insists that all development proposals be quantified using the core measures.[2]
First, the telco checks if the proposal plan is realistic. This plan data allows a quantified baseline contract to be agreed upon. The supplier is then contractually required to provide progress data at least every month. The progress data is used to evaluate and report progress. The goal is to ensure that delivery of the full function is on time and within budget, and that the software is delivered with high reliability.
Naturally, suppliers are motivated to get the telco’
s business, and hence to supply
the data on the plans and progress. The core data allows the telco to quantitatively assess each supplier proposal. These measures complement the SEI’
s Capability
Maturity Model (CMM) Maturity Levels,[3] which are also used to by the telco to assess the qualitative factors in the supplier’
s development process.
In the development described here, it was the first time the supplier had been requested to provide the plan data using the core measures. In particular, the requirement to estimate the expected size range of software was completely new.
It is worth noting a recent report dealing with software purchasing to understand why purchasers of software development should be motivated to use the core measures.[4] This report is a highly critical evaluation of the software purchasing competence of the U.S. Federal Aviation Administration (FAA). The report sets out how the FAA is exposed commercially if it does not get and use core data.
[1 ]Carleton, A.D., Park, R.E., and Goethert, W.B., “The SEI core measures,” The Journal of the Quality Assurance Institute, July 1994.
[2 ]Kempff, G.W., “Managing Software Acquisition,” Managing System Development, July 1998.
[3 ]Humphrey, W.S., “Three Dimensions of Process Improvement. Part 1: Process Maturity,” CROSSTALK The Journal of Defense Software Engineering, February 1998.
[4 ]GAO Report to the Secretary of Transportation: Air Traffic Control GAO/AIMD-97-20.
THE PERFECT PROJECT PLAN DATA
Before the contract is awarded, the supplier is required to estimate the size range of the software to be developed. The size range is expressed in logical input statements (LIS; i.e., what is to be written by the team), and estimated as the minimum, most likely, and maximum values.[5] This takes into account the uncertainties in the requirement specification by estimating the size range of each software module. The supplier did this based on the 18 modules identified for development. The result is shown in Exhibit 1.
Exhibit 1. Module Size Range Estimate Data
SIZE ESTIMATE IN LOGICAL INPUT STATEMENTS (LIS)
Complete for all All Modified and New Modules Estimated
Modules
Size Range Modified + New LIS
Module ID
Least
Most Likely
Most
1
MOD 1
2000
2500
4000
2
MOD 2
2000
3000
6000
3
MOD 3
2800
3000
4000
4
MOD 4
1000
1200
1500
5
MOD 5
2000
3000
4000
6
MOD 6
2000
2000
3000
7
MOD 7
800
1000
1200
8
MOD 8
2000
3000
6000
9
MOD 9
1500
2000
2500
10
MOD 10
1000
2000
3000
11
MOD 11
500
1000
1500
12
MOD 12
2500
3000
4000
13
MOD 13
500
1000
1500
14
MOD 14
300
500
800
15
MOD 15
500
1000
1500
16
MOD 16
500
1000
1500
17
MOD 17
500
1000
1500
18
MOD 18
500
1000
2000
TOTAL
22900
32200
49500
The proposed development staffing plan for 13 months is provided and shown in Exhibit 2.
Exhibit 2. The Software Main Build (MB) Time and Effort
Using the Basic Measures to Evaluate the Plan
The core plan data of size, time, and effort allows comparison against industry reference measures available for different application types.[6] In this case, the comparison is made against telecommunication (telecom) developments, and the plan can be confirmed as realistic and within the bounds of known industry values.