The Process Advisor Assessment Model
The Process Advisor assessment model enables self-directed assessment for those organizations that want to begin software engineering technology transition activities without incurring substantial initial expense. Unlike the SEI assessment questionnaire (which contains only Boolean questions), the process advisor model incorporates qualitative, quantitative, and Boolean questions. The qualitative and quantitative assessment questions follow the structure discussed earlier in this chapter. Responses to these questions are assessed using a quasi-expert system that is built into the model. Each response to the questionnaire is compared with a set of typical responses. The quasi-expert system provides a set of inferences that help an organization to develop findings and recommendations based on the response.
Boolean questions address eight process attributes: organizational policies, training, software development process, quality assurance, project management, software engineering methods, computer-aided software engineering tools, and software metrics and measurement. Responses to the Boolean questions generate process
attribute grades for each of the eight attributes. These form a process maturity footprint for a software organization.
The process maturity footprint (Exhibit 2) indicates the relative strengths and weaknesses in the process attribute areas and enables an organization to compare itself with common software engineering practice and best practice, which is the level of software engineering practice found in the top 10 to 15 percent of all software development organizations. Process advisor also provides guidance for establishing an education strategy, creating technology selection and justification models, and building an effective transition plan.
Exhibit 2. A Process Attribute Footprint
PROCESS MATURITY
The majority of assessment models (including the two described in the preceding sections) enable an organization to compute its process maturity. Management must then know how to use this number.
All too often, a senior manager decides that a specific process maturity level should become an organization goal. That is, an organization that currently has a process maturity of 1.0 is chartered with becoming a 3.0 organization within 24 months.
Although there is nothing inherently wrong with setting process maturity goals, focusing solely on improving the process maturity value misses the point. The goals of every software development organization should be to improve the quality of the applications it builds, to satisfy its customers and users, and to accomplish work on time. Improving process ma turity helps in achieving these goals, but it should not become the goal.
In general, process maturity (and process attribute grades) should be used in the following ways:
§ To target areas of strength and weakness
§ To raise management consciousness
§ To define areas in which further investigation (e.g., assessment meetings with relevant staff) may be needed
§ To provide a comparison to industry common and best practice
§ To serve as a baseline for reassessment later in the transition life cycle Using process maturity in these ways, an organization can establish a foundation on which the technology transition plan is built.
DEVELOPING FINDINGS AND RECOMMENDATIONS THAT
IMPROVE PROCESS MATURITY
Findings and recommendations are derived from the results of the assessment.
However, it is sometimes difficult to interpret the assessment results in a manner that leads to pragmatic recommendations for change. A self-directed assessment approach must provide a set of inference-based guidelines that are tied to different maturity levels for each of the process attributes under assessment. Once the assessment has been completed, the maturity grade for each process attribute is determined. The grade range provides a solid indication of both findings and recommendations. To illustrate inference-based guidelines, the sample findings and recommendations are reproduced from the Process Advisor workbook.
THE SOFTWARE DEVELOPMENT PROCESS
Questions in the software engineering process section of the questionnaire focus on standards as a way to determine whether an organization has codified its approach.
Examine the grade and place it in the context of the grade ranges: Grade Range
Identifier
Below 1.65
E
1.65 to 2.25
D
2.26 to 2.75
C
2.76 to 3.25
B
Above 3.26
A
Interpretation:
§ Grades E and D — It is unlikely that the organization has developed a written description of its process or defined a process in any explicit manner.
o
Action: The organization should create a skeletal framework for software engineering, that is, a set of activities, deliverables,
milestones, and QA actions that can be applied as software is being developed. A description of the framework is then needed to solicit comments and recommendations from managers and technical staff.
Over time, the framework should be reworked and more detail added until it evolves into a standard.
§ Grades C and B — The organization has codified many of the activities associated with software development. It is likely that the same approach is applied across different projects and that project planning, control, and software quality assurance are easier to achieve as a result. However, just because standards exist does not mean that the process is effective or properly characterized.
o
Action: Each of the standards should be reviewed to determine whether it reflects modern software engineering practice and whether there are aspects that can be streamlined or that do not work well.
Time should be spent polling development staff to determine whether the standards are being used as widely as these grade ranges imply.
Specific technical areas without standards can be determined by reviewing responses to individual questions. It may be worthwhile to develop a framework approach for a specific technical area (e.g., testing) in a manner similar to that described in the action paragraph for E and D ranges.
QUALITY ASSURANCE ACTIVITIES
Questions in the QA activities section of the process assessment questionnaire explore the emphasis of an organization on documentation, reviews, and other QA functions. Examine the grade and place it in the context of the grade ranges (listed in the previous section). If one or more of the subsection grades are significantly different from the overall section grade, further investigation into that area is warranted.
Interpretation:
§ Grades E and D — Software quality and the activities needed to ensure it are not a primary focus within the software development organization.
Documentation is probably weak, because there are no standard formats to guide developers. Effective reviews are not being conducted and the results of reviews are not applied to improve the process. Software quality assurance is not a formally defined activity.
o
Action: The organization should develop a plan to improve documentation, reviews, and software quality assurance. Beginning with documents and reviews, the first action is to pick one or two documents and develop a standard format (being brief is best) and then develop a set of review guidelines for them. Over time, the actions can be broadened until most important documents are defined, are being produced, and are being reviewed.
§ Grade C — The organizational approach to predictable documentation, effective reviews, and basic quality assurance activities is coming together.
o
Action: The organization should review responses to each of the subsections to determine which areas need the most improvement.
Quality assurance functions are likely to need further imp rovement; if
so, focus on establishing mechanisms for ensuring compliance with documentation and process standards. The organization might also broaden its review approach, if this can be done cost-effectively. At the same time, computer-aided software engineering tools should be employed to create effective documentation in a more productive manner.
§ Grade B — The organization is at the state of practice in the QA area.
However, it may not be using quantitative data to analyze the software engineering process.
o
Action: One idea to consider is a fledgling program in statistical QA for software. By first collecting data on defects uncovered through other QA activities, the organization can work to improve methods to reduce defects. It can then acquire tools that will enable the organization to build quality software more effectively.
MANAGING PROCESS ASSESSMENT
The process assessment may be conducted by an internal consulting organization, by outsiders, or in a self-directed fashion. Regardless of the approach that is chosen, the assessment must be coordinated by IS management.
It is vital that management prepare an organization for the process assessment activity before the activity begins. The intent of the assessment, and the benefits to be derived from it, should be communicated to technical managers and staff. The types of assessment data to be collected and the use of assessment data should be thoroughly explained. The results of the assessment should be shared with all participants. All involved must understand that the purpose of the assessment is to establish a basis for process improvement, not to punish or judge technical proficiency.
RECOMMENDED COURSE OF ACTION
If an organization is committed to software development process improvement, its first step is to assess the current state of software engineering practice. The IS
manager is advised to:
§ Set the stage for process improvement by providing preliminary training in software engineering for managers, technical staff, and users. Training should introduce process and technology options, emphasize the benefits of improved software quality, and explain the technology transition cycle.
§ Select a local champion to manage the assessment approach from the inside.