The vision of Watts Humphrey, Capability Maturity Models were developed to assess the capabilities of organizations in taking up large development projects of Department of Defense, USA. SW-CMM addresses the process improvement in software development organizations. SW-CMM identifies a set of guidelines that need to be implemented for producing quality software. The SW-CMM framework has been accepted internationally as a very comprehensive framework for quality processes implementation.
Software capability is defined as “range of expected results that can be achieved by following a software process”. The actual performance achieved by following these processes is known as Software Process Performance. “The extent to which a process is explicitly defined, managed, measured, controlled and effective” is defined as Software Process Maturity [Reference: The Capability Maturity Model: Guidelines for Improving the Software Process, CMU-SEI, Addison-Wesley, 1994].
Fig. 1 TQM and CMM
In organizations which develop both hardware and software, Total Quality Management (TQM) has to be followed. As shown in Fig. 1, CMM addresses the management of quality of software aspects of the project i.e., CMM is confined to software quality management of the organization whereas TQM addresses both hardware and software quality management.
Levels of Software Process Maturity
Based on the software process maturity, an organization can be at one of the 5 levels. The levels are shown in Fig. 2.
|
|
Fig. 2 Levels of Software Process Maturity in Capability Maturity Model
Level 1: Initial
Organizations at level 1 do not have any defined process. Such organizations are characterized by fire fighting, i.e., everything is done at the last minute resulting in chaos. However, organizations at this level may be able to produce good results, mainly due to the ‘heroics’ of one or more people in the organization. Success solely is dependent on one or few highly competent people and if they quit the organization, the organization may collapse.
Level 2: Repeatable
In level 2 organizations, software development is done through a disciplined approach. To execute a project, a set of processes is defined and followed. So, level 2 organizations are characterized by “defined processes” for execution of projects.
Level 3: Defined
Level 3 organizations define a standard development process that is applicable across the organization. For a specific project, a development process is defined which is tailored from the organization’s standard process. “Standard consistent processes” are used to execute the projects and hence consistent results can be expected from organizations at this level. The processes are qualitative, and hence it is likely that the performance depends on the individuals.
Level 4: Managed
Organizations at this level use quantitative measures for various processes and hence it is easy to measure the effectiveness of the process resulting in predictability. Quality of software is also managed using quantitative measures.
Level 5: Optimizing
Organizations at this level define the processes for continuous improvement. Each of the projects are analyzed to find the scope for improvement and through a feedback mechanism, the management takes steps to continuously improve the processes resulting in continuous improvement in quality.
CMM Architecture
CMM proposes a stage-by- stage improvement of the process quality. For each level, Key Process Areas (KPAs) are defined. The KPAs at each level are given in Table I.
An organization, interested in following CMM, has to take up KPAs at level 2 and implement them, go on to the third level KPAs and so on. It is not advisable to skip the levels.
For each KPA, the following template is defined:
Goals: the goals to be achieved by implementing the KPA
Commitment to perform: To implement the KPA, commitments of the management are given in this section. These are the pre-requisites for effective implementation of the KPA.
Activities performed: The activities to be carried out for the implementation of the KPA.
Measurement and analysis: The metrics to be used for quantification of the process.
Verifying implementation: Steps to be taken to ensure that the KPA is being implemented.
|
CMM |
Level |
Key Process Areas |
|
Level |
1 : Initial |
|
|
Level |
2 : Repeatable |
Requirements management Software project planning Software project tracking and oversight Software subcontract management Software quality assurance Software configuration management |
|
Level |
3: Defined |
Organization process focus Organization process definition Training program Integrated software management Software product Engineering Inter-group coordination Peer reviews |
|
Level |
4: Managed |
Quantitative process definition Software quality management |
|
Level |
5: Optimizing |
Defect prevention Technology change management Process change management |
Table I. Capability Maturity Model: Levels and Key Process Areas Each of these Key Process Areas is described below. Level 2 KPAs Requirements Management
Most software projects get into trouble because the development work starts without having full understanding of the user requirements. This KPA ensures that a formal understanding between client and software developer is reached. A document “Software Requirements Specifications” is prepared which the client and developer sign. Since it is not feasible to ‘freeze’ the SRS, a baseline SRS is prepared and any changes to this document have to be done through a defined procedure and the changes have to be intimated to all the persons working on the project. Planning the software project is done based on the SRS. Along with SRS, acceptance test plan is prepared which is the basis for testing the software. SRS is reviewed periodically and metrics such as number of changes proposed/accepted are recorded.
Software Project Planning
A project plan document is prepared for every project covering the goals, objectives, estimates for size, time, effort, cost, risk items, life cycle model to be used, standards to be followed for quality assurance, configuration management, software tools to be used etc. All changes made to the plan are also recorded. The plan is reviewed periodically and any deviations are analyzed and contingency plan is worked out. Metrics such as estimates and actual values for time, effort, size and cost are recorded.
Project Tracking and Oversight
Using the software development plan as the basis, periodic reviews are conducted for each project, and if there are any changes to commitments (such as delivery date), these changes are communicated to the persons concerned (the client or marketing team). Likely changes in cost, size and effort are also tracked and corrective action is taken. Metrics such as percentage change in estimates versus actuals are recorded.
Software Subcontract Management
When a portion or complete software is given to a subcontractor for development, it has to be ensured that the subcontractor develops quality software. The prime contractor must have a documented procedure for identifying the subcontractor. This procedure should specify the criteria for selection of subcontractor such as technical expertise, past experience, staff, quality standards being followed etc. After selecting the subcontractor, a written contractual agreement must be signed. This agreement should clearly specify the deliverables, time schedule, quality standards, test plan etc. The management as well as Quality Assurance (QA) group of the prime contractor must review the agreement periodically.
Software Quality Assurance
A separate Quality Assurance (QA) team has to be formed and the QA manager should be independent of the development team. A software QA plan is prepared during the project planning stage itself and this plan is used to test the software product. The QA team has to ensure that the software has been developed as per the QA plan and any defect found by the QA team has to be informed to the development team for corrective action. Software QA activities are reviewed periodically by the senior management. An expert group (preferably, external to the organization) has to audit the QA activities periodically.
Software Configuration Management
A Software Configuration Control Board (SCCB) has to be established, which has to approve all the changes to be made to the various work products such as SRS, design document, test plan etc. A software configuration management plan is prepared in which configuration items are identified. The configuration items are SRS, design document,
QA plan, development plan, source code, test plan etc. If any change has to be made to any of the work products, an Engineering Change Request (ECR) has to be submitted to the SCCB in a specified format. The SCCB will study the effect of the likely change on the schedule, cost etc. and approve or reject the proposed change. If the change is accepted, it is communicated to all the project team members through an Engineering Change Note (ECN). All the changes have to be documented. The Software Configuration Management activities are reviewed periodically.
Level 3 KPAs
At level 3, organization-wide processes are defined for software development. For each project to be executed, a process is defined which is based on the organization-wide processes. In other words, the project-specific process is a tailored version of the organization-wide process.
Organization Process Focus
A separate group is established to define, modify and manage the organization-wide processes for software development. Steps are taken to assess the effectiveness of these processes and make improvements. All employees are trained in the processes to be used across the organization.
Organization Process Definition
A standard software process for the organization is developed and maintained. This process should cover the entire software development life cycle and the various life cycle models that can be used for individual projects. Guidelines for tailoring this process for individual projects are also developed.
Training Program
A training group is formed which will identify the training needs of the employees in development tools, application domain and software engineering standards. All the employees will be imparted training in the identified areas and training records are maintained which describe the effectiveness of the training.
Integrated Software Management
This KPA ensures that for every project, a tailored version of the organization’s software process is used. The software development plan for each project should indicate how this tailoring has been done and should indicate the life cycle model selected, estimation techniques selected, design methodology selected etc.
Software Product Engineering
The necessary tools for software development (such as compilers, configuration management tools, documentation tools) are identified. Methodologies for requirements engineering (object-oriented, functional, prototype) are identified. Design methodologies (object-oriented, structural, prototyping) are also identified. Code is developed according to a defined process. Testing is performed according to a defined process and test strategies (functional, structural) are worked out. Defined processes are followed for integration testing, system testing and acceptance testing, documentation and collection of data for defects.
Inter-group Coordination
This KPA ensures that various groups associated with software development (client, development team, quality assurance team, configuration management team etc.) resolve all the issues of common concern. These groups meet periodically and review the activities. Issues not resolved are handled according to a documented procedure.
Peer Reviews
Peer reviews (reviews by development engineers at the same level in the hierarchy of the organization) are very effective to resolve most of the technical issues. Peer reviews are conducted periodically and results are recorded.
Level 4 KPAs
At this level, emphasis is on quantifying the effectiveness of various processes used in the software development.
Quantitative Process Management
A separate group to manage the quantitative process management activities is formed whose responsibility is to collect, record and analyze the data for various processes and suggest process improvements. A plan is prepared for Quantitative Process Management and the group carries out the activities according to the plan. The QA group audits the activities of this group.
Software Quality Management
To manage quality, it is necessary to measure quality. Software quality management ensures that for every project, a software quality plan is available in which metrics are identified to measure the quality. The quantitative goals (such as reliability, availability) are documented and compared with actual results. Issues such as cost of poor quality (for example, increase in the maintenance costs) are addressed.
Core Competency in Software Process Management Level 5 KPAs
At this level, the KPAs address continuous process improvement. Defect Prevention
At organization level, a team is formed which addresses the defect prevention activities. The team coordinates with different project teams. At the beginning of each task of a project, the members work out the strategies required to prevent defects. Analysis of causes for defects is carried out and necessary steps are taken to ensure that the causes for defects are eliminated/reduced. Necessary changes to the various processes are carried out to eliminate the causes for defects.
Technology Change Management
For an organization to survive and grow, it is necessary to adapt the latest technology. To manage the changes in technology is very crucial and it is to be planned. A group responsible for Technology Change Management is formed which evaluates new technologies and identifies the technologies that can be inducted in the organization. Pilot projects are implemented, if necessary, to evaluate and select new technologies. If the pilot projects indicate substantial benefits to the organization, these technologies are incorporated into the organization’s process.
Process Change Management
To prevent defects and to induct new technologies based on the lessons learned due to past mistakes, the organization processes need to be changed periodically. This KPA addresses this activity. Process Change Management is to be carried out according to a documented procedure. The group responsible for software process activities coordinates this activity also. Proposals received for process improvements are reviewed, the effects of the changes in processes are studied through pilot projects and the changes to the processes are informed to all the employees.
Implementation of CMM
Organizations, which do not have any standard defined processes for software development, need to start with level 2 KPAs and perform each of the activities for all the KPAs. All the employees need to be trained on software engineering principles and CMM. Each individual has to actively participate in the implementation of CMM.
For each KPA,
• There must be a written policy indicating the commitment of the management.
• The management must ensure that necessary infrastructure, funds and training
are provided to the team members.
• The management should ensure that there is a review process in place for its
effective implementation and also verify the implementation periodically.
• Measurements have to be taken to quantify the process and based on the
measurements, the process effectiveness has to be evaluated.
It is advisable to have an organization structure with clear-cut responsibilities. For effective implementation of CMM, every organization should have
• A development group responsible for the development of the software, with a
project manager who is responsible for day-to-day management of the project
and is the decision-making authority on technical issues.
• Software engineering process group responsible for management of the
software processes.
• Software test group responsible for testing the system. It is advisable to have
an independent group responsible for testing.
• Software QA group to carry out the quality assurance activities. In some
organizations, the software engineering process group is also responsible for
QA activities.
• Software configuration management group which will carry out the
configuration management activities.
• Training group to train the staff in processes, development tools and
application domain.
•
Shortcomings of CMM
The CMM is an outcome of Department of Defense (of USA) initiative to evaluate software subcontractors for large defense projects to be executed at large software development organizations. So, critics of CMM argue that this model cannot be directly applied to small organizations executing small or medium size software projects. Contrary to these arguments, even small organizations will be benefited by implementing KPAs at levels 2 and 3. However, CMM in its present form still has some deficiencies, and the CMMI (Capability Maturity Model Integration) is a substantially improved model.
Consider an organization which has implemented 5 of the 6 KPAs at level 2 and 6 of the 7 KPAs at level 3. It will be assessed as a level 1 organization, which is unfair. This is due to grading on a scale of 1 to 5 which is very simplistic. Also, it is difficult to measure the effectiveness of KPAs at level 4 and level 5 as there are very few organizations in the world which have been assessed at level 4 and level 5. In some organizations, different divisions may be at different levels—one division may be at level 1 and another at level 3. Because of these issues, CMM is considered as ‘too simplistic’ when it comes to grading the organizations. It is important to note that the level 1 or 2 or 4 is not the important issue, whether the processes are in place and whether they are being followed and continuously improved are the issues.
Capability Maturity Model for Software (SW-CMM) Page 10 of 16
Core Competency in Software Process Management
ELELTECH
In software engineering, risk management has become a key area of focus because of the complexity of the software projects and the risk associated with them. However, in CMM, risk management is not a KPA. Installation and delivery of the software are also not addressed by CMM.
More than these limitations, it has been felt by many small organizations that CMM results in too much overhead when applied to small projects involving a few persons. To apply the framework of CMM for small projects, SEI developed Personal Software Process (PSP) which addresses the individual.
The field of capability maturity assessment is still immature, however CMM is a giant leap forward in objectively assessing an organization for its capability maturity in delivering quality software.
Process Change Management
A quality conscious organization must realize that the process defined for software development is not a static document that cannot be modified. Continuous process improvement—whereby the organization will improve its profitability, the individuals will improve their productivity, the technological developments can be inducted into the organization- is a must for every vibrant organization.
The quality models and standards, ISO 9000 or CMM, are also aimed at a continuously improving organization’s capabilities. To continuously improve, one has to continuously learn. Such learning organizations will grow with changing times.
To change the process for betterment, the project manager has to define a process for process change management. Changes to the defined process should not be done on ad hoc basis, but using formal procedures. For instance, one engineer may come up with an idea of a new way of project estimation. He may claim that it is more accurate than the existing method. To prove the effectiveness of the new model, the model has to be studied along with the old model for at least a few projects and once the effectiveness is ascertained, then it can be introduced for all the other projects. Such a systematic way of process change management would help in ensuring that no errors are introduced in the process.
Inputs for Process Change
The whole process of software development is reflected in the project plan document. In organizations which are at CMM level 3 or 4 or 5, there will be an organization-wide process definition from which the specific project plan is derived. In small organizations, for each project, a project plan is prepared based on its own process. In either case, based on the experience of executing the projects, the process definition can be improved. However, the process change has to be done systematically to ensure that the change does not have a negative impact.
The inputs for the process change are:
• Any event that triggered a change in the project plan.
For instance, the project is getting delayed and the time frame needs to be changed. The old estimate is based on the assumption that the time required for testing is about 25% of the total project duration. In the present project, the 25% of the time turns out to be very less, at least 40% of the time is required. This event is an input to the process change. However, wrong conclusions should not be arrived at. This particular project needed higher time for testing, perhaps because it is an embedded software development work. So, though the event triggered a process change, the actual change should be made based on a more detailed study and also taking into consideration inputs from other groups.
• The postmortem analysis report
Once the project is completed, the lessons learned during the project execution can provide important inputs to the process change. The estimated values versus actual values would give the inputs as regards the process of estimation of project duration and effort, productivity values, defect estimations and actual values, reliability figures etc.
• Inputs from the literature
In some cases, well-researched software engineering practices are published in literature. An excellent example is [Feature articles on Software Engineering in the Small, IEEE Computer, Vol. 32, No. 10, October 1999] which gives the process implementation in small organizations. As small organizations cannot afford the overheads of ISO 9000 and CMM and also because the projects may not need such detailed process standards, scaled-down versions of these standards can be implemented. Certainly, such well-researched work can be tried out. However, tailoring of the process to one’s requirement rather than the blind following is required.
• Technology inputs
Due to technological advancements, new products are released into the market which help do software engineering better. A large number of tools are available for analysis, design, automatic code generation, test case generation and testing, configuration management etc. Using such tools will greatly enhance the productivity of the people and the organization.
Core Competency in Software Process Management
Introduction of the new technologies and tools should not be done based on hear-say and the hype created in the marketing briefs and media. Each technology or tool has to be evaluated by conducting a pilot study and its impact on the process; the pros and cons of the new technology need to be studied; the additional investment involved in training the engineers is to be looked into. Then, these inputs can be translated into the process definition.
Measuring Impact of Process Change
The process change is reflected in the following parameters:
• The accuracy of estimates in the projects for time and effort as a result
of which the cost overruns and time overruns will not occur or at least
the overruns will be to a lesser extent.
• Increased customer satisfaction. The customer feedback should be
better in terms of the grading on the product performance and the
number of service calls should be less.
• The productivity of the individual employees which will show an
improvement.
• Lesser manpower attrition. In organizations where the process
definitions are not clear or do not exist, there will be too many work
pressures and loosing tempers as a result of which employee morale
will be low and hence they tend to leave the organization. In process-
oriented organizations where development is done systematically,
there will be less heartburn and employees tend to stay longer.
• Profitability on the projects: Through a better process, the costs can be
controlled and hence the profit margins will be higher. The cumulative
effect of higher productivity, faster development and less wastage is
higher profitability.
So, for every project, these parameters need to be measured. As the process improves, the effect is reflected on these parameters. At both organizational level and project level, the measures mentioned above would help in quantitatively measuring the process quality.
Managing the Process Change
The process change has to be monitored and reviewed by the senior management. Staff should be encouraged and rewards can be introduced for suggestions on process change. Procedure for handling process change should be clearly defined:
1. Submission of proposal as to why the change is proposed-based on customer
feedback, defect report analysis, external auditor remarks or internal QA
recommendations.
2. Evaluation of each proposal and documentation.
3. Decision on the priority of the proposal.
Core Competency in Software Process Management
4. Implementation plan including responsibility and time frame.
5. Tracking the status of each proposal.
The format for process improvement is given in Table II. For every proposed change, this form has to be filled up.
Process Improvement Proposal
Proposed by
Group
Proposal date
Acknowledgement to the proposal sent on
Proposed area
Description of the proposal
Sources of data
Expected benefits: productivity, quality, improvement in development time, end user
satisfaction, reduction in wastage, ethical values etc.
Impact analysis of the proposal (risks involved, need for pilot project, when/how to
terminate the pilot project)
Recommendation
Resources allocated (budget, time, personnel etc.)
Results of pilot project
Date of incorporating the change in the organization process definition
Table II. Format for Process Improvement Proposal
The senior management has to take periodic reviews to study the impact of the process changes on productivity, quality, development time etc.
Quality Process Implementation Issues
The essence of ISO 9000 series standards and the CMM framework is the same. Define a process, implement it and periodically review the process to check its effectiveness and based on the review of the results, improve the process. As the software development process is a highly complex activity, the process is divided into various sub-activities and each sub-activity (call it clause or Key Process Area, it does not matter), follow a defined methodology. As it is humanly not possible or advisable to rely on our memories all the time, we should document all these process definitions.
Unfortunately, in some organizations, management gives scope for ‘process bureaucrats’. These guys insist on following the process documents blindly without any concern for the process improvement.
In some organizations that achieved CMM Level 4 and 5, many employees suddenly become unhappy and they feel that the process is being given importance and not the people. Judy Bamberger says, “CMM is one of the most misunderstood pieces of technical literature in existence” [J. Bamberger, Essence of the CMM, IEEE Computer, Vol. 30, No. 6, June 1997, pp.112-114]. A culture needs to be developed whereby the engineers and managers get the ‘essence’ of CMM rather than a blind following of the process standards. .
Large organizations working on large projects can afford to implement the CMM or ISO 9000 standards though it results in lot of overheads. Certainly, many small organizations cannot afford nor do they want to obtain certification due to lack of resources. Even then, these organizations will be benefited by adapting these standards and if required suitably modifying them.
In small organizations a simple trick works very well: the project manager has to follow CMM meticulously, without bugging the engineers with too much of process documentation. The project manager has to plan the project and share it; write the SRS and share the contents through a lecture; make the engineers sit together for a peer review; and so on. It works well, and the engineers do not get bogged down with the process documentation.
The project manager has to realize the importance of the quality standards and impress upon the team members the importance of following these standards. Standards implementation should be a culture rather than an add-on to the development.
Implementation of Quality Standards in Small Organizations
The quality standards such as CMM and ISO 9000 have been formulated to bring in process orientation for large-scale projects, executed by large organizations. The CMM framework, in particular, has been developed to assess the capabilities of subcontractors for large defense projects. To implement these quality standards, there will be lot of overhead. Additional work force is required for the process engineering group, QA group etc. Lot of documentation is involved for developing and maintaining organization-wide processes, and lot of administrative work is involved in maintaining the library for process documentation, configuration management etc. Most of the small organizations cannot afford this overhead as it eats away the profit margin. Small organizations working on small development projects, can avoid this overhead if they follow a customized software engineering methodology. In small organizations, everyone does everything—even senior level persons may be involved in coding. Small organizations work on high technology products and hence innovation and creativity are the key elements. Following standards with a bureaucratic approach hinders the creativity. Hence, small organizations need to tailor the quality standards for their specific needs.
For working on small projects having a high technology orientation, the constitution of the project team is crucial. To reduce overheads, the project team will not be linked to a separate quality team or process team. The development team members themselves need to act as quality engineers and process engineers. This calls for:
• Every team member has to be rigorously trained on software
engineering and quality standards.
• Every team member should be made to follow the Personal Software
Process to bring in discipline and quality in the work.
• Peer reviews should be encouraged to bring in quality in every phase
of development.
• Project reviews should be carried out with high periodicity.
• Strong communication channels should be established amongst all the
team members.
• Quantitative measures have to be introduced at every stage for
measuring the quality of the process and product.
• The emphasis should be on product quality and not too much on
compliance to CMM.
Small is Beautiful
Technological innovation takes place in small organizations. Many large organizations increase their intellectual capital by taking over the small organizations. Small organizations can take up high technology projects and carry out development with shorter lead times if they can take fast decisions, avoid administrative overheads and also avoid process bureaucracy. However, they still need to follow software engineering methodology, but avoid the unnecessary overheads. Even large organizations need to create virtual small organizations so that they can work with the necessary intellectual freedom devoid of process bureaucracy.
To summarize, CMM provides the necessary framework for delivering quality software. Every organization, big or small, needs to understand the essence of this quality standard and follow it. Commitment of the management and also each and every employee to the quality of the product is a must.
