Know Your Enemy: Software Risk Management

Karl E. Wiegers
Process Impact
www.processimpact.com

Software engineers are eternal optimists. When planning software projects, we often assume that everything will go exactly as planned. Or, we take the other extreme position: the creative nature of software development means we can never accurately predict whatís going to happen, so whatís the point of making detailed plans? Both of these perspectives can lead to software surprises, when unexpected things happen that throw the project off track. In my experience, software surprises are never good news.

Risk management is becoming recognized as a best practice in the software industry for reducing the surprise factor. While we can never predict the future with certainty, we can apply structured risk management practices to peek over the horizon at the traps that might be looming, and take actions to minimize the likelihood or impact of these potential problems. Risk management means dealing with a concern before it becomes a crisis.

Symptoms that an organization is not effectively practicing risk management include a continual state of project instability, constant fire-fighting, multiple schedule slippages because of recurring surprise factors, and constantly operating in a high-stress, crisis management role. Formal risk management greatly improves the likelihood of successful project completion, and it reduces the potential negative consequences of those risks that cannot be avoided.

What Is Risk?

A simple definition of a "risk" is a problem that could cause some loss or threaten the success of our project, but which hasnít happened yet. (And weíd like to keep it that way.) These potential problems might have an adverse impact on the cost, schedule, or technical success of the project, the quality of our software products, or project team morale. Risk management is the process of identifying, addressing, and eliminating these potential problems before they can damage our project.

We need to differentiate risks, as potential problems, from the current problems facing the project, because different approaches are taken for addressing these two kinds of issues. For example, a staff shortage because you havenít been able to hire people with the right technical skills is a current problem, but the threat of your top technical people being hired away by the competition is a risk.

Current, real problems require prompt corrective action, whereas looming risks can be dealt with in several different ways. We might choose to avoid the risk entirely by changing our project approach or even canceling the project. Or, we could elect to simply absorb the risk and take no specific actions to avoid or minimize it. More often, though, weíll want to contemplate actions that will either minimize the likelihood of the risk turning into a problem, or reduce the negative impact of that problem on our project.

Whether we tackle them head-on or keep our heads in the sand, risks have a potentially huge impact on many aspects of our project. For example, project estimates too often assume that everything will go just as planned. The eternal optimism of software developers leads us to focus on the exciting technical challenges facing the project, relying on our boundless ability to concoct clever solutions to whatever problems arise. Even when developers recognize that uncertainty poses a threat to our best-case schedules, managers sometimes only hear the smallest estimates we present.

This tacit assumption that nothing untoward will derail our project is simply not realistic. Our estimates should incorporate our best judgment about the potentially scary things that could happen on each project, and managers need to respect the assessments we make. Plans that do not include an analytically derived contingency budget of money and time are fine for Fantasyland, but not for the real world.

We do far too much pretending in software. We pretend we know who our users are, we know what their needs are, that we wonít have staff turnover problems, that we can solve all technical problems that arise, that our estimates are achievable, and that nothing unexpected will happen. Risk management is about discarding the rose-colored glasses and confronting the very real potential of undesirable events conspiring to throw our project off track.

Why Manage Risks Formally?

A formal risk management process provides a number of benefits to the project team. First, it gives us a structured mechanism to provide visibility into threats to project success. By considering the potential impact of each risk item, we can make sure we focus on controlling the most severe risks first. A team approach allows the various project stakeholders to collaboratively address their shared risks, and to assign responsibility for risk mitigation to the most appropriate individuals. We can combine risk assessment with project estimation to quantify possible schedule slippage if certain risks materialize into problems. Without a formal approach, we cannot ensure that our risk management actions will be initiated in a timely fashion, completed as planned, and effective. The net result of these activities is to help avoid preventable surprises late in the project, and therefore improve the chance of meeting our commitments.

The entire development organization also enjoys benefits from risk management. Sharing what does and does not work to control risks across multiple projects helps projects avoid repeating the mistakes of the past. Members of the organization can pool their experience and identify opportunities to control our most common risks, through education, process improvement, and application of improved software engineering and management techniques. Over time, you can build a checklist of risk items and mitigation strategies from multiple projects that can help future projects look for the vipers that might be lurking underfoot.

Risks and Uncertainty

Much risk is attributable to uncertainty about the things we sometimes pretend are under control. Itís what we donít know that can hurt us. Uncertainty is a normal and unavoidable characteristic of most software projects. It can result from the continuously increasing complexity of the products we create, and the haste with which we sometimes dive into the source code editor. When youíre living on the bleeding edge of rapidly changing technology or business conditions, uncertainty permeates your life. Lack of practical knowledge about the software development techniques and tools we are using presents an additional source of uncertainty.

Controlling risk partly means reducing uncertainty. Of course, reducing uncertainty has a cost. We need to balance such costs against the potential cost we could incur if the risk is not addressed and does indeed bite us. It may not be cost-effective to reduce uncertainty too much. For example, if we are concerned about the ability of a subcontractor to deliver an essential component of our product on time, we could engage multiple subcontractors to increase the likelihood that at least one will come through on schedule. Thatís an expensive remedy for a problem that may not even exist. Is it worth it? It depends on the down side we incur if indeed the subcontractor dependency causes the project to miss its planned ship date. Only you can decide for each individual situation.

Proactive risk management doesnít necessarily mean avoiding projects that could incur a high level of risk. Prospering in todayís software marketplace often means that high-risk projects are precisely the ones we need to undertake. Formal risk management makes sure we go into such projects with our eyes open, so we know what kinds of things could go wrong, and weíve done our best to make sure those factors wonít prevent the ultimate success of the project.

Typical Software Risks

The list of evil things that can befall a software project is depressingly long. The enlightened project manager will acquire extensive lists of these risk categories to help the team uncover as many concerns as possible early in the planning process. Possible risks to consider can come from group brainstorming activities, or from a risk factor chart accumulated from previous projects. In one group Iíve worked in, individuals came up with very insightful descriptions of their risk factors, which I edited together and we then reviewed as a team. The Software Engineering Institute has assembled a taxonomy of hierarchically-organized risks in 13 major categories, with about 200 thought-provoking questions to help you spot the risks facing your project [Carr, 1993].

Following are several typical risk categories and risk items that may threaten your project [McConnell, 1996]. If any of these things have happened to you, put them on your master risk factor checklist to remind each of your future project managers to ask themselves if it could happen to them. There are no magic solutions to any of these risk factors, so we need to rely on past experience and a strong knowledge of contemporary software engineering and management practices to control those risks we are most concerned about.

Capers Jones has identified the top five risk factors that threaten projects in different application sectors [Jones, 1994]. Table 1 illustrates those factors, and the approximate percent of projects to which they apply, in the management information systems (MIS) and commercial software sectors.


Table 1. Most Common Risk Factors for Various Project Types

Project Sector

Risk Factor

Percent of Projects at Risk

MIS

Creeping user requirements

80%

 

Excessive schedule pressure

65%

 

Low quality

60%

 

Cost overruns

55%

 

Inadequate configuration control

50%

Commercial

Inadequate user documentation

70%

 

Low user satisfaction

55%

 

Excessive time to market

50%

 

Harmful competitive actions

45%

 

Litigation expense

30%

Dependencies

Many risks arise because of dependencies our project has on outside agencies or factors. We cannot usually control these external dependencies, so mitigation strategies may involved contingency plans to acquire a necessary component from a second source, or working with the source of the dependency to maintain good visibility into status and detect any looming problems. Here are some typical dependency-related risk factors:

  • customer-furnished items or information
  • internal and external subcontractor relationships
  • inter-component or inter-group dependencies
  • availability of trained, experienced people
  • reuse from one project to the next

Requirements Issues

Many projects face uncertainty and turmoil around the productís requirements. While some of this uncertainty is tolerable in the early stages, the threat to success increases if such issues are not resolved as the project progresses. If we donít control requirements-related risk factors, we might either build the wrong product, or build the right product badly. Either situation results in unpleasant surprises and unhappy customers. Become familiar with established requirements gathering and management practices, and watch out for these risk factors:

  • lack of clear product vision
  • lack of agreement on product requirements
  • unprioritized requirements
  • new market with uncertain needs
  • new applications with uncertain requirements
  • rapidly changing requirements
  • ineffective requirements change management process
  • inadequate impact analysis of requirements changes

Management Issues

Although management shortcomings inhibit the success of many projects, donít be surprised if your risk management plan doesnít list very many of these. After all, the project manager is usually the person who is writing the risk management plan, and most people donít wish to air their own weaknesses (assuming they even recognize them) in public. Nonetheless, issues like those listed here can make it harder for projects to succeed. If we donít confront such touchy issues, we shouldnít be surprised if they bite us at some point. Defined project tracking processes, and clear roles and responsibilities, can address some of these risk factors.

  • inadequate planning and task identification
  • inadequate visibility into actual project status
  • unclear project ownership and decision making
  • unrealistic commitments made, sometimes for the wrong reasons
  • managers or customers with unrealistic expectations
  • staff personality conflicts
  • poor communication

Lack of Knowledge

The rapid rate of change of software technologies, and the increasing shortage of skilled staff, mean that our project teams may not have the skills we need to be successful. The key is to recognize the risk areas early enough so that we can take appropriate preventive actions, such as obtaining training, hiring consultants, and bringing the right people together on the project team. These factors might apply to your team:

  • inadequate training
  • poor understanding of methods, tools, and techniques
  • inadequate application domain experience
  • new technologies or development methods
  • ineffective, poorly documented, or neglected processes

Other Risk Categories

The list of potential risk areas is long, but weíre better off knowing our enemy than being surprised. Some other areas you should consider include these:

  • unavailability of development or testing equipment and facilities
  • inability to acquire resources with critical skills
  • turnover of essential personnel
  • unachievable performance requirements
  • problems with language translations and product internationalization
  • technical approaches that may not work

Risk Management Approaches

An organization can select one of five possible approaches to dealing with risks [McConnell, 1996]. The default response is crisis management, the knee-jerk reaction undertaken when a previously unidentified or unmanaged risk develops into a clear and present danger. Only slightly better is to fix the product when a failure is encountered. A more proactive approach is to identify the risks facing your project and plan how youíll respond to them if they raise their ugly heads. Still better is to take concrete actions to prevent those identified risks from ever causing trouble. The ultimate in risk management is to eradicate the root causes that cause certain risks to chronically threaten your organizationís projects.

Risk management is the application of appropriate tools and procedures to contain risk within acceptable limits. Risk management consists of several sub disciplines.

  • Risk assessment is the process of examining a project and identifying areas of potential risk. Risk identification can be facilitated with the help of a checklist of common risk areas for software projects, or by examining the contents of an organizational database of previously identified risks and mitigation strategies (both successful and unsuccessful).. Risk analysis involves examining how project outcomes might change with modification of risk input variables.

    Risk prioritization helps the project focus on its most severe risks by assessing the risk exposure. Exposure is the product of the probability of incurring a loss due to the risk and the potential magnitude of that loss. This prioritization can be done in a quantitative way, by estimating the probability (0.1 Ė 1.0) and relative loss, on a scale of 1 to 10. Multiplying these factors together provide an estimation of the risk exposure due to each risk item, which can run from 0.1 (donít give it another thought) through 10 (stand back, here it comes!). The higher the exposure, the more aggressively the risk should be tackled. It may be easier to simply estimate both probability and impact as High, Medium, or Low. Those items having at least one dimension rated as High are the ones to worry about first.
  • Risk avoidance is one way to deal with risk: donít do the risky thing! You may avoid risks by not undertaking certain projects, or by relying on proven rather than cutting edge technologies.
  • Risk control is the process of managing risks to achieve the desired outcomes. Risk management planning produces a plan for dealing with each significant risk, including mitigation approaches, owners, and timelines. Risk resolution is execution of the plans for dealing with each risk. Finally, risk monitoring involves tracking your progress toward resolving each risk item.

Simply identifying the risks facing your project is not enough. We need to write them down in a way that lets us communicate the nature and status of risks throughout the affected project stakeholder community over the duration of the project. Figure 1 shows a form I have found to be convenient for documenting risks. This risk list could be included as a section in your software project plan, or it could remain as a standalone document. As an alternative, the form in Figure 2 can be used to document individual risk factors in more detail.

When documenting risk statements, use a condition-consequence format. That is, state the risk situation (condition) that you are concerned about, followed by at least one potential adverse outcome (consequence) if that risk should turn into a problem. Often, people suggesting risks may state only the condition ("the customers donít agree on the product requirements") or the consequence ("we can only satisfy one of our major customers"). Pull those together into the condition-consequence structure: "The customers donít agree on the product requirements, so weíll only be able to satisfy one of our major customers."

The P, L, and E columns in Figure 1 provide locations to capture the probability of a risk materializing into a problem (P), the loss that could be incurred due to the risk (L), and the overall risk exposure (P times L). You can sort the risk list in descending order of exposure, so the top priority risk items are at the top of the list. You canít worry about every risk item, so use this prioritization mechanism to learn where to focus your risk control energy. Set goals for determining when each risk item has been satisfactorily controlled. For some risk items, your mitigation strategies may focus on reducing the probability, while the approach for other items may emphasize reducing the impact.

In the next column of the form, document any observation that could constitute a first indicator that the risk factor is indeed becoming a current problem. By monitoring these first indicators, you can get the earliest possible notice that a more aggressive risk management approach may be needed.

The column headed Risk Mitigation Approaches allows you to identify the actions you intend to take to keep the risk item under control. With any luck, some of your mitigation approaches can be used to address multiple risk factors. For example, one group Iíve worked in identified several risks related to failures of components of their web delivery infrastructure (servers, firewall, e-mail, and so on). A mitigation strategy that addressed many of those risks was to implement an automated monitoring system that could check the status of the servers and communication functions periodically and alert us to any failures. Other mitigation approaches may include identifying alternatives and options for technical risks, or identifying alternative resource sources for staffing risks. In addition to mitigating the risks, you may be able to prevent them entirely, or you may wish to develop contingency plans of action to take if the risk materializes into a problem despite your best efforts to control it.

Figure 1. Risk Documentation Form.

ID

Risk Description

P

L

E

First Indicator

Risk Mitigation Approach

Who

Due

 

List each major risk facing the project. Describe each risk in the form "condition Ė consequence".

Example:

"Subcontractorís staff does not have sufficient technical expertise, so their work is delayed for training and slowed by learning curve."

*P

*L

*E

  • For each risk, describe the earliest indicator or trigger condition that might indicate that the risk is turning into a problem.
  • For each risk, state one or more approaches to control, avoid, minimize, or otherwise mitigate the risk.
  • Risk mitigation approaches should yield demonstrable results, so you can measure whether the risk exposure is changing.
  • Assign each risk miti- gation to an individual.

    State a date by which the mitigation approach is to be imple- mented.

     

    Figure 2. Another Form for Documenting Individual Risk Items.

    Risk ID: <sequence number>

    Classification: <risk category, e.g., from SEI taxonomy>

    Report Date: <date this risk report was last updated>

    Description: <Describe each risk in the form "condition Ė consequence".>

     

    Probability: <Whatís the likelihood of this risk becoming a problem?>

    Impact: <Whatís the damage if the risk does become a problem?>

    Risk Exposure: <Multiply Probability times Loss to estimate the risk exposure.>

    First Indicator: <Describe the earliest indicator or trigger condition that might indicate that the risk is turning into a problem.>

    Mitigation Approaches: <State one or more approaches to control, avoid, minimize, or otherwise mitigate the risk. Mitigation approaches may reduce the probability or the impact.>

    Date Started: <State the date the mitigation plan implementation was begun.>

    Date to Complete: <State a date by which the mitigation plan is to be implemented.>

    Owner: <Assign each risk mitigation action to an individual for resolution.>

    Current Status: <Describe the status and effectiveness of the risk mitigation actions as of the date of this report.>

    Contingency Plan: <Describe the actions that will be taken to deal with the situation if this risk factor actually becomes a problem.>

    Trigger for Contingency Plan: <State the conditions under which the contingency plan will begin to be implemented.>

    Each mitigation action should be assigned to an individual for implementation and monitoring of effectiveness. Also, it can be helpful to identify a target date for completion of each mitigation approach. This gives you something to track against, so you can determine whether the planned risk management approaches are being implemented in an effective and timely fashion.

    As with other project management activities, you need to get into a rhythm of periodic monitoring. You may wish to appoint a "risk czar" for the project, who is responsible for staying on top of the things that could go wrong, just as the project manager is staying on top of the activities leading to project completion. While the simple act of thinking through your projectís risk factors and documenting them is a good start, they wonít cure themselves. The operating principle for risk management is the same as that for process improvement: "Gentle pressure, relentlessly applied."

    Keep the top ten or so risks highly visible, and track the effectiveness of your mitigation approaches regularly. As the initial list of top priority items gradually gets beaten into submission, new items may float up to the top ten list. Donít kid yourself into concluding that a risk is controlled simply because the selected mitigation action has been completed. Controlling a risk may mean that you have to change your mitigation strategy when you conclude that it is ineffective. You can drop the risk off your threat-detection radar when you conclude that your mitigation approaches have reduced the loss exposure from that item to an acceptable level.

    Risk Management and Schedule Estimation

    You may have had the experience of preparing an estimate for some body of work, only to have it cut in half by a skeptical manager. Similarly, arbitrary "fudge factors" to account for the possibility of unknown events may be discarded by managers who always expect the best possible outcome. Quantitative risk management can build the case for contingency buffers that are more likely to be taken seriously by the decision makers.

    When estimating the potential loss associated with each risk item, consider the impact a materialized risk could have on your schedule. Consider a project dependency on having a necessary component delivered from a subcontractor on schedule. You estimate the potential schedule loss if this delivery doesnít happen as 4 weeks, and the probability as 0.3. The risk exposure from this item is therefore 4 weeks times 0.3, or 1.2 weeks. By performing a similar analysis for each of your top 10 risk factors, you can total the individual risk exposures to estimate the cumulative risk exposure from those items, in units of calendar time (or, perhaps, actual dollars), as shown below.

    total risk exposure = sum of (Probability x Impact)

    You canít know exactly which of the identified risk factors might turn into problems. However, by quantifying the overall risk exposure facing your project in this fashion, you are better prepared to justify a realistic contingency buffer that should be built into your schedule. This will let you respond to some of the problems that might arise, without totally trashing your timing plans.

    Risk Management Can Be Your Friend

    The skillful project manager will use risk management as a technique for raising the awareness of conditions that could cause her project to go down the tubes. Consider a project that begins with an unclear product vision and a lack of customer involvement. The astute project manager will spot this situation as posing potential risks, and will document them in the risk management plan. Early in the projectís life, the impact of this situation may not be too severe. However, if time continues to pass and the lack of product vision and customer involvement are not improved, the potential threat to the project will steadily rise.

    By reviewing the risk plan periodically, the project manager can adjust the probability and/or impact of these risks. They may rise to the top ten, and can be brought to the attention of senior managers or other stakeholders who are in a position to either stimulate corrective actions, or make a conscious business decision to proceed in spite of the risks. Weíre keeping our eyes open and making informed decisions, even if we canít control every adverse condition facing the project.

    Learning from the Past

    While we cannot predict exactly which of the many threats to our projects might come to pass, most of us can do a better job of learning from previous experiences to avoid the same pain and suffering on future projects. As you begin to implement risk management strategies on your projects, also keep records of your risk management activities for future reference. Try these suggestions:

    • Record the results of even informal risk assessments, to capture the thinking of the participants on each project.
    • Keep records of the mitigation strategies attempted for each of the risks you chose to pursue, noting which approaches worked well and which did not pay off.
    • Conduct post-project reviews to identify the unanticipated problems that arose. Should you have been able to see them coming through a better risk management approach, or would you likely have been blindsided in any case? Do you think these same problems might occur on other projects? If so, add them to your growing checklist of potential risk factors that the next project can think about.

    Anything you can do to improve your ability to avoid or minimize previous problems on future projects will improve your companyís business success and reduce the chaos and frustration that reduces the quality of worklife in so many software organizations.

    Bibliography

    Boehm, Barry W. Software Risk Management. Los Alamitos, Calif.: IEEE Computer Society Press, 1989.

    Carr, Marvin J., Suresh L. Konda, Ira Monarch, F. Carol Ulrich, and Clay F. Walker. "Taxonomy-Based Risk Identification," CMU/SEI-93-TR-006. Pittsburgh: Software Engineering Institute, 1993.

    Gemmer, Art. "Risk Management: Moving Beyond Process," IEEE Computer (May 1997), pp. 33-43.

    Hall, Elaine M. Managing Risk: Methods for Software Systems Development. Reading, Mass.: Addison-Wesley, 1998.

    Jones, Capers. Assessment and Control of Software Risks. Englewood Cliffs, N.J.: PTR Prentice-Hall, 1994.

    Kitchenham, Barbara, and Stephen Linkman. "Estimates, Uncertainty, and Risk," IEEE Software (May/June 1997), pp. 69-74.

    McConnell, Steve. Rapid Development: Taming Wild Software Schedules. Redmond, Wash.: Microsoft Press, 1996.

    Schulmeyer, G. Gordon, and James I. McManus, eds. Total Quality Management for Software. Boston: International Thompson Computer Press, 1996. [Chapter 8 is on software risk mitigation].

    Williams, Ray C., Julie A. Walker, and Audrey J. Dorofee. "Putting Risk Management into Practice," IEEE Software (May/June 1997), pp. 75-82.


    (This article was originally published in Software Development magazine, October 1998. It is reprinted here (with modifications) with permission from Software Development).