Maximizing the Value of Schedule Risk Assessments (SRAs)
Quick Summary
- SRAs are only as valuable as the intent and quality behind them. Treating them as a check-the-box compliance requirement produces meaningless results that get filed away and never used.
- Real-world examples show how cost pressure and last-minute guidance changes at kickoff can derail an SRA, leaving project teams with outputs no one trusts or acts on.
- Focusing three-point estimates on tasks that drive critical and near-critical paths delivers real insight without the prohibitive cost of analyzing every task in a large IMS.
- Documented assumptions for selected tasks produce derived duration factors that are far more accurate than gut-feel percentages, and the difference shows up directly in the SRA results.
A previous blog, Schedule Risk Assessment Fundamentals, highlighted that when properly used, schedule risk assessments (SRAs) are a powerful management tool that can contribute to project success. It does, however, require a high-quality integrated master schedule (IMS) that is integrated with a disciplined risk management process. When conducted with the intention to gain a realistic view of schedule outcomes and risk drivers, it provides management with additional insight and an early warning indicator of potential threats to meeting schedule objectives.
The Challenge with Conducting SRAs
Schedule risk assessments are frequently a topic of discussion in project management circles and sometimes for the wrong reason. A well-run SRA can provide the confidence levels for achieving different schedule end dates as well as valuable insight into the IMS. However, the SRA is particularly susceptible to the “garbage in – garbage out” (GIGO) principle.
H&A consultants have recently observed discussions that indicate SRAs are not being properly performed or used to help manage projects. Some of these observations reinforce the GIGO principle.
One H&A senior consultant sat in on a meeting with the government program manager for a large project where the consultant, having worked on the just-completed SRA, asked the government program manager what they would do with the results. The amazingly honest answer was, “Sadly, it will just be filed. It is seen as a check-the-box thing we have to do.”
Using the results of the SRA to stuff the drawer might not be as wild a response as it seems. Too many times, we see the SRA being done in a perfunctory manner using inputs that will not yield useful management information and insight. One way people are cutting corners on the SRA is by applying global factors to the existing duration estimates and running the simulation with those durations. Applying a formula across the board is not the same as analysis. Yes, the SRA can be done so poorly it is meaningless. This is especially true if the applied factors are not realistic.
An Example of What to Avoid
One of our consultants observed that in one case the best of intentions were thwarted in the SRA by cost pressures and lack of management commitment. The manager of the SRA provided a form to each CAM for them to provide the best case, worst case, and most likely case duration estimates and provide written explanations for all three cases. The instructions were for the CAMs to use the form for every incomplete task and future task.
There were about 10,000 such tasks distributed among 70 or so CAMs. On average, that would be 140+ tasks for each CAM. Doing three-point estimates for 140 tasks would be a large expense and consume a lot of valuable time, without even mentioning the cost.
The government program manager, who pays the bills, was present at the kickoff meeting for the SRA and intervened immediately when he heard the directions being provided. He stated that he would not pay for all that effort; it would be too costly. Unfortunately, there was no probing to find out what would be reasonable to this manager in terms of details for the SRA.
Instructions for this SRA were revised on-the-fly and the SRA was done. It was done poorly; in some parts due to the disruption at the kickoff meeting and the poor guidance. The intervention of the government program manager had left the impression that the SRA “was not worth it.” That impression was wrong.
The idea of documenting the three-point estimates is a good one, but too time-consuming to be applied to every task. There are valid options that can still benefit from this detailed look at the durations, yet avoid the significant expense of analyzing all the tasks.
A Better Approach
One approach is to do a detailed analysis of the three-point estimates for specific selected tasks that reduces the number of tasks that require detailed manual estimation. The focus should be on the tasks that provide insight into the part of the IMS most likely to cause the end date or a major event date to change. Examples include those tasks that are:
- On one or more of the top number of critical paths.
- On the path to the next major event (these tasks can be found using the driving path approach).
- Known to be or assumed to be prone to duration risk.
- Known to require scarce resources that may have limited availability.
- Believed to be drivers of duration risk for other reasons.
A simple example will help to understand why documenting the three-point estimates for some tasks is basic to achieving a useful SRA. If you, acting as a CAM, were asked to estimate the time required to drive 10 miles from your home to work by car in a hypothetical town, you would want to understand the scope of the effort. A drive of 10 miles through the countryside, or on a freeway, or on city streets can be very different.
Some help with the assumptions could improve your estimate. If you were told that the first 5 miles of the drive was in town, where the speed limit is 25 mph with the potential for red lights along the way, and the last 5 miles is on the freeway, where the speed limit is 60 mph, this would help you produce a better estimate.
In fact, you will get a better estimate by understanding the scope, the assumptions, the risks, and so on. When doing the three-point estimate, you would employ a process that includes these steps that consider:
- The nature of the task. What exactly is to be done in the task?
- Past experience. Have we done this or similar work before?
- Capabilities. For example, can you drive, does your car work well, can you go up to 60 mph in your car, do you have gas, and are you equipped for potential weather?
- Assumptions. How many traffic lights are there along the way? How long would you wait at a red light? What time of year is it? What day of the week? What time of day?
- The risks. Are there possible road issues, such as construction? How about traffic issues? Accidents?
Now, suppose you were required to document your estimated durations. Using the assumption details from above, you might end up with this:
| Case | Duration | Assumptions |
|---|---|---|
| Best | 17 minutes | A dry day, early in the morning before traffic, you have all green lights, and you obey the speed limit. |
| Most Likely | 23 minutes | The road is fine, first 5 miles is at 25 mph. Only 2 red lights with a wait time of 2 minutes each, and 7 minutes for freeway travel at 45 mph. |
| Worst Case | 32 minutes | The road is slippery and you can only travel at 20 mph. You must stop at 3 red lights and sit for 2 minutes each. You have one unexpected stop for 4 minutes because of other drivers. Also, the freeway speed is only 45 mph. |
You now have a set of durations you can use in the SRA. You also have the details needed for explaining the duration estimate. Additionally, there is enough information to be able to change the estimates if presented with new facts or revised assumptions. For example, perhaps the project’s period of performance moves to the right and the work will now be performed in the winter. You can adjust your estimates for winter weather impacts.
You also have enough information to be able to derive factors to be used in formulae to generate three-point estimates for other tasks. Be careful to make sure you only use the information to generate a factor on similar work. In this case, the factors would be -26% and +39%. Those are derived factors. For comparison, a common “gut feel” reaction to the question of what factors should be used is usually more like minus 5% and plus 10%. Using derived factors versus a “gut feel” will yield significant differences in the SRA and the value of the results.
Recommendations to Increase the Value of an SRA
Begin with the intention to treat the SRA as the important and valuable tool it is. Choose to change the approach from a “check-the-box” or compliance mentality into a straightforward process that helps to produce a more realistic and executable IMS. No one likes schedule or resulting cost surprises as discussed in another blog, Maintaining a Credible Estimate at Completion (EAC), that also addresses why a credible forecast completion date is equally important. Here are a few suggestions to improve your approach to conducting SRAs:
- Provide clear, specific directions to project personnel on what is expected. Highlight why the SRA is an important step.
- Verify a quality IMS has been established.
- Validate the risk information.
- Do focused analysis of discrete tasks on a given number of critical and near-critical paths and document the rationale for the best/worst/most-likely case durations.
- Do focused analysis on driving path tasks if not on the critical paths.
- Do focused analysis on known risks.
- Use realistic factors derived from reality when applying factors to the larger body of IMS tasks.
Taking Action
Producing a quality IMS takes skilled master schedulers that understand the management and predictive value of a well-constructed schedule. The next step up to improve the realism of the IMS is to conduct an SRA when it makes sense. Examples include conducting an SRA as part of the process to establish the baseline schedule, when there is major change, or before a major event such as a Critical Design Review (CDR).
It is not an easy task to distill the steps to conduct a value-added SRA into a well-defined and useful process. H&A master schedulers and risk subject matter experts often work with clients to establish a pragmatic SRA process. They also train and mentor project teams on how to use the SRA outputs to produce realistic schedules with a higher probability of success. Call us today to get started.
Maximizing the Value of Schedule Risk Assessments (SRAs) Read Post »
Deltek Acumen – Top-Level Menus
