Risk Assessment

Why perform a schedule risk assessment (SRA)? 

SRAs are critical in understanding the likelihood of meeting baseline or forecasted completion dates as well as identifying which risks to focus on for protecting the project schedule. 

An SRA is a Monte Carlo based simulation of the schedule using the existing logic but with different duration inputs in repeated “walk-throughs” of the schedule. In each run through of the schedule a different duration may be used for every task therefore the end date is usually different. When enough simulations are performed, a picture emerges of the distribution of potential outcomes from the shortest, to the longest, and to all those in-between. 

A simple way to think of this is to imagine driving to the grocery store on the same route a thousand times. Sometimes it may be rainy or snowy or maybe even clear and dry. There may be accidents, road work, traffic lights, or other issues. Each trip can have a different duration but overall, with enough trips, we will end up with good picture of how much time we should plan for the trip. We can even have different plans for different conditions.

Of course, you do not have the luxury of performing your project a thousand times. You get one opportunity. But an SRA can help you understand what the journey through your project can look like through simulation.

Performing SRAs and managing the project based on the results can provide project managers with powerful tools to successfully meet project objectives. SRAs are a discipline that supports critical risk and schedule decision making.

Projects are often long durations of five years or more, involve one-of-a-kind systems, facilities, or integrations, and have numerous constraints ranging from security, regulatory, to environmental. In addition, these projects have oversight from organizations such as the Government Accountability Office (GAO), Office of Management and Budget (OMB), and even Congress with expectations set to meet internal milestone commitments such as critical decisions (CDs) and initial operational capabilities (IOCs). While this is a subset of the many challenges facing these projects, they all rely on project managers and the project team meeting schedule commitments.

Purposes of the SRA

The main purposes of the SRA are twofold: 1) to understand the schedule and 2) to understand the probability of achieving the end date in the schedule. 

In a recent H&A workshop a very high-level executive in one of the large government agencies said of the SRA, “I don’t do it, but I receive the “P” numbers.” What she meant was that she receives the result in terms of the probability numbers developed in the SRA. She is informed of the probability of achieving the schedule as part of the program status review.

Understanding the SRA

In simple terms, an SRA is a structured probabilistic analysis by simulation of a project’s schedule that quantifies the likelihood of meeting critical dates. Initially the SRA focuses on the baseline and the probability of achieving the baselined dates. Once progress has been added to the schedule, the emphasis shifts to the current or working schedule and away from the baseline. The SRA uses the latest available information.

Uncertainty can exist in the schedule in terms of duration uncertainty, unplanned events, merge bias, and other such factors. The SRA tries to account for duration uncertainty by using a three-point approach to the task durations. The owners of the tasks are challenged to provide best case, most likely case, and worst-case durations for use in simulating the schedule. 

The SRA uses the project’s schedule logic as well as uncertainty assigned to activities along with the project’s risk register to produce probabilistic outcomes. A probabilistic analysis is a way of understanding outcomes when the future is uncertain. Instead of assuming there is only one single outcome for the schedule, probabilistic analysis takes variation into account based on durations, risks, and other events through simulation. This simulation is then repeated hundreds, sometimes thousands of times to produce statistical analysis that defines what the range of results could be. This statistical analysis will reveal, based on inputs to the model, a distribution of possible outcomes with confidence intervals expressed like this:

• There is a 50% chance of finishing the project by this date, or
• There is an 80% chance of meeting this milestone.

The SRA can be used to evaluate an entire project schedule or a subset of the schedule which may be a specific milestone, deliverable, or even a single work package. The scope of the SRA depends upon what part or whole of the schedule you would like to examine based on application needs. 

For this reason, SRAs are useful for project managers, control account managers (CAMs), schedulers, risk managers, and other key stakeholders. Stakeholders can use SRAs to meet compliance requirements, run “what-if” scenarios, or provide inputs into vendor selections and vendor performance management decisions. While the SRA is an excellent management decision tool, it should not be used to provide a static critical path review. It should never be used to create a deterministic “best-case” schedule, and it should not be dependent upon a specific software tool.

One important output of the SRA is commonly called the “Tornado Chart” because of the shape of the data provided. This analysis identifies the number of times a task acts as a driver in the outcome of the schedule. If a task is often driving the outcome, it should be considered carefully. Potential other approaches should be considered or at least a detailed review of the durations of the task should be performed.

Foundations

The SRA will only be as credible as the schedule it evaluates and the integrity of the risk information that is applied. Prior to the execution of a successful SRA, the schedule integrity should meet established quality standards. 

The DCMA EVMS Compliance Metrics (DECM) specific to schedule data provides a widely accepted benchmark for evaluating integrated master schedule (IMS) health. The NDIA Integrated Program Management Division (IPMD) Planning and Scheduling Excellence Guide (PASEG) includes a section on performing schedule health assessments with a list of often used metrics. Some project teams may customize their IMS quality metrics based on their established and approved EVMS documentation. Areas typically evaluated include logic integrity, critical path integrity, schedule realism, and data quality.

Logic integrity includes the proper use of predecessors and successors with minimal usage of leads and lags and a clear path from start to finish. Critical path integrity ensures that there is an understood set of critical and near-critical paths with logical drivers of key milestones and avoidance of excessive float. Standards for schedule realism include reasonably short activity durations and a limited use of constraints. Data quality requires valid dates, calendars and status updates. All these elements are covered in the generally accepted schedule quality metrics and most SRA tools will validate the schedule as well ensure that the schedule quality is adequate to perform a successful SRA.

Equally important to the SRA is the risk information. Risks should be identified and clearly characterized; their association with the schedule should be mapped. When feasible, mitigation plans should be incorporated into the schedule along with any variations they may have. The identified risks should succinctly state the event, and the cause and effect of the risk. Each risk should be characterized by its probability of occurring and the impact to the schedule if it were to occur. This characterization should be grounded in credible assumptions, usually based on the project’s rubrics for scoring likelihood and impact.

Input of risk events into the schedule provides additional realism. In terms of the simulation there will be times when the risk event happens and times when it does not with different durations as well.

Once quality schedule and risk information is validated, these inputs are imported into the probabilistic tool used by the project team. Since probabilistic analysis is based on the Monte Carlo methodology which uses statistical math, any tool that accommodates schedule and risk inputs should be sufficient. 

Frequency of SRA Application

SRAs can be expensive when time is taken to do the best case, most likely, and worst-case duration analysis by the CAMs. The cost is a limiting factor in the use of the SRA. It is most common for SRAs to be performed:

1. At the time of baselining to understand the baseline schedule and the probability of success.
2. When a major change is made to the project schedule.
3. Before a major event such as the Critical Design Review (CDR) where the nature of the effort changes from design to build and the team wants to understand the probability of success for the remaining effort.

Do not accept a contract that requires performing the SRA when requested by the customer. That is too open ended of a requirement and does not allow you to estimate the cost or control the cost. A contract should specify the number of SRAs to be included so that any additional SRAs can be treated as a compensated change.

Conclusion

When grounded in a high-quality IMS and integrated with disciplined risk management, SRAs provide stakeholders with a realistic view of schedule outcomes and the risk drivers. While often thought of as a compliance requirement, the SRA provides management decision insights and is useful in understanding, as an early warning indicator, any threats to meeting schedule objectives. When properly used, the SRA is a powerful management tool that can contribute to project success.Interested in learning more? H&A master schedulers and risk subject matter experts often assist clients with establishing their SRA process and mentoring project teams to use the SRA outputs to create more realistic schedules with a higher probability of success. Call us today to get started.