schedule risk assessment

Maximizing the Value from Integrated Baseline Review (IBR) Investments 

, , , , , , , ,

A previous blog, How Integrated Baseline Reviews (IBRs) Contribute to Project Success, provided an overview of the purpose and scope of IBRs as well as the benefits of conducting an IBR. This blog adds to the discussion on the benefits of conducting an IBR. It reflects observations gathered from our earned value consultants while assisting clients to prepare for IBR events

As a reminder, IBRs provide the opportunity to verify the:

  • Contractor and the customer have a common understanding of the scope of work, technical requirements, and accomplishment criteria. 
  • Contractor has established an executable performance measurement baseline (PMB) for the entire contractual scope of work that accurately reflects how they plan to accomplish the work within the contractual period of performance, negotiated contract cost, and funding profile. 
  • Required resources have been identified and assigned to the project to accomplish the project’s objectives. For example, the staffing plan accurately reflects the sequence of work as well as resource availability and demand.  
  • Technical, schedule, and cost risks/opportunities have been identified, assessed, and captured in a risk/opportunity register. Risk mitigation actions have been incorporated into the PMB to reduce known threats to an acceptable level. This is often the most valuable component of the IBR to ensure all parties have an understanding of the risks/opportunities, assumptions, and risk mitigation or opportunity capture plans. 

Factors that Contribute to a Successful IBR

Treating an IBR as just a contractual requirement limits its value to all parties. IBRs are essential to the successful execution of any project. IBRs require a focused mindset to clearly define as well as assess the measurable benefits gained for the time and effort invested in the IBR. From our observations, contractors that defined what they expected to gain from an IBR, whether the IBR was contractually required or not, made a measurable difference in the outcomes from the IBR. The effectiveness of an IBR is contingent upon management’s commitment to excellence in implementing their EVMS and their desire to ensure they have reliable and useful data for management visibility and control. And that begins with establishing an executable PMB. 

The following list of factors often influence the perceived value of an IBR and hence the approach a contractor takes to planning and conducting their IBRs. 

  • Recognizing the relative importance of the review.
  • Defining the value or measurable benefits they expect to gain from conducting the review.
  • Well defined risk/opportunity management process. 
  • Timely and sufficient review planning and preparation.
  • Joint or collaborative planning and preparation.
  • Well defined objectives as well as entrance and exit criteria. 
  • Tailoring the IBR approach to best accomplish the review objectives.
  • Communication and expectation management.

These factors were ultimately indicative of whether the IBRs were considered value-added (retrospective assessment by the participants) based on the level of understanding, investment in or attention to, or the degree of success in implementing these factors. Based on H&A earned value consultant’s observations, the single factor that tends to drive the IBR approach is clearly defining the value the contractor expects to gain beyond what is mandatory or contractually required. 

IBR Investment Value

The term “IBR investment value” is purposefully used here. The intent is to invite you to re-assess how IBRs are viewed apart from simply meeting government agency IBR requirements. “IBR investment value” is used to mean a qualitative assessment that encapsulates the value-add or measurable benefits teams often have difficulty defining as well as to help provide the impetus and guiding direction for conducting an IBR. It has both intrinsic and extrinsic properties. 

The intrinsic value of the IBR investment resides in those specific elements of information (as identified by the customer in the form of questions or concerns) that are either exchanged, clarified, or refined through the course of discussions between the customer and performing contractor teams. This intrinsic value can be measured by how well the exchanged information supports:

  • A complete, clear and mutual understanding of the work to be accomplished.
  • The resources needed to get the work done.
  • The detailed plan to perform the work.
  • What resources are available to support the plan.
  • What’s missing or unknown that is needed to complete the work correctly and on time.
  • What risks, issues, concerns, or opportunities are associated with contractor’s concept that need to be fully considered to make the plan work. 

The extrinsic value of the IBR Investment rests wholly in the quality of the exchanges (discussions), and the resulting actions generated from the discussions. This extrinsic IBR value addresses how appropriate, rich and comprehensive the information exchanges were, and answers to questions, such as:

  • Were the discussions responsive to a list of customer information requirements and concerns? 
  • Were the right discussions held? At the right level of detail?
  • Were the right people involved in each discussion? 
  • Did the discussions provide sufficient context? Were they comprehensive? Complete?
  • Did the discussions address associated risks, issues, opportunities or other concerns? Relationships to other discussions/elements?
  • Were all the customer’s questions or concerns answered to their satisfaction?
  • Were the discussions documented to support decisions? Alternatives? Changes? Studies?

The exchanges of essential information (intrinsic value) and the quality of those exchanges (extrinsic value) when combined directly translate to the investment value achieved from the IBR. It characterizes how well the information exchanged provides both teams with the necessary details to successfully define, schedule, budget, and manage the contracted effort relative to the investment into the IBR process. A realistic, risk adjusted PMB helps to prevent schedule delays and cost overruns during project execution that often impact a contractor’s profit margins and tarnishes their credibility with their customers. 

What are the characteristics of a value added IBR approach?  

A successful approach H&A earned value consultants have observed contractors implement is a structured process corporate management actively participates in to ensure they gain the most value from all IBR events. 

This is often an outgrowth from corporate initiatives to retain top project management talent and establishing an EVMS self-governance process. It is part of a corporate culture that is committed to excellence in project management and sustaining a best in class EVMS – becoming efficiently expert at EVM

What are some common characteristics of their IBR approach?

  • A chartered authority or corporate team responsible for assisting project personnel with IBR events in addition to EVMS implementation, self governance, and customer surveillance events. A good practice we have seen implemented is to establish rotating members on the IBR teams from different projects as a means to pollinate best practices across projects. It also provides an opportunity to mentor top talent on track to move up to higher management positions.  
  • A standard repeatable process with defined measurable outcomes that can be tailored to the unique project requirements or objectives. This includes maintaining a set of materials for the internal IBR team to effectively plan and execute an IBR as well as to close out any action items. Examples include training materials to prepare project personnel, process description with team member roles and responsibility assignments, data call list, role based interview question forms with assessment criteria, data quality assessment materials and tools, list of data traces to be performed, schedule risk assessment tools, risk/opportunity evaluation criteria, defined assessment criteria (technical, schedule, cost, resources), in-briefing and out-briefing templates, and template to capture action items to track to closure. The corporate team is often responsible for actively maintaining the content for the IBR teams and conducting training. 
  • They place an emphasis on two components that directly impact the quality of the schedule and cost data.  This includes:
    • Well-documented data driven basis of estimates (BOEs) that can be substantiated using historical or bench-marked data with the goal of reducing expert judgement cost estimates to the lowest level possible as a risk reduction strategy.  
    • The quality of the risk/opportunity management plan and the content in the risk/opportunity register. This content directly affects the ability of all parties to gain a better understanding of the risks/opportunities and best options to mitigate a risk or capture an opportunity. A well constructed schedule is required to be able to perform schedule risk assessments (SRAs). SRAs help to identify where duration risk exists in the schedule and to determine a level of confidence in meeting major project milestones as well as the project completion date.  
  • They perform internal IBRs as a standard practice on all projects regardless of contractual requirements. This is particularly important when subcontractors are performing a substantial percentage of the work effort. The corporate team often assists Project Managers with conducting a joint IBR with major subcontractors.  

Need help establishing a corporate IBR process?

H&A earned value consultants often help clients to establish a corporate EVM council or center of excellence with defined responsibilities to ensure project personnel effectively implement their EVMS, integrate risk/opportunity management into the EVMS, as well as define and implement a standard repeatable process for IBRs and self-governance. Clients often need assistance establishing a repeatable process for conducting schedule risk assessments, an essential component of the IBR process. A defined process that clearly articulates the expected measurable outcomes from conducting IBRs is one way to ensure all parties gain the most value from the event with the end objective of ensuring a realistic and executable PMB has been established.  

Call us today to get started.  

Maximizing the Value from Integrated Baseline Review (IBR) Investments  Read Post »

Video Release – Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 2 of 2

, , , , , , , , , , ,

The conclusion of our review of the foundational elements of performing a schedule risk assessment (SRA) using Acumen Risk 6.1

0:17 – Risk Exposure Chart
1:03 – Tornado Chart
2:14 – Parting Thoughts

Read the blog post at:

Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 2 of 2

Video Release – Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 2 of 2 Read Post »

Video Release – Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 1 of 2

, , , , , , , , , , ,

 

How confident are you that your project will finish on time? Review the foundational elements of performing a schedule risk assessment (SRA) using Acumen Risk 6.1

2:22 – Schedule Health Diagnostics
4:55 – Duration Uncertainty
6:35 – Risk Events
8:20 – Simulation Process
 
Read the blog post at:

Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 1 of 2

Video Release – Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 1 of 2 Read Post »

Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 1 of 2

, , , , , , , , , , ,

Why Perform Schedule Risk Assessments? EVMS and Agile implementations within the same company or on the same project.

Before a project is ready to be baselined, a typical question the customer asks the project manager is, “How confident are you that the project will finish on time?”

This is a more difficult question than you might think.  In competitive environments, guessing is not an option.  The probability of success on a project must be quantified.  The risks that impact the odds for success must also be quantified.  If the risk is managed, the probability of completing the project on time and under budget is improved.

Customers are not blind to the importance of risk management.  This is evidenced by recent changes in government contracting requirements that call for formal risk assessments of project schedules.  Even if risk management were not a contractual requirement, it would be irresponsible for any project manager to ignore the need for risk management and proceed without identifying and assessing the project’s risks.

Schedule risk exists in every project.  This risk can be quantified, analyzed, and mitigated, or it can be ignored.  However, ignoring schedule risk does not make it go away.  Fortunately, there are advanced software tools, such as Deltek’s Acumen Risk, that can help model the expected impacts of risk in the schedule. Then, the answer to “how confident are you that the project will finish on time?” can be answered with quantifiable information.

In the following sections, a few of the foundational elements of performing a schedule risk assessment (SRA) using Acumen Risk 6.1 will be discussed.  The software was designed with the understanding that not everyone is an expert in schedule risk analysis.  The software provides beginners with an easy to follow path to perform in-depth schedule risk analysis as well as advanced features for experienced risk experts.

Along with quick start guides and help documentation, the menu structure is laid out like a schedule maturity timeline.  From left to right, the menu selections take one from the start-up steps of importing the schedule, to analyzing the schedule, assessing schedule risk, accelerating the schedule, and advanced customization features.

Deltek_Acumen-Top-Level_MenusDeltek Acumen – Top-Level Menus

 

Schedule Health Diagnostics

Before delving into schedule risk assessments, let’s take one minor detour from risk into schedule diagnostics.

Would you trust a broken watch to tell you the correct time?  The same goes for a schedule risk assessment.  A broken schedule network cannot be trusted to yield reliable, and therefore actionable, SRA results.

The National Defense Industrial Association (NDIA) Integrated Program Management Division (IPMD) Planning & Scheduling Excellence Guide (PASEG), is widely regarded as one of the premier references on scheduling best practices.  The PASEG was created by a joint team of both government and industry scheduling experts, thus it has no particular point of view to promote or defend.  One of the scheduling best practices the PASEG discusses is that the integrated master schedule (IMS) should be validated before any SRA is performed.  “Validated” means that the tasks, logic, durations, constraints, and lags in the IMS should be analyzed and corrected as necessary.

Acumen Fuse provides a complete set of schedule diagnostics.  When I first clicked on the “Diagnostics” tab, I saw an initial set of metrics.

EVMS: Acumen Fuse Schedule diagnostics

Each one of these metrics was applied to the project’s timeline that which makes it easy to see both where and when the issues occur.  What I did not notice at first was that these metrics were just one subset; I was only looking at the “Schedule Quality” subset of the diagnostics.  There were similar subsets in the areas of Logic, Duration, Constraints, Float, and the DCMA 14-point Schedule Assessment, just to name a few.  All of these diagnostic tests can be modified to reflect your company or customer’s standards.

Before leaving the topic of schedule health, there are a few words of caution.  No matter how useful a schedule analysis tool may be, there is no substitute for the task managers taking ownership of the IMS and ensuring that it is in good working order.  For example, analysis software can be used to check to determine if a task has a predecessor and a successor, but only someone familiar with the effort can determine if a task has the “correct” predecessor and successor.  Analysis software is becoming more and more sophisticated, but people still control the success or failure of the project.

Duration Uncertainty

Once a sound schedule has been developed, the next foundational elements of an SRA are the duration uncertainty estimates.  There are two widely accepted methods of assigning duration uncertainty.

The preferred and more precise method is to obtain three-point duration estimates (best case, worst case, and most likely) from the task owners.  At a minimum, this should be performed on all critical and near-critical tasks (and driving and near-driving tasks supporting significant events).  For larger schedule networks, it may not be reasonable to gather this type of information for every task.  If custom three-point estimates are not available, templated duration uncertainty could be applied based on the type of work, the task owner, historical performance, or any other applicable task characteristic.

Acumen Risk handles both methods very easily.  Custom three-point estimates can be entered for each task in days (or hours), or as a percentage of the current remaining duration of the task.  Standard duration uncertainty templates are easily applied to a task by selecting the appropriate risk level on the calibration bar.  To streamline the process, by setting the calibration at any summary level, the uncertainty template is cascaded down to all the “children” tasks.

Description. Calibration.

Risk Events

One thing traditional Critical Path Method (CPM) networks do poorly is model unexpected results.  For example, if there is a 90% success rate on fatigue testing, the IMS will generally be constructed to assume the test will be successful, with no disruption to downstream tasks.

EVMS: Critical Path Method

But what happens if the test fails?  While unlikely, there is still a very real possibility that the results will be unfavorable.  If the test does return unfavorable results, there will likely be a significant delay while re-work is performed in the areas of design, build and test.  A traditional CPM network can model a successful test or an unsuccessful test, but not both.  This is not a problem with a schedule risk assessment.  Information from the project’s risk register can be used to model the likelihood of a test failure, as well as the consequence, or delay to downstream tasks resulting from that failure.

EVMS: CPM Risk Events Consequence

Is this an acceptable risk?  An SRA can quantify the risk and provide information on the likelihood of successful deliveries.  Acumen does not stop there though.  One of its newest features is to organize and track all risk events within its built-in risk register, as well as to track the steps being taken to help mitigate that risk.  Or, if your organization already maintains an external risk register in Excel, it can be imported into Acumen to eliminate the duplicate tracking of risk events.  Whether the risk register is imported from Excel or built from scratch within Acumen, a single risk event can then be mapped to one or more activities, or a single activity can be associated with one or more risk events.

EVMS: risk registers 

 

Simulation ProcessEVMS: Simulation Process

A typical SRA uses Monte Carlo techniques to simulate hundreds or thousands of potential project outcomes using the risks and uncertainties that have been supplied.

For most users, simply accepting the default settings and pushing the “Run Risk Analysis” button would be sufficient.  But if terms like “Convergence”, “Correlation Coefficient”, “Central Limits Theorem” and “Seed Value” are part of your normal working environment, Acumen provides a variety of settings that can be customized to tune the SRA to best model your project.

No matter which approach you take, the Acumen toolset provides a quick and easy simulation process.

 

 

 

 

 

 

 

 

 

 

What to Expect in Part 2

Part 2 of this blog will delve into the interpretation of SRA results.

 

Yancy Qualls, PSP

Engagement Director, Schedule Subject Matter Expert (SME)

Humphreys & Associates, Inc.

Assessing Schedule Risk Using Deltek’s Acumen Risk 6.1 | Part 1 of 2 Read Post »

Scroll to Top