EVMS

Merging Earned Value Management System Descriptions

By / / , , ,
, , ,

Are there best practices that apply when a company with an approved/certified Earned Value Management System (EVMS) acquires another company that also has an approved/certified EVMS in place? What happens with the EVM System Descriptions as well as related processes and procedures? What about the various project control tools being used? How do you level set the project control proficiency levels of personnel using the EVMS? Schedule and cost level of detail and data architecture also come into play. For example, project performance data is often used at the corporate level for financial analysis, portfolio analysis, and external reporting and may require data to be organized in a specific manner. Is the EVMS providing reliable status, forecast completion date (FCD), and estimate at completion (EAC) information to management? 

What are your options?

H&A earned value consultants have observed different approaches and often assist companies with determining their strategy. Assuming you are the acquiring company, you could:

  1. Require the acquired company to use your EVMS. 
  2. Let them continue using their EVMS for an agreed upon timeframe or indefinitely.
  3. Take the best of both and establish a new and improved EVMS.  

Each option has its pros and cons. There are also other implications for at least the acquired company. DCMA will need to conduct an Integrated Baseline Review (IBR) and/or a compliance review if the acquired company’s EVMS assessment is no longer applicable. If you make significant modifications to your EVM System Description, DCMA will need to review the revised System Description to determine whether it still complies with the EIA-748 Standard for EVMS guideline requirements.

Things to Consider

  • What do you want to achieve?

    If the goal is to establish a common EVM System Description across the corporation, the strategy will need to reflect that. Define the business objectives that clearly articulate the benefits of using a standardized approach that can help to create and implement the plan to achieve your goal. In this example, that could narrow your path forward to either option 1 or 3 depending on the state of your EVMS.

  • What is the state of your current EVM design and System Description?

    Do you already have a best in class corporate level system in place? If yes, option 1 is a good fit. The strategy would be to create a plan to transition the acquired company to your EVMS. A single EVMS is easier to maintain and to train people on how to use it effectively. Commonality makes it easier to move personnel between projects.

    Perhaps your company is fine with different EVM Systems at a business unit level. For example, perhaps the business units have a different customer base (DoD versus DOE), and the requirements are different. In this case, it may make sense to go with option 2. We recommend being prepared to do an in-depth assessment of the acquired company’s EVMS to become familiar with it, gain an understanding of how project personnel use it, and evaluate the quality of the schedule and cost data. It is imperative that you have a good understanding of the strengths and weaknesses of the acquired company’s EVMS. You may find best in class practices that you could incorporate into your EVMS. On the other hand, you may discover issues you need to address with a corrective action plan. Some of them may be as simple as providing desktop instructions for the schedulers or control account managers (CAMs). The more difficult are actions taken to change the culture such as resistance to providing visibility into the data.

    Option 3 may be good path in situations where you know there are components in your EVMS that need to be streamlined or enhanced. It provides an opportunity to fix known issues with your EVM design or System Description. It could also be an opportunity to replace a mix of software tools or home-grown tools with a standard set of commercial off the shelf (COTS) schedule, cost, and analysis as well as risk tools. Integration with a standard Agile tool may also come into play. In this case, your strategy may be to create a working group from both companies to create a best in class corporate EVMS. 

  • Structure of the EVM System Description.

    There may be “layers” to it that makes it easier to accommodate unique business unit environments. For example, perhaps you have established a corporate level System Description that states what the company does to comply with the EIA-748 guidelines when an EVMS is contractually required or what is required to satisfy internal management needs for project/portfolio analysis (no external customer management system or reporting requirements). The corporate level system should define specific rules all business units are expected to follow. The business units define how they comply with the corporate requirements (their specific process). A good approach is to also allow project managers to define project directives to specify project unique requirements as long as they comply with the corporate and business unit requirements.

    In this example, option 1 is a good fit. The strategy would be to help the acquired company to establish revised EVM processes that align with the corporate requirements similar to other business units in the corporation.

Other Considerations

Your strategy and tactical plan must address identified risks and opportunities. A common challenge is resistance to change. A potential risk mitigation approach could be to bring in the acquired company’s personnel as part of a joint corporate management team with the goal to create a single best in class EVMS. It is essential to establish ownership in the new or revised process. An example from one H&A client illustrates the importance of taking ownership in the EVMS as part of a successful transition.

“We didn’t force what we had on them, nor did we give in. We have a corporate EVM System Description. When we acquired the company, we brought them in to do a revision of the System Description, as the decision was made that we will operate as one company. They are now using that System Description and are using the same EVM cost tool. We are working other initiatives to harmonize other systems. It was surprisingly not contentious. We incorporated their leads into the organization with minimal disruption. We also have corporate training, which they supported and some of their legacy folks are leading that. The company as a whole changed, rather than forcing our way on them. Not many major differences between us, but inclusion of the folks from the acquired company as well as business groups was key. Frankly, one of our legacy divisions was harder to work with than anyone from the company we acquired.” – EVMS Director, A&D Contractor

While this is an example of where things went well, your risk mitigation approach should be prepared for situations where the teams do not agree upon the documented process, tools, or training that could result in an impasse. Knowledge of the current internal environment and personnel mix can help to determine the best mitigation strategy. A strong leadership team must be in place to ensure teams are working to achieve common objectives and to amicably resolve differences with a target completion date.

The tactical plan must also include a robust training plan that covers the revised EVMS process, procedures, and any new tools. This is critical to ensure project personnel gain a good understanding of what changed, who is responsible for what, workflow, requirements such as data coding or level of data detail, and how to use the tools effectively. Role based training is often useful to ensure project control personnel, schedulers, CAMs, and others are following the documented procedures specific to their day-to-day tasks. Desktop instructions are also useful to ensure project personnel are using the software tools effectively in alignment with the documented process and procedures.

What to do if you find yourself in this situation?

It often helps to start with a gap analysis of your or the acquired company’s EVM design and System Description as well as assess how project personnel implement the system and the quality of the data. H&A earned value consultants often conduct an EVMS gap analysis to provide a fact-based and independent analysis of the EVMS, project personnel proficiency levels, and quality of the schedule and cost data. Once you are able to identify and quantify the strengths and weaknesses of the system, you are in a better position to determine your best strategy that aligns with your corporate business objectives and goals.

Over the years, H&A earned value consultants have observed first-hand what strategies and tactics for designing and implementing a best in class EVMS ensures success in a variety of business environments. We can also help you avoid common pitfalls that can derail the best laid plans – it is often the case a client didn’t realize there were hidden risks, or they had made incorrect assumptions.

We can help you determine the right strategy for your situation. Call us today at (714) 685-1730 to get started.

Merging Earned Value Management System Descriptions Read Post »

Management Reserve Best Practice Tips

By / / , , , , , ,
, , , , , ,

A recurring theme H&A earned value consultants find themselves discussing with clients is emphasizing that management reserve (MR) is a very precious budget set aside that must be protected and used appropriately. Unfortunately, MR is often used inappropriately, and quickly depleted in the early stages of a project.

What happens when MR is consumed for other uses than what it was intended? There is no budget available for appropriate uses of MR such as for emerging work, rework, redesign, or make/buy adjustments within the scope of the contract when it is needed in the latter stages of a project. When that happens, a project manager is forced to create a “home” for actual costs for these activities. This results in other inadvisable actions such as:

  • Zero budget work packages which are also known as estimate to complete (ETC) only work packages.
  • De earning the budgeted cost for work performed (BCWP) and opening completed work packages to accept charges.
  • Culling budgets from future unopened work packages, and if they exist, planning packages, summary level planning packages (SLPP), and undistributed budget (UB).

These actions will call into question the integrity of the EVMS and EVM data. The customer conducting EVMS surveillance will also be quick to point out this deficiency in the EVMS implementation and raise the issue to ensure it has management’s attention to correct. The inappropriate use of MR has created a cascade of problems that could have been avoided. In some instances, project personnel were simply not following the rules for the use of MR found in the contractor’s EVM System Description. That’s an easier problem to resolve than other root causes.

The Role of Risk and Opportunity Management in Establishing MR

What H&A earned value consultants often uncover as the root cause of inappropriate uses of MR was that a robust risk and opportunity (R&O) management process would have made a difference in establishing a quantified set aside for MR to handle realized risks. Proactively identifying and managing risks improves project performance. The expectation of specific risks occurring leads to risk handling plans that lower the likelihood and impact of risks. It also provides an informed basis to establish an adequate amount of MR that reflects identified and assessed risks.

The risk assessment provides additional information that assists a project manager’s decision making process to validate a request to use MR is appropriate and has the backup data needed to justify the use of MR and the amount of MR allocated. This detail is necessary for the baseline change request (BCR) approval process as well as the Integrated Program Management Report (IPMR) Format 5 or Integrated Program Management Data and Analysis Report (IPMDAR) Performance Narrative Report (PNR). A project manager is required to identify the changes to MR during the reporting period and provide a brief explanation of the change. This explanation has the potential to pique the interest of the customer to gain a better understanding of why MR was used and the potential impact to the integrity of the EVM data.

Note: MR may increase or decrease for a variety of reasons. The primary use of MR is to handle realized risks within a control account that is within the statement of work (SOW) for the contract. All MR debits or credits should be tracked in a log for full traceability for the entire life of the project. Remember that MR can never be a negative value.

Acceptable Uses of MR

As highlighted in an H&A article titled “The Effective Use of Management Reserve,” examples of the appropriate uses of MR include:

  • Newly identified work is authorized and assigned to a control account manager (CAM). It may be that once the work begins, one or more tasks that were missed in the original planning process now need to be scheduled and resource loaded. Newly identified work could also be the result of internal replanning that required a change in approach or resource requirements.

    An example of this could be a project manager issued a work authorization to a CAM to conduct three tests to meet the requirements in the contract SOW. In the middle of the first test, it becomes clear to the CAM and project manager that a fourth test will be necessary. The project manager and CAM should be aware of this potential risk and be prepared to implement their risk handling strategy as a result of the R&O management process. The CAM can quickly prepare a BCR that the project manager can immediately approve to allocate MR budget to complete the fourth test. 
  • It is necessary to redo a task. This may include unanticipated redesign, remake, or retest. Hopefully, the project’s risk register identified the potential risks associated with the original tasks and management was prepared for the realized risk. 
  • Make/buy adjustments.  This could result in an MR debit or credit. 
  • Statement of work transfers from one organization to another. This could result in an MR debit or credit. 

Inadvisable Uses of MR Commonly Allowed

Although it is often allowed in a contractor’s EVM System Description, it is inadvisable to use MR for direct and indirect rate changes in the future. Note: MR should never be used to make any rate adjustments (or any other adjustments) to historical budgeted cost for work scheduled (BCWS) or BCWP data.

A rate change is not a change to the SOW for a CAM. It is merely a change to the cost of that work. Cost variances that occur because of direct and/or indirect rate changes can easily be explained in a Variance Analysis Report (VAR). Ironically, this use of MR is typically treated as a one-way street. Contractors apply MR when the direct and/or indirect rates are going up in the future but do not return to MR when the rates are projected to go down.

When a contractor’s EVM System Description allows MR to be used for future direct and/or indirect rate changes, ideally, the likely rate changes are identified as a risk and quantified when the initial MR is established for a project. This requirement should be noted in the EVM System Description. That way the set aside for MR includes budget for corporate rate adjustments that are outside of the control of the project manager or CAM. 

Another example of a commonly allowed but inadvisable use of MR is to “true up” a purchase order that is in excess of the original budget at completion (BAC) for material, equipment, or purchased services. For example, a project manager issues a work authorization to a CAM that includes purchasing material, equipment, or services from a supplier. The CAM then reaches an agreement with a supplier with scope, schedule, and budget. If that agreement is greater or less than the BAC, MR should not be applied, nor should budget be returned to MR to make the BAC match the PO value. Assuming the scope does not change, then MR should not be used to wipe out a cost variance whether positive or negative. The cost variance can be easily explained and the EAC can be increased or decreased. This is another example where contractors are treating this as a one-way street; they apply MR when it goes up, but do not return to MR when it goes down. A contractor would not “true up” for internal work overruns/underruns so why “true-up” for material or services provided by a supplier? 

Best Practice Tips

The following is a short list of best practices H&A earned value consultants often recommend clients implement for managing MR.

  • The EVM System Description should clearly spell out what are appropriate and inappropriate uses of MR. It should also provide guidance to eliminate instances of the “one way street” debit from MR. If needed, provide supplemental procedures, decision trees, or other work instructions to help project personnel follow EVM best practices and preserve MR for handling realized risks which typically occur in latter stages of a project.
  • Ensure that the R&O management process is integrated with the EVMS and provides the necessary risk identification and assessment information for the project manager to establish a realistic MR set aside based on quantifiable information. Where applicable, ensure likely rate changes are captured as a potential risk to the project and considered when the initial MR for the project is established if they intend to use MR for rate changes in the future.
  • Conduct recurring training to reinforce the purpose for MR and the appropriate use of MR. A recommended approach is to discuss a variety of use cases with project personnel so they know how to handle various situations that may occur on a project. 

Have you noticed “creative” uses of MR that are contrary to EVM best practices? Hopefully, you identified those situations as part of your EVMS self-governance process and were able to quickly implement corrective actions before your customer pointed out the issue to you. H&A earned value consultants often assist clients with producing procedures or work instructions that clearly spell out how to use MR appropriately. We also offer a range of EVMS training to reinforce EVM best practices including the appropriate use of MR. Call us today to get started.

Management Reserve Best Practice Tips Read Post »

Level of Effort (LOE) Best Practice Tips

By / / , , ,
, , ,
Level of Effort (LOE) Best Practice Tips

Clients are often seeking advice from our earned value consultants about implementing a practical approach in response to government customer requirements to proactively manage level of effort (LOE) tasks. The DoD EVMS Interpretation Guide (EVMSIG), NASA guidance, and DOE guidance such as the Compliance Assessment Governance (CAG) document clearly state the requirements for contractors related to planning, maintaining, and managing LOE. DOE also specifies a limit to the percentage of LOE allowed within a control account to avoid skewing performance measurement of the discrete work effort. In addition, both the DCMA and DOE EVMS data quality test metric specifications include manual and automated tests with thresholds specific to LOE.

Common accepted best practices for LOE include:

  • Reducing the amount of LOE to the lowest level possible to minimize the number of activities that need to be actively managed. Objective measures of performance are always preferred.
  • When LOE activities are included in the schedule, they should not drive the date calculations of discrete activities in the integrated master schedule (IMS). They should also not appear on the critical path.
  • LOE must be segregated from discrete work effort. In practice, this means a work package can only be assigned a single earned value method. The work package is one of three types. It is either 1) discrete effort with an assigned earned value technique such as the Milestone or Percent Complete technique, 2) apportioned effort, or 3) LOE. 
  • It must be verified it is truly LOE, i.e., it is management or sustainment type of activity that has no identifiable end products or established relationship to other measurable effort. It is clearly not discrete effort or apportioned effort. Remember that with LOE, the passage of time is the only measurement criteria. At the end of the performance month, the budget value for that month is earned. For this reason, LOE is the least desirable earned value method. 
  • The budget or estimate to complete the work effort is time phased and reflects the planned or forecast period of performance. The period of performance and resource requirements must be substantiated. Determining the basis of estimate for the LOE activity can also help to verify the work is truly LOE.

So, what is the problem? 

Common situations H&A earned value consultants run into are contractors where:

  • Managing the LOE is put on “auto pilot.” This might work for project management type of activities that span the duration of the project. It does not work so well when the LOE is associated with the occurrence of discrete work effort that is subject to change – i.e., the discrete work effort duration changes or the start date and/or the complete date changes. The result?
  • LOE tasks may incur actual cost of work performed (ACWP) with no budgeted cost for work performed (BCWP);
  • LOE tasks earn BCWP with no ACWP; or
  • The estimate at completion (EAC) is greater than the ACWP with BCWP equal to the budget at completion (BAC).

    Any one of these conditions would trip the DCMA and DOE test metrics and should be avoided. These types of situations were illustrated in a previous blog, “Level of Effort Decision Tree” that discusses how to properly replan LOE. 
  • Their EVM System Description doesn’t provide sufficient guidance to project personnel on what proactive management of LOE means. What are the rules for planning and maintaining LOE? How is LOE handled differently from discrete work packages?

    Some System Descriptions allow LOE replanning to occur within the “freeze period,” usually defined as the current reporting period and often plus one additional month. This is contrary to other best practice guidance about how to handle changes for open discrete effort work packages. For discrete effort work packages, changes within the freeze period are not allowed and the work package must be closed to replan the remaining work. What’s the process for handling that open LOE work package? What about retroactive changes when the LOE work occurs earlier or later than planned, or the duration is different than planned? Then what?

    When project personnel lack guidance, then arguments often ensue about what is the “correct” interpretation of the wording in governing documents or test metric specifications that are often inconsistent.
  • Validation checks are not routinely performed. This includes validation checks to ensure that control account managers (CAMs) are selecting the appropriate earned value method for a work package following the EVM System Description guidance during the work definition and planning phase. It also includes routine monthly data checks to identify common data anomalies typically associated with LOE such as ACWP and no BCWP or BCWP with no ACWP. The goal is to fix problems in the current reporting month and avoid making any retroactive changes. You should be catching and fixing avoidable DCMA or DOE EVMS test metric “triggers” every reporting period.

Best Practice Tips

Here is a short list of best practice tips that H&A earned value consultants have helped clients to implement over the years to ensure LOE is properly planned and proactively managed. The approach is tailored for each client to reflect the type of work the company typically performs. This is documented in their EVM System Description, related procedures, and recurring training to ensure project personnel have clear, specific guidance they can follow.

  • Consider using the Percent Complete earned value technique instead of LOE. A best practice is to identify quantifiable backup data (QBD) for a work package using the Percent Complete earned value technique. The QBD for the LOE type of work package could be the milestones identified for the discrete effort work package the LOE work package is supporting. This helps to ensure the work packages are reviewed and managed together.
  • Could the Apportioned Effort method be used instead of LOE? Is it possible to establish a direct relationship between the discrete effort and supporting effort? For example, is historical data available to document that the support number of hours is a given percentage of the discrete effort labor hours? If so, then using the Apportioned Effort method is a much better alternative. When the discrete work package is statused, the apportioned effort work package would be automatically statused as well. 
  • Consider shorter durations for the LOE when that LOE is supporting discrete effort. Should the first occurrence of the LOE trigger a data anomaly test metric, it can be proactively handled along with any future replanning. The remaining LOE would already be in one or more separate work packages so there won’t be any criticism for changing open work packages. Any adjustments can be made in the current reporting period avoiding any retroactive changes that would trigger other data metric tests. What is considered to be “short duration” should be defined in the EVM System Description. An example would be LOE work packages of 3 to 4 months in duration. Be sure to provide specific guidance to project personnel on how to process these types of current reporting period LOE replanning adjustments. The LOE work package breakpoints should be technically related. For example: “Phase I Support,” “Drawing Support,” and so forth instead of generic descriptions such as “April Support,” “May Support,” or “June Support.”
  • Use rolling wave planning. This is by far one of the better solutions. This helps to ensure the discrete tasks and any supporting LOE tasks are planned together before the work is authorized to begin. Shorter durations for the LOE tasks are often used to align with the forward planning window.
  • Incorporate LOE earned value method checks into your routine status and data analysis process. Identify any upcoming LOE activities (for example, the next 60 to 90 days), along with the CAM responsible for the work effort to verify they accurately reflect the current plan. It is always better to proactively replan future LOE when needed instead of defaulting to an “auto pilot” mode. The CAM should understand this is part of their responsibilities.

Does your EVM System Description or training materials need a refresh to include specific guidance for project personnel that documents the preferred approach for planning, maintaining, and managing LOE? H&A earned value consultants frequently help clients with EVM System Description content enhancements or creating specific procedures that reflect your unique business environment. Call us today at (714) 685-1730 to get started.

Level of Effort (LOE) Best Practice Tips Read Post »

Incorporating IMS Information Directly into Independent Estimate at Completion (IEAC) Formulas

By / / , , ,
, , ,
Incorporating IMS Information Directly into Independent Estimate at Completion (IEAC) Formulas

“When you need to discuss the schedule, look at the schedule.”

– A Scheduler’s Lament

There are many existing formulas for calculating an Independent Estimate at Complete (IEAC) from earned value data. A recent study of a sample of projects found that the calculated IEACs analyzed at the 25%, 50%, and 75% complete points were not accurate when compared to the final actual cost of work performed (ACWP). The following table lists the thresholds used to assess the accuracy of the IEACs at the different complete points for the sample projects.

Percent CompleteAccuracy Threshold
25%Within +/- 10% of final ACWP
50%Within +/- 7% of final ACWP
75%Within +/- 5% of final ACWP

While working on that study of the accuracy of commonly applied IEAC formulas as well as on a small project as an analyst for a customer, the idea for using data directly from the integrated master schedule (IMS) in conjunction with the cost performance data to create a new IEAC formula emerged.

Using Data Directly from the IMS to Calculate an IEAC

It should be noted that none of the generally used IEAC formulas use data directly from the IMS. The IEAC formulas use data found in the cost performance portion of the earned value monthly reports to customers.

IMS data is only used indirectly in the IEAC formulas. When a task is started and progress updated, the earned value (the budgeted cost for work performed or BCWP) is developed from the progress reported. This is measured against the cost baseline (the budgeted cost for work scheduled or BCWS).

At the same time, in the IMS environment, the schedule analysts are calculating the Baseline Execution Index (BEI) for task completions/finishes. BEI (for finishes) measures how many of the tasks baselined to be completed by the cut-off date were completed. If all the tasks were done (BEI = 1), their value would have been earned. Of course, other tasks could have started, progressed, and maybe even finished. For this example, the Schedule Performance Index (SPI) calculated at that point (BCWP/BCWS) should be at least 1 and potentially higher. The SPI reflects the baseline value of completed tasks plus the in-process claimed baseline value. The in-process claimed value can be subjective in some cases.

The argument, if there were one, might be there is no need to try and include BEI or similar schedule measures in the IEAC formulas since they already include SPI.

However, there is a whole different and unique set of information coming from the IMS that is not currently used in the IEAC formulas. That information is what we chose to call “Duration Performance” and “Realism Ratio.” These are measures of the actual duration for completed tasks and the forecast duration for future tasks.

Calculating Duration Performance

The IMS data includes the baseline number of days assigned to each task as well as the actual number of days to complete each task. If a task is baselined to take 10 days (Baseline Duration = 10) and the task took 15 days to complete (Actual Duration = 15) then it is taking 150% of baseline to do the work.

This is similar to the Cost Performance Index (CPI) that uses the BCWP and the ACWP to determine how efficient the work performance has been. The formula BCWP/ACWP shows how the work accomplished compares to the cost of that work performed.

If we assume, for labor at least, that taking longer to complete a task often leads to costing more than baselined, we can use the Duration Performance to develop an IEAC.

To develop the Duration Performance, we would use the IMS from the month being analyzed to perform the following actions:

  1. Filter out all summary tasks and look only at real work tasks.
  2. Decide what to do with level of effort (LOE) – keep it or ignore it.
  3. Filter for all tasks that are completed (100% complete).
  4. Add up the baseline duration in days for all these completed tasks.
  5. Add up the actual duration days for these same completed tasks.
  6. Compare the actual duration days used to the baseline duration days.

An example would be:

  • 100 completed tasks
  • Total baseline days duration = 1,000
  • Total actual days duration = 1,500
  • Duration Performance = 1,000 / 1,500 = .67

One of the common IEAC formulas is the “SPI times CPI” that is calculated like this: ACWP + Budgeted Cost of Work Remaining (BCWR) / (CPI x SPI) where BCWR = Budget at Completion (BAC) – cumulative to date BCWP.

Now that we have a duration performance factor, we can develop a new IEAC. The Duration Performance IEAC would be done using the CPI from the same month as the IMS where ACWP + BCWR / (CPI x Duration Performance Index).

Using some actual data from a project for a single month we see:

  • Duration Performance Index = .82
  • BEI = .72
  • CPI = .92
  • SPI = .94 (significantly higher than the BEI)
  • ACWP = $9.2M
  • BCWR = $18.3M
  • IEAC using standard formula with CPI x SPI = $9.2 + $18.3 / (.92 x .94) = $30.3M
  • IEAC (Duration Performance) = $9.2 +$18.3 / (.92 x .82) = $33.5M

Assessing the Realism Ratio

When we look at the remaining tasks to be completed, we can use the Realism Ratio to assess how the future forecast durations compare to the performance so far.

The data needed are the baseline duration and the forecasted duration for all tasks that have not been started. This concept excludes in-process tasks. In our example from before, the data we created looked like this:

  • 100 completed tasks
  • Total baseline days duration = 1,000
  • Total actual days duration = 1,500
  • Duration Performance = 1,000 / 1,500 = .67

We would use the same IMS to do this:

  1. Filter out all summary tasks and look only at real work tasks.
  2. Decide what to do with LOE – keep it or ignore it.
  3. Filter for all tasks that are not started.
  4. Add up the baseline duration in days for all these tasks not started.
  5. Add up the forecasted duration days for these same tasks not started.
  6. Compare the forecasted duration days to the baseline duration days.

Let’s say there were 100 tasks not started. If the forecasted days were 1,000 and the baseline days were 1,000 that would yield 100%. When we did the example, the Duration Performance was .67. This means that performance to date was .67 but the future will be 100% or 1. You can see the disconnect. That disconnect we call the Realism Ratio (in this example, .67/1).

Data from the actual project for the same month as discussed earlier shows:

  • Duration Performance = 122% of baseline
  • Future Performance = .86 or 86% of baseline.

This means that the future durations are cut significantly.

We would use this data to develop a factor called a Realism Ratio (86/122 = .70) and that would be used to develop an IEAC using this formula: IEAC (Realism Ratio) = ACWP + BCWR / (CPI x Realism Ratio).

Using the same sample project data from above and adding in an assessment of the forecasted durations for the remaining work, we see:

  • Duration Performance = .82
  • BEI = .72
  • CPI = .92
  • SPI = .94 (significantly higher than the BEI)
  • ACWP = $9.2M
  • BCWR = $18.3M
  • Realism Ratio = .70
  • IEAC using standard formula with CPI x SPI = $9.2 + $18.3 / (.92 x .94) = $30.3M
  • IEAC (Duration Performance) = $9.2 +$18.3 / (.92 x .82) = $33.5M
  • IEAC (Realism Ratio) = $9.2 +$18.3 / (.92 x .70) = $37.6M

The project is not complete, so the final ACWP position is not known. There is a dramatic difference between the three IEACs. The difference between BEI and SPI indicates that in-process tasks and other factors such as LOE are potentially affecting SPI.

What can we learn from this sample project?

In this example, additional investigation is warranted. There are potential issues with the realism of the baseline and current schedule that are signaling a cost growth issue is likely to occur. Relying on just the time-phased cost data for IEAC calculations may not be sufficient to assess whether a contractor’s range of EACs included in their monthly cost performance reports are realistic. For more discussion, see the blog on Maintaining a Credible Estimate to Completion (EAC) and the blog on Using EVM Performance Metrics for Evaluating EACs.

Are there lurking cost growth surprises in your projects? You may want to consider revisiting your estimate to complete (ETC) and EAC process to verify there is an integrated assessment of the schedule and cost data to identify potential disconnects. H&A earned value consultants can provide an independent assessment of the quality of the data as well processes and procedures to help you verify your EACs are realistic. Call us today at (714) 685-1730.

Incorporating IMS Information Directly into Independent Estimate at Completion (IEAC) Formulas Read Post »

Establishing a Robust EVMS Self-Governance Process

By / / , , ,
, , ,
Establishing a Robust EVMS Self-Governance Process

A previous blog, Benefits of an EVMS Self-Governance Process, discussed why establishing a self-governance or self-surveillance process is important and how an effective process builds confidence with the customer. With a structured and repeatable process in place, effective self-governance demonstrates management’s commitment to maintaining the EVMS and open communications with the government customer. Self-disclosure and quickly addressing EVMS compliance issues are essential.

H&A earned value consultants often assist contractors to implement a robust self-governance process as their level of EVMS maturity increases over time. This blog highlights how H&A provides support and technical expertise to help a DOE contractor to do just that.

Developing the Self-Governance Process and Tools

H&A is a strong teammate in the development and implementation of a robust EVMS self-governance process for TRIAD at the Los Alamos National Laboratory (LANL). TRIAD is the prime contractor that provides laboratory management and operations for LANL. H&A is involved in developing tools, refining processes, establishing business rhythms, and summarizing data necessary to support the implementation of a leading-edge EVMS self-surveillance capability. The H&A team is also instrumental in developing the tools necessary to analyze, review, and act on the monthly data set TRIAD provides to the DOE EVMS compliance team.

These tools generate the DOE EVMS compliance metrics (automated and manual) in accordance with the DOE EVMS Metric Specification to ensure TRIAD is able to view the data the same way as their DOE counterparts. Once the tools generate the DOE EVMS compliance metrics, the results are passed to the TRIAD System Surveillance Officers (SSOs) to review and confirm flagged items are either actual fails or exempted/justified based on the rationale captured in the tool. For failed metrics, the SSOs and the project teams use the source data from the tool to identify the root cause and proactively correct EVMS compliance issues. Each month the EVM compliance data is collected across projects, summarized, and graded at the TRIAD level, and then gathered into an EVMS compliance dashboard for TRIAD leadership review and action.

Monthly Self-Surveillance Process

The monthly self-surveillance process includes the following activities.

  1. For each project, the tool generates the automated metrics from the DOE compliance flat files and then collects the results of manual testing into a single file with all 183 DOE metrics. This tool enables an SSO to review the flags, access the source cost and schedule data, apply exemptions/waivers, and then share the data with the project team to resolve issues. By trending this data across the project’s life cycle and capturing SSO exemptions and monthly actions, the team can analyze the data, determine root causes, address issues, and capture historical EVMS compliance actions in one place.
  2. An EVMS summarization tool then collects the results from each project and rolls the lower-level results into a summary TRIAD level. Each metric grade (Pass/Fail/Caution) considers weighted EVMS performance across multiple projects to ensure grading is aligned with the exit criteria for the DOE corrective action plans. In addition to TRIAD level grading for each metric, the summary tool also rolls up the metrics to the 10 EVMS Maturity Subprocess areas and 56 Attributes of an EVMS which is documented in the Compliance Assessment Governance (CAG) Appendix to the DOE EVMS Compliance Review Standard Operating Procedure (ECRSOP). This summarization tool provides the subprocess area and attribute grading at both the project and TRIAD levels. By viewing the data across projects and time, the EVMS core team can quickly identify systemic or project level issues.
  3. A set of tailored EVMS compliance summarization metrics is presented in a “dashboard” configuration for the EVMS core group and senior leadership to review. Leadership uses this summary data to determine where they need to dive deeper into the data and whether TRIAD is meeting their EVMS compliance targets.

Figure 1 illustrates this management level dashboard view. 

Figure 1: Example of the Summary Level Compliance Metrics Across Projects

Figure 1: Example of the Summary Level Compliance Metrics Across Projects

  1. The team also developed and uses a flat file analysis tool that is aligned to the DOE data integrity and quality checks (DIQs). This tool is used for projects transitioning into DOE Critical Decision (CD) Milestones 2 or 3 execution phases that require submittals to the DOE Project Assessment and Reporting System (PARS). This tool ensures the project flat files meet the DOE data quality standards. Like the 183 metrics tool, the flat file tool enables analysts to isolate data quality issues, review the source data, and then determine and track how the team will resolve or justify each issue. In addition to preparing for PARS submittals, these DIQ assessment metrics are also generated monthly to help assess on-going system integration integrity.

Establishing a Best-in-Class Self-Governance Process

In addition to supporting the monthly self-surveillance process, these tools and processes are instrumental in supporting the active surveillance portion of TRIAD’s self-governance efforts. The active surveillance team uses the same tools to summarize and review the “data call” sets in preparation for their reviews. Just like their DOE counterparts, the TRIAD active surveillance team analyzes the 183 DOE compliance metrics to focus their inquiries and document review findings. The H&A team was instrumental in planning, executing, and closing out the recent TRIAD active surveillance that was observed and lauded by the DOE EVMS compliance team.

By supporting the design, development, planning, and execution of all facets of a leading-edge DOE self-governance process, the H&A team helped to ensure our LANL customer has the robust EVMS compliance capability necessary to meet the rigorous DOE EVMS compliance requirements.

As this case study demonstrates, with H&A’s help, TRIAD successfully implemented a structured and repeatable self-governance process with analysis tools that capture objective measures and metrics to actively demonstrate compliance and issue resolution to their customer.

H&A earned value consultants can do the same for you. Call us today at (714) 685-1730 to get started. 

Establishing a Robust EVMS Self-Governance Process Read Post »

Scroll to Top