A previous blog, Maintaining a Credible Estimate at Completion (EAC), discussed why producing a realistic EAC is essential to managing the remaining work on a contract. Internal management and the customer need visibility into the most likely total cost for the contract at completion to ensure it is within the negotiated contract cost and funding limits.
As noted in the earlier blog, one common technique to test the realism of the EAC is to compare the cumulative to date Cost Performance Index (CPI) to the To Complete Performance Index (TCPI).
Example of Using the Metrics for Evaluating Data
One example of documented guidance to industry for evaluating the realism of the EAC is the DOE Office of Project Management (PM) Compliance Assessment Governance (CAG) 2.0, and the related DOE EVMS Metric Specifications they use to assess the quality of schedule and cost data. This blog highlights the use of this guidance and how any contractor can incorporate similar best practices to verify EACs at a given WBS element, control account, or project level are realistic.
To refresh, the CPI is the efficiency at which work has been performed so far for a WBS element, control account, or at the total project level. The formula for the cumulative to date CPI is as follows.
Best practice tip: To ensure a valid CPI calculation, verify the BCWP and ACWP are recorded in the same month for the same work performed.
The TCPI provides the same information, however, it is forward looking. While the CPI is the work efficiency so far, the TCPI is the efficiency required to complete the remaining work to achieve the EAC. The formula for the TCPI is as follows.
Best practice tip: To ensure a valid TCPI, verify the BCWP and ACWP are recorded in the same month for the same work performed, and the BAC and EAC are for the same work scope. In other words, the scope of work assumptions are the same for the budget and remaining cost. This is why anticipated changes should not be included in the EAC.
The DOE uses the CPI in two of their assessment metrics and the TCPI in one, however, these are critical metrics partly because they are the only ones used to assess two different data evaluations: 1) commingling level of effort (LOE) and discrete work, and 2) EAC realism.
Commingling LOE and Discrete Work
The first use of CPI (no TCPI in this metric) falls under the Budgeting and Work Authorization subprocess. The primary purpose is to evaluate the effect of commingling LOE and discrete work scope has on control account metrics. The basic premise for this metric is that if the CPI for the LOE scope is significantly different than that for the discrete, the mixture of LOE in that control account is likely skewing overall performance reporting.
Here is the formulation DOE uses.
C.09.01: Control Account CPI delta between Discrete and LOE >= ±0.1
X = Number of incomplete control accounts (WBS elements) in the EVMS cost tool, where
- The LOE portion of the budget is between 15% and 80% of the total budget, and
- The difference between the CPI for the discrete work and the LOE work is >= ±0.1.
Threshold = 0%
Best practice tip: Run this metric quarterly on your control accounts that commingle LOE and discrete work packages. When there is a significant discrepancy between the performance of the LOE versus discrete work effort, consider isolating the LOE effort from the discrete effort at the earliest opportunity. An example could be the next rolling wave planning window or as part of an internal replanning action. Alternatively, it may be necessary to perform the calculations at the work package level to assess the performance of just the discrete effort when it is impractical to isolate by other means.
Process and procedure tip: Ensure the LOE work packages within a control account are kept to minimum (typically less than 15%), during the baseline development phase. This helps to prevent discrete work effort performance measurement distortion during the execution phase. A useful best practice H&A earned value consultants have helped contractors to implement during the budget baseline development process is to perform an analysis of the earned value methods used within a control account and the associated work package budgets. This helps to verify any LOE work packages are less than the 15% threshold for the control account. In some instances, it may be logical to segregate the LOE work effort into a separate control account. The objective is to identify and resolve the issue before the performance measurement baseline (PMB) is set.
EAC Realism
One DOE metric uses the TCPI and this involves a comparison to the CPI. This falls in the Analysis and Management Reporting subprocess. This DOE EVMS Metric Specification states: “This metric confirms that estimates of costs at completion are accurate and detailed.” As noted above, the metric compares the cost performance efficiency so far to the cost efficiency needed to achieve the EAC and is specific to the EAC a control account manager (CAM) would review for their scope of work. Depending on the level actual costs are collected, this analysis may need to be performed at the work package level instead of the control account level.
Here is the formulation DOE uses assuming actual costs are collected at the work package level.
F.05.06: Work Package CPI – EAC TCPI > ±0.1
X = Number of incomplete (>10% complete) work packages where CPI –TCPI > ±0.1.
Y = Number of incomplete (>10% complete) work packages in the EVMS cost tool.
Threshold = 5%
There is no requirement that the forecast of future costs has a linear relationship with past performance. While there may be legitimate reasons why future cost performance will fluctuate from the past, outside reviewers who receive EVM data will look for a trend or preponderance of data that would indicate the EACs are not realistic. When a significant number of active work packages are outside the ±0.1 CPI-TCPI threshold, it is an indication that the EACs are not being maintained or are driven by factors other than project performance.
Best practice tip: Run this metric every month for each active work package prior to month-end close. For those work packages outside the ±0.1 threshold, review the EAC to ensure it is an intentional forecast of costs given the current conditions.
Process and procedure tip: One of the training courses H&A earned value consultants often conduct is a Variance Analysis Reporting (VAR) workshop. This workshop is designed to help CAMs become more proficient with using the EVM metrics to assess the performance to date for their work effort, identify the root cause of significant variances, and document their findings as well as recommended corrective actions. This analysis includes verifying their estimate to complete (ETC) is a reasonable assessment of what is required to complete the remaining authorized work and their EACs are credible.
Additional References
Further discussion on using the CPI and TCPI to assess the EAC realism at the project level can be found in the DOE CAG, Analysis and Management reporting subprocess, Estimates at Completion. This section provides a good overview of comparing the cumulative to date CPI to the TCPI as well as comparing an EAC to calculated independent EACs (IEACs) for further analysis to assess the EAC credibility.
Interested in learning more about using EVM metrics as a means to verify EACs at the detail or project level are realistic? H&A earned value consultants can help you incorporate best practices into your processes and procedures as well as conduct targeted training to improve your ETC and EAC process. Call us today at (714) 685-1730.