Common Problems Found in EVM Systems – Recommended Corrective Actions – Part 1

Humphreys & Associates (H&A) has the opportunity to work with a broad spectrum of clients who operate their Earned Value Management System (EVMS) under the contractual authority of a variety of customers.  Many clients have a surveillance program conducted by the Defense Contract Management Agency (DCMA), the Department of Defense (DoD) Executive Agent for EVMS which also has a reciprocal agreement with other agencies.  Some agencies, such as the Department of Energy (DoE), have their own surveillance programs.  H&A also works with many companies to help them “tune up” their EVMS as a part of sound business practices.

In the course of conducting and supporting client reviews, H&A has identified several recurring themes that many organizations allow into their EVMS, and because we often support the resolution of these issues through the Corrective Action Planning process, we can also recommend the most common remedies to prevent or correct them.

This will be a 5 part series that will cover three topics in each article.

The anticipated topics for parts two through five (subject to change) are:

            Part 2:  Poor use of Percent Complete; Data Integrity Issues; Poor Scope Language.

            Part 3:  IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse.

            Part 4:  Misalignment between BCWP and ACWP; Freeze Period Violations; Failed Data Traces.

            Part 5:  Inappropriate use of PERT and LOE; Misuse of Management Reserve: Administrative CAMs.

1) The Estimate at Completion (EAC) is systemically out of alignment with cumulative performance with no justification.

The Earned Value Management System Guidelines (EVMSG) define the EAC as the sum of the contract’s cumulative to date Actual Cost of Work Performed (ACWP) plus the company project manager’s best estimate of the time-phased resources (funds) required to complete the remaining authorized work, the Estimate to Complete (ETC).  One of the measures that is used to provide a “sanity check” of the reasonableness of the EAC is to compare the To Complete Performance Index (TCPI) to the Cost Performance Index (CPI).

• CPI = BCWP / ACWP: An indicator of the cost efficiency at which work is being performed.
• TCPI = (BAC-BCWP) / (EAC-ACWP): The cost efficiency that would have to be attained in order to achieve the EAC value being used in the formula.

According the DCMA publication “EVMS Program Analysis Pamphlet” (DCMA-EA PAM 200.1, July 2012), a mathematical difference of 0.10 or greater is used as an early warning indication that the contractor’s forecasted completion cost could possibly be unrealistic, stale, or not updated recently.  This is generally measured at all levels of the project, but is often the focus at the control account level.

Having a difference greater than 0.10 does not mean the EAC must be adjusted.  Often the TCPI formula will give unusually high or low values when the account being measured is at the beginning or end of its progress.  There may be a very good reason for the forecasted efficiency of an Estimate to Complete (ETC) to be out-of-line with its historical efficiency.  But often surveillance reveals a large number of accounts that are out-of-line by this measurement, indicating a lack of discipline in an organization’s EAC process.

Most Common Corrective Action Plans:

The first action should be to ensure that the Earned Value Management System stipulates an update to the ETC on at least a monthly basis, and the Control Account Managers (CAMs) are held accountable for their EACs’ realism.  This is often accomplished through training, updating processes, and the development of easy to use tool sets.  The organization could also require the CAM to justify any control account’s ETC that is out of tolerance with an EAC realism check.

Care must be taken to ensure that the EACs are not being set just to avoid tripping this metric.  It is easy to “calculate” an EAC by dividing the BAC by the CPI; however, this does not meet the intent of the guideline requirements.

2) Poor Variance Analysis and Reporting (VAR)

This is probably the most common Corrective Action cited by reviewers performing EVMS surveillance.  Because the variance analysis relies completely on the discipline of the CAMs and their support staffs, the process often atrophies for various reasons. Within the variance analysis report (VAR) process, probably the most troublesome is the identification and tracking of Corrective Actions to their logical completion.  A well written variance analysis will explain the root cause, impact and corrective action associated with a cost (current period, cumulative, at completion) or a schedule (current period, cumulative) variance.  Cost variance explanations should provide a breakdown of the rate versus hours for labor or the price versus usage for material.  Schedule variance explanations should focus on identifying the effort that is not being accomplished per the plan, and forecast the Estimated Completion Date (ECD) for when the schedule variance will go to zero.  Most important in all of this process is the identification and tracking of the Corrective Actions that are being taken to either mitigate the variance or at least ensure that the variance does not worsen.

Reviewers of VARs often find that there is a lack of clear and concise treatment of the problem, and that many writers of VARs simply reiterate the variance as indicated by the data, such as “The cost overrun is because more hours were expended than originally planned”.  Many VARs will point a finger back to the system that generated the data.  But the worst offender is the lack of a corrective action plan that includes mitigations steps that address the issue.

Most Common Corrective Action Plans

Poor variance analysis is primarily a discipline issue with the CAMs and their support staffs. Like most discipline issues, this can be addressed by training, adding structure, and incorporating reviews of the VAR process.  The writing of VARs is one area where annual refresher training may be needed in order to verify that the concepts are still being practiced.  This training should involve the managers in the organizational chain above the CAMs who are responsible for reviewing and approving the variance analysis reports.

It can also be beneficial to add a structured approach to the variance explanation page.  This should include required inputs on Root Cause, Impact, and Corrective Action, and may include a link to a corrective action tracking system that allows the user to create and assign mitigation steps plus provide status of previously identified corrective actions.  Some organizations have also introduced the scoring of VARs using a scoring template as a feedback mechanism for those writing the reports.

3) Lack of Effective Subcontract Management

The Interim DoD Instruction for 5000.02, “Operation of the Defense Acquisition System” (November 25, 2013), reiterates the requirements for earned value application on all cost/incentive contracts greater than $20M including subcontracts.  In the DCMA Instruction 1201, “Corrective Action Process” (September 23, 2013), the importance of subcontract management is stressed with the following:

“Prime contractors have wide latitude as to how they control their supply chain and are ultimately responsible for flow down and execution of contract requirements. When DCMA discovers a noncompliance requiring a Level I or II CAR at a subcontract level, the appropriate CAR (Level I or II) shall be issued directly to the subcontractor with notification to the prime contractor via the prime CMO. The notification to the prime contractor shall be redacted as needed to prevent disclosure of subcontractor proprietary information. In situations where a noncompliance(s) at the subcontract level meets the criteria for a Level III CAR, the CAR shall be issued to the prime contractor.”

Over the past few years, the lack of effective and consistent management of subcontractors with an EVM flow down requirement has been the source of numerous discrepancy reports (DRs), and has been cited as a significant attributor to EVMS disapprovals/de-validations.  As noted in the language above, prime contractors are not only responsible for including the correct clauses in subcontracts but also have responsibility for the execution of the contract requirements.  When a subcontractor does not follow its own system description, is not in compliance with the EIA-748 Guidelines, or submits reports that do not meet the requirements of the contractual Data Item Description (DID), it is the responsibility of the prime to enforce compliance.

The reasons for ineffective subcontract management are many, but in some organizations enforcement of contractual requirements is the domain of a subcontract administrator/buyer who may not have a firm understanding of the EVMS requirements.  Many subcontractors also often resist the surveillance attempts by their prime, or attempt to reject any efforts by their prime to enforce the execution of these types of contractual requirements.

Most Common Corrective Action Plans

The first step in corrective action is a thorough understanding of the contractual requirements and language in the subcontracts.  H&A has found numerous examples of incorrect contractual language, or even language implying that the subcontractor reports are for “information only”, thus diminishing the ability of the prime to enforce compliance.  The contract should include: 1) A requirement for compliance with the EVM system guidelines in EIA-748, preferably by including the appropriate DFARs clauses (DFARs 252.234-7001 and 7002); 2) the appropriate DID, currently DI-MGMT-81861, referenced in the Contract Data Requirements List (CDRL) with any special tailoring or reporting guidance documented (for contracts issued prior to June, 2012, the Contract Performance Report DID is DI-MGMT-81466A and the IMS DID is DI-MGMT-81650); 3) language which establishes the right of the prime contractor to conduct reviews, such as Integrated Baseline Reviews (IBRs) and recurring system surveillance.

The prime contractor must then establish a disciplined surveillance program that includes review of all 32 EIA-748 Guidelines on an annual basis at each subcontractor’s facility.  This process should also include the procedure for issuing and tracking corrective actions.  These surveillance reviews are often conducted jointly with the cognizant DCMA/DCAA for each subcontractor.

And finally the prime must review all reports delivered by the subcontractor for compliance with the appropriate DID, and be prepared to take action when the submitted reports are noncompliant with the CDRL and/or DID Instructions.  This action can take the form of a subcontract administrator’s letter requiring adjustments in future reports, rejection and resubmittal of non-compliant reports, or rejection and contractual remedies; such as payment withholds or Award Fee impact, as applicable.

Please contact Humphreys & Associates if you have any questions on this article.

April 25th, 2014 – Part 2:  Common Problems found in EVMS | Poor use of Percent Complete; Data Integrity Issues; Poor Scope Language.