by Humphreys & Associates | April 25, 2014 10:15 am
This is the second part of a five part series regarding common findings discovered in EVMS reviews and the recommended corrective actions to mitigate those findings. In the previous article, EAC alignment issues, poor variance analysis, and lack of subcontract management were discussed. This second part focuses on: poor use of the percent complete earned value technique, data integrity issues, and poor scope language.
The topics anticipated for parts three through five are:
Part 3: IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse.
Part 5: Inappropriate use of PERT and LOE; Misuse of Management Reserve; Administrative CAMs.
Here are the three topics for this installment;
Earned Value techniques (EVT) fall into three distinct classifications:
Within the Discrete technique there are methods ranging from objective to subjective. The most objective methods are those that employ a “yes/no” question as the primary measure of accomplishment, e.g., “Has the activity begun?” or “Was the hardware delivered?”. In these cases, there is no subjectivity in the evaluation of performance.
The percent complete technique can introduce very subjective criteria as the source of measuring accomplishment. These are sometimes called “Manager’s Estimate”, or, derisively, “The Percent Guess” technique. The problem with this application of percent complete is that visibility into accurate work package performance may be hidden until it is too late to take effective managerial action. Worse, managers responsible for activities linked downstream from the percent complete work package may not discover the true status of a critical predecessor until it is too late to implement effective mitigation actions. While a key mantra for effective project management is “no surprises”, misuse of the percent complete earned value technique can introduce the possibility for a great deal of surprise.
The goal in selecting the most appropriate EVT is to use the most objective rationale possible given the nature of the work. There are key strategies that should be employed to reduce the subjectivity of the percent complete technique.
To standardize its reviewing organizations and the reduction in resources available to conduct on-site reviews, the DCMA is transitioning to establishing predefined success criteria and testing methods to evaluate the compliance of each guideline using data submitted by the contractor. In addition, as mentioned above, the DCMA has released to its staff a “Program Analysis Pamphlet” that contains 17 measures of data integrity for contractor supplied data. A few of these are:
In many cases, there may be scenarios that result in a temporary data condition that trips one of the above thresholds; such as Level of Effort (LOE) but these should be the exception rather than the rule.
Because of the growing interest in data integrity, it is important to screen for these conditions prior to submitting the data to the customer. If possible, an organization should allow time in its monthly business rhythm to correct the easy-to-fix errors prior to submittal of the reports. Those data flags that are legitimate or cannot be corrected prior to submittal should be reported to the customer in Format 5 of the Integrated Program Management Report (IPMR) or Contract Performance Report (CPR). Some issues can be more systemic and require a longer correction process. These should also be identified to the customer with a corrective action plan which includes a date for when the corrections will be in effect.
And finally the prime contractor must review all reports delivered by the subcontractor for compliance with the appropriate DID Instructions, and be prepared to take action when the reports are noncompliant with the CDRL and/or DID Instructions. This action can take the form of a contracting officer’s letter requiring adjustments in future reports, rejection and resubmittal of non-compliant reports, or rejection and contractual remedies; such as payment withholds or Award Fee impact, as applicable.
The original scope for the project/contract is defined in the Contract Statement of Work (CSOW) and then organized within the Work Breakdown Structure (WBS). The WBS Dictionary defines the technical boundaries of each WBS element and identifies the work products. At a more detailed level, the scope is also identified in the control account plan and the work authorization documents. There is an expectation that the scope language will become more specific and descriptive as it migrates from the project’s top level, through the WBS, down to the control accounts.
In EVM Surveillance Reviews, as well as Integrated Baseline Reviews (IBR)s, a normal set of activities includes data traces. The project’s scope, schedule, and budget are traced across all the pertinent documents and reports to ensure that the EVMS is operating as a single, integrated system. Included in the scope trace is a review of lower level (WBS element or control account) scope statements to ensure they are consistent with the CSOW. It is also expected that the WBS Dictionary will be maintained and current with all external and internal changes that add scope to WBS elements or control accounts. Often an organization begins a project with a well-documented set of scope statements, only to let them languish in the face of constant and voluminous changes.
Like Variance Analysis and Reporting, having accurate, current, and descriptive scope statements in all pertinent documents requires a disciplined approach and persistent review. The first step is establishing a procedure that makes this an important part of the earned value process, including the importance of maintaining the currency of the WBS Dictionary by technically qualified personnel. The quality of these documents is generally the responsibility of the CAMs in concert with the system engineering organization or equivalent; however, the Project Controls staff plays an important role in assisting the CAMs and reviewing the products.
One of the most common findings in this area is the failure to maintain the WBS Dictionary. This document must reflect the current scope and incorporate all approved changes. The contractor should establish a process in the monthly business rhythm that “sweeps” all known changes, generally through the work authorization documents, and evaluates those for incorporation into the Dictionary.
Please contact Humphreys & Associates if you have any questions on this article.
Source URL: https://blog.humphreys-assoc.com/common-problems-found-in-evms-reviews-and-recommended-corrective-actions-part-2/
Copyright ©2023 Humphreys & Associates unless otherwise noted.