Update #1 November 10, 2017
Update #2 December 15, 2020
- BP0 – Earned Value Management Systems (Overarching guidance, overview of the Business Practices)
- BP1 – Pre-Award EVM System Plan Review
- BP2 – System Description Review
- BP3 – Program Support
- BP4 – System Surveillance
- BP5 – Review for Cause
- BP6 – Compliance Review Execution
- BP7 – Compliance Metric Configuration Control (DCMA internal document)
BP6 defines a uniform process for the DCMA EVMS Center personnel to conduct the planning, execution, and reporting of an initial contractor EVMS Compliance Review (CR), following 23 Steps in 5 major phases or Process Steps (see Figure 1 below from BP6):
The focus of this discussion is on the second phase shown in Figure 1, “Execute Pre-Event Activities,” Steps 6 (Request Other Data Sources) and 7 (Initiate Data Analysis). The old CR Processes included the contractor’s Self-Assessment (SA) as a more formalized part of their Compliance Review Steps, but the Business Practices no longer include the SA. Any self-assessment conducted by the contractor is included in the Box 6 in Figure 1 “Request Other Data Sources,” but only as a possible source of Corrective Action Requests (CARs) that could become part of the CR process. Data Analysis is still a major part of the new CR Process (Box 7 in Figure 1) and plays a significant role in DCMA’s “data-driven” reviews. Even though the DCMA no longer formalizes the SA they still encourage contractors to conduct some type of evaluations of their own systems prior to any customer review taking place.
Since the contractor should be using the same data the DCMA will be using for their Data Analysis (DA), it is essential to provide quality data in response to the DCMA data call and during any internal evaluations the contractor conducts. Remember, the DCMA can halt the CR process if the data are found to be lacking. One item to note regarding a contractor’s internal evaluation: if the contractor self-identifies a non-compliant issue and takes timely and appropriate steps to correct it, DCMA does not intend to issue a Corrective Action Request (CAR) for that issue. They will however, continue to monitor the progress in correcting the noncompliance through the Corrective Action Plan (CAP) reviews.
The DCMA Data Analysis step is an assessment of the contractor’s readiness to undergo the remainder of the CR Process, and will influence the timing of the rest of the review process. A contractor’s internal EVMS assessment of the same data that will be delivered to the DCMA should give the contractor an accurate assessment of their readiness for the review. Unfortunately, it is sometimes difficult for personnel in a company to objectively criticize their own processes and personnel implementing those processes. This article presents an effective and important pre-Compliance Review internal assessment for contractors – the Mock EVMS Review.
With the Self-Assessment (SA) no longer being a formal part of the CR process, the Mock EVMS Review is an excellent way for the contractor to get an independent assessment of their readiness to undergo the Compliance Review. Since the Mock Review must be a thorough assessment of their system and implementation, the amount of effort required to conduct the Mock Review is significant.
The Mock Review should simulate as closely as possible the Compliance Review the DCMA will conduct. The Mock Review is typically done by an outside third party reviewer that will take an objective look at the contractor’s system and implementation. This can be a parent organization or a sister organization that has experience conducting such reviews. This type of experience within a company is not common, however, so companies often come to third party organizations, like Humphreys & Associates, Inc., who have done these sorts of assessments for many years.
The Mock EVMS Review should simulate everything DCMA will do, including:
§ A comprehensive data call (using the DCMA’s current data call list);
§ A rigorous Data Analysis of those data call items;
§ Thorough evaluation of the EVM System Description and implementing procedures;
§ Use of Discrepancy Report (DR) and Corrective Action Request (CAR) forms with instructions;
§ Use of various checklists and EIA-748 guideline evaluation forms;
§ Use of role specific interview forms;
§ Conducting simulated interviews across various functional levels;
§ Conducting data traces by guideline; and
§ Use of briefing and report templates to convey the results.
The Mock EVMS Review must include a thorough scrub of the schedule and cost data. All data must trace through all project artifacts. The best way to prepare for this, as well as well as continually ensure data integrity, is to incorporate regular data quality checks into your normal project control activities. A few basic checks you should always perform include:
§ Verifying the Integrated Program Management Data and Analysis Report (IPMDAR) cost and schedule electronic deliverable datasets or the data included in every contractual performance report format such as the Contract Performance Report (CPR) Formats 1 to 5 or Integrated Program Management Report (IPMR) Formats 1 through 7 before delivery to the customer. Trace and reconcile all summary values to verify the source detailed data matches what is included in a dataset or report format. Check for math errors, missing elements, or other invalid data, such as: the BCWScum (budget) or BCWPcum (earned value) is greater than the budget at completion (BAC) or the ACWPcum (actual cost) is greater than the estimate at completion (EAC).
§ Verifying all project logs (contract budget base, management reserve, undistributed budget, etc.) and confirming summary numbers match the reporting deliverable information (CPR/IPMR/IPMDAR).
§ Confirming the schedule and cost data tell the same story and are reconciled to other project artifacts, such as the Work Breakdown Structure (WBS) Dictionary and work authorization documents.
§ Confirming the change logs are up to date ensuring approved changes are incorporated in the schedule and cost data in a timely manner. All changes must be fully traceable.
§ Conducting schedule data quality checks to verify logic, status updates, accurate coding, calendar assignments, reasonable descriptions, and the like.
§ Verifying subcontractor schedule and cost data for accuracy and timeliness.
It takes effort to produce and maintain quality schedule and cost data for the life of a project. It requires management focus and attention to detail. With the wealth of toolsets available and the amount of data available electronically, implementing steps to make it easier to maintain data integrity can make a difference.
The processes that companies implement have the potential to hinder the process to produce and maintain quality data. Always be prepared to update desktop procedures or other process steps to create a single data source that is fully traceable. Where possible, eliminate duplicate steps, duplicate data entry, or manual rekeying that has the potential to compromise data integrity or the ability to conduct data traces.
Whether you have a customer or contractor review coming up or not, data integrity is a fundamental requirement for an effective EVMS.
Call Humphreys & Associates to assist with the contractor Mock EVMS Review process including reviewing the system documentation, performing data quality checks, and helping prepare the data packages for delivery to DCMA. We can be reached at (714) 685-1730 or email us.