Earned Value Management System Compliance Reviews Instruction 208

EVMS Compliance Review Series #4 – Training to Prepare for Interviews

By / / , ,
, ,
Updated November 10, 2017
Updated December 15, 2020

DCMA Compliance Review Series

This is the fourth topic in a series discussing the DCMA Compliance Review (CR) process. It is the second update to the original series #4 Blog that was based on the instruction book from the DCMA Website that identified the “DCMA 16 Step (and then their 8-Step [UPDATE #2]) EVMS Compliance Review Process.” This change was documented in the DCMA “Earned Value Management System Compliance Reviews Instruction (DCMA-INST 208).” This Instruction has been rescinded and replaced with a set of DCMA Business Practices (BP). These Business Practices split out topics the old DCMA Instruction 208 covered in one document. Whether you are a contractor new to the EVM contracting environment or a seasoned veteran, if the Earned Value Management System (EVMS) compliance and acceptance authority is the Defense Contract Management Agency (DCMA), these new Business Practices apply to you.

This is a continuing discussion about the interviews that are part of the optional contractor self-assessments and as part of conducting any Compliance Review, Surveillance Review, or Review for Cause (RFC). The DCMA uses a standard set of interview forms for the control account managers (CAMs), project managers, integrated product team (IPT) leads, risk managers, business managers, and accounting. Contractors should use similar interview questionnaires to simulate the DCMA interview process as closely as possible to prepare the CAMs and others for the DCMA interviews.

CAM Interviews are Critical

While it is important to have successful interviews with everyone, the CAM interviews are critical. The CAMs are the primary interview subjects during a compliance review. The CAMs must be able to demonstrate they have a full understanding of their scope of work and show how they use the company’s EVMS artifacts to manage their work on a day-to-day basis, in addition to the normal weekly or monthly business rhythm. The CAMs need to have a good understanding of their role to provide acceptable responses to interview questions and to be comfortable with the interview process.

CAMs must be able to demonstrate a working knowledge of:

  • EVM,
  • The 32 guidelines in the EIA-748 Standard for Earned Value Management Systems,
  • The company’s EVMS,
  • How to perform the applicable data traces for their scope of work, and
  • How they fit into the grand scheme of the project.

The CAMs must also have a level of proficiency with the project control toolsets. Many companies use electronic CAM notebooks integrated into the EVMS Storyboard to assist the CAMs.

Three-Step Training for CAMs

Conducting training for the CAMs is an important preparation step. Based on our long history of successfully preparing CAMs, project managers, and others for compliance reviews, Integrated Baseline Reviews (IBRs), or surveillance reviews, we suggest a three-step training process for the CAMs.

Set Expectaions

Step 1. Conduct training sessions for the CAMs on what is expected during a DCMA interview. This training can include conducting an example group interview the CAMs and others can observe. The group interview session typically includes an open discussion at the end to critique the interview process.

  • What were the good techniques?
  • What could be improved?
  • What are some things to avoid?

This helps to set expectations and provides a preparation guide for the people to be interviewed.

White Hat Interview

Step 2. Conduct “white hat” interview sessions with the CAMs. These are more one-on-one mentoring sessions to help the CAMs through the interview process. Providing feedback at the end of each session reinforces the learning process and increases their level of proficiency. This dry run helps the CAMs to feel more comfortable with the interview process. It also helps to increase their confidence level and ability to think on their feet.

Black Hat Interview

Step 3. Conduct “black hat” interview sessions with the CAMs. For these sessions, the interviewers conduct the interview as if they were members of the DCMA review team. If the CAM is unable to undergo a thorough interview, the session evolves into a mentoring session to help them through the process. The “black hat” interview sessions should continue until each CAM is comfortable with the interview process.

Identify Deficiencies

A summary of the findings from the interviews can help a company to be proactive in addressing known issues. The goal is to identify any deficiencies whether in the EVMS design, implementation, end-user knowledge, or data quality. Deficiencies can be reported in self-assessment Discrepancy Report (DR) forms along with the corrective action plans that will be implemented to resolve the issue.

It is far better for the company to find and address any issues as part of the internal self-assessment process than to have DCMA find them during the formal Compliance Review. This can make a difference in whether or not DCMA issues Corrective Action Requests (CARs) or recommends moving to the next step in the compliance review process.

Contact H&A to help train your Control Account Manager (CAM) with one-on-one mentoring, conducting mock review interviews, or as part of preparing for an Integrated Baseline Review (IBR). We can be reached at (714) 685-1730 or email us.

EVMS Compliance Review Series

EVMS Compliance Review Series #4 – Training to Prepare for Interviews Read Post »

EVMS Compliance Review Series #3 – Using Storyboards to Depict the Entire EVMS

By / / , , ,
, , ,
UPDATE#1 NOVEMBER 10, 2017
UPDATE#2 DECEMBER 15, 2020

DCMA Compliance Review

This is the third topic in a series discussing the DCMA Compliance Review (CR) process. It is the second update to the original series #3 Blog that was based on the instruction book from the DCMA Website that identified the “DCMA 16 Step (and then their 8-Step) EVMS Compliance Review Process.” This change was documented in the DCMA “Earned Value Management System Compliance Reviews Instruction (DCMA-INST 208).” This Instruction has been rescinded and replaced with a set of DCMA Business Practices (BP). These Business Practices split out topics the old DCMA Instruction 208 covered in one document. Whether you are a contractor new to the EVM contracting environment or a seasoned veteran, if the Earned Value Management System (EVMS) compliance and acceptance authority is the Defense Contract Management Agency (DCMA), these new Business Practices apply to you.

EVMS Storyboards

The focus of this discussion is on using EVMS Storyboards to assist with the interviews and data traces that are part of any internal EVMS review, as well as during the conduct of any of Compliance Review, Surveillance Review, or Review for Cause (RFC). Storyboards can also be useful in supporting on-going surveillance activities. Even more important is their usefulness for training contractor and government personnel in how the EVM System operates.

Depict the Management System

An EVMS Storyboard depicts the entire management system as stratified flow diagrams with artifacts illustrating the inputs and outputs that demonstrate the system in operation. It is meant to clearly illustrate:

  • How the entire system functions (the flow),
  • Who is responsible for doing what (decision points and actions),
  • Products (forms, reports, outputs, and other artifacts), and
  • How all the process groups are interrelated to create a fully integrated EVMS.

Single Thread

The storyboard should also illustrate a single thread trace to demonstrate data integrity from beginning to end using actual project artifacts. Storyboards are an essential tool for training all levels of users and is invaluable for demonstrating to a customer or a review team how a contractor’s EVM System functions.

Formats

Storyboards can take many forms. The entire process can be displayed on panels or poster boards on a conference room wall. It can be contained in the EVM System Description document as an overall flow, or as segmented flows in the system description sections corresponding to the process flow. It can also be electronic or online flow diagrams with links to live artifacts.

The DCMA EVMS Center has made it clear that EVMS Storyboards are an important part of the compliance review process. During a compliance review, the contractor would be expected to use these stratified flow documents to walk through the various system processes. This includes the project manager, integrated product team (IPT) leads, control account managers (CAMs), and other functional leads. The various users of the EVMS should be able to use the storyboard flow documents to help tell their “story” and demonstrate data traceability using live project artifacts.

Compliance Review

When preparing for any compliance review, be sure to step through your EVMS Storyboard to verify:

  • It is complete and clearly illustrates how all the process groups are interrelated,
  • Roles and responsibilities are unambiguous,
  • Entrance and exit points are clearly marked,
  • Artifacts are up to date,
  • It accurately reflects how project personnel are using the EVMS to manage their work, and
  • All the data represented on the storyboard traces properly.

What if I don’t have an EVMS Storyboard?

If you don’t have an EVMS Storyboard in place, consider starting with the flow diagrams in your EVM System Description and building them out into a storyboard. An EVMS Storyboard is an essential tool to successfully navigate the compliance review process. It can help to prevent DCMA Discrepancy Reports (DRs) or Corrective Action Requests (CARs) being written because the DCMA review team didn’t know how to locate certain documents, or they didn’t understand how a process worked.

Humphreys & Associates can help you develop a new EVMS Storyboard or enhance your existing EVMS Storyboard, an important and cost saving tool. To contact us, (click here) or call (714) 685-1730.

 

EVMS Compliance Review Series

 

EVMS Compliance Review Series #3 – Using Storyboards to Depict the Entire EVMS Read Post »

EVMS Compliance Review (CR) Series #2 – Preparing for the EVMS CR

By / / , , ,
, , ,
Update #1 November 10, 2017
Update #2 December 15, 2020

http://www.dcma.mil/This is the second topic in a series discussing the DCMA Compliance Review (CR) process. It is the second update to the original series #2 Blog that was based on the instruction book from the DCMA Website that identified the “DCMA 16 Step (and then their 8-Step) EVMS Compliance Review Process.” This change was documented in the DCMA “Earned Value Management System Compliance Reviews Instruction (DCMA-INST 208).” This Instruction has been rescinded and replaced with a set of DCMA Business Practices. These Business Practices split out topics the old DCMA Instruction 208 covered in one document. Whether you are a contractor new to the EVM contracting environment or a seasoned veteran, if the Earned Value Management System (EVMS) compliance and acceptance authority is the Defense Contract Management Agency (DCMA), these new Business Practices apply to you. The overall EVMS review processes are covered in the following eight Business Practices (BP):

  • BP0 – Earned Value Management Systems (Overarching guidance, overview of the Business Practices)
  • BP1 – Pre-Award EVM System Plan Review
  • BP2 – System Description Review
  • BP3 – Program Support
  • BP4 – System Surveillance
  • BP5 – Review for Cause
  • BP6 – Compliance Review Execution
  • BP7 – Compliance Metric Configuration Control (DCMA internal document)

 

BP6 defines a uniform process for the DCMA EVMS Center personnel to conduct the planning, execution, and reporting of an initial contractor EVMS Compliance Review (CR), following 23 Steps in 5 major phases or Process Steps (see Figure 1 below from BP6):

Figure 1 - EVMS Compliance Review (CR), following 23 Steps in 5 major phases or Process Steps
Figure 1

The focus of this discussion is on the second phase shown in Figure 1, “Execute Pre-Event Activities,” Steps 6 (Request Other Data Sources) and 7 (Initiate Data Analysis).  The old CR Processes included the contractor’s Self-Assessment (SA) as a more formalized part of their Compliance Review Steps, but the Business Practices no longer include the SA. Any self-assessment conducted by the contractor is included in the Box 6 in Figure 1 “Request Other Data Sources,” but only as a possible source of Corrective Action Requests (CARs) that could become part of the CR process. Data Analysis is still a major part of the new CR Process (Box 7 in Figure 1) and plays a significant role in DCMA’s “data-driven” reviews. Even though the DCMA no longer formalizes the SA they still encourage contractors to conduct some type of evaluations of their own systems prior to any customer review taking place.

Since the contractor should be using the same data the DCMA will be using for their Data Analysis (DA), it is essential to provide quality data in response to the DCMA data call and during any internal evaluations the contractor conducts. Remember, the DCMA can halt the CR process if the data are found to be lacking. One item to note regarding a contractor’s internal evaluation: if the contractor self-identifies a non-compliant issue and takes timely and appropriate steps to correct it, DCMA does not intend to issue a Corrective Action Request (CAR) for that issue. They will however, continue to monitor the progress in correcting the noncompliance through the Corrective Action Plan (CAP) reviews.

The DCMA Data Analysis step is an assessment of the contractor’s readiness to undergo the remainder of the CR Process, and will influence the timing of the rest of the review process.  A contractor’s internal EVMS assessment of the same data that will be delivered to the DCMA should give the contractor an accurate assessment of their readiness for the review.  Unfortunately, it is sometimes difficult for personnel in a company to objectively criticize their own processes and personnel implementing those processes.  This article presents an effective and important pre-Compliance Review internal assessment for contractors – the Mock EVMS Review.

With the Self-Assessment (SA) no longer being a formal part of the CR process, the Mock EVMS Review is an excellent way for the contractor to get an independent assessment of their readiness to undergo the Compliance Review. Since the Mock Review must be a thorough assessment of their system and implementation, the amount of effort required to conduct the Mock Review is significant.

The Mock Review should simulate as closely as possible the Compliance Review the DCMA will conduct. The Mock Review is typically done by an outside third party reviewer that will take an objective look at the contractor’s system and implementation. This can be a parent organization or a sister organization that has experience conducting such reviews. This type of experience within a company is not common, however, so companies often come to third party organizations, like Humphreys & Associates, Inc., who have done these sorts of assessments for many years.

The Mock EVMS Review should simulate everything DCMA will do, including:

§  A comprehensive data call (using the DCMA’s current data call list);

§  A rigorous Data Analysis of those data call items;

§  Thorough evaluation of the EVM System Description and implementing procedures;

§  Use of Discrepancy Report (DR) and Corrective Action Request (CAR) forms with instructions;

§  Use of various checklists and EIA-748 guideline evaluation forms;

§  Use of role specific interview forms;

§  Conducting simulated interviews across various functional levels;

§  Conducting data traces by guideline; and

§  Use of briefing and report templates to convey the results.

The Mock EVMS Review must include a thorough scrub of the schedule and cost data. All data must trace through all project artifacts. The best way to prepare for this, as well as well as continually ensure data integrity, is to incorporate regular data quality checks into your normal project control activities. A few basic checks you should always perform include:

§  Verifying the Integrated Program Management Data and Analysis Report (IPMDAR) cost and schedule electronic deliverable datasets or the data included in every contractual performance report format such as the Contract Performance Report (CPR) Formats 1 to 5 or Integrated Program Management Report (IPMR) Formats 1 through 7 before delivery to the customer. Trace and reconcile all summary values to verify the source detailed data matches what is included in a dataset or report format. Check for math errors, missing elements, or other invalid data, such as: the BCWScum (budget) or BCWPcum (earned value) is greater than the budget at completion (BAC) or the ACWPcum (actual cost) is greater than the estimate at completion (EAC).

§  Verifying all project logs (contract budget base, management reserve, undistributed budget, etc.) and confirming summary numbers match the reporting deliverable information (CPR/IPMR/IPMDAR).

§  Confirming the schedule and cost data tell the same story and are reconciled to other project artifacts, such as the Work Breakdown Structure (WBS) Dictionary and work authorization documents.

§  Confirming the change logs are up to date ensuring approved changes are incorporated in the schedule and cost data in a timely manner. All changes must be fully traceable.

§  Conducting schedule data quality checks to verify logic, status updates, accurate coding, calendar assignments, reasonable descriptions, and the like.

§  Verifying subcontractor schedule and cost data for accuracy and timeliness.

It takes effort to produce and maintain quality schedule and cost data for the life of a project. It requires management focus and attention to detail. With the wealth of toolsets available and the amount of data available electronically, implementing steps to make it easier to maintain data integrity can make a difference.

The processes that companies implement have the potential to hinder the process to produce and maintain quality data. Always be prepared to update desktop procedures or other process steps to create a single data source that is fully traceable. Where possible, eliminate duplicate steps, duplicate data entry, or manual rekeying that has the potential to compromise data integrity or the ability to conduct data traces.

Whether you have a customer or contractor review coming up or not, data integrity is a fundamental requirement for an effective EVMS.

Call Humphreys & Associates to assist with the contractor Mock EVMS Review process including reviewing the system documentation, performing data quality checks, and helping prepare the data packages for delivery to DCMA. We can be reached at (714) 685-1730 or email us.

EVMS Compliance Review Series

EVMS Compliance Review (CR) Series #2 – Preparing for the EVMS CR Read Post »

EVMS Compliance Review Series #1 – Prep for the DCMA Compliance Review Process

By / / , , ,
, , ,
Update #1 November 7, 2017
Update #2 December 15, 2020

This is the first topic in a series discussing the DCMA Compliance Review (CR) process. It is the second update to the original series #1 Blog that was based on the instruction book from the DCMA Website that identified the “DCMA 16 Step (and then their 8-Step [UPDATE #2]) EVMS Compliance Review Process.” This change was documented in the DCMA “Earned Value Management System Compliance Reviews Instruction (DCMA-INST 208).” This Instruction has been rescinded and replaced with a set of DCMA Business Practices (BP). These Business Practices split out topics that the old DCMA Instruction 208 covered in one document. Whether you are a contractor new to the EVM contracting environment or a seasoned veteran, if the Earned Value Management System (EVMS) compliance and acceptance authority is the Defense Contract Management Agency (DCMA), these new Business Practices apply to you.

The overall EVMS review processes are covered in the following eight Business Practices (BP):

  • BP0 – Earned Value Management Systems (Overarching guidance, overview of the Business Practices)
  • BP1 – Pre-Award EVM System Plan Review
  • BP2 – System Description Review
  • BP3 – Program Support
  • BP4 – System Surveillance
  • BP5 – Review for Cause
  • BP6 – Compliance Review Execution
  • BP7 – Compliance Metric Configuration Control (DCMA internal document)

As noted in BP6, “[a] CR is a comprehensive assessment of a contractor’s system, comprised of a System Description (SD), command media, processes, related tools, and the contractor’s ability to implement the EVM system on contracts with an EVMS requirement. The intent is to demonstrate and document compliance to the 32 Guidelines in the EIA-748 EVMS standard. There are three steps for evaluating compliance:

  • Assess whether the contractor’s EVM SD adequately documents how its system meets the intent of the 32 Guidelines;
  • Evaluate the contractor’s ability to demonstrate the EVMS implementation as described in the SD and supplemental procedures;
  • Verify whether the EVMS is providing timely, accurate, reliable and auditable data.”

BP6 defines a uniform process for the DCMA EVMS Center personnel to conduct the planning, execution, and reporting of an initial contractor EVMS Compliance Review (CR), following 23 Steps in 5 major phases or Process Steps (see Figure 1 below from BP6):

Figure 1 - EVMS Compliance Review (CR), following 23 Steps in 5 major phases or Process Steps

This figure shows that the BP6 approach is essentially a combination and consolidation of the 16 Steps and the 8 Step processes in the two prior CR instructions. The tables below show those compliance review process approaches of the old 16 Steps and the 8 Steps:

Table 1. EVMS 16 Step CR Process
Earned Value Management System Review Process
Table 2. EVMS 8 Step CR Process
Process StepDescription
1Notification and Data Call
2Planning and the Data Analysis (DA)
3Opening Meeting
4Fieldwork
5Communication
6Report Writing
8Close-Out

The BP6 process adds clarity on the Review Planning process prior to the Notification steps shown in both the old sets of steps, and then the rest of the BP6 process is consistent with both of the older processes. This is much like the joke about the man who ordered a medium pizza for himself, and the pizza proprietor asked if he would like that cut into 8 pieces or 16 pieces or 20 pieces. The man replied “Hmmm. Eight pieces are probably not enough — I am really hungry, you had better do 20 slices!”

This Business Processes also remains consistent with the change the “8 Step process” made to the “16 Step Process” in that DCMA has removed all lower-level activities that previously involved DCAA and remain DCMA responsibilities. In addition, as part of the data analysis, the current Business Practices incorporate the use of the DCMA EVMS Compliance Metrics (DECM) to isolate any data issues for discussion during the review.

A frequent and common question that H&A consultants are asked is how long it takes to complete the DCMA Compliance Review process. In all the years H&A has been in business, it has varied based on:

§  The current contractual environment,

§  DCMA process requirements, and

§  The resources DCMA has available to do compliance reviews.

In the past, a contractor with an aggressive EVMS implementation plan could complete the process in coordination with the DCMA in about 18 months.

It is a different story today. DCMA is short on the resources they need to do compliance reviews and they have been updating their processes and requirements. The result: it can take up to three years to complete the process. This can have serious implications in those instances where a contractor is contractually required to have a DCMA EVMS Acceptance in place for a project. In some instances, the contract may be near completion or already completed by the time DCMA sends off a notification letter to the contractor.

Unfortunately, the modifications to the CR process have simply cut that “pizza” into 16 slices, then 8 slices, and now 23 slices, meaning that virtually the same process is required, no matter how you cut it. The problem was not the number of steps, but the length of the process (the pizza itself!).  The DCMA now intends to do data intensive reviews and to conduct parts of the EVMS review remotely, which could save time and money, hopefully without jeopardizing the integrity of the review process.

What do you do if you find yourself in this situation? One suggestion is to have a discussion with the Director of the EVMS Center to request a waiver from the system acceptance contractual requirement because the compliance review process cannot be completed before the contract completion date.

One critical implication is that each of these process approaches assumes successful accomplishment of each step. Any failure along the way will extend the time it will take a contractor to attain acceptance of their system and implementation. A contractor must be fully prepared to prevent a repeat of any of the steps in the process. It is essential to do things right the first time to prevent further delays. This is a discussion for the next topic in this series – preparing for the EVMS Compliance Review process by conducting internal mock reviews. Since “self-assessments” are no longer formalized in the CR Process, DCMA expects contractors to do internal surveillance and self-identify (and address) EVMS issues to possibly avoid formal Corrective Action Requests (CARs) from DCMA.

Contact Humphreys & Associates for questions about preparing for a DCMA compliance review. Go to our Contact page or email us.

EVMS Compliance Review Series

EVMS Compliance Review Series #1 – Prep for the DCMA Compliance Review Process Read Post »

Scroll to Top