earned value management

How to Get the Most from Your IMS: What Makes an IMS Useful?

By / / , , , ,
, , , ,
How to Get the Most from Your IMS: What Makes an IMS Useful?

A major challenge with an Integrated Master Scheule (IMS) is making the most out of this powerful project management tool. Large and even not-so-large projects are required to have an IMS which adheres to the requirements of the Integrated Program Management Report (IPMR) or Integrated Program Management Data and Analysis Report (IPMDAR) Data Item Description (DID) and meets the schedule data quality checks available. But having met the DID requirements and passing the quality checks does not mean the IMS is being used to get the most information into the hands of decision makers on the project. 

The IPMR or IPMDAR DID, even though detailed and thorough, is generic. The same DID is used on a contract whether it is for an important new hardware-based system, a new software system, or some other goal. The DID misses the point that the type of product on the contract means that unique topics may be the most important. There is no focus in the DID; it is high level and comprehensive. What’s missing are the special interest or special focus schedules that can be drawn from the IMS with good coding, grouping, filtering, and sorting techniques.

Remember that the statement of work (SOW) for the project is really comprised of promises made in the various plans submitted in the proposal and updated after contract award. The System Engineering Management Plan (SEMP), the Software Development Plan (SDP), Make/Buy Plan, Procurement Plan, Test and Evaluation Management Plan (TEMP), and others all contain commitments that should be translated into actions in the IMS. There should be access to the portions of the IMS that show how these commitments will be accomplished.

Examples of Special Interest Schedules

On a new hardware-based project, key pieces of information revolve around getting designs done and into a form to plan for the procurement of the various things needed to build the end items. This information often resides in the IMS but must be pulled out into a “drawing release” schedule. That is a schedule showing the transition from design to 3-D models or drawings. Pinpointing the times and links in the IMS where design becomes actionable for procurement enables the project team and their suppliers to be prepared to execute the procurements. The teams or engineers (designers) and procurement people should be focused on the drawing release schedule with frequent, at least weekly meetings to coordinate.  

One of the serious issues I witnessed on a project was the lack of preparedness in having trained and qualified drawing checkers who could sign-off and release drawings. Because of the need for flight safety considerations, the drawing checker position could only be filled by qualified people. Instead of preparing months ahead for the wave of drawings, the waves were allowed to crash into the far-too-few checkers. What was supposed to be a short cycle became weeks long as drawings sat waiting for release.

As the transition is made from design to production, then the procurement schedule becomes another focused schedule which should show the information about what is being procured, from whom, and when it will be delivered. This information is used by the engineers, procurement, and inventory control people including receiving to understand the timing and volume of inbound items. Frequent coordination with suppliers and internal team members using this focused schedule helps to ensure a smooth process. Will the factory be ready to receive, inspect, process, and store all the inbound items?

What about software development as a special topic focus schedule? This is often needed. The project personnel must be able to quickly understand where in the cycle the software is and when releases will be made as well as what the release is needed for in the overall project. 

Think about all the other important focus areas that could exist on a project. For example, the training effort should be in a focused schedule showing the development of the training material (courseware), training aids, facilities, instructor preparation and anything else needed to execute the statement of work related to training.

Testing is possibly more complicated than training. This includes development of test plans and procedures, as well as the creation of test fixtures and tools. The preparation of test personnel capable of performing complicated tests should be in the focused schedule.

By now you get the idea. The IMS is the combination of all these schedules, the thing that coordinates between and among them. But the focused schedules are the real bread-and-butter of the schedule discipline. It is shocking to see a project where these do not exist. Maybe the managers don’t know they are needed or possibly that they are available. Maybe the schedule team did not prepare for these extractions from the IMS.

If your project does not have focused schedules and does not use them, the project is in jeopardy. These schedules should be generated and used frequently; weekly at least.  Even if the IMS itself is only undergoing a monthly update, the detailed schedules should be much more alive and part of the weekly communication between work teams on the project.

Tips and Suggestions

  • How to get started. Begin with the source documents and their authors, the SEMP, the TEMP, and so on. Make a list of the likely focused schedules that need to be drawn from the IMS. Read them and extract the information you need to build your IMS. If you are just starting the IMS, you could build individual schedules with the authors and then integrate them into the IMS. If you already have built the IMS, you can find the tasks you need and code them so they will appear on the focused schedule. Once coded, extract the focused schedule from the IMS using the applicable coding filters and verify it matches the commitments that were made in the source documents from the various authors. 
  • Hold “report court.” Rather than reading and sorting through all the various plans, schedule a project meeting with the key team members and ask them what reports and what schedules they need to do their jobs. Remind them of the commitments they made in the plans. They can bring or provide a list and description to you, and you can decide “in court” which items you can or cannot incorporate into the IMS. That can help to streamline what is included in the IMS and who is responsible for what. 
  • Create a schedule data dictionary if you haven’t already done so. This is essential to identify standard as well as project unique activity, milestone, or resource coding and how the coding is used so there is a common understanding of the content. The customer will need this information as well as when the IMS is provided as a monthly performance reporting data deliverable (see the IPMDAR Section 2.4.2.20, Data Dictionary for Native Schedule File). It is a prerequisite to ensure consistency in use as well as to establish a level of discipline throughout the IMS development and maintenance process. Ensure tasks always include the necessary coding. Ideally, you did your homework on the likely special topic schedules you need to draw from the IMS before starting to build the IMS. Otherwise you may need to determine various sorting and filter techniques to identify the tasks that require additional coding details. As noted above, once the tasks are coded, you or other project personnel will be able to extract the various special interest schedules from the IMS as needed. 

Need help?

Building a useful IMS for complex projects is not easy. Up front planning for the development of the IMS can help to identify the necessary outline codes and other coding to be able to group, filter, or sort the activities to extract the special topic or special focus details from the IMS. The IMS is an essential communication tool for everyone on the project. How the schedule is constructed and coded makes a difference. H&A scheduling subject matter experts (SMEs) have decades of experience in a variety of complex project environments and can help you avoid common pitfalls. Contact us today.

How to Get the Most from Your IMS: What Makes an IMS Useful? Read Post »

Establishing Milestones in the Integrated Master Schedule (IMS) Appropriately

By / / , , , ,
, , , ,

The purpose of the Integrated Master Schedule (IMS) is to model and communicate the plan to accomplish a project’s objectives. A key part of that model is the identification of key events that are represented as milestones. The selection of these milestones should be done with consideration for its purpose – what does the milestone represent and communicate? You should be aware of the intent of each milestone that is entered into the IMS. The IMS is a critical communication tool to ensure everyone on the project has a common understanding of the project’s work flow. Too many times I have witnessed a scheduler slam a milestone into the IMS without regard to how it impacts the schedule logic. This could be due to haste but, in my experience, it is often due to a lack of understanding of the purpose of the milestone.

Figure 1 illustrates a common diagram for a milestone. 

Gate Review Milestone. C may not proceed until the Gate Review has been completed.
Figure 1 Example of a Gate Milestone

As illustrated in Figure 1, the milestone is a gate and will hold up work in task C until the milestone is claimed as finished.

If the intent is to have the milestone act as an indicator instead of a gate, then the diagram in Figure 2 could satisfy that intent. If a successor is needed for the indicator milestone, something like the “End of Project” milestone could be added.

Indicator Milestone - C may proceed as soon as A is completed. Until the Milestone is claimed finished, it will move along with the data date leaving its baseline behind and indicating it has not been claimed.
Figure 2 Example of an Indicator Milestone

An accomplished scheduler knows the dangers of a Merge in the IMS. The Merge introduces schedule risk. Imagine the damage to the schedule risk assessment (SRA) if a Merge were entered as illustrated in Figure 3.

MERGE in the IMS - Neither C no D may proceed until the Gate Review has been completed. Does C really depend on B or does D really depend on A?
Figure 3 Example of Merge Risk in the IMS

In addition to adding risk, as the question indicates in Figure 3, the situation portrayed may not be true.

Is the purpose of a milestone clear to everyone?

What is the real purpose of the milestone? That must be defined first so the diagram can be entered properly into the schedule, and the IMS can model the correct steps for the project. Along with the definition of the purpose, the completion criteria should be defined and documented.

Unfortunately, this problem often extends to others on the project and even to those most responsible for the project – the Program/Project Managers (PMs). A case in point. Some years ago, a high-level customer PM challenged me, in my role as the contractor IMS architect, after the PM’s schedule subject matter expert (SME) expressed their concern that milestones in the IMS were being input incorrectly.

The milestone in dispute was the Preliminary Design Review (PDR). The PM and the PM’s schedule SME said the review was a gate and therefore should be modeled as illustrated in Figure 4. Note: In the real schedule, there were many more predecessors and successors to the milestone. Figure 4 simplifies the schedule content for clarity.

MERGE in the IMS - Neither C nor D may proceed until the Gate Review has been completed. Does C really depend on B or does D really depend on A?
Figure 4 Impact of a Gate Review Milestone

They both agreed that Tasks A and B were tasks to be done during the review itself and that the Milestone was to represent the satisfactory completion of the review. According to the definition of the PDR in the Integrated Master Plan (IMP) entrance/exit criteria, the review would lead to a letter of acceptance. The letter of acceptance was the definition of done in this case. When asked how long it would take from the time the review in A and B (and all predecessors) would be held until the letter was received, the answer was something in the order of weeks.

A literal reading of the IMS would go like this: “Hold the review in tasks A and B (and all predecessors) then wait for the approval letter before starting any other work.”

When asked if it was the PM’s intent for the several hundred engineers and others working on the project to put down their pencils after the review and wait for the letter while doing nothing as shown in PM’s desired version of the milestone in the IMS, the immediate reaction of the PM was shocked silence. Of course not. The project could not go on hold for even one week waiting for a letter. The teams would disband, and the workforce would be gone. Work would stop.

I then told the PM it was not the contractor’s intention to go parade rest and wait for the letter even if he had thought that was what was supposed to happen. If the review in A and B was deemed successful with some reasonable set of action items, then the teams would proceed. It might be that they would proceed on risk, but they would proceed anyway. I then showed the PM and the PM’s schedule SME how we would model the review in the IMS to show proceeding on risk. It would look like the example in Figure 5 if we implemented the milestone as an indicator milestone.

Review as Indicator Milestone - C and D may proceed when their respective predecessor is completed. The milestone is not a gate.
Figure 5 Review as an Indicator Milestone

The PM thought that could work but was concerned there was no gate review aspect to this diagram and PM control of the project would be weakened or lost. I then showed him how we could put the review into the IMS as an indicator with a delayed gate effect. In other words, work would proceed while the letter was being prepared but would stop at some point if the letter was not received. That diagram looked like the example in Figure 6. 

Review as Indicator Milestone but also a Gate - C and D may proceed on risk when their respective predecessor is completed. E and F however may not proceed until their immediate predecessor (C or D) and the Gate Milestone are finished.
Figure 6 Review as Indicator and as a Gate

The letter could be prepared while the teams worked on tasks C and D. If issues arose then the teams would be compelled to stop after tasks C and D individually. In this case an issue with task C might not hold up task D and conversely, an issue with task D might not hold up task C. This was a measure of control the PM thought would be adequate when the need for the approval letter in the milestone was also added.

Talking Through the IMS to Verify the Intent of Milestones

The point is that the IMS is a model of the project that should define exactly what is supposed to happen. What exactly is the IMS telling us to do? Is the review a gate? Is it just an indicator? What do the documents and agreements say about the milestone? This is definitely not the time to quickly slam a milestone into the schedule logic without taking the time to think about its purpose or what you want to communicate to someone else on the project.

This story also highlights the importance of ‘reading’ or ‘talking through’ the IMS. When it was explicitly stated that the project would be put on hold if the schedule depicted in Figure 4 were followed, the team quickly realized the need for a better approach, leading to the development of a more effective plan.

Interested in Learning More?

There is an art and skill that is honed over time for creating integrated master schedules that accurately reflect the work to be performed and clearly communicates that plan to everyone on the project. There is always more to learn. H&A offers basic and advanced scheduling workshops taught by senior master schedulers with decades of experiences in all types of scheduling environments that can be tailored for the scheduling tools you are using. Give us a call today to get started.

Establishing Milestones in the Integrated Master Schedule (IMS) Appropriately Read Post »

Management Reserve Best Practice Tips

By / / , , , , , ,
, , , , , ,

A recurring theme H&A earned value consultants find themselves discussing with clients is emphasizing that management reserve (MR) is a very precious budget set aside that must be protected and used appropriately. Unfortunately, MR is often used inappropriately, and quickly depleted in the early stages of a project.

What happens when MR is consumed for other uses than what it was intended? There is no budget available for appropriate uses of MR such as for emerging work, rework, redesign, or make/buy adjustments within the scope of the contract when it is needed in the latter stages of a project. When that happens, a project manager is forced to create a “home” for actual costs for these activities. This results in other inadvisable actions such as:

  • Zero budget work packages which are also known as estimate to complete (ETC) only work packages.
  • De earning the budgeted cost for work performed (BCWP) and opening completed work packages to accept charges.
  • Culling budgets from future unopened work packages, and if they exist, planning packages, summary level planning packages (SLPP), and undistributed budget (UB).

These actions will call into question the integrity of the EVMS and EVM data. The customer conducting EVMS surveillance will also be quick to point out this deficiency in the EVMS implementation and raise the issue to ensure it has management’s attention to correct. The inappropriate use of MR has created a cascade of problems that could have been avoided. In some instances, project personnel were simply not following the rules for the use of MR found in the contractor’s EVM System Description. That’s an easier problem to resolve than other root causes.

The Role of Risk and Opportunity Management in Establishing MR

What H&A earned value consultants often uncover as the root cause of inappropriate uses of MR was that a robust risk and opportunity (R&O) management process would have made a difference in establishing a quantified set aside for MR to handle realized risks. Proactively identifying and managing risks improves project performance. The expectation of specific risks occurring leads to risk handling plans that lower the likelihood and impact of risks. It also provides an informed basis to establish an adequate amount of MR that reflects identified and assessed risks.

The risk assessment provides additional information that assists a project manager’s decision making process to validate a request to use MR is appropriate and has the backup data needed to justify the use of MR and the amount of MR allocated. This detail is necessary for the baseline change request (BCR) approval process as well as the Integrated Program Management Report (IPMR) Format 5 or Integrated Program Management Data and Analysis Report (IPMDAR) Performance Narrative Report (PNR). A project manager is required to identify the changes to MR during the reporting period and provide a brief explanation of the change. This explanation has the potential to pique the interest of the customer to gain a better understanding of why MR was used and the potential impact to the integrity of the EVM data.

Note: MR may increase or decrease for a variety of reasons. The primary use of MR is to handle realized risks within a control account that is within the statement of work (SOW) for the contract. All MR debits or credits should be tracked in a log for full traceability for the entire life of the project. Remember that MR can never be a negative value.

Acceptable Uses of MR

As highlighted in an H&A article titled “The Effective Use of Management Reserve,” examples of the appropriate uses of MR include:

  • Newly identified work is authorized and assigned to a control account manager (CAM). It may be that once the work begins, one or more tasks that were missed in the original planning process now need to be scheduled and resource loaded. Newly identified work could also be the result of internal replanning that required a change in approach or resource requirements.

    An example of this could be a project manager issued a work authorization to a CAM to conduct three tests to meet the requirements in the contract SOW. In the middle of the first test, it becomes clear to the CAM and project manager that a fourth test will be necessary. The project manager and CAM should be aware of this potential risk and be prepared to implement their risk handling strategy as a result of the R&O management process. The CAM can quickly prepare a BCR that the project manager can immediately approve to allocate MR budget to complete the fourth test. 
  • It is necessary to redo a task. This may include unanticipated redesign, remake, or retest. Hopefully, the project’s risk register identified the potential risks associated with the original tasks and management was prepared for the realized risk. 
  • Make/buy adjustments.  This could result in an MR debit or credit. 
  • Statement of work transfers from one organization to another. This could result in an MR debit or credit. 

Inadvisable Uses of MR Commonly Allowed

Although it is often allowed in a contractor’s EVM System Description, it is inadvisable to use MR for direct and indirect rate changes in the future. Note: MR should never be used to make any rate adjustments (or any other adjustments) to historical budgeted cost for work scheduled (BCWS) or BCWP data.

A rate change is not a change to the SOW for a CAM. It is merely a change to the cost of that work. Cost variances that occur because of direct and/or indirect rate changes can easily be explained in a Variance Analysis Report (VAR). Ironically, this use of MR is typically treated as a one-way street. Contractors apply MR when the direct and/or indirect rates are going up in the future but do not return to MR when the rates are projected to go down.

When a contractor’s EVM System Description allows MR to be used for future direct and/or indirect rate changes, ideally, the likely rate changes are identified as a risk and quantified when the initial MR is established for a project. This requirement should be noted in the EVM System Description. That way the set aside for MR includes budget for corporate rate adjustments that are outside of the control of the project manager or CAM. 

Another example of a commonly allowed but inadvisable use of MR is to “true up” a purchase order that is in excess of the original budget at completion (BAC) for material, equipment, or purchased services. For example, a project manager issues a work authorization to a CAM that includes purchasing material, equipment, or services from a supplier. The CAM then reaches an agreement with a supplier with scope, schedule, and budget. If that agreement is greater or less than the BAC, MR should not be applied, nor should budget be returned to MR to make the BAC match the PO value. Assuming the scope does not change, then MR should not be used to wipe out a cost variance whether positive or negative. The cost variance can be easily explained and the EAC can be increased or decreased. This is another example where contractors are treating this as a one-way street; they apply MR when it goes up, but do not return to MR when it goes down. A contractor would not “true up” for internal work overruns/underruns so why “true-up” for material or services provided by a supplier? 

Best Practice Tips

The following is a short list of best practices H&A earned value consultants often recommend clients implement for managing MR.

  • The EVM System Description should clearly spell out what are appropriate and inappropriate uses of MR. It should also provide guidance to eliminate instances of the “one way street” debit from MR. If needed, provide supplemental procedures, decision trees, or other work instructions to help project personnel follow EVM best practices and preserve MR for handling realized risks which typically occur in latter stages of a project.
  • Ensure that the R&O management process is integrated with the EVMS and provides the necessary risk identification and assessment information for the project manager to establish a realistic MR set aside based on quantifiable information. Where applicable, ensure likely rate changes are captured as a potential risk to the project and considered when the initial MR for the project is established if they intend to use MR for rate changes in the future.
  • Conduct recurring training to reinforce the purpose for MR and the appropriate use of MR. A recommended approach is to discuss a variety of use cases with project personnel so they know how to handle various situations that may occur on a project. 

Have you noticed “creative” uses of MR that are contrary to EVM best practices? Hopefully, you identified those situations as part of your EVMS self-governance process and were able to quickly implement corrective actions before your customer pointed out the issue to you. H&A earned value consultants often assist clients with producing procedures or work instructions that clearly spell out how to use MR appropriately. We also offer a range of EVMS training to reinforce EVM best practices including the appropriate use of MR. Call us today to get started.

Management Reserve Best Practice Tips Read Post »

Introduction to the Cost and Software Data Reporting (CSDR) Reporting Requirements

By / / , , , ,
, , , ,

A common client request is to assist them with sorting through the various DoD contractual reporting requirements and contract value reporting thresholds that apply. We frequently run into situations where a contractor needs clarification on why they have a Cost and Software Data Reporting (CSDR) requirement and whether they should seek to waive the requirement. Subcontractors to a prime often question the requirement to provide actual cost data directly to the DoD, especially for Firm Fixed Price (FFP) contracts.

Background

CSDRs are the primary means the DoD uses to collect data on the development, production, and sustainment costs incurred by contractors performing DoD acquisition contracts. It is a DoD system for collecting actual costs, software data, and related business data. The resulting data repository serves as the primary source for contract cost and software data for most DoD resource analysis efforts including cost database development, applied cost estimating, cost research, program reviews, analysis of alternatives (AoAs), and life cycle cost estimates.

CSDR reporting requirements are determined by the contract value regardless of the acquisition phase and contract type. In general, CSDR reporting is required for Acquisition Category I-II programs and Information System (IS) programs valued at more than $50M. They can also be required for Middle Tier Acquisition programs (greater than $20M) and other programs (greater than $100M). Risk can also be a determining factor regardless of the contract value.

DoD Instruction (DoDI) 5000.73, Cost Analysis Guidance and Procedures (March 2020), provides additional details about the cost data reporting. Table 1 in the 5000.73 lists the cost reporting requirements contract value thresholds. The DoD Manual 5000.04 Cost and Software Data Reporting (May 2021) is the primary requirements document for the development, implementation, and operation of the DoD CSDR system to ensure data reported is accurate and consistent.

About CADE

The Office of the Secretary of Defense Cost Assessment and Program Evaluation (OSD CAPE) established the Cost Assessment Data Enterprise (CADE), a secure web-based information system that hosts the controlled unclassified CSDR repository, the Defense Acquisition Cost Information Management System, and the forward pricing rate library. CADE also contains a selected acquisition report database, a contracts database, data analytics capabilities, and a library containing cost estimating content such as cost analysis requirement descriptions and cost estimates. CADE is access-controlled, and available through the public-facing CADE Portal website.

Similar to the cost estimating and proposal pricing functions within contractor’s organizations that rely on historical actual costs to assess the validity of a proposed cost estimate, independent and sound cost estimates are vital for effective DoD acquisition decision making and oversight. CADE plays a critical role in capturing the expenditure, technical, and programmatic data after contract execution in a consistent manner to enable independent cost estimating and analysis. This cost estimate data is essential to support efficient and effective resource allocation decisions throughout the planning, programming, budgeting, and execution process for the DoD.

CSDR Reporting Requirements

There are a series of Data Item Descriptions (DIDs) for this reporting requirement.  Some forms are submitted electronically using DoD defined XML schemas, Excel, or JSON encoded data in accordance with a File Format Specification (FFS) and Data Exchange Instruction (DEI). The list of DIDs are as follows. These DIDs can be downloaded from the CADE website.

  • Contract Work Breakdown Structure, DI-MGMT-81334D (May 2011).
  • Cost Data Summary Report, DI-FNCL-81565C (May 2011), DD Form 1921, XML Schema.
  • Functional Cost-Hour Report, DI-FNCL-81566C (September 2015), DD Form 1921-1, XML Schema.
  • Progress Curve Report, DI-FNCL-81567C (May 2011), DD Form 1921-2, XML Schema. 
  • Sustainment Functional Cost-Hour Report, DI-FNCL-81992 (May 2011), DD Form 1921-5, XML Schema.
  • Contractor Business Data Report, DI-FNCL-81765C (March 2021), DD Form 1921-3, Excel. 
  • Software Development Report, DI–MGMT-82035A (October 2022), DD Form 3026-1, XML Schema. 
  • Software Maintenance Report, DI–MGMT-82035A (October 2022), DD Form 3026-2, XML Schema.
  • Enterprise Resource Planning (ERP) Software Development Report, DI-MGMT-82035A (October 2022), DD Form 3026-3, XML Schema.
  • Cost and Hour Report (FlexFile), DI-FNCL-82162 (November 2017), JSON encoded data file following FFS and DEI.
  • Quantity Data Report, DI-MGMT-82164 (November 2017), JSON encoded data file following FFS and DEI.
  • Maintenance and Repair Parts Data Report, DI-MGMT-82163 (November 2017), Excel.
  • Technical Data Report, DI-MGMT-82165 (November 2017), Excel.

The Cost and Hour Report (FlexFile) and Quantity Data Report play a critical role in collecting cost data from contractors for the DoD data repository because they use JSON data encoding to organize the content. They are intended to replace the legacy 1921 series of paper-based formats including the DD 1921, 1921-1, 1921-2, and 1921-5. It also requires contractors to provide significantly more historical cost data than the 1921 formats. As a result, the DoD cost estimating community has additional insight into historical costs. The goal is to establish a common framework and standard nomenclature to collect data from different contractors, all of them with unique cost accounting structures, that are mapped to the DID, FFS, and DEI requirements for use in the data repository.

Establishing a Consistent, Repeatable Process to Produce the CSDR Data Deliverables

For contractors new to the CSDR reporting requirements and in particular, the FlexFile JSON data encoding, can appear to be daunting. That’s where software tools such as those from Midnite Dynamics can help. Midnite Dynamics specializes in assisting contractors with producing the CSDR data deliverables. 

Their software tool, C*CERT+, streamlines, automates, validates, and produces the legacy 1921 family of Excel and XML reports as well as the FlexFile and Quantity Data Report JSON submittals. C*CERT+ eliminates what otherwise is a manually intensive, resource draining, tedious and costly effort subject to recurring rejections. It is one thing to create the required legacy reports or FlexFile JSON files for submittal, it is another to pass the submittal validation process. C*CERT+ provides numerous data validations and analysis reports to ensure the data is 100% compliant before it is submitted. For example, the software includes over 90 FlexFile validations to ensure data compliance as illustrated in Figure 1.

Figure 1: Example of FlexFile data validation results.
Figure 1: Example of FlexFile data validation results.

The software includes a Validation and Remarks utility to analyze the source data details that could result in a Validation Trip. Remarks can be entered directly into the validation module for anything that requires an explanation. This is illustrated in Figure 2. This narrative is included with the data submittal.

Figure 2: Example of providing remarks about the FlexFile data content.

C*CERT+ also interfaces with existing EVM cost tools and accounting systems to produce the existing legacy 1921 reports, the FlexFile, and other data submittals as well as to consolidate separate projects/CLINs/task orders into a single contract report.

Once the C*CERT+ Standard Category Mapping Rules are set up, they can be shared throughout the corporation or business unit to establish a standard and repeatable process for producing the data deliverables. This mapping process translates the contractor’s source data into an output that matches the CSDR data submittal format rules. This saves a tremendous amount of time and makes it much easier to consistently produce the CSDR data deliverables. An example of the Mapping Rules is illustrated in Figure 3.

Figure 3: Mapping Rules translate contractor unique cost data into a format that matches the CSDR data submittal requirements.

Do your process and procedures or training materials need an update to include specific guidance for project control teams to produce required DoD contractual reports or data submittals using your tool sets of choice? Give us a call today at (714) 685-1730 to get started. 

Introduction to the Cost and Software Data Reporting (CSDR) Reporting Requirements Read Post »

Life After EVMS Certification – Surveillance

By / / ,
,

Your company has just spent a year or more pursuing the EVM System Certification, going through the formal Compliance Review by the DCMA, and clearing the Corrective Action Requests identified during that review.  Finally, the company has received that coveted System Certification notification from the DCMA!  So, now the company is finally through with that process, right? 

Well… not exactly!

Now comes the really hard part – living up to that certification throughout the life of the contract(s). 

Can you lose DCMA System Certification?

All too often contractors, who have suffered through many months of trying to get their EVM Systems accepted by the government, have tended to let their guard down once certification is attained, letting their implementation of EVMS fall into disrepair.  Doing so can only end badly: poor or late data submittals, cost and/ or schedule surprises (overruns/ missed delivery dates, etc.), and generally unhappy customers.  When the customer is not happy, nobody is happy!  Failure to correct this situation on the company’s part could ultimately result in what is known as a Review For Cause conducted by the DCMA to determine if the company’s EVM Certification should be withdrawn.  Losing the EVMS Certification not only means the contractor can no longer claim in proposals that they have a certified EVMS, it can also cause a company several unwanted contractual, monetary, and reputation impacts – and the recertification process itself is no picnic either.

EVMS Business Practices

To help avoid the above unpleasant consequences, part of the DCMA’s EVMS review series includes ongoing Surveillance of a contractor’s EVM System, starting immediately after it has been certified.  The DCMA has a series of EVMS related Business Practices (BP) [updated May 2020]

  • BP0 – Earned Value Management Systems-Overall guidance on DCMA EVMS assessments
    • BP1 – Pre-Award EVM System Plan Review -of contractor proposal for EVM compliance
    • BP2 – System Description Review –of contractors EVM System and related documents
    • BP3 – Program Support – DCMA’s general EVMS-related support (IBRs, data review, etc.)
    • BP4 – System Surveillance – post-EVMS ongoing review of continued EVMS compliance
    • BP5 – Review for Cause – to assess if an EVMS acceptance is to be withdrawn
    • BP6 – Compliance Review Execution – how a Compliance Review will be run

The focus of this article is BP4 – System Surveillance, but do note that there is also BP5 on conducting a Review For Cause (RFC) discussed briefly above.

The stated purpose of BP4 is that it: “Defines the process to evaluate contractor EVMS compliance through continuing surveillance.” 

Yes, this means it is a compliance review after a contractor has gone through their Compliance Review! The basic intent of this process is to keep some pressure on the contractor to make sure EVMS implementation remains high quality throughout the life of the contract.

Drive the Car

Using our building-a-car analogy from Part 1 of this series: we have designed and built our car and trained our people how to drive, so it is now time to DRIVE OUR CAR. If we do not maintain all the car’s systems properly we are likely to encounter a lot of warning indicator lights (poor performance). If we ignore those indicators, the car is likely to fail on us. If the car blows up, we may have to go through the whole car building process again.

The DCMA does not want a contractor’s EVM System to fall into disrepair either.  The Review For Cause (RFC) and overall recertification process is unpleasant for the DCMA, too. This is also why, unlike in the past, the EVMS Center team players for Surveillance reviews are pretty much the same ones who participated in the formal Compliance Review:

  • Director, EVMS Center
  • Group Lead, EVMS Center
  • Team Lead, EVMS Center
  • Team Member, EVMS Center

There will likely be fewer Team Members than were on the full Compliance Review team, but as you can see, the Leaders will be the same. The DCMA has been placed in charge of all surveillance activities, and Government Program if team members may be needed to augment the DCMA Team.

The Compliance Review (CR) process [BP6] consisted of 5 phases:

  1. Plan
  2. Execute Pre-Event activities
  3. Execute – CR Onsite Activities
  4. Report
  5. Closing Actions

The Surveillance Review process [BP4], however, is comprised of only 3 phases:

  1. Plan
  2. Conduct
  3. Report

The Surveillance Process is primarily conducted off-site, although some on-site interviews could be required if follow-up action is required because potential non-compliances exist as a result of the data analysis.  Typically, on-site actions would be interviews of CAMs or other contractor personnel.

Plan phase:

  • Identify Requirements:  The EVMS Center Team Member works with the contractor’s EVMS point of contact to establish or update an Annual EVMS Surveillance Plan (SP).  They will jointly identify the contracts with EVMS requirements that may be subject to the surveillance activities, which may include existing contracts, new contracts, modifications to existing contracts, or even subcontracts with EVMS requirements.
  • Risk Assessment: The EVMS Center Team Member will use a Risk Assessment Worksheet to “identify a population of representative contracts” against which the DCMA can test all aspects of the system.
  • Create/ Update the Surveillance Plan (SP):  Before the start of each Fiscal Year, the EVMS Center Team Member will create or update the Surveillance Plan to document the surveillance activities using all the candidate contracts for the upcoming Fiscal Year.  The Plan is to ensure the complete evaluation of all 32 Guidelines for the site’s EVMS over a 3-year cycle to support the site re-certification.  This evaluation includes data analysis and the DCMA EVMS Compliance Metrics (DECM) for the particular Guidelines identified in the plan for each surveillance event.  This means that over the 3-year cycle 10 or 11 Guidelines must be evaluated each year in order to cover the 32 Guidelines.  This Surveillance Plan must be reviewed and approved by the EVMS Center Group Lead.

Conduct Phase:

  • Data Call:  At least 45 days prior to each surveillance event, data will be required based on the specific Guidelines being covered in that particular event.  This will typically be a subset of the Data Call required for the Compliance Review.
  • Execute Data Analysis:  Using data provided in the Data Call, the Team Member will use the tests identified in the DECM for the Guidelines being reviewed in each surveillance event.  The contractor should be running these metric checks as part of their own self-evaluation, and the EVMS Center Team Member can use those results in addition to running independent calculations on those metrics.  This is often done if Joint Surveillance is used by the DCMA, but the DCMA Team Member must still perform independent assessments and document results of those checks.
  • Evaluate Results:  Identify potential compliance issues resulting from data analysis metric tests.
  • Follow-up:  Discuss issue with the contractor, get more data samples to test further, conduct interviews of contractor personnel, as needed.  Convey data analysis concerns to the contractor before the surveillance event.
  • Closeout:  To close out the action, one of these will take place:
  • Accept as the correct execution of the contractor system
  • Highlight as a risk for future surveillance
  • Issue a Corrective Action Request (CARs)

The EVMS Center Team Member provides an out brief at the end of the surveillance event.

Report Phase:

  • Document results:  summarize/ provide the following:
  • A report with an Executive Summary
  • Results from all metrics evaluated
  • Follow-up actions taken on metric results
  • Summary of CARs issued/ observations made
  • Summary of contractor internal CARs/ findings
  • Summary of reasons for a Risk Assessment/ Surveillance Plan update (if applicable)
  • Identified risks for evaluation at future surveillance events
  • CAR/ Corrective Action Plan (CAP) status for the site

The EVMS Center Team Member identifies any significant deficiencies to the Team Lead, and then (as necessary) to the Group Lead and the Director for concurrence and processing.  A letter will be issued to the Contracting Officer (CO) notifying them of the completion of the 3-year EVMS compliance assessment, current system status, and any outstanding Corrective Action Requests (CARs) or Corrective Action Plans (CAPs).  The EVMS Center Team Member enters the approved plans/ reports to the Integrated Workflow Management System (IWMS) and for subsequent distribution to appropriate offices and the contractor (as required).

Compliance Review Support

As you can see, this seems almost as involved as getting ready for a full-up Compliance Review.  This is probably by design, since the DCMA wants to emphasize the importance of maintaining the EVM System in a compliant state for the life of the contract.  As a contractor, it is equally important to you because if your system application falls apart and you lose your EVMS certification (via BP5), the recertification process (BP6) starts all over again.

Humphreys & Associates has people ready to help you wade through the government requirements for EVMS certification, whether you are new to EVMS, or you are an experienced contractor needing help getting ready for a Compliance Review, System Surveillance, or even a Review For Cause.  We also have a wide range of training courses for beginners in EVMS or advanced courses for experienced Earned Value professionals.  We can also prepare your company for Surveillance, an IBR, an RFC, or even a full blown Compliance Review. Contact us today at (714) 685-1730 or e-mail us.

Life After EVMS Certification – Surveillance Read Post »

Scroll to Top