Common Problems Found in EVM Systems and Recommended Corrective Actions – Part 4

By / /
HA_Blog-Common-Problems-part-4-Twitter

This is the fourth part of a five part series regarding common problems found in EVM Systems and the recommended corrective actions to help mitigate those findings.  The previous three articles discussed:

common problems found in evm systems - part 4

The topics anticipated for part five are: Inappropriate use of PERT and LOE; Misuse of Management Reserve: Administrative CAMs.

1)  Misalignment between BCWP and ACWP

The Earned Value Management System Description Document (EVM SDD) should include a statement that requires Actual Cost of Work Performed (ACWP) to be reported within the same accounting period as Budgeted Cost for Work Performed (BCWP) is earned; which is most applicable for material.  Both ACWP and BCWP contain the term “Work Performed”.  The ACWP is not a measure of how much has been spent but rather reflects how much it cost to accomplish the scope of work reflected in the BCWP.

Accounting systems generally record actual costs for material when invoices are paid; this may or may not align with when earned value is claimed for that material.  If material earned value is claimed at point of usage, it may be necessary to collect actual costs in a holding account and then delay recording ACWP in the earned value system until the material is used.

When material earned value is taken at the point of receipt, invoice payments may be delayed for 45 days (or more). The actual costs associated with this material will be recorded in the accounting system after the earned value credit is taken.  In this case, recording ACWP in the earned value system must be accelerated.  The process of delaying or accelerating the recording of ACWP in the earned value system is often called using “Estimated Actuals” or, more appropriately, “Estimated ACWP”.

There are two obvious examples of this process being done incorrectly.  The first is in the data where BCWP is claimed without corresponding ACWP in the current period, or vice versa.  This may be below the threshold level for variance explanation and is often attributable to Level of Effort (LOE) control accounts, but it creates a situation that attentive customers will need to understand.  The second example is more direct, and occurs when contractors simply explain the situation in Variance Analysis Reports that are subsequently summarized in the Contract Performance Report (CPR) or Integrated Program Management Report (IPMR) Format 5.  The Control Account Manager (CAM) will use words such as “billing lag,” “accrual delay,” or “late invoicing” in the explanation of a cost variance.  Consequently, any time that financial billing terms are used to explain a cost variance, it raises a flag regarding a potential misalignment between BCWP and ACWP.

One issue with ACWP and BCWP misalignment is that it invalidates the use of the earned value data for predictive purposes.  Unless both data elements are recorded within the same accounting period, using indices such as the CPI, TCPI, or IEAC  (Independent Estimate at Completion) will deliver erroneous results.  The time and effort of the CAMs in the variance analysis process should be spent on managing the physical progress and efficiencies of the work, not having to explain payment or accounting system irregularities.

Most Common Corrective Action Plans

When this issue is reported, the best response is to develop a disciplined Estimated ACWP process, including logs and a monthly trace from the Accounting General Ledger to the EVM ACWP.  It is also important to train the CAMs and support staff on how to record and subsequently retire those entries in an Estimated ACWP log book.  Reviewers of the Variance Analysis Reports should be trained to screen for entries that indicate an inappropriate alignment between BCWP and ACWP.  In addition, as indicated in the blog discussion on Data Integrity (Part 2 of this series), situations where there is BCWP without corresponding ACWP, or vice versa, at the control account level, should be flagged and justified by the CAM prior to submittal of the CPR/IPMR to the customer.

2)  Freeze Period Violations

“Freeze Period” refers to future accounting periods, including the current accounting period, in which baseline changes should be strictly controlled.  This is also sometimes called the “Change Control Period”.  The definition of this period should be in the company’s EVM SDD, but will usually have a time-frame such as “current accounting period plus the next accounting period”.  The SDD should specify what kinds of changes are allowed within this period, how they are to be documented in the CPR/IPMR, and any necessary customer notification or approval requirements when these changes are incorporated.  The SDD should require that customer approval is necessary for changes to open work packages that affect BCWS or BCWP in the current or prior accounting periods, and any changes to LOE data in prior periods or in the current period if the LOE account has incurred charges (ACWP).

There is an additional requirement specific to retroactive adjustments which includes the current period.  The EIA-748-C Guideline 30 specifically stipulates the requirement that these types of changes be controlled, and that adjustments should be made only for “correction of errors, routine account adjustments, effects of customer or management directed changes, or to improve the baseline integrity and accuracy of performance measurement data”.  Again, the reasons allowed for the changes should be specified in the EVM SDD.  However, regardless of the reason, it is a requirement that all retroactive changes be reflected in the current period data in the CPR/IPMR Formats 1 and 3, and that Format 5 include the related explanations (National Defense Industrial Association (NDIA), Integrated Program Management Division (IPMD), Earned Value Management Systems Intent Guide, August 2012).

Some projects have a great deal of volatility.  The incorporation of subcontractor data (especially if that data lags the prime contractor reporting period) and accounting system adjustments often create retroactive (including current period) adjustments.  The operation of change boards may also result in changes, both internal and external, which require immediate implementation.  EVM compliance in this environment is a matter of disciplined incorporation of changes, including visibility and communication to the customer (and sometimes prior approval) of any impacts to the baseline.

Most Common Corrective Action Plans

When discrepancies are found with freeze period noncompliances, the first action should be to ensure that procedures are in place that are compliant with the EIA-748.  The discipline required by these procedures must be communicated to the program team so that a consistent change control processes is maintained.  Key to compliance is visibility and communication of freeze period changes via CPR/IPMR Formats 3 and 5.

H&A has seen a loose interpretation of the guideline allowance for adjustments to “improve the baseline integrity and accuracy of performance measurement data”.  Care must be taken that adjustments falling under this category are not made to avoid variances.

3)  Failed Data Traces

The reviews associated with EVM surveillance and compliance have become increasingly data centric for the past several years.  One of the first steps in a review is submittal to the customer of a complete set of EVM data so analysis can be conducted against predefined success criteria prior to conducting an on-site review.  When there is an on-site review, the data trace portion of that review can be a major component at the company, project, and Control Account Manager levels.

The primary purpose of the data traces is to evaluate the Earned Value Management System.  Is the EVMS operating as a single integrated system that can be counted on for reliable and valid information?  The data traces performed generally follow three separate threads: Scope, Schedule, and Budget.  There are a variety of documents and reports that contain this information, but the reviewers will look for a single thread of data to flow and be traceable throughout the system.

All systems are different, but a common strategy for data traces might be as follows:

  • Scope:  WAD → WBS Dictionary → Contract Statement of Work.
  • Schedule:  WAD → IMS → CAP.
  • Cost (Budget):  RAM → WAD → IMS → CAP → CPR/IPMR Format 1 → CPR/IPMR Format 5.
  • Cost (ACWP):  CAP → Internal Reports → CPR/IPMR (Formats 1 & 2) → General Ledger.

If there are also supplemental sources of data that flow into the EVMS, such as subcontractor, manufacturing, or engineering reports, then these should also be a part of the data trace.

The key to this process is the concept of “traceability”.  The easiest path to prove traceability is if the data are an exact match; however, this is not always possible.  Prime contractors often have to make adjustments to subcontractor data, use of estimated ACWP often will not allow a match with the accounting ledger, and supplemental schedules often “support” the IMS while not matching exactly.  These are normal and explainable disconnects in the data.  When submitting data for review, it is important to know where the data does not match and to pass that information on to the reviewers.  If preparing for an on-site review, the CAMs and others who may be scheduled for discussions should perform a thorough scrub of the data and have quick explanations available when a trace is not evident in that data.

Most Common Corrective Action Plans

It is important that any special circumstances that cause traceability issues be relayed to the review team with the data submittal.  The people who conduct the analysis often operate independently until they are on-site for the review, and it is possible to avoid misunderstandings by identifying any issues with the submitted data set.  This type of communication has the potential to eliminate unnecessary findings.

A short term response to a data trace issue is to establish a process to screen the EVM data before submission to the customer.  Starting with the accounting month end, the statusing and close-out process requires a comparative analysis of the various databases containing the same information.  Because of the volume of data contained in most systems, this should be automated.  There should be time in the monthly business rhythm to allow for corrections and data reloads to improve the accuracy across the various data locations.

The best approach to improved data traces is to design a system that minimizes the number of entries for a single set of data.  For example, H&A found one contractor with over 10 different databases where the CAM’s name was hand entered which resulted in a configuration control nightmare for that data element.  The process of system design should include a complete listing of common data elements that are included in the storyboarding of the process flow.

The topics anticipated for Part 5 are: Inappropriate use of PERT and LOE; Misuse of Management Reserve: Administrative CAMs.

To read previous installments:

  • Part 1 – EAC Alignment Issues, Poor Variance Analysis, Lack of Effective Subcontract Management
  • Part 2 – Poor use of Percent Complete; Data Integrity Issues; Poor Scope Language
  • Part 3 – IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse

Common Problems Found in EVM Systems and Recommended Corrective Actions – Part 4 Read Post »

Common Problems Found in EVM Systems and Recommended Corrective Actions – Part 3

By /

Part 3: IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse

This is the third part of a five part series regarding common findings discovered in Earned Value Management Systems (EVMS) reviews and the recommended corrective actions to help mitigate those findings.  The previous two articles discussed:

Part 3: IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse

The topics anticipated for parts four and five are:

Part 4: Misalignment between BCWP and ACWP; Freeze Period Violations; Failed Data Traces.

Part 5: Inappropriate use of PERT and LOE; Misuse of Management Reserve; Administrative CAMs.

1.  IMS Health Problems 

Several years ago the Defense Contract Management Agency (DCMA) issued the 14-Point Assessment Metrics. Twelve of the metrics are related to the “health” of the Integrated Master Schedule (IMS) while the remaining two (critical path length index and baseline execution index) are “tripwire” metrics for schedule performance.  Of the 12 health metrics, H&A has found that most discrepancy reports (DRs) are associated with missing logic, high total float, and high duration.

Missing Logic:  The DCMA uses logic checks to identify any incomplete tasks that are missing a successor or predecessor, or both.  As a rule of thumb, all activities should be tied to at least one predecessor and one successor with the exception of the first and last activities (respectively) in the project.  By the DCMA’s standards, there is an allowance of 5% for activities not having these types of relationships; but some believe that may be too loose.  In the Planning and Scheduling Excellence Guide or PASEG (National Defense Industrial Association, June 2012, version 2.0) states that all discrete tasks (excluding receipts/deliveries, LOE and summary tasks) should have at least one predecessor and one successor as even one missing logical tie could adversely affect the program’s ability to successfully execute the contract.

High Total Float:  Total float (or “total slack” for Microsoft Project users) is the amount of time an activity can be delayed or expanded before the finish date of the project is affected.  In the DCMA Program Analysis Pamphlet (DCMA-EA PAM 200.1, April 2012), any incomplete tasks with total float greater than 44 working days are considered as having high total float;  an allowance of 5% is also given before this metric trips a red flag.  The primary drivers for inappropriate high total float are missing successor linkages or planning well in advance of need.  For this metric, however, there are many conditions that may drive a high total float value that are perfectly legitimate.  This is especially true for longer projects and production projects that often receive materials well advance of need.

High Duration:  The DCMA Program Analysis Pamphlet classifies incomplete activities with a baseline duration greater than 44 working days as having “high duration”, and again have applied a 5% threshold to the metric.  The fear of very long tasks is that they may not provide enough precision for measurement of accomplishment and will introduce subjectivity into the statusing process.  As with total float, there may be conditions that drive high duration activities that are justifiable.  This is often the case when activities are representative of schedules outside of the IMS, such as at subcontractors or manufacturing planning systems.

Most Common Corrective Action Plans

The IMS is a critical management tool, and the purpose of the health metrics is to ensure that it provides an accurate plan and reliable forecasting for program management and execution.  The basic approach to resolving DRs written for IMS health issues is to first take all the necessary steps to improve the real health of the schedule using the metrics as indicators.  This includes a thorough review of the linkages, relationships, and task durations on an ongoing basis.  Organizations should establish a health check “rhythm”, to be used to review the IMS prior to customer submittal.  This process should also require the CAMs and their scheduling support staff to justify any conditions that may drive tripping a metric.

The contractor should work with its customer to gain a mutual understanding of the conditions that may legitimately result in high total float and high duration activities.  Contractors should try to avoid taking illogical actions, such as adding unnecessary linkages or arbitrarily breaking tasks into small durations simply to meet the metric requirements.  The IMS health metrics are simply indicators of potential issues.  If the nature of the program were one where relatively higher total float values or high durations are to be expected, the appropriate thresholds for tripping a metric may be higher than the standard 44 days.  In these cases, it is worth having a discussion with the customer to establish  new metric thresholds.

2. Data Item Noncompliance

The reports that are generated from an EVMS or IMS, and delivered to the customer, are usually placed on the contract by the incorporation of a Data Item Description (DID) and included in the Contract Data Requirements List (CDRL).  As of June, 2012, both earned value and schedule reporting are included in the Integrated Program Management Report (IPMR), DI-MGMT-81861.  It is important to generate the system reports in accordance with the appropriate DID as the requirements have changed with progressive releases.

Prior to the IPMR DID, IMS reporting was required per DI-MGMT-81650.  For EVM reporting, the previous DID was DI-MGMT-81466A, Contract Performance Report (for contracts established between March, 2005 and June, 2012), and before that was DI-MGMT-81466 or the Cost Performance Report.  The release of the Contract Performance Report DID in March 2005 also eliminated the use of the Cost/Schedule Status Report (C/SSR, DI-MGMT-81467) for new contracts.  There are, however, active contracts which use any one of the above DIDs as the requirements document for earned value and schedule reporting, and compliance of the submitted reports is evaluated against the DID that is required on each contract.

Data Item Descriptions are not just guidelines for reporting, they stipulate the contractual requirements for the documents. There are 203 uses of the word “shall” in the current IPMR DID, and some of these “shall statements” refer to a list of many requirements.  Any planned deviation, or tailoring, from the DID must be approved by the Procuring Authority and documented in the CDRL (DD 1423-1 on DoD contracts).  Section 3.0 of the “IPMR Implementation Guide” (OUSD AT&L PARCA, January 24, 2013) provides tailoring guidance for the IPMR.

Software programs used to generate the IPMR formats have reduced the amount of data specific errors in the reports; however, there are many requirements in the IPMR that are not related to data reporting.  In the requirements for the Format 5 (Explanations and Problem Analyses), for example, there are nine discussion requirements in addition to the required explanations for cost and schedule variances that exceed the variance thresholds.  The narrative portions of the IPMR cannot be generated by a software tool.

A contributing factor in the delivery of poor data items is when the customer encourages noncompliance or does not provide feedback on submitted reports.  It is easy to fall into apathy regarding compliance to the DID when there is no motivation to do so.  This situation, however, does little to convince other reviewers, such as the DCMA, that noncompliance is allowable.

Most Common Corrective Action Plans

When DID noncompliance is found and communicated to the contractor, the best immediate approach is correction and resubmittal of the document.  Noncompliance is most likely a discipline issue which requires a structured approach to developing the report, training the personnel who are responsible, and a thorough review prior to submittal.  Many organizations develop checklists that are used to ensure that all the requirements have been met prior to submittal.  There are also training materials available which can provide cell-by-cell instructions to make the proper entries into these reports (H&A has a DVD titled “Contract Performance and Funds Status Reports (CPR/CFSR) Completion and Reconciliation”).  It may be worthwhile to develop a “buddy system” with another program or another part of the company to exchange outside review and evaluation of data items.  This type of accountability can be mutually beneficial.

3. Planning Package Mismanagement

A planning package is far-term effort in a control account that cannot yet be subdivided into detailed work packages.  Planning packages share similar attributes as work packages, such as a time phased budget, a scope of work, start and finish dates, and must have enough detail in the IMS to support the development of a critical path.  There can be no accomplishment or actual costs recorded against the scope and budget that is defined in a planning package.  When enough information is available to detail plan the planning packages, they are converted to work packages.  This is done through a process called “Rolling Wave Planning”, and it is a good practice to have the detailed information available for at least six months in advance.  Advanced detailed planning is an effective approach to avoid unpleasant surprises, such as lack of availability of the necessary resources or the necessity to begin a hiring exercise.  In addition, near term lack of detail in the Integrated Master Schedule may drive improper or incomplete logic ties, which will impact total float and critical path analyses.

Company EVMS System Description Documents (SDDs) should provide guidance for rolling wave planning, including rules for any baseline adjustment in the current or near term periods.  It is important that planning packages are not allowed to exist in the current or past periods.  It is also improper for any actual costs (ACWP) or performance (BCWP) to be recorded against a planning package.  Most earned value engine software tools prohibit this, but some contractors have been known to override that prohibition in the toolset.  In addition to ACWP and BCWP, there should also be no cumulative BCWS in the current period for any planning package.  Cumulative BCWS is the most noticeable evidence that a planning package was not converted to a work package in a timely manner.

Most Common Corrective Action Plans

The most common corrective action is to conduct a monthly analysis of the EVM data to identify planning packages that are nearing the planning period.  While it is the responsibility of the control account manager (CAMs) to convert planning packages to work packages, Project Controls can easily provide the CAMs with a list of planning packages needing conversion.  If there is no guidance or process written for rolling wave planning, these should be developed to provide instructions to the CAMs and the support staff.  It is also critical that organizations maintain the restrictions in the earned value engines to prohibit the accrual of earned value or actual costs for planning packages.

Please contact Humphreys & Associates if you have any questions on this article.

Common Problems Found in EVM Systems and Recommended Corrective Actions – Part 3 Read Post »

Common Problems Found in EVMS Reviews and Recommended Corrective Actions – Part 2

By /

Part 2:  Poor use of Percent Complete; Data Integrity Issues; Poor Scope Language

This is the second part of a five part series regarding common findings discovered in EVMS reviews and the recommended corrective actions to mitigate those findings.  In the previous article, EAC alignment issues, poor variance analysis, and lack of subcontract management were discussed.  This second part focuses on: poor use of the percent complete earned value technique, data integrity issues, and poor scope language.

The topics anticipated for parts three through five are:

Part 3:  IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse.

Part 4:  Misalignment between BCWP and ACWP; Freeze Period Violations; Failed Data Traces.

Part 5:  Inappropriate use of PERT and LOE; Misuse of Management Reserve; Administrative CAMs.

Here are the three topics for this installment;

1. Poor Use of the Percent Complete Earned Value Technique

Earned Value techniques (EVT) fall into three distinct classifications:

  • Discrete
  • Apportioned
  • Level of Effort (LOE)

Within the Discrete technique there are methods ranging from objective to subjective.  The most objective methods are those that employ a “yes/no” question as the primary measure of accomplishment, e.g., “Has the activity begun?” or “Was the hardware delivered?”.  In these cases, there is no subjectivity in the evaluation of performance.

The percent complete technique can introduce very subjective criteria as the source of measuring accomplishment.  These are sometimes called “Manager’s Estimate”, or, derisively, “The Percent Guess” technique.  The problem with this application of percent complete is that visibility into accurate work package performance may be hidden until it is too late to take effective managerial action.  Worse, managers responsible for activities linked downstream from the percent complete work package may not discover the true status of a critical predecessor until it is too late to implement effective mitigation actions.   While a key mantra for effective project management is “no surprises”, misuse of the percent complete earned value technique can introduce the possibility for a great deal of surprise.

Most Common Corrective Action Plans

The goal in selecting the most appropriate EVT is to use the most objective rationale possible given the nature of the work.  There are key strategies that should be employed to reduce the subjectivity of the percent complete technique.

  1. Quantifiable Backup Data (QBD):  This is also sometimes referred to as “Predetermined Rationale”, and essentially requires the establishment of lower level milestones, activities, or steps to determine the percent complete.  These steps are established and weighted prior to beginning the effort. The percent complete is determined when the steps for the percent complete work package are accomplished; i.e. the weight assigned to the steps is earned.  It is recommended that a work package has at least one objective measurement point in each earned value period (generally monthly).
  2. Shorter Duration Activities:  Because one of the dangers of using the percent complete technique is a potential delay in reporting accurate status until it is too late to take management action, it is recommended that these types of work packages be relatively short in duration.  The DCMA publication “EVMS Program Analysis Pamphlet”(DCMA-EA PAM 200.1, July 2012) defines any activity more than 44 working days to be a “high duration”activity.  The choice of 44 days generally means that an activity with a duration of that or less will start and finish within 2 months.  Included in the pamphlet is the recommendation that the IMS contain no more than 5% of activities greater than 44 days.
  3. Percent Complete Threshold:  One method often employed when using percent complete is the introduction of a percentage ceiling that limits the reported accomplishment until the effort is 100% complete.  This limits the “99% Complete”scenario that can introduce false status and create the often incorrect impression that completion is imminent.  A common threshold in this mandate is to limit the percent complete to 75% or 80% until the activity is 100% complete. Some even limit the work package budget; example, maximum of 300 hours and the duration not to exceed three months.

2)  Data Integrity Issues

To standardize its reviewing organizations and the reduction in resources available to conduct on-site reviews, the DCMA is transitioning to establishing predefined success criteria and testing methods to evaluate the compliance of each guideline using data submitted by the contractor.  In addition, as mentioned above, the DCMA has released to its staff a “Program Analysis Pamphlet” that contains 17 measures of data integrity for contractor supplied data.  A few of these are:

Data Integrity Issues by Humphreys & Associates

Most Common Corrective Action Plans

In many cases, there may be scenarios that result in a temporary data condition that trips one of the above thresholds; such as Level of Effort (LOE) but these should be the exception rather than the rule.

Because of the growing interest in data integrity, it is important to screen for these conditions prior to submitting the data to the customer.  If possible, an organization should allow time in its monthly business rhythm to correct the easy-to-fix errors prior to submittal of the reports.  Those data flags that are legitimate or cannot be corrected prior to submittal should be reported to the customer in Format 5 of the Integrated Program Management Report (IPMR) or Contract Performance Report (CPR).  Some issues can be more systemic and require a longer correction process. These should also be identified to the customer with a corrective action plan which includes a date for when the corrections will be in effect.

And finally the prime contractor must review all reports delivered by the subcontractor for compliance with the appropriate DID Instructions, and be prepared to take action when the reports are noncompliant with the CDRL and/or DID Instructions.  This action can take the form of a contracting officer’s letter requiring adjustments in future reports, rejection and resubmittal of non-compliant reports, or rejection and contractual remedies; such as payment withholds or Award Fee impact, as applicable.

3) Poor Scope Language in the WBS Dictionary and/or Work Authorization Documents

The original scope for the project/contract is defined in the Contract Statement of Work (CSOW) and then organized within the Work Breakdown Structure (WBS).  The WBS Dictionary defines the technical boundaries of each WBS element and identifies the work products.  At a more detailed level, the scope is also identified in the control account plan and the work authorization documents.  There is an expectation that the scope language will become more specific and descriptive as it migrates from the project’s top level, through the WBS, down to the control accounts.

In EVM Surveillance Reviews, as well as Integrated Baseline Reviews (IBR)s, a normal set of activities includes data traces.  The project’s scope, schedule, and budget are traced across all the pertinent documents and reports to ensure that the EVMS is operating as a single, integrated system.  Included in the scope trace is a review of lower level (WBS element or control account) scope statements to ensure they are consistent with the CSOW.  It is also expected that the WBS Dictionary will be maintained and current with all external and internal changes that add scope to WBS elements or control accounts.  Often an organization begins a project with a well-documented set of scope statements, only to let them languish in the face of constant and voluminous changes.

Most Common Corrective Action Plans

Like Variance Analysis and Reporting, having accurate, current, and descriptive scope statements in all pertinent documents requires a disciplined approach and persistent review.  The first step is establishing a procedure that makes this an important part of the earned value process, including the importance of maintaining the currency of the WBS Dictionary by technically qualified personnel.  The quality of these documents is generally the responsibility of the CAMs in concert with the system engineering organization or equivalent; however, the Project Controls staff plays an important role in assisting the CAMs and reviewing the products.

One of the most common findings in this area is the failure to maintain the WBS Dictionary.  This document must reflect the current scope and incorporate all approved changes.  The contractor should establish a process in the monthly business rhythm that “sweeps” all known changes, generally through the work authorization documents, and evaluates those for incorporation into the Dictionary.

Please contact Humphreys & Associates if you have any questions on this article.

Common Problems Found in EVMS Reviews and Recommended Corrective Actions – Part 2 Read Post »

Common Problems Found in EVM Systems – Recommended Corrective Actions – Part 1

By /
HA_Blog-Common Problems part 1

Humphreys & Associates Corrective ActionsHumphreys & Associates (H&A) has the opportunity to work with a broad spectrum of clients who operate their Earned Value Management System (EVMS) under the contractual authority of a variety of customers.  Many clients have a surveillance program conducted by the Defense Contract Management Agency (DCMA), the Department of Defense (DoD) Executive Agent for EVMS which also has a reciprocal agreement with other agencies.  Some agencies, such as the Department of Energy (DoE), have their own surveillance programs.  H&A also works with many companies to help them “tune up” their EVMS as a part of sound business practices.

In the course of conducting and supporting client reviews, H&A has identified several recurring themes that many organizations allow into their EVMS, and because we often support the resolution of these issues through the Corrective Action Planning process, we can also recommend the most common remedies to prevent or correct them.

This will be a 5 part series that will cover three topics in each article.

The anticipated topics for parts two through five (subject to change) are:

            Part 2:  Poor use of Percent Complete; Data Integrity Issues; Poor Scope Language.

            Part 3:  IMS Health Problems; Data Item Non-Compliance; Planning Package Misuse.

            Part 4:  Misalignment between BCWP and ACWP; Freeze Period Violations; Failed Data Traces.

            Part 5:  Inappropriate use of PERT and LOE; Misuse of Management Reserve: Administrative CAMs.

1) The Estimate at Completion (EAC) is systemically out of alignment with cumulative performance with no justification.

The Earned Value Management System Guidelines (EVMSG) define the EAC as the sum of the contract’s cumulative to date Actual Cost of Work Performed (ACWP) plus the company project manager’s best estimate of the time-phased resources (funds) required to complete the remaining authorized work, the Estimate to Complete (ETC).  One of the measures that is used to provide a “sanity check” of the reasonableness of the EAC is to compare the To Complete Performance Index (TCPI) to the Cost Performance Index (CPI).

  • CPI = BCWP / ACWP: An indicator of the cost efficiency at which work is being performed.
  • TCPI = (BAC-BCWP) / (EAC-ACWP): The cost efficiency that would have to be attained in order to achieve the EAC value being used in the formula.

According the DCMA publication “EVMS Program Analysis Pamphlet” (DCMA-EA PAM 200.1, July 2012), a mathematical difference of 0.10 or greater is used as an early warning indication that the contractor’s forecasted completion cost could possibly be unrealistic, stale, or not updated recently.  This is generally measured at all levels of the project, but is often the focus at the control account level.

Having a difference greater than 0.10 does not mean the EAC must be adjusted.  Often the TCPI formula will give unusually high or low values when the account being measured is at the beginning or end of its progress.  There may be a very good reason for the forecasted efficiency of an Estimate to Complete (ETC) to be out-of-line with its historical efficiency.  But often surveillance reveals a large number of accounts that are out-of-line by this measurement, indicating a lack of discipline in an organization’s EAC process.

Most Common Corrective Action Plans:

The first action should be to ensure that the Earned Value Management System stipulates an update to the ETC on at least a monthly basis, and the Control Account Managers (CAMs) are held accountable for their EACs’ realism.  This is often accomplished through training, updating processes, and the development of easy to use tool sets.  The organization could also require the CAM to justify any control account’s ETC that is out of tolerance with an EAC realism check.

Care must be taken to ensure that the EACs are not being set just to avoid tripping this metric.  It is easy to “calculate” an EAC by dividing the BAC by the CPI; however, this does not meet the intent of the guideline requirements.

2) Poor Variance Analysis and Reporting (VAR)

This is probably the most common Corrective Action cited by reviewers performing EVMS surveillance.  Because the variance analysis relies completely on the discipline of the CAMs and their support staffs, the process often atrophies for various reasons. Within the variance analysis report (VAR) process, probably the most troublesome is the identification and tracking of Corrective Actions to their logical completion.  A well written variance analysis will explain the root cause, impact and corrective action associated with a cost (current period, cumulative, at completion) or a schedule (current period, cumulative) variance.  Cost variance explanations should provide a breakdown of the rate versus hours for labor or the price versus usage for material.  Schedule variance explanations should focus on identifying the effort that is not being accomplished per the plan, and forecast the Estimated Completion Date (ECD) for when the schedule variance will go to zero.  Most important in all of this process is the identification and tracking of the Corrective Actions that are being taken to either mitigate the variance or at least ensure that the variance does not worsen.

Reviewers of VARs often find that there is a lack of clear and concise treatment of the problem, and that many writers of VARs simply reiterate the variance as indicated by the data, such as “The cost overrun is because more hours were expended than originally planned”.  Many VARs will point a finger back to the system that generated the data.  But the worst offender is the lack of a corrective action plan that includes mitigations steps that address the issue.

Most Common Corrective Action Plans

Poor variance analysis is primarily a discipline issue with the CAMs and their support staffs. Like most discipline issues, this can be addressed by training, adding structure, and incorporating reviews of the VAR process.  The writing of VARs is one area where annual refresher training may be needed in order to verify that the concepts are still being practiced.  This training should involve the managers in the organizational chain above the CAMs who are responsible for reviewing and approving the variance analysis reports.

It can also be beneficial to add a structured approach to the variance explanation page.  This should include required inputs on Root Cause, Impact, and Corrective Action, and may include a link to a corrective action tracking system that allows the user to create and assign mitigation steps plus provide status of previously identified corrective actions.  Some organizations have also introduced the scoring of VARs using a scoring template as a feedback mechanism for those writing the reports.

3) Lack of Effective Subcontract Management

The Interim DoD Instruction for 5000.02, “Operation of the Defense Acquisition System” (November 25, 2013), reiterates the requirements for earned value application on all cost/incentive contracts greater than $20M including subcontracts.  In the DCMA Instruction 1201, “Corrective Action Process” (September 23, 2013), the importance of subcontract management is stressed with the following:

“Prime contractors have wide latitude as to how they control their supply chain and are ultimately responsible for flow down and execution of contract requirements. When DCMA discovers a noncompliance requiring a Level I or II CAR at a subcontract level, the appropriate CAR (Level I or II) shall be issued directly to the subcontractor with notification to the prime contractor via the prime CMO. The notification to the prime contractor shall be redacted as needed to prevent disclosure of subcontractor proprietary information. In situations where a noncompliance(s) at the subcontract level meets the criteria for a Level III CAR, the CAR shall be issued to the prime contractor.”

Over the past few years, the lack of effective and consistent management of subcontractors with an EVM flow down requirement has been the source of numerous discrepancy reports (DRs), and has been cited as a significant attributor to EVMS disapprovals/de-validations.  As noted in the language above, prime contractors are not only responsible for including the correct clauses in subcontracts but also have responsibility for the execution of the contract requirements.  When a subcontractor does not follow its own system description, is not in compliance with the EIA-748 Guidelines, or submits reports that do not meet the requirements of the contractual Data Item Description (DID), it is the responsibility of the prime to enforce compliance.

The reasons for ineffective subcontract management are many, but in some organizations enforcement of contractual requirements is the domain of a subcontract administrator/buyer who may not have a firm understanding of the EVMS requirements.  Many subcontractors also often resist the surveillance attempts by their prime, or attempt to reject any efforts by their prime to enforce the execution of these types of contractual requirements.

Most Common Corrective Action Plans

The first step in corrective action is a thorough understanding of the contractual requirements and language in the subcontracts.  H&A has found numerous examples of incorrect contractual language, or even language implying that the subcontractor reports are for “information only”, thus diminishing the ability of the prime to enforce compliance.  The contract should include: 1) A requirement for compliance with the EVM system guidelines in EIA-748, preferably by including the appropriate DFARs clauses (DFARs 252.234-7001 and 7002); 2) the appropriate DID, currently DI-MGMT-81861, referenced in the Contract Data Requirements List (CDRL) with any special tailoring or reporting guidance documented (for contracts issued prior to June, 2012, the Contract Performance Report DID is DI-MGMT-81466A and the IMS DID is DI-MGMT-81650); 3) language which establishes the right of the prime contractor to conduct reviews, such as Integrated Baseline Reviews (IBRs) and recurring system surveillance.

The prime contractor must then establish a disciplined surveillance program that includes review of all 32 EIA-748 Guidelines on an annual basis at each subcontractor’s facility.  This process should also include the procedure for issuing and tracking corrective actions.  These surveillance reviews are often conducted jointly with the cognizant DCMA/DCAA for each subcontractor.

And finally the prime must review all reports delivered by the subcontractor for compliance with the appropriate DID, and be prepared to take action when the submitted reports are noncompliant with the CDRL and/or DID Instructions.  This action can take the form of a subcontract administrator’s letter requiring adjustments in future reports, rejection and resubmittal of non-compliant reports, or rejection and contractual remedies; such as payment withholds or Award Fee impact, as applicable.

Please contact Humphreys & Associates if you have any questions on this article.

April 25th, 2014 – Part 2:  Common Problems found in EVMS | Poor use of Percent Complete; Data Integrity Issues; Poor Scope Language.

Common Problems Found in EVM Systems – Recommended Corrective Actions – Part 1 Read Post »

Factoring Subcontractor Data

By /

Factoring Subcontractor Data by Humphrey's & AssociatesIf you are a prime contractor Control Account Manager (CAM), how do you plan Budgeted Cost for Work Scheduled (BCWS) and claim Budgeted Cost for Work Performed (BCWP) when the negotiated subcontract value you are assigned to manage differs from your control account budget?

First, let’s understand the facts. You are assigned to manage a deliverable hardware subsystem reporting element in the contract work breakdown structure that will also be reported in the Integrated Program Management Report (IPMR).  The history of your subcontract is:

  • The subcontractor proposed a total price of $110M (Cost $102M and Fixed Fee $8M).
  • For various reasons, your program manager believed that the final subcontract could be negotiated for 18% less than the proposed price, so your authorized budget is $90M (including fee).
  • You placed the authorized budget in a planning package until the completion of subcontract negotiations.
  • The subcontractor will report its IPMR data at the total price level (including fee).
  • The subcontract was negotiated for a total price of $100M (Cost $95M and Fixed Fee $5M).
  • You requested that the program manager make up the budget difference from Management Reserve (MR), but the program manager declined.

Factoring the subcontractor’s data is the best approach in this situation.  Factoring the subcontractor’s data simply means applying a consistent multiplier to the subcontractor’s budget values: Budgeted Cost for Work Scheduled (BCWS) and Budget at Completion (BAC).  The multiplier is developed by dividing the available control account budget by the subcontract price.  In the example, the multiplier is .9 and was derived by dividing the CAM’s budget of $90M by the total subcontract value for the hardware system of $100M.  This multiplier is applied to the time-phased budget provided by the subcontractor as shown in the table below.  This factored budget becomes the prime contractor’s control account budget.

Factoring Subcontractor Data Budget (Example)

The calculation of the prime contractor’s earned value, also known as the Budgeted Cost for Work Performed (BCWP), simply requires applying the same factor to the subcontractor’s cumulative BCWP each month as shown in the table below.  In the example, for the month of March, cumulative subcontractor BCWP of 32 multiplied by the factor of 0.9 yields the prime contractor’s factored BCWP of 29. When calculating the BCWP, the value in the prime contractor’s Earned Value Management System must reflect the same percent complete (BCWP/BAC x 100) as the subcontractor’s reported data; this is illustrated by the highlighted cells in the data example.

Factoring Subcontractor Data - BCWP

Note that factoring does not apply to Actual Cost of Work Performed (ACWP), the Estimate to Complete (ETC) or the Estimate at Completion (EAC) because those values represent actual costs rather than budgeted amounts that must reconcile with the subcontract Target Cost.

Factoring can occur whether the prime contractor’s budgeted amount for the subcontract effort is either greater or less than the subcontract negotiated price.  Another instance where factoring is appropriate is when the subcontractor provides IPMR data without fee.  The subcontractor’s fee is a cost to the prime contractor and should be included in the prime contractor’s Performance Measurement Baseline (PMB), so factoring is an appropriate technique in this situation.

In summary, remember the following when factoring subcontractor data:

  • Factoring ensures that the subcontract factored BCWS equals the prime contractor’s budget.
  • The prime contractor’s factored BCWP must yield a percent complete consistent with the subcontractor’s percent complete.
  • Factoring does not apply to the ACWP, ETC or the EAC.

Thank you for reading our blog. You can also sign up for our EVMS Newsletter. Give Humphrey’s & Associates a call with questions or to inquire about our classes, certifications and services.

 

Factoring Subcontractor Data Read Post »

Earned Value: Fun with Numbers or Real Management Data – Answers (part 2)

By /

This is the second in a two part article on Earned Value: Fun with Numbers Part 1 / Answers

Answers: Earned Value terms used in the context of this article

Earned Value: Fun with numbers pt 2

Management Reserve

Correct Answer: N

  • In the Earned Value Management System (EVMS) vernacular, Management Reserve (MR) is the budget set aside for Known Unknowns – not Unknown Unknowns. This distinction is important, as the budget at complete (BAC) plus MR equals the Contract Budget Base (CBB).
  • Management reserve is typically used when an identified risk is realized (Known Unknowns). Once a risk has become a reality, such as re-work, re-test, re-make, more lines of software required, etc., the newly identified work required to satisfy the existing contract SOW must be scheduled and resource loaded (BCWS). This additional budget must be distributed to a CAM (or CAMs) via a work authorization document. The source for this budget is MR and not UB, as UB must have previously been logged with a predetermined budget and associated SOW.
  • While the Unknown Unknowns could be estimated using simulations, models, etc., the project does not have the luxury to have such a budget set aside initially for an Unknown Unknown occurrence. If an Unknown Unknown becomes a Known, thus the newly identified risk becomes reality, the contractor could use the existing MR to budget this newly identified task or tasks to satisfy the contract requirement or prepare a change proposal for their external customer.
  • When there is no MR, the contractor could implement an Over Target Baseline (OTB) in the event newly identified risks are realized with prior customer approval, as there is usually insufficient MR remaining.

Schedule Variance

Correct Answer: G

  • Schedule Variance (SV) = BCWP – BCWS

–      The resulting answer will identify the schedule position on the program, a negative answer indicates a behind schedule condition some or all of the program tasks. A positive result indicates an ahead of schedule condition for some or all of the program tasks.
–      Always use this information to supplement the Program Schedule tools data which uses actual days/week/months to identify the programs schedule position.

Budget At Completion

Correct Answer: D

  • The total contract value of all the time phased planned work

Contract Budget Base/Contract Target Cost

Correct Answer:M

  • The total contract value of all the time phased planned work, plus Management Reserve

Budgeted Cost for Work Scheduled

Correct Answer:A

  • The Budgeted Cost for Work Scheduled, or BCWS, is the time phased value expressed in hours and/or dollars for all of the authorized budgeted work scheduled to be accomplished on a program. This is the time phased baseline from which all work performed is measured.

Schedule Slip

Correct Answer: J

  • The amount of time/duration that the planned completion dates have been missed by, the total number of days/weeks/months that an activity is behind to the original baseline dates.

 Variance At Completion

Correct Answer: I

  • Variance at Completion (VAC) is the difference between the Budget at Completion (BAC) and the Estimate at Completion (EAC).  VAC can be calculated at any level from the control account up to the total contract. It represents the amount of expected overrun (negative VAC) or underrun (positive VAC)
  • The VAC is computed by subtracting the EAC from the BAC. A negative result is unfavorable indicates the tasks being measured are forecasting an overrun at completion. A positive result is favorable indicates the tasks being measured are forecasting an underrun at completion.

Estimate At Completion

Correct Answer: F

  • The Earned Value Guidelines define the EAC as the sum of the contracts cumulative to date Actual Cost of Work Performed (ACWP) plus the company project manager’s best estimate of the time-phased resources (funds) required to complete the remaining authorized work, the Estimate to Complete (ETC).  This relationship is often expressed by the formula EAC = ACWP + ETC.  Thus, the EAC is a forecast of the project’s final cost.  The project manager may revise work priorities, replan remaining tasks on the project schedule and/or adjust the technical approach to complete the project’s goals within the estimated remaining resources.  The goal is to complete all of the contract work scope within the Contract Budget Base–CBB(cost) and Contract Completion Date–CCD (schedule).

Actual Cost of Work Performed

Correct Answer: B

  • The costs actually incurred and recorded in accomplishing the work performed on the program. The costs include Labor, Material/ Sub-Contracts, Other Direct Costs (ODC) and the associated Indirect costs applied to each category.

Estimate To Complete

Correct Answer: E

  • The Estimate to Complete (ETC) is the company project manager’s best estimate of the time-phased resources (funds) required to complete the remaining authorized work.

Cost Variance

Correct Answer: H

  • Cost variance (CV) which is calculated as BCWP minus ACWP.  A result greater than 0 is favorable (an underrun), a result less than 0 is unfavorable (an overrun).

Program Overrun

Correct Answer: O

  • The difference between the total Budget and the Estimated Costs at Completion, a negative number indicates an overrun to the program. (A positive number indicates an underrun to the program)

Time Now

Correct Answer: L

  • Time Now is the end of the current (and cumulative to date) period that the data is being measured against.

Budgeted Cost for Work Performed

Correct Answer: C

  • The Budgeted Cost for Work Performed (BCWP) is the value of work completed based on the value of the BCWS assigned to that work. This is equal to the sum of the budgets for completed work

Forecasted Program Schedule Slip

Correct Answer: K

  • The total number of days/months that the program is estimating the completion date of all authorized efforts will exceed the planned Contract Completion Date (CCD). Compare the CCD date to the Estimated Completion Date (ECD)

Estimated Completion Date

Correct Answer: P

  • The estimated date that all authorized efforts will be completed.

Discussion of the displayed data

The Program began in January, the Time Now (L) is June or approximately 6 months into the effort. There are unfavorable Schedule (G) and Cost (H) Variances. The program was planned (BCWS-A) to complete in March. The Current Estimated Completion Date–ECD (P) is June which indicates a 3 month Forecasted Program Schedule Slip (K). The Budget At Completion (D) Was about $39M (including Management Reserve –MR (N), the Estimate At Completion–EAC (F) is about $56M resulting in a Program Overrun (O) of  $17M.

As you can see, selecting key measurement data metrics empowers the program manager with the information to estimate the impacts of early trends in the program. Using the data from analytical tools such as “EMPOWER” allows the program manager to evaluate the current status and develop corrective action and mitigation plans to help minimize the impacts to the overall contract and keep all customers aware of possible outcomes.

We hope you found this two part article on Earned Value: Fun with Numbers useful. Feel free to share or call Humphrey’s & Associates for more information. 

Earned Value: Fun with Numbers or Real Management Data – Answers (part 2) Read Post »

Scroll to Top