Timely Subcontractor Data – Mission Impossible?

By / / , , ,
, , ,
Timely IPMDAR Subcontractor Data – Mission Impossible?

With the arrival of the Integrated Program Management Data and Analysis Report (IPMDAR) requirements for electronic cost and schedule dataset submittals, DoD contractors with EVMS or EVM reporting contractual requirements have a tighter time frame for submitting their month end data. A previous blog, Introduction to the IPMDAR Data Deliverable – Tips for Producing the Outputs summarizes these data reporting requirements. This includes the Contract Performance Dataset (CPD) for the time phased cost data and the Schedule Performance Dataset (SPD) along with a native file export out a schedule tool.

For month end data submittals, the IPMDAR Data Item Description (DID), DI-MGMT-81861C (20210830) states:

1.8.1 Monthly Submission Requirement. IPMDAR data shall be required at least monthly. The reporting frequency shall be specified in the Contract Data Requirements List (CDRL). All reports shall reflect data from the same accounting period and shall be provided at any time after the close of the contractor’s accounting period, but no later than sixteen (16) business days after the contractor’s accounting period end date.

On the surface, you might say requiring data delivery 16 business days after the contractor’s accounting period end date doesn’t sound unreasonable or even much different from the previous Integrated Program Management Report (IPMR) DID (DI-MGMT-81861A), and you would be right.

What is the issue?

Some people think that because the IPMDAR submittals are electronic datasets instead of report formats it is easier to generate and report that information. That is not necessarily true, and shortening the data turnaround time exacerbates the problem. The tighter time requirements also apply when there are EVM reporting subcontractors providing performance data to a prime contractor.

The third sentence of 1.8.1 above states: “All reports shall reflect data from the same accounting period…” This requirement is very challenging, especially when a subcontractor operates on a different month end accounting calendar; for example, a “5-4-4” versus the prime’s “4-4-5” calendar. Even when the prime and subcontractor are on the same month end calendar, for the prime to submit IPMDAR data in 16 business days, the subcontractor has less time to provide their data to the prime. It becomes even more challenging on very large programs that have several tiers of subcontractors.

Subcontractors cry “foul” because they don’t have enough time to get all the performance data ready in the reduced time. Prime contractors cry “foul” because they are held accountable for data that may or may not come from one or more tiers of subcontractors in time for them to conduct basic data analysis and deliver month end data for the IPMDAR. The government customer still insists all the data must be for the same accounting month end date, even though that may not be well defined. Customers also do not want the subcontractor data delayed by a month just to get the subcontractor data “caught up” – i.e., “comparing apples to oranges.”

Is incremental delivery of IPMDAR the answer?

The government suggests that incremental delivery could resolve this dilemma. The DoD IPMDAR Implementation and Tailoring Guide (August 24, 2021) expands on the paragraph from the IPMDAR DID:

1.8.1.1 Incremental Delivery. Reports may be provided incrementally, including preliminary data, with the number of days for delivery of each submittal tailored in the CDRL. Data delivered is not considered authoritative until the final submission and signature. The recommended incremental delivery process is the Schedule, followed by the CPD and the Executive Summary, Government review of submittals, Government directed Detailed Analysis, Contractor Detailed Analysis delivery and all final data.

The IPMDAR Implementation and Tailoring Guide also provides a notional example of how an incremental delivery could be handled:

1. SPD – To be delivered with native file five (5) working days after the end of the contractor’s accounting period (may be labeled preliminary)

2. CPD – To be delivered with the Executive Summary ten (10) working days after the end of the contractor’s accounting period (may be labeled preliminary)

3. Contracting Office to select items for detailed analysis (variances) – to contractor thirteen (13) working days after the end of the contractor’s accounting period

4. Performance Narrative Analysis – to be delivered NLT sixteen (16) working days after the end of the contractor’s accounting period along with any other “final” versions of previously submitted files

Note: The notional incremental delivery plan above is not additive.

Doing the above might demonstrate to the government customer that the prime contractor is at least trying their best to make the prime/subcontract situation work – even though they would be using “estimated data” until the final versions come in from the subcontractors. Does this approach really make the timely delivery of the data easier to attain? The bottom line does not change. Per number 4 above, the prime still has to deliver all the data in “final versions” by the 16th business day following the close of their accounting calendar. The note at the bottom specifies the days indicated in each step are not “additive” – i.e., the contractor does not get 5+10+13+16 = 44 business days.

In some circumstances, incremental delivery might allow some subcontractors a bit more time to get the data to the prime contractor, but there would still have to be tighter delivery dates for the incremental deliveries, so the problem does not really go away.

What are your options?

This difficult situation arises because few contractors consider the implications of having to get all data by their accounting month end. Not all the subcontractor work elements are set up the same way. Contractors who have the EVM reporting requirement, who do or will have EVM reporting subcontractors, should address this basic difference as part of the contract negotiation process. One possible part of this negotiation could be to use the IPMDAR DID, paragraph 1.4, to help level the field for reporting purposes. This paragraph states:

1.4 Direct Reporting Contractor Role.

1.4.1 A Direct Reporting Contractor is any contractor required to provide the IPMDAR directly to the Government. This includes prime contractors, subcontractors, intra-government work agreements, and other agreements, based on the contract type, value, duration, nature of the work scope, and the criticality of the information. In this document, instances of “Contractor” are synonymous with “Direct Reporting Contractor.”

There is a footnote to this paragraph that states:

In the event that the Direct Reporting Contractor is a contractor other than the prime, the Direct Reporting Contractor will additionally report to the prime. Subcontractor data shall be provided to the prime in a manner that supports the contractor’s submission to the Government.

One solution is to negotiate to have each EVM reporting subcontractor deemed a “Direct Reporting Contractor” that submits their data directly to the government, including the customer, as well as to the prime contractor. The prime and subcontractors are each submitting their IPMDAR electronic deliverables to the DoD EVM Central Repository (EVM-CR).

Each level of contract, the prime through however many tiers of subcontractors there may be, will have the same 16 business days after their own accounting month end dates to provide all interested parties with the EVM data. The prime contractor still must at least get estimated subcontractor data to do their monthly assessment, making corrections in the next month after they have received the final version of the data from their subcontractors.

Should the government customer want analysis performed on subcontracted effort, the IPMDAR dataset submittals will be in the DoD EVM-CR. They can do that analysis independently of the prime contractor’s analysis that would be provided after the prime’s 16th business day.

This approach would put pressure on the prime because they will not have seen the subcontractor’s data prior to it being delivered to the government, but that could be addressed in the next reporting period’s “errata” variance analysis narrative. The government would also have the detailed data from the subcontractors when the subcontractor is providing a reduced set of data such as only total cost data to the prime. Should the government customer not want to do that level of analysis, the government customer may need a different contracting solution to avoid requiring EVM reporting down through various levels of contracts. This is often determined by the contract value and risk factors associated with the subcontractor. 

This approach also requires the subcontractor to produce two deliverables. One for the prime contractor in an agreed upon format and one for the government customer following the IPMDAR DID electronic submittal requirements for the DoD EVM-CR. These reporting requirements should be negotiated with the subcontractor well in advance; the subcontractor needs to know their data deliverable and reporting requirements when they bid on the work effort for the prime.

This subcontractor data incorporation issue has been around for many years and can be very confusing. H&A earned value consultants can help you work through the various responses to this requirement in the best possible way for your situation. Call us today at (714) 685-1730 to get started.

Timely Subcontractor Data – Mission Impossible? Read Post »

Navigating EVMS Certification: A Step-by-Step Guide to Compliance

By / / ,
,
Navigating EVMS Certification

In the complex landscape of project management, ensuring compliance with the EIA-748 Standard for Earned Value Management Systems (EVMS) Guidelines is a critical step for companies seeking to secure and successfully manage government contracts. Humphreys & Associates are leaders in earned value consulting, providing comprehensive strategies and solutions tailored to meet the rigorous requirements of the EVMS approval or certification process by a Cognizant Federal Agency (CFA). This article is the first part of a three-part series aimed at guiding organizations through the process of implementing a compliant EVMS and successfully completing a CFA EVMS compliance review.

Understanding the EVMS Approval or Certification Process

Achieving EVMS approval or certification by a CFA such as the Defense Contract Management Agency (DCMA) or the Department of Energy (DOE) is a structured process that requires careful planning and execution. The process begins with a thorough understanding of the EIA-748 Standard for EVMS Guidelines, which is the foundation for determining whether an EVMS is compliant. The EIA-748 Guidelines define the requirements to establish and maintain an effective EVMS. The approval or certification process involves several key steps, beginning with the initial application and concluding with the CFA formal determination a contractor’s EVMS complies with the Guidelines. Throughout this journey, organizations must demonstrate a thorough understanding of the Guideline requirements and how they are implemented within their project management framework as documented in their EVM System Description.

Steps to Achieve EVMS Certification

  1. Preparation and Self-Assessment: Conducting an internal review of the current project management processes and comparing them against the EIA-748 Standard for EVMS Guideline requirements is an important first step to identify gaps in the system that will need to be addressed.
  2. Training and Education: It is crucial for the team responsible for EVMS implementation to receive proper training to understand the Guideline requirements and how to apply them.
  3. System Description Development: A comprehensive EVM System Description that explains how the organization’s processes meet the EIA-748 Guideline requirements must be developed.
  4. Implementation: The EVMS must be implemented on a project, demonstrating the project team’s ability to use the EVMS and EVM data to manage the project, as well as the system’s functionality in a real-world scenario.
  5. Mock Compliance Review: An internal review, often with the help of an independent third party, should be conducted to verify the EVMS complies with the Guideline requirements as well as to verify the quality of the project’s schedule and cost data to provide timely and actionable information for managing the project.
  6. Formal Compliance Review: A CFA conducts a formal review of the EVMS. This includes reviewing the EVM System Description, performing a detailed examination of project schedule and cost data, conducting interviews with project personnel, and assessing how the EVMS has been implemented.

Importance of Complying with the EIA-748 Guidelines

Meeting the EIA-748 Guideline requirements is not just about compliance; it is about integrating a system that enhances the project management capabilities of an organization. A well-implemented EVMS facilitates better project control, provides early warning signs of performance issues, and supports informed decision-making. Compliance with the Guidelines is often a prerequisite for bidding on government contracts, making a formal EVMS approval or certification a strategic necessity for companies in the defense, aerospace, and construction industries, among others.

Key Requirements for an EIA-748 Compliant EVMS

The EIA-748 Standard for EVMS provides the basic guideline requirements for organizations to establish and implement a system that integrates project work scope with the schedule and cost components to enhance project planning and control. The Guidelines are organized into these process areas:

  • Organization: The organization guidelines focus on establishing the framework for decomposing a project’s scope of work to level where it is possible to identify management responsibility for the work scope, schedule, and cost components.
  • Planning, Scheduling, and Budgeting: The organization guidelines are the framework for the planning, scheduling, and budgeting process required to establish the performance measurement baseline, a common point of reference for measuring completed work and communicating the project’s current status.
  • Accounting Considerations: These guidelines focus on the recurring processes for determining progress and collecting the actual costs for work performed. The objective is to ensure alignment between the budget plan, performance claimed (earned value), and actual costs to continually assess and analyze project performance for potential corrective action.
  • Analysis and Management Reports: These guidelines facilitate the analysis and use of the performance data to proactively manage the project. An EVMS generates variance data that helps management to focus on areas that are not performing to plan for potential corrective action.
  • Revisions and Data Maintenance: The integrity of the performance measurement baseline must be maintained to manage the remaining work on a project. These guidelines focus on establishing a controlled process to document, authorize, track, and manage revisions to a project’s scope, schedule, and budget.

Frequency and Importance of Surveillance Reviews

Surveillance reviews are an important part of maintaining a compliant EVMS. Once the CFA has approved or certified a contractor’s EVMS, the contractor is responsible for implementing an annual self-surveillance or self-governance process to ensure the EVMS continues to be implemented on projects in an effective and consistent manner in compliance with the EIA-748 Guidelines. It also ensures the contractor’s EVMS process and procedures, training, and tools are actively maintained over time. The government customer also conducts surveillance for the life a contract, typically on an annual basis. This is a recurring evaluation of the contractor’s management control practices and samples of internal and external reported data. The focus is typically on major system activities, problem identification, and tracking any corrective actions to closure.

The successful implementation of an EVMS that complies with the EIA-748 Guideline requirements is a testament to an organization’s commitment to project excellence. It demonstrates a capability many government agencies require, making it an essential investment for businesses looking to expand their opportunities within this sector. The subsequent articles in this series will delve into the specifics of the EVMS implementation process and the intricacies of preparing for EVMS compliance and surveillance reviews.

Stay tuned to learn more about effective EVMS implementation for government contracts in our next blog, “Effective EVMS Implementation for Government Contracts: Roles and Challenges,” and how to prepare for surveillance or compliance reviews in “Preparing for EVMS Reviews: Strategies for Success with Humphreys & Associates.”

This article has provided an overview of the EVMS approval or certification process by a CFA and the importance of complying with EIA-748 Standard for EVMS Guidelines. 

Navigating EVMS Certification: A Step-by-Step Guide to Compliance Read Post »

Preparing for EVMS Reviews: Strategies for Success with Humphreys & Associates 

By / /
Preparing for EVMS Reviews

Facing an Earned Value Management System (EVMS) compliance review can be an imposing prospect for any organization with EVMS contractual requirements. With strategic preparation and expert guidance from Humphreys & Associates, renowned for their leadership in earned value consulting, this necessity can be transformed into an opportunity for process improvement to ensure an effective and efficient EVMS can be implemented on any project. This comprehensive guide, the first in a three-part series, takes a closer look at the strategic approach necessary for preparing for a Cognizant Federal Agency (CFA) EVMS compliance review. CFAs include the Defense Contract Management Agency (DCMA) for DoD and NASA contracts and the Department of Energy (DOE). A CFA conducts a formal review to determine whether the contractor’s EVMS is compliant with the EIA-748 Standard for EVMS Guidelines. A formally approved or certified EVMS is an EVMS the CFA has determined to be compliant with the EIA-748 Guidelines.

Key Elements in Preparing for an EVMS Compliance Review

A successful EVMS compliance review is predicated on several fundamental elements. Being well-prepared in these areas can make the difference between a review that goes smoothly and one that uncovers issues that require corrective action.

  • Thorough Documentation: Proper documentation is the backbone of any review process. It’s essential to maintain an EVM System Description that explains how the system complies with the EIA-748 Guideline requirements as well as any agency specific EVMS requirements. This includes documented policies and procedures as well as process flowcharts illustrating system inputs and outputs with responsibility assignments.
  • Mock Reviews: Conducting internal mock reviews can serve as a rehearsal for the actual review. These practice runs help to uncover any weaknesses in the system and provide a chance to correct them beforehand. Mock reviews also help familiarize the team with the compliance review process, reducing anxiety and ensuring that everyone knows what to expect.
  • Continuous Training: Regular and comprehensive training ensures that project personnel are familiar with the EVMS processes, know how to use the EVM data, and understand the importance of their role in maintaining quality schedule and cost data. This training should be updated regularly to reflect any changes in EVMS requirements, software tools, or company procedures.
  • Data Integrity: The accuracy and completeness of project data are crucial. Regular validation checks should be conducted to ensure data in the EVMS are valid, reliable, and traceable. This includes verifying the project data aligns with the corporate financial records and that the system accurately reflects the project’s current status.
  • Stakeholder Engagement: Effective reviews require the cooperation and understanding of all stakeholders. Engaging them early in the review process helps ensure everyone is on the same page and that the roles and responsibilities are clearly defined. This engagement includes regular communication and involvement in the compliance review preparation process.

Assistance in the Review Preparation Process

Humphreys & Associates provides comprehensive services designed to support organizations throughout the compliance review preparation process. This includes:

  • Compliance Review Readiness Assessments: These assessments are crucial in determining the readiness of an organization’s system, personnel, and data quality for a compliance review. The assessment identifies areas of strength and those requiring improvement, enabling targeted action to enhance readiness.
  • Preparation Workshops: Workshops conducted by experienced consultants can train and prepare the team for the compliance. These sessions cover everything from the basics of EVM to the nuances of the review process, tailored to the specific needs of the organization.
  • Documentation Review: Prior to an compliance review, it’s beneficial to have an external review of the documentation that will be presented. This review can identify areas where additional information is needed and ensure that the documentation accurately reflects the EVMS and compliance with the EIA-748 Guidelines.
  • Support Services: During the review, having expert support available can alleviate the challenges that may arise. This includes on-the-spot advice and assistance in addressing the CFA’s questions and concerns. Should the CFA issue formal corrective action requests (CARs), Humphreys & Associates can assist with producing and implementing the corrective action plans (CAPs) to resolve the deficiencies as quickly as possible.

Common EVMS Review Findings and Remediation

There are several common findings that an EVMS compliance review might reveal that could potentially impact an organization’s ability to obtain a formal CFA EVMS compliance approval or certification. Being aware of these potential findings and understanding how to address them is key to success.

  • Inadequate Baseline Control: Maintaining a current and accurate performance measurement baseline is essential. When a government review team finds baseline control to be lacking, it’s often due to inadequate processes for incorporating changes into the baseline or failing to maintain traceability of the changes.
  • Insufficient Documentation: Documentation gaps can lead to findings of non-compliance. Government review teams need to see evidence that a complete set of processes are in place and that these processes are being followed. This includes having proper version control and evidence of management approvals.
  • Data Discrepancies: Differences between what’s reported in the project reports and the actual project data can indicate serious issues with data management systems. Ensuring that the EVMS software is properly configured, data validation checks are routinely performed, and that schedule and cost data align can prevent such discrepancies.

To address and prevent these common issues, Humphreys & Associates recommends a proactive stance, with robust change control processes, continuous enhancement of documented practices, and improvements to data management systems to ensure accuracy and traceability.

Preparing for an EVMS compliance review is a critical task that can significantly impact the management and success of government contracts. With the strategies provided here and the support of Humphreys & Associates, organizations can confidently navigate the EVMS compliance review process.

Explore the nuances of the implementation phase in “Effective EVMS Implementation for Government Contracts: Roles and Challenges,” and enhance your understanding of the certification process in “Navigating EVMS Certification: A Step-by-Step Guide to Compliance.”

Preparing for EVMS Reviews: Strategies for Success with Humphreys & Associates  Read Post »

Effective EVMS Implementation for Government Contracts: Roles and Challenges

By / / , ,
, ,
Effective EVMS Implementation for Government Contracts

Embarking on implementing an Earned Value Management System (EVMS) for government contracts is a complex task. With the expert guidance of Humphreys & Associates, leaders in earned value consulting, organizations can effectively navigate this intricate process. This detailed exploration, the second in a three-part series, delves into the best practices, key roles, and common challenges of implementing an EVMS on government projects.

Best Practices for Implementing an EVMS on Government Projects

The implementation of an EVMS for a government project demands team work and strategic planning. Following best practices facilitates the EVMS compliance process and enhances overall project performance.

  1. Begin with a Clear Strategy: Starting with a clear strategy requires a full understanding of the scope of contract requirements and aligning project management processes with the EIA-748 Standard for EVMS Guidelines. This means establishing clear technical, schedule, and cost objectives as well as establishing a performance measurement baseline that are in harmony with contractual obligations.
  2. Use a Phased Approach: A phased approach to EVMS implementation allows for better management of resources and more focused attention on each aspect of the system. By prioritizing critical areas, such as organizing and decomposing the entire contractual scope of work into manageable product-oriented elements that can assigned to responsible managers, organizations can ensure that the foundational elements of an EVMS are solid before expanding to other areas.
  3. Involve All Stakeholders Early: The early involvement of all stakeholders, including project managers, project controls team, finance, procurement, and even suppliers, ensures that everyone understands the EVMS requirements and their role in the implementation. This early buy-in helps to streamline the integration of EVMS into existing processes and encourages collaborative problem-solving.
  4. Ensure Adequate Training: Comprehensive training programs are essential to equip all team members with the necessary knowledge of EVMS principles as well as how to use the schedule and cost tools that support the EVMS. This training should be tailored to the various roles within the team and include practical exercises that reflect the challenges they will face during implementation.
  5. Focus on Data Quality: High-quality data is the cornerstone of an effective EVMS. Ensuring accuracy, timeliness, and reliability of project data involves setting up rigorous data collection and processing systems, continuous data verification, and validation processes.
  6. Continuously Improve: Continuous improvement involves regularly reviewing and refining the EVMS processes based on project performance, self-governance feedback, and lessons learned. It’s about fostering a culture of constant enhancement to adapt to project changes and industry advancements.

Who’s Who in EVMS Development and Deployment

A successful EVMS implementation relies on the collaborative efforts of a dedicated team, each member plays a critical role.

  • Project Managers: They are the linchpins in ensuring that the EVMS is implemented as intended on the project. They coordinate between different teams, manage resources, and ensure that project contractual objectives are being met.
  • Control Account Managers (CAMs): CAMs have a focused role in managing specific project segments of work. They are responsible for the scope of the work, schedule, and budget for their control accounts that are decomposed into detail work packages or planning packages.  They are critical in ensuring work completion, developing and implementing corrective actions when needed, and providing accurate work status information.
  • EVMS Analysts: These specialists monitor and analyze project performance against the established baselines. They provide forecasts, identify variances, and offer insights that guide decision-making and corrective actions.
  • Finance Personnel: The finance team ensures the integrity of project accounting and its alignment with EVMS requirements. They are responsible for cost recording, allocation, and reporting, playing a vital role in the financial aspect of project control.

Common Challenges and Solutions for an EVMS Implementation

While implementing an EVMS, organizations may encounter several challenges, but with the right strategies, these can be mitigated.

  • Promote Organizational Buy-In: Resistance to change is a common barrier. Overcoming this requires demonstrating the value of the EVMS to the stakeholders, highlighting its benefits in terms of improved project visibility and control. Engaging team members by making them part of the implementation process can foster acceptance and support.
  • Maintain Data Integrity: To ensure the accuracy and completeness of data, it’s imperative to establish standard data governance practices. This includes documenting best practices to help project personnel develop and maintain quality schedule and cost data, performing regular data quality assessments, and conducting continuous training.
  • Seek Expert Advice: Leveraging the knowledge and expertise of EVMS consultants like Humphreys & Associates can be invaluable. Consultants can offer guidance, best practices, and training that are tailored to the organization’s specific needs and challenges.

The implementation of an EVMS is a critical step towards achieving project success, especially in the highly regulated government contracting environment. The insights provided here, coupled with the expertise of Humphreys & Associates, can help organizations to navigate the EVMS implementation landscape effectively.

For a deeper dive into this topic, read our full article, “Navigating EVMS Certification: A Step-by-Step Guide to Compliance.” And stay tuned for the final installment in this series, where we will explore strategies for preparing for compliance or surveillance reviews in “Preparing for EVMS Reviews: Strategies for Success with Humphreys & Associates.”

Effective EVMS Implementation for Government Contracts: Roles and Challenges Read Post »

Incorporating IMS Information Directly into Independent Estimate at Completion (IEAC) Formulas

By / / , , ,
, , ,
Incorporating IMS Information Directly into Independent Estimate at Completion (IEAC) Formulas

“When you need to discuss the schedule, look at the schedule.”

– A Scheduler’s Lament

There are many existing formulas for calculating an Independent Estimate at Complete (IEAC) from earned value data. A recent study of a sample of projects found that the calculated IEACs analyzed at the 25%, 50%, and 75% complete points were not accurate when compared to the final actual cost of work performed (ACWP). The following table lists the thresholds used to assess the accuracy of the IEACs at the different complete points for the sample projects.

Percent CompleteAccuracy Threshold
25%Within +/- 10% of final ACWP
50%Within +/- 7% of final ACWP
75%Within +/- 5% of final ACWP

While working on that study of the accuracy of commonly applied IEAC formulas as well as on a small project as an analyst for a customer, the idea for using data directly from the integrated master schedule (IMS) in conjunction with the cost performance data to create a new IEAC formula emerged.

Using Data Directly from the IMS to Calculate an IEAC

It should be noted that none of the generally used IEAC formulas use data directly from the IMS. The IEAC formulas use data found in the cost performance portion of the earned value monthly reports to customers.

IMS data is only used indirectly in the IEAC formulas. When a task is started and progress updated, the earned value (the budgeted cost for work performed or BCWP) is developed from the progress reported. This is measured against the cost baseline (the budgeted cost for work scheduled or BCWS).

At the same time, in the IMS environment, the schedule analysts are calculating the Baseline Execution Index (BEI) for task completions/finishes. BEI (for finishes) measures how many of the tasks baselined to be completed by the cut-off date were completed. If all the tasks were done (BEI = 1), their value would have been earned. Of course, other tasks could have started, progressed, and maybe even finished. For this example, the Schedule Performance Index (SPI) calculated at that point (BCWP/BCWS) should be at least 1 and potentially higher. The SPI reflects the baseline value of completed tasks plus the in-process claimed baseline value. The in-process claimed value can be subjective in some cases.

The argument, if there were one, might be there is no need to try and include BEI or similar schedule measures in the IEAC formulas since they already include SPI.

However, there is a whole different and unique set of information coming from the IMS that is not currently used in the IEAC formulas. That information is what we chose to call “Duration Performance” and “Realism Ratio.” These are measures of the actual duration for completed tasks and the forecast duration for future tasks.

Calculating Duration Performance

The IMS data includes the baseline number of days assigned to each task as well as the actual number of days to complete each task. If a task is baselined to take 10 days (Baseline Duration = 10) and the task took 15 days to complete (Actual Duration = 15) then it is taking 150% of baseline to do the work.

This is similar to the Cost Performance Index (CPI) that uses the BCWP and the ACWP to determine how efficient the work performance has been. The formula BCWP/ACWP shows how the work accomplished compares to the cost of that work performed.

If we assume, for labor at least, that taking longer to complete a task often leads to costing more than baselined, we can use the Duration Performance to develop an IEAC.

To develop the Duration Performance, we would use the IMS from the month being analyzed to perform the following actions:

  1. Filter out all summary tasks and look only at real work tasks.
  2. Decide what to do with level of effort (LOE) – keep it or ignore it.
  3. Filter for all tasks that are completed (100% complete).
  4. Add up the baseline duration in days for all these completed tasks.
  5. Add up the actual duration days for these same completed tasks.
  6. Compare the actual duration days used to the baseline duration days.

An example would be:

  • 100 completed tasks
  • Total baseline days duration = 1,000
  • Total actual days duration = 1,500
  • Duration Performance = 1,000 / 1,500 = .67

One of the common IEAC formulas is the “SPI times CPI” that is calculated like this: ACWP + Budgeted Cost of Work Remaining (BCWR) / (CPI x SPI) where BCWR = Budget at Completion (BAC) – cumulative to date BCWP.

Now that we have a duration performance factor, we can develop a new IEAC. The Duration Performance IEAC would be done using the CPI from the same month as the IMS where ACWP + BCWR / (CPI x Duration Performance Index).

Using some actual data from a project for a single month we see:

  • Duration Performance Index = .82
  • BEI = .72
  • CPI = .92
  • SPI = .94 (significantly higher than the BEI)
  • ACWP = $9.2M
  • BCWR = $18.3M
  • IEAC using standard formula with CPI x SPI = $9.2 + $18.3 / (.92 x .94) = $30.3M
  • IEAC (Duration Performance) = $9.2 +$18.3 / (.92 x .82) = $33.5M

Assessing the Realism Ratio

When we look at the remaining tasks to be completed, we can use the Realism Ratio to assess how the future forecast durations compare to the performance so far.

The data needed are the baseline duration and the forecasted duration for all tasks that have not been started. This concept excludes in-process tasks. In our example from before, the data we created looked like this:

  • 100 completed tasks
  • Total baseline days duration = 1,000
  • Total actual days duration = 1,500
  • Duration Performance = 1,000 / 1,500 = .67

We would use the same IMS to do this:

  1. Filter out all summary tasks and look only at real work tasks.
  2. Decide what to do with LOE – keep it or ignore it.
  3. Filter for all tasks that are not started.
  4. Add up the baseline duration in days for all these tasks not started.
  5. Add up the forecasted duration days for these same tasks not started.
  6. Compare the forecasted duration days to the baseline duration days.

Let’s say there were 100 tasks not started. If the forecasted days were 1,000 and the baseline days were 1,000 that would yield 100%. When we did the example, the Duration Performance was .67. This means that performance to date was .67 but the future will be 100% or 1. You can see the disconnect. That disconnect we call the Realism Ratio (in this example, .67/1).

Data from the actual project for the same month as discussed earlier shows:

  • Duration Performance = 122% of baseline
  • Future Performance = .86 or 86% of baseline.

This means that the future durations are cut significantly.

We would use this data to develop a factor called a Realism Ratio (86/122 = .70) and that would be used to develop an IEAC using this formula: IEAC (Realism Ratio) = ACWP + BCWR / (CPI x Realism Ratio).

Using the same sample project data from above and adding in an assessment of the forecasted durations for the remaining work, we see:

  • Duration Performance = .82
  • BEI = .72
  • CPI = .92
  • SPI = .94 (significantly higher than the BEI)
  • ACWP = $9.2M
  • BCWR = $18.3M
  • Realism Ratio = .70
  • IEAC using standard formula with CPI x SPI = $9.2 + $18.3 / (.92 x .94) = $30.3M
  • IEAC (Duration Performance) = $9.2 +$18.3 / (.92 x .82) = $33.5M
  • IEAC (Realism Ratio) = $9.2 +$18.3 / (.92 x .70) = $37.6M

The project is not complete, so the final ACWP position is not known. There is a dramatic difference between the three IEACs. The difference between BEI and SPI indicates that in-process tasks and other factors such as LOE are potentially affecting SPI.

What can we learn from this sample project?

In this example, additional investigation is warranted. There are potential issues with the realism of the baseline and current schedule that are signaling a cost growth issue is likely to occur. Relying on just the time-phased cost data for IEAC calculations may not be sufficient to assess whether a contractor’s range of EACs included in their monthly cost performance reports are realistic. For more discussion, see the blog on Maintaining a Credible Estimate to Completion (EAC) and the blog on Using EVM Performance Metrics for Evaluating EACs.

Are there lurking cost growth surprises in your projects? You may want to consider revisiting your estimate to complete (ETC) and EAC process to verify there is an integrated assessment of the schedule and cost data to identify potential disconnects. H&A earned value consultants can provide an independent assessment of the quality of the data as well processes and procedures to help you verify your EACs are realistic. Call us today at (714) 685-1730.

Incorporating IMS Information Directly into Independent Estimate at Completion (IEAC) Formulas Read Post »

Establishing a Robust EVMS Self-Governance Process

By / / , , ,
, , ,
Establishing a Robust EVMS Self-Governance Process

A previous blog, Benefits of an EVMS Self-Governance Process, discussed why establishing a self-governance or self-surveillance process is important and how an effective process builds confidence with the customer. With a structured and repeatable process in place, effective self-governance demonstrates management’s commitment to maintaining the EVMS and open communications with the government customer. Self-disclosure and quickly addressing EVMS compliance issues are essential.

H&A earned value consultants often assist contractors to implement a robust self-governance process as their level of EVMS maturity increases over time. This blog highlights how H&A provides support and technical expertise to help a DOE contractor to do just that.

Developing the Self-Governance Process and Tools

H&A is a strong teammate in the development and implementation of a robust EVMS self-governance process for TRIAD at the Los Alamos National Laboratory (LANL). TRIAD is the prime contractor that provides laboratory management and operations for LANL. H&A is involved in developing tools, refining processes, establishing business rhythms, and summarizing data necessary to support the implementation of a leading-edge EVMS self-surveillance capability. The H&A team is also instrumental in developing the tools necessary to analyze, review, and act on the monthly data set TRIAD provides to the DOE EVMS compliance team.

These tools generate the DOE EVMS compliance metrics (automated and manual) in accordance with the DOE EVMS Metric Specification to ensure TRIAD is able to view the data the same way as their DOE counterparts. Once the tools generate the DOE EVMS compliance metrics, the results are passed to the TRIAD System Surveillance Officers (SSOs) to review and confirm flagged items are either actual fails or exempted/justified based on the rationale captured in the tool. For failed metrics, the SSOs and the project teams use the source data from the tool to identify the root cause and proactively correct EVMS compliance issues. Each month the EVM compliance data is collected across projects, summarized, and graded at the TRIAD level, and then gathered into an EVMS compliance dashboard for TRIAD leadership review and action.

Monthly Self-Surveillance Process

The monthly self-surveillance process includes the following activities.

  1. For each project, the tool generates the automated metrics from the DOE compliance flat files and then collects the results of manual testing into a single file with all 183 DOE metrics. This tool enables an SSO to review the flags, access the source cost and schedule data, apply exemptions/waivers, and then share the data with the project team to resolve issues. By trending this data across the project’s life cycle and capturing SSO exemptions and monthly actions, the team can analyze the data, determine root causes, address issues, and capture historical EVMS compliance actions in one place.
  2. An EVMS summarization tool then collects the results from each project and rolls the lower-level results into a summary TRIAD level. Each metric grade (Pass/Fail/Caution) considers weighted EVMS performance across multiple projects to ensure grading is aligned with the exit criteria for the DOE corrective action plans. In addition to TRIAD level grading for each metric, the summary tool also rolls up the metrics to the 10 EVMS Maturity Subprocess areas and 56 Attributes of an EVMS which is documented in the Compliance Assessment Governance (CAG) Appendix to the DOE EVMS Compliance Review Standard Operating Procedure (ECRSOP). This summarization tool provides the subprocess area and attribute grading at both the project and TRIAD levels. By viewing the data across projects and time, the EVMS core team can quickly identify systemic or project level issues.
  3. A set of tailored EVMS compliance summarization metrics is presented in a “dashboard” configuration for the EVMS core group and senior leadership to review. Leadership uses this summary data to determine where they need to dive deeper into the data and whether TRIAD is meeting their EVMS compliance targets.

Figure 1 illustrates this management level dashboard view. 

Figure 1: Example of the Summary Level Compliance Metrics Across Projects

Figure 1: Example of the Summary Level Compliance Metrics Across Projects

  1. The team also developed and uses a flat file analysis tool that is aligned to the DOE data integrity and quality checks (DIQs). This tool is used for projects transitioning into DOE Critical Decision (CD) Milestones 2 or 3 execution phases that require submittals to the DOE Project Assessment and Reporting System (PARS). This tool ensures the project flat files meet the DOE data quality standards. Like the 183 metrics tool, the flat file tool enables analysts to isolate data quality issues, review the source data, and then determine and track how the team will resolve or justify each issue. In addition to preparing for PARS submittals, these DIQ assessment metrics are also generated monthly to help assess on-going system integration integrity.

Establishing a Best-in-Class Self-Governance Process

In addition to supporting the monthly self-surveillance process, these tools and processes are instrumental in supporting the active surveillance portion of TRIAD’s self-governance efforts. The active surveillance team uses the same tools to summarize and review the “data call” sets in preparation for their reviews. Just like their DOE counterparts, the TRIAD active surveillance team analyzes the 183 DOE compliance metrics to focus their inquiries and document review findings. The H&A team was instrumental in planning, executing, and closing out the recent TRIAD active surveillance that was observed and lauded by the DOE EVMS compliance team.

By supporting the design, development, planning, and execution of all facets of a leading-edge DOE self-governance process, the H&A team helped to ensure our LANL customer has the robust EVMS compliance capability necessary to meet the rigorous DOE EVMS compliance requirements.

As this case study demonstrates, with H&A’s help, TRIAD successfully implemented a structured and repeatable self-governance process with analysis tools that capture objective measures and metrics to actively demonstrate compliance and issue resolution to their customer.

H&A earned value consultants can do the same for you. Call us today at (714) 685-1730 to get started. 

Establishing a Robust EVMS Self-Governance Process Read Post »

Scroll to Top