Earned Value Management (EVM)

Hiring the Right EVM Professional

By / / , , , , , ,
, , , , , ,

Submarine on top of ocean with sailors on deck

EVM Hiring, Not Selling

You are searching for the right person to fill that critical EVM program management or project controls position on one of your newer or one of your tough projects. So, what does the interview sound like? Probably like so many I have witnessed. But there is a much better way to conduct the interview and get the right person.

Many of the interviews I have participated in consisted largely of the interviewer telling the potential candidate about the position, about the company, and almost making the interview a selling situation. It sometimes seemed like the theme was “How can we convince this person to come on board?”

Always Clarify

Of course, some time in the interview must be spent explaining the situation to the candidate’s satisfaction. You would not want to make an offer to someone only to have them come back at you expressing confusion about the position or the project. That happened to me years ago. I was interviewing with a major computer firm for the position of “program manager.” Obviously, the ad I answered, and the screening process were flawed. I arrived at the interview and within a few minutes the interviewing manager was commenting on the fact I had no software programming experience. They were looking for a manager for a software development (programming) effort. They did not even understand the term program manager as it related to project management. We agreed to end the interview on good terms although I am sure we both realized we had wasted a lot of time.

Often an interviewer will focus on the certifications the interviewee has achieved. If the person is a PMP from the PMI, that is a good thing. But more than once I have met and worked with people who are certified and credentialed, but who really have no earned value training and cannot get the job done in the real world. Be careful and dig deeper. The right interview can help do that for you.

Can They Get the Job Done?

But the most frequent observation I have made about a defective interview process is the failure to verify that the candidate can do the job. The best illustration of this is from the book “Peopleware” by Tom DeMarco and Timothy Lister. The example is in Chapter 16 and is called “Hiring a Juggler.” It presents the story of the hiring manager, it was the circus manager, asking a lot of questions about other circuses the juggler has worked for, the things the juggler can juggle, how many things can he keep in the air at one time, and so on. At the end of the interview, the manager is satisfied he has found his new juggler and offers him the job. The surprised juggler asks one question only, “Don’t you want to see me juggle?”

At H&A, when we are looking at new individuals for our scheduling practice, we actually give them a test. They are provided a written description of an interview with a CAM in which the CAM explains what is supposed to happen in his or her control account. From that written discussion, the interviewees are asked to get into the scheduling software with which they are proficient and build the plan described by the CAM. With that plan, they are asked to determine the end date, locate the critical path, and otherwise verify that the schedule is a high-quality schedule. In other words, we ask our interviewees to show us they can juggle.

EVM Expert Questions

So what kinds of things would you want to talk about in an interview for a project manager candidate, an EVMS candidate, or a scheduling candidate? What direction could you take in the interview that would be more oriented to seeing if the person can juggle? How about some of these questions? Or at least how about the general direction of these questions?

Question List

  1. In your opinion, who are the stakeholders for the project WBS?
  2. What are the pitfalls that you would encounter while building the right WBS? How can they impact your project?
  3. Tell me about the System Engineering Technical Review (SETR) process and how that would be part of your project?
  4. How would you assess whether the amount of Management Reserve withheld on your project was the right amount?
  5. What, from your experience, do you think is the single biggest project-killing issue, and how would you prevent or minimize it on your project?
  6. In addition to that issue, what are three more serious potential problems that can cause failure?
  7. What is total float (total slack) and how would you use that as a manager of a project?
  8. What is a “driving path” and why would that be important to you on your project?
  9. How would you evaluate a control account EAC on your project?
  10. When you issue ground rules for developing a new project plan, what confidence level do you set for duration estimates and cost estimates from your teams?
  11. What process would you recommend for developing the project-level best case, worst case, and most likely EACs?
  12. From your point of view, what are the main duties of a control account manager?
  13. What are some measures of cost and schedule performance-to-date in a control account and what do they mean to you as a manager?
  14. When a control account has a CPI (cumulative) of .75, and SPI (cumulative) of 1.1, and a VAC of -20%, what does it mean?
  15. What are some of the Generally Accepted Scheduling Principles (GASP)?
  16. What is TCPI and what do you use it for?
  17. Can you explain some of the key measures in a project schedule that you can use to assess its quality?
  18. Please explain how a Schedule Risk Assessment is conducted and how the results are used.
  19. What professional organizations do you belong to?
  20. What is the last book you read about project management?

Extend

Now that you have had a chance to think about those questions, undoubtedly others have come to mind. An interview with the give-and-take generated from discussing a list of questions like those would be very revealing. At the end of that interview you should know if the interviewee can juggle. You will know where they have good understanding and where they might not be ready.

Does the interviewee have to be exactly right on every topic? Not at all. But the answers and the discussion can help you assess how much development is still needed for this candidate to be able to shine in the open position you are trying to fill. Not everyone knows everything. Experience is a great teacher, but it comes from the situations where the interviewee has been directly exposed. Or perhaps from their leaning.

Take a moment and think about the interviewing practices at your company. Are they like the ones we just discussed? Can they be improved? Where are they weak? Where are they strong?

Hiring the Right EVM Professional Read Post »

Project Management: Earned Value Consulting; Could You Use Some?

By / / , , , , ,
, , , , ,

Failed Projects

A recent article discussed the results of a survey on the reasons that projects failed. The definition of failure was that the project was abandoned. Abandonment does not occur frequently in the world of government projects; especially defense projects where there should be strong “must have” needs driving the project. These projects tend to persist until completed even though the outcomes are not satisfactory. But there is a lot to learn from the list of reasons for failure.

Of the sixteen reasons listed, the top four had to do with changes to the environment that had given rise to the project. For example, changes in the company priorities was the most often cited reason for abandonment. In that same vein were issues with changing objectives and inaccurate definition of requirements. These types of failures are not topics for this blog since they do not immediately involve execution of the project.

Execution Problems

Lower on the list of those environment related reasons for failure were the ones more related to execution problems. These are of significant interest to a project management using an earned value consulting company such as Humphreys & Associates (H&A). These reasons related more to issues of poor project management that could have been corrected. In this area were reasons like “poor change management,” “inaccurate cost estimates,” “inaccurate time estimates,” and “inexperienced project management.”

The answers given in a survey situation depend very much on the mindset of the person responding. Is the reason really “inaccurate cost estimates” or should it have been “failure to execute to the estimate”? How many times have you seen a problem in execution “swept under the carpet” as being an inaccurate estimate or plan? One of these two answers points to the estimating system and process while the other points to project management. The estimates were generated; and, at some point, they were deemed to be sufficiently detailed to launch the project. If a scrubbed and blessed estimate is “inaccurate” that would still be a failure of project management. If the problem were really a failure to execute, then how easy would it be to blame the problem on poor estimates? This blog will discuss the cited failures as if they were execution failures.

Earned Value Management (EVM) Consultant Specialists

There are situations in life where the need for specialized advice is common and well accepted by us all. When your doctor is unsure of the medical issues, the doctor will send you to a specialist. The reason is obvious. The specialist has learned so much more about a specific problem and has so much experience diagnosing and treating the problem that it would be foolish not to secure the services of that specialist. In fact, it might be malpractice. A project management consultant can be thought of much like a medical specialist.

There are similar situations in business where the need for specialized knowledge is critical. Large companies tend to have in-house legal departments to cover the day-to-day legal issues and tasks that are central to their businesses. However, the need to go to outside counsel for large or unusual issues is accepted. Companies do not hesitate to engage the services of outside law firms to help them through troubled times. Project management consultants are like outside counsel.

What if there were a project management or earned value management situation you have never encountered before? A good example would be the times that H&A has been called in to help clients navigate the unhappy circumstances of needing to go over-target. Going through the over-target-baseline (OTB) or over-target-schedule (OTS) process is not a common experience. It is a tense time when careers can be on the line and the company reputation might also be at risk. It takes specialized knowledge to get it right. In some cases, it even takes the objective view of an outsider to help make the right decisions.

Specialized Knowledge

Another example of specialized knowledge being crucial is when the customer has deemed some issue on the project to be deficient. In some situations, a customer’s Corrective Action Request (CAR) can result in cost penalties and damaged reputations; possibly even worse consequences could result. Engaging the services of an EVM consultant with experience in identifying problems, building Corrective Action Plans (CAP)s, and leading or helping implementing the corrective actions is often a valuable and necessary action. Ask yourself how smart it would be to assume that those who were involved in causing the issue would be capable of creating a satisfactory solution.

These scenarios are aligned with the idea of project management consulting being something you only need in a crisis. There are other non-crisis needs for specialized support. Often H&A is engaged simply to help a client prepare a proposal. A proposal situation puts heavy demands on the company staffing levels and can require areas of specialized knowledge not available in the company. What if the company has never created a fully compliant Integrated Master Schedule (IMS) and they could use help the first time? What if there are not enough trained and experienced schedulers to work on the proposal? What if the company does not have a documented project management system?

Make or Break Opportunities

Projects can be huge and risky. They can be make-or-break opportunities to a company. Where so much can depend on good project management, smart companies recognize the need for an outside opinion and outside talent. Just like the internal legal department, the internal project management group sometimes needs to call on outside subject matter experts. While it might be obvious, let’s look at some reasons why this is true.

There are more ordinary everyday reasons to engage a project management consultant. Perhaps an organization just managed to win a new project bigger than any they have won before. In this case, they may not be ready to handle the project in terms of experience, systems, and even just talented headcount. A project management consulting company such as H&A can bring solutions to your earned value woes. It can also provide temporary training staff to get things going until the client is ready to take over.

Poor Communication

Let’s get back to the survey of reasons that projects failed. Are there issues on the list where project management consulting could have made a difference? Imagine an improved project management process and staff after a period of consulting to support creating or improving systems and training personnel?

The fifth most frequent reason for failure is “poor communication.” A good project management system with trained personnel is all about communication. Communication of plans, communication of progress, communication of issues, and communication of corrective actions are all actions required in a project management system. Quite often the problem of “poor change management,” cited as the sixth most common reason for failure, is reduced or eliminated after using the consulting services of a specialist?

What about the twelfth cited problem of “inadequate resource forecasting”? Would a well built and maintained resource-loaded Integrated Master Schedule (IMS) go a long way in providing forecasts of resource needs and the impacts of not having the resources? In fact, a proper IMS would help with several of the cited reasons for failure, such as inaccurate duration estimates. In fact, the application of a process, such as Schedule Risk Analysis (SRA), with the help of an experienced consultant can identify such issues in advance while there is still time to take action.

Earned Value Training

Disregarding the threat of failure as a motivator, the need for constant improvement should be enough reason to consider a project management consultant. We can all laugh at the time-worn clichés of “not-invented-here” or “we’ve never done it that way;” however, these are clichés for a reason. There is resistance to outside help and there is resistance to change. But outside help can be a great logjam breaker. An experienced and knowledgeable consultant can be your voice when you need someone who has, to use another cliché, “been there and done that.”

In fact, our consultants can laugh when they say they have “been there” and they have more than a T-shirt to prove it.

Project Management: Earned Value Consulting; Could You Use Some? Read Post »

Formal Reprogramming – What Happened?

By / / , , , , , , , , , , ,
, , , , , , , , , , ,

Graph of an Increasing Budget

A long time ago, in a galaxy far, far away….an Over Target Baseline (OTB) – by design – was a rare occurrence (and the OTS concept did not even exist as part of Formal Reprogramming). Formal Reprogramming was a very difficult and cumbersome process that most contractors (and the government) really did not like to consider. The government, in its 1969 Joint Implementation Guide, said:

“Reprogramming should not be done more frequently than annually and preferably no more frequently than once during the life of the contract.”

The Office of the Under Secretary of Defense (OUSD) Acquisition, Analytics and Policy (AAP) – formerly PARCA – , in their latest OTB/OTS guide, states that Formal Reprogramming now has expanded to include an Over Target Schedule (OTS).  However, in that guide, it is stated in Paragraph 1.3.8:

“Ideally, formal reprogramming should be done no more than one time during the life of a contract. However, there may be instances where another formal reprogramming is warranted… When formal reprogramming is accomplished in accordance with the procedures in this guide, with a realistic cost and schedule estimate established for the remaining work, it should not be necessary to undergo formal reprogramming again.”

Today, though, whenever contractors incur a significant cost or schedule variance, instead of resolving the variance cause, the first words seem to be: “Let’s do an OTB or OTS.”  The lure of “getting rid of cost and schedule variances” seems too good to pass up.  Unfortunately, an OTB/OTS implementation has never been an instantaneous process. With AAP’s 12 step OTB/OTS process, it is obvious that the contractor will not be able to start today and incorporate the OTB/OTS in the next Integrated Program Management Data and Analysis Report (IPMDAR) dataset. In fact, AAP’s OTB/ OTS guide states in paragraph 3.8:

“It may be difficult to ascertain the length of time it will take to implement a new baseline based on the scope of the effort. It is not uncommon for the entire process to take up to six months which would be too long of a period without basic cost reporting.”

The last line of the above cited paragraph was referencing the reporting requirements to the customer when an OTB/OTS is being implemented.

The IPMDAR Implementation and Tailoring Guide (5/21/2020) even recognizes the issues with timeliness of implementing an OTB/OTS:

2.3.2.5.5  Formal Reprogramming Timeliness. Formal reprogramming can require more than one month to implement. During formal reprogramming, reporting shall continue, at a minimum, to include ACWP, and the latest reported cumulative BCWS and BCWP will be maintained until the OTB/OTS is implemented. 

So why does it take so long to implement the OTB/OTS?  Can the contractor just adjust the bottom line variances and move on?  Actually no, nothing is really that simple.  This is one of the reasons that implementing an OTB and OTS should not be taken lightly.   The AAP OTB/OTS Guide addresses adjustments this way:

“3.5.6.2 Adjusting Variances: A key consideration in implementing an OTB is to determine what to do with the variances against the pre-OTB baseline. There are essentially five basic options. This is a far more detailed effort than these simple descriptions imply, as these adjustments have to be made at the detail level (control account or work package).”

When considering the number of control accounts and work packages involved in a major contract, a Formal Reprogramming can become a rather daunting task.  The contractor also has to report the effects of the Formal Reprogramming in the IPMDAR Reprogramming Adjustments columns. These adjustment columns appear on both Format 1 and Format 2 of the IPMDAR database, which means the contractor must undertake the assessment for both the contract’s WBS and the OBS – for each WBS element and for each OBS element reported.  This can be further complicated if the OTB/OTS exercise were flowed down to subcontractors for a given program.  The AAP OTB/ OTS Guide, paragraph 3.8 also states:

“The customer should be cognizant of the prime contractor’s coordination complexities and issues with its subcontractors. The time to implementation may be extended due to accounting calendar month overlaps, compressed reiterations of contractor ETC updates, internal reviews, subcontractor MR strategy negotiations, senior management approvals, etc., all while statusing the normal existing performance within a reporting cycle.”

In the early days, when implementing an OTB with variance adjustments, the company and the customer agreed on a month-end date to make the data adjustments.  Then the contractor ran two CPRs or IPMRs (now the IPMDAR): (1) the first report as though no OTB had been implemented [to determine the amount of adjustments to cost variance (CV) and schedule variance (SV) at all the reporting levels] and, (2) the second report [after the OTB implementation had been completed – no matter how long it took] showing the Column 12 adjustments plus whatever BAC changes were being implemented.

Under the current OTB/OTS Guide, it appears as though this process is being done all at once. As stated in the AAP OTB/ OTS Guide paragraph 3.8 above, this implementation could take up to 6 months to complete, so lagging the second report until the OTB/OTS implementation is completed seems logical. The last sentence in paragraph 3.8 also stipulates that regardless of how long implementation takes, the contractor and customer will agree on interim reporting that will be required, further stating that:

“In all cases, at least ACWP should continue to be reported.”

Perhaps this agreement with the customer should also specify the content of the first IPMDAR following OTB/OTS implementation.

All things taken into account, the process of requesting and getting approval for an OTB or OTS can be a long and difficult process, especially if, at the end of it all, the contractor’s request is denied.  Even if it were approved and the contractor implements and works to the newly recognized baseline, immediately doing another one is not a pleasant thought – and remember, it was not intended to be pleasant. Reprogramming was always supposed to be a last resort action, when reporting to the current baseline was totally unrealistic.

Now, what about those cases where a contract has one or two elements reporting against totally unrealistic budget (or schedule) baselines?  The AAP OTB/ OTS Guide does cover a partial OTB, but reiterates that this is still an OTB because the Total Allocated Budget (TAB) will exceed the Contract Budget Base (CBB).  In the early days, however, the government allowed what was called Internal Operating Budgets (IOBs) for lower level elements (control accounts, or specific WBS elements, etc.) that were having problems resulting in an unrealistic baseline for the work remaining. The 1987 Joint Implementation Guide, paragraph 3-3. I (5) described IOBs as follows:

“(5) Internal Operating Budgets. Nothing in the criteria prevents the contractor from establishing an internal operating budget which is less than or more than the total allocated budget. However, there must be controls and procedures to ensure that the performance measurement baseline is not distorted.

(a) Operating budgets are sometimes used to establish internal targets for rework or added in-scope effort which is not significant enough to warrant formal reprogramming. Such budgets do not become a substitute for the [control] account budgets in the performance measurement baseline, but should be visible to all levels of management as appropriate. Control account managers should be able to evaluate performance in terms of both operating budgets and [control] account budgets to meet the requirements of internal management and reporting to the Government.

(b) Establishment and use of operating budgets should be done with caution.  Working against one plan and reporting progress against another is undesirable and the operating budget should not differ significantly from the [control] account budget in the performance measurement baseline. Operating budgets are intended to provide targets for specific elements of work where otherwise the targets would be unrealistic. They are not intended to serve as a completely separate work measurement plan for the contract as a whole.”

Current literature no longer specifically addresses Internal Operating Budgets (IOBs), but with the recent trend of contractors jumping to the OTB/OTS conclusion, it might be a better alternative to have individual instances of unrealistic budgets (or schedules) that do not otherwise push the total program to the need for a complete OTB and/or OTS implementation.

These could be good discussion topics for future AAP and DCMA meetings with industry representatives, to determine if there are ways to streamline the process, or at least reduce the amount of requests to implement Formal Reprogramming.  Variances are, after all, performance measurement indicators that should not just be routinely and artificially eliminated.

Formal Reprogramming – What Happened? Read Post »

Humphreys and Assoc Reviews 7 Principles of Earned Value Management Tier 2 System Implementation Intent Guide

By / / , , , , , , ,
, , , , , , ,

In this video we review the 7 Principles of Earned Value Management Tier 2 System Implementation Intent Guide published by the Assistant Secretary for Preparedness and Response, or ASPR.

This Guide is primarily used by the Biomedical Advanced Research and Development Authority, or BARDA, on countermeasure R&D contracts that have a total acquisition cost greater than $25 million and a Technical Readiness Level of less than 7.

7 Principles of Earned Value Management Tier 2 System Implementation Intent Guide -- EVM Cross Reference Guide

Humphreys and Assoc Reviews 7 Principles of Earned Value Management Tier 2 System Implementation Intent Guide Read Post »

Scroll to Top