Agile EVMS

Earned Value Management and Agile Integration – Work Scope as the Foundation

By /

A previous H&A blog, “EVM (Earned Value Management) vs. Agile Project Management,” provided an introduction to the differences between an Earned Value Management System (EVMS) and Agile development methodologies.

At first glance, it may seem that EVM and Agile development methodologies are incompatible. Agile is all about rapid incremental product deliveries and responding quickly to an evolving understanding of the desired deliverable or outcome. Depending on how an EVMS is implemented, the EVMS can often seem rigid in comparison.

Can the two methodologies coexist and complement each other to create an effective integrated project management system? The answer is yes, provided you have thought through how you intend to use the two systems together and map how and where they integrate. The goal is to leverage the benefits of each system and without forcing either system to do something it wasn’t meant to do.

There is a natural top down and bottom-up process integration when defining the scope of work and acceptance criteria. This process integration continues for planning and scheduling the feature estimates of effort, establishing the budget baseline, and ultimately maintaining the estimate to complete (ETC) in the EVMS. It also supports an integrated process for measuring completed work. The Agile daily standup meetings provide current information about accomplishments and impediments at the lowest level of the project. The Agile system provides the quantifiable backup data for claiming earned value in the EVMS for performance reporting.

The following image illustrates the relationship between the two systems.

The source for this image is the NDIA IPMD Industry Practice Guide for Agile on EVM Programs, Revision May 2019, Figure 2-4.

The Foundation for Integrating EVM and Agile: Defining the Scope of Work

A work breakdown structure (WBS) is commonly used to organize and decompose a project’s scope of work into manageable, product-oriented elements. It is an essential communication tool for the customer and contractor so they have a common frame of reference to capture and manage requirements as well as expected deliverables or outcomes. It establishes a common basis for measuring progress and defining accomplishment criteria. It is the framework for developing a project’s schedule (timing of tasks), identifying resources to accomplish the scheduled tasks, creating cost estimates as well as the budget baseline, and identifying risks. In an EVMS, the WBS is often decomposed to the control account level which is further decomposed into work packages or planning packages. Once extended to the lower level, it provides a framework for tracking technical accomplishments, measuring completed work, and identifying variances from the original plan to complete the work.

There is a similar planning hierarchy to Agile projects – the Product Backlog is the foundation for defining the scope of work. The Product Backlog starts at the Epic or Capability level and is further defined through product planning. The process includes prioritizing the capabilities and defining the sequence of deliverables to create the Product Roadmap (timing). The capabilities are decomposed into features along with an estimate of the effort to deliver the feature. Features should include exit criteria (definition of done) and have minimal dependencies. At the lowest level, features are decomposed into Sprint Stories and related tasks forming the basis for the schedule and measuring completed work in the EVMS. As illustrated in the image above, in the Product Backlog hierarchy, an Epic/Capability relates to the control account level in the EVMS. Features relate to the work packages in the EVMS.

In an integrated environment, there can be a natural mapping between the WBS and the Product Backlog regardless of the starting point. When starting from the Agile system, the Product Backlog could be used to create the project’s WBS. The customer may also pre-define the top level WBS elements that could form the backlog structure. The DoD uses MIL-STD-881, Work Breakdown Structures for Defense Materiel Items, for this purpose so they have a set of common templates they can use across programs to capture historical actual cost data for cost estimating and should cost analysis. Even though the top level WBS elements may be pre-defined, the lower-level content can reflect an outcome based Agile structure that focuses on customer driven deliverables.
An example is illustrated here.

Possible Agile software development MIL-STD-881 WBS breakout. The source for this image is the DoD ADA Agile and Earned Value Management: A Program Manager’s Desk Guide, Revision November 2020, Figure 2.

What’s common between the two systems?  You have:

  • Decomposed a common understanding of the scope of work into manageable, product or outcome-oriented elements of work.
  • Defined the agreed upon accomplishment criteria or definition of done for a given deliverable or outcome.
  • Built a framework to determine the timing of tasks, identifying the resource requirements, creating a cost estimate to do the work, and means to objectively measure completed work.  Ideally, you have also identified the risks or uncertainties so you know where potential issues may surface.

Best Practices for Integrating the WBS and Product Backlog

Keeping in mind you have a similar decomposition of work between the two systems, you can set up the WBS and Product Backlog to align with each other.  The goal is to ensure traceability so you can easily support the EVMS requirements. 

Here are a few best practices for integrating the WBS and Product Backlog.

  1. Verify the WBS aligns with the prioritized Product Backlog to at least the Epic/Capability level (control account level in the WBS).  Determine how you intend to maintain traceability between the WBS elements and the work items in the Product Backlog so it doesn’t matter which system you use to review or confirm the agreed upon scope of work.  Identify and document how you intend to map the WBS elements to a work item in the Product Backlog so you always have the necessary cross references identified.  Keep it simple.  Where possible, minimize the need to enter something more than once. 
  2. Use the Product Backlog to tailor the WBS.  This is particularly true for projects where Agile development is central to the final deliverable.  Consider your two end objectives: you need to provide project performance reporting via the EVMS and organize the items in the Product Backlog so you can decompose them from the Epic/Capability level to the Feature level and then to the Sprint Story and task level.
  3. Watch the level of detail in the WBS.  Avoid driving the WBS to too low a level of detail.  Depending on the project, it may be appropriate to go no lower than the Epic/Capability level (control account level).  The reasoning: in the Agile system, the lower-level Stories and tasks are flexible and subject to change.  You are focused on completing current near-term work effort within a Sprint.  Better to let the Agile system manage and track what is happening at the rapidly changing detail level – it is designed to do that.  For EVMS purposes, project monitoring and control should be at a higher level where scope can be consistently defined, aligned to the schedule and budget, and claimed as done (technical accomplishment).  The WBS should reflect the level of detail that is aligned to the permissible variation in requirements or configuration.  Otherwise, you create too much change “noise” in the EVMS.
  4. Confirm the WBS and Product Backlog contains 100% of the contract scope of work.  Verify the statement of work has been mapped to the WBS elements or work items in the Product Backlog.  Why is this important?  You need to demonstrate you have captured the entire technical scope of work to at least the Epic/Capability level.  You need this for internal planning purposes so your development teams have an understanding of the technical requirements and expected outcomes as well as managing changes.  Your customer also needs confidence that you have an understanding of the entire scope of work and have planned accordingly for the duration of the project. 
  5. Verify you have defined the Feature accomplishment criteria or definition of done so it is easy to measure completed work.  This could be in the WBS dictionary or documented in the Agile system Product Backlog.  Determine where that information can be found and document the process to maintain it.  Ideally it is in one place so you only have to maintain it in one system and everyone on the project knows they are referencing current information.  Clearly defined accomplishment criteria mean you can objectively measure accomplishments – in both systems.  Enable that capability right from the start when you set up a new project in the two systems.

Are your EVM and Agile systems are sharing useful information?  Perhaps you have learned the hard way the WBS and Product Backlog aren’t sufficiently mapped for you to maintain traceability between the two systems.  We can help.  Call us today at (714) 685-1730

Earned Value Management and Agile Integration – Work Scope as the Foundation Read Post »

Agile Techniques for Hardware & Software Development

By / / ,
,
Agile Video Cover

Introduction

This white paper presents best practices and techniques to overcome hardware and software development challenges encountered as organizations move towards increased agility. Agile transformations typically start in pockets within the same discipline, such as Software or IT departments. As the Agile teams become more efficient and require increased support from inter-related departments, the supporting functions will likely adapt and begin to use Agile techniques to keep up with increased demand. Expanding Agile across the value stream results in the need for the organization to assess their current state of agility, create cross-functional teams, decide how to scale, craft vertical slices of work, adapt new development techniques, and increase customer involvement. Teams will need to be restructured to allow for end-to-end development, which is particularly difficult for hardware teams.

Team Structure & Composition

Hardware team structures are typically multi-disciplined in very diverse areas of competence. The Agile Coach should analyze the team composition and ensure all skills necessary to fulfill the team vision are present. It is possible for a hardware team to be end-to-end, with the use of techniques such as deployment of people with scarce skills, support from Subject Matter Experts (SMEs) and functional roles at the scaled level to ensure that the team can be self-reliant. If a skill set gap is present the coach can facilitate training, skill pairing, team deployment, or hiring so that the team can independently produce working product.

The first step towards cross-functionality on an Agile team is to pair similar skills together on as many stories as possible. An Electrical Engineer and a Mechanical Engineer would be an example of a natural pairing as their education and working experience have many parallels. As time goes on the team members will break down development barriers that typically exist between departments. This encourages the team members to become more T-shaped, which means deep in one area of competence and knowledgeable in other areas. They will understand the impact of “throwing work over the wall”. They will create more efficient ways to do work. They will need access to tools and assets within their own discipline that have been restricted to one or two individuals. An example of where restricted access may occur is within the Mechanical Engineering discipline. The development team may be separate from the Test Lab team, meaning they share the same discipline but only the Test Lab team has access and training on test equipment.

Software team structures are typically rooted in the same discipline; however, they are segregated by stage of software development and access to tools. For example, you may have a software team that is composed of an embedded software engineer, a software development engineer, and an application developer. Some team members may have experience in legacy products and access to specific tools while newer team members may be focused on new product development. It is much easier to break down inter-departmental barriers than it is to break down cross-departmental barriers.

Crafting Vertical Slices of Product Functionality

Product Roadmap development forms the foundation for Agile product architecture that allows for carefully crafted user stories and slices of work so that multiple disciplines can work on the same scope. The first layer of the roadmap should consist of Epics which are composed of large pieces of scope. Epics can be developed in a modular, phased, or business / user experience approach to help establish initial product architecture that enables teams to avoid regressing to waterfall development techniques.

For new programs, customers can support Agile development approaches by using a Work Breakdown Structure (WBS) that is product based, focuses on outcomes, and is compliant with Mil-STD-881. Customers should play a major role in Product Roadmap Development and ensure that Epics are architected to support vertical slicing and are not sequential/departmental pieces of work. The requirements for technical reviews should be modified to allow for incremental and iterative reviews and will require more frequent participation by the customer.

For existing programs that have recently transitioned to Agile development techniques, the customer should request Agile–Earned Value hierarchy and mapping. The Product Backlog must cover all requirements for the work that is being executed by development teams. The requirements are satisfied by the Stories, Features, and Epics. They should map to the Statement of Work paragraphs and Work Breakdown Structure. The customer should make time for more frequent Product Owner interaction and may need to access the Agile tools to have a better understanding of the Product Backlog and Release Plan. Attending the Release Backlog Reviews and major Release demonstrations will provide the Agile teams with crucial feedback that is necessary for continuous improvement.

Before the Integrated Baseline Review (IBR), the customer should have an in-depth understanding of the Agile techniques, frameworks, terminology, hierarchy, mapping and product architecture that is planned to be used on the program. This will allow the IBR discussion to be focused on how realistic and achievable the baseline is instead of discussing the chosen Agile approach.

Waterfall Epics consist of departmental or sequential pieces of scope. It would be difficult to design, develop, test, integrate and produce incremental pieces of working product using this type of product architecture. Teams would have to complete all Epics to arrive at working product.

Figure 1- Waterfall Epics
Figure 1- Waterfall Epics

In the above diagram, the team may be cross-functional and fully able to develop the product end-to-end but the way the work has been architected encourages the team to work within their departmental functions in a sequential fashion.

Epics can be architected into “modules” or parts of a product. Each module or part needs a Known Stable Interface (KSI) between itself and other modules or parts. This helps ensure the integrity of the relationships between them. The Product Ownership team serves as the KSI between modules and handles inter-dependencies and system integration at a scaled level. Modules will need to conform to the constraints surrounding the entire system or product. An example of this would be envelope constraints. If a module needs to fit into an existing product it would have to conform to a specific envelope constraint; otherwise it would not fit.

Figure 2- Modular Epics

In the above diagram, the Epics have been architected into individual parts or modules, so there must be significant focus on integration and system level thinking to avoid extensive re-work. Breaking Epics into modules is one of the easiest approaches that a Product Owner can use to group work because it allows them to visualize and manage the pieces of scope and interfaces in a way that can be easily understood. The weakness in this approach is that the roadmap is not as customer centric as other techniques.

Epics can be architected into waves or “phases” of capability. This is a technique that is best suited for a product that is delivered to a customer in various states or is delivered to multiple phases of customers. The first phase is typically a minimum viable product (MVP). Additional phases build on the MVP to refine and improve the product allowing it to be released to other customers or end users for different purposes. This continues until the product meets the primary end user or customer’s needs.

Figure 3- Phased Epics
Figure 3- Phased Epics

Epics can also be architected as a part of the existing business flow or desired user experience. Epics are organized around the steps the business or customer currently takes, or desires to take, to create or use a product. Epics formed in this manner will draw the organization’s focus to the customer’s desired outcome.

Figure 4- Customer Personas
Figure 4- Customer Personas
Figure 5 and 6

Regardless of the chosen method for Epic development, each Epic can be further decomposed into Features that represent a slice of meaningful functionality to a customer or end user. When the teams are ready to work on a Feature they will break it in to user stories.

Figure 7 - User Story Formation

User stories are typically written in narrative form and should be a testable vertical slice of functionality. The story should encompass requirements, design, development, test, integration and analysis.

Each user story should have a Definition of Done and an Acceptance Criteria. The Definition of Done is a checklist for user stories of similar types and ensures quality and completeness. The Acceptance Criteria helps the team have a clear understanding of the problem they are trying to solve and what test they will have to pass before work will be considered complete. The Acceptance Criteria clarifies the “what” and ensures there is a common understanding of the requirements prior to the start of work.

Vertical slices allow the team to quickly pivot as they learn more about the product they are developing. Because the product is being incrementally tested and vetted by the customer, there is a higher probability that the customer will be happier with the end product and that the product itself will be higher quality. Once vertical slices are ready to be worked by the team, there are many techniques they can use to perform the work with increasing agility.

Hardware Development Challenges

Hardware development is physical in nature therefore it is significantly different from the intangible development of software products. Changes to hardware are generally more expensive due to wasted development efforts, raw material costs, shipping and handling, expediting fees, inventory, and material management costs. Drastic software changes are virtual in nature and so they typically result in re-work and wasted development efforts. This significant difference in the cost of altering a design makes it imperative that hardware teams have less expensive ways to test out their ideas early in the product development process, before ordering more expensive prototype, pilot, or production materials. This is one reason why teams brainstorm many design concepts for components and use virtual modeling, 3-D printing, and simulations to fail the concepts as early as possible, obtaining meaningful customer feedback along the way. They then arrive at the best design solutions and continue to use virtual modeling, 3-D printing, and simulations during sourcing lead times to gain important product feedback about the slice of functionality they are developing. The teams push to build “something” demonstratable to the customer to solicit feedback regardless of prototype, pilot, or production part availability.

Hardware development introduces a special challenge in the form of lead time associated with obtaining a supply of raw materials. As the team rapidly iterates toward developed products, they need a shorter supply chain with vetted and available rapid sourcing suppliers. Adjusting the supply chain to accommodate quick turn suppliers and rapid prototyping takes time. An initial workaround for shortening the onboarding time of prototype suppliers in the Enterprise Resource Planning (ERP) system is the use of purchase cards available to the development team with an associated spend limit. This allows the teams to purchase low cost prototype materials from the supplier of their choice without the red tape associated with getting interim suppliers in the ERP system. A more lasting solution is the development of the sourcing skillset on development teams or the inclusion of sourcing at a scaled level. This will allow the teams to either source materials themselves or access more timely sourcing at the team of teams.

Agile Development Techniques

Many of the agile best practices and techniques that software teams use can be modified and applied to hardware development to create more opportunities for agility along the value stream.

Pair Programming is an agile software development technique in which two programmers work together at one workstation. One person writes code (Driver) while the other reviews each line of code as it is typed (Observer/Navigator) and strategizes about future work or improvement opportunities. The two are both capable of writing code and regularly switch roles. Because the two software team members pair together, bugs are less prevalent and the two are more accountable to efficiently complete work. Many studies have been conducted on the effectiveness of Pair Programming and the results show that pairs have ~15% less defects in their code. However, pairing can become overused if team members spend all day working together and are unable to independently approach work. It can also be distracting to use pair programming for more complex code in which solo work is better suited.

The concept of “pairing” is derived from the Pair Programming technique and can be used to facilitate Development Team Members working together to complete stories for the purpose of shared learning, awareness of the “big” picture, collaboration, and increased knowledge saturation. Pairing is an easy way to start making gains towards a more cross-functional team.

Test Driven Development is a software development technique, also known as red-green refactoring, that requires developers to write and run tests before they write code. The new test is run and subsequently failed to ensure that it captures the necessary logic to be confident in the results. The minimum code is now written to pass the test and the test is repeated to ensure the code passes. If the code fails it will be adjusted until it passes. The code is re-factored, non-functional attributes of the code are improved, and the result is solid code, reduced re-work, and improved quality.

Agile hardware teams borrow from this concept by writing stories with a test in mind and using red-green testing techniques to develop the product to the point that it passes the test. This is often referred to as the Minimum Viable Product (MVP) and it helps prevent over-engineering of the solution. This approach also ensures that work is completely done each sprint and it raises the quality of the final product because the pieces are tested along the way.

Scaling and Infrastructure

Agile development techniques can improve the efficiency within teams; however, there may be dependencies between teams, departments, or supporting functions that need to be resolved in order to arrive at complex, large scale product solutions. Scaling can help address bottlenecks and significantly improve time to market as waste is removed from current development processes. Some examples of organizational waste include handoffs, specializations, multi-tasking, context switching, and decision making by parties not involved in the details of development.

Agile Heat Mapping is a technique where the location and maturation of Agile, lean techniques, frameworks and best practices are mapped out and recorded across the organization. This provides visibility into pockets of Agility and allows for enhanced collaboration, learning, and sharing. It also serves as a baseline for Horizontal Scaling. Horizontal Scaling refers to the spread of Agile approaches across the organization. Teams that are using Scrum, Kanban, various lean and agile techniques, and scaled frameworks can be present within the same organization and yet they are oblivious to each other’s existence. By mapping out the current state of organizational agility, progress can be tracked over time.

Figure 8- Agile Heat Mapping
Figure 8- Agile Heat Mapping

As Agile pockets are discovered and teams push for greater increase in agility, they need to combine efforts in order to efficiently utilize resources and eliminate waste. Value Stream Mapping is an activity where the flow of work through the existing structure is laid out and sequenced. It is all-encompassing from product inception to product production. It helps make inefficiencies, waste, and bottlenecks visible and highlights improvement opportunities that can lean out the value stream. Value stream mapping can provide insight into organizational areas that would benefit from additional Agile infrastructure and scaling. Vertical Scaling establishes infrastructure in the form of an Agile Framework to support development teams as they work. Vertical scaling provides alignment and structure to manage dependencies, resolve impediments, prioritize work and align teams along the value streams. Without this type of infrastructure, the teams are less efficient at system level integration, have more hand-offs, and do not operate as a whole unit with the big picture in mind. 

Conclusion

In conclusion, there are many best practices and techniques that organizations can use to overcome hardware and software development challenges during an Agile transformation. The migration from departmentalization to cross-functionality can be accelerated using techniques such as value stream mapping, skill gap analysis, pairing, and the deployment of scarce skills at the scaled level.

Modifying Agile software development techniques for use in hardware can improve the quality of product and reduce time to market. The difference between software development and hardware development is intangible versus physical product, which can result in costly incremental design changes. Access to virtual modeling, 3D printing, and simulations allows teams to iterate and minimize hardware development expenses.

Customers and contractors can work together to develop the product roadmap and architect work in a way that enables teams to develop vertical slices of functionality. The inclusion of definition of done and acceptance criteria, and the use of user story narratives ensures that the work is completed with higher quality and fulfills all requirements.

Some of the greatest challenges in Agile Transformation surround team composition, product architecture, stable infrastructure, and customer involvement.

Agile Techniques for Hardware & Software Development Read Post »

Agile Manifesto

By / / , , , , ,
, , , , ,

Agile Alliance

On February 2001, seventeen people convened at the Snowbird Ski Resort in Utah to find common ground and develop an alternative approach to document-driven, heavyweight software development processes. The group included representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development and Programmatic Programming. They named themselves the “Agile Alliance” and created and signed the Agile Manifesto which became the cornerstone for the Agile movement.

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

Individuals and Interactions over Processes and Tools
Working Software over Comprehensive Documentation
Customer Collaboration over Contract Negotiation
Responding to Change over Following a Plan

That is, while there is value in the items on the right, we value the items on the left more.

Agile Streamlines Documentation

Historically, enormous amounts of time were invested in documenting the product for development and delivery. The list was extensive and included technical specifications, requirements, prospectus, interface design documents, test plans, documentation plans, and untimely approvals, resulting in bottlenecks and long cycle times. Agile does not eliminate documentation, rather, it streamlines documentation by providing the developer with just the essentials that are needed to do the work.

People Over Process

Valuing people more highly than processes or tools is easy to understand because it is people who respond to business needs and drive the development process. If the process or the tools drive development the team is less responsive to change and less likely to meet customer needs.

Negotiation vs. Collaboration

Negotiation is the period when the customer and the product manager work out the details of a delivery with points along the way where the details may be renegotiated. Collaboration is a different creature entirely, with development models such as waterfall, customers negotiate the requirements for the product often in great detail prior to any work starting.

This meant the customer was involved in the process of development before development began and after it was completed but not during the process. The agile manifesto describes a customer who is engaged and collaborates throughout the development process, making it much easier for developers to meet the needs of the customer.

Welcoming Change

Traditional software development regarded change as an expense and avoided it whenever possible. The intention was to develop detailed elaborate plans with a defined set of features and dependencies so that the team had a clear understanding of what should be developed and when it should be worked.

Agile welcomes changing requirements, even late in the development and uses rolling wave planning to decompose work just-in-time, without going into elaborate decomposition of the high-level plan.

12 Principles

The Agile Alliance also created 12 principles that support and expand on the four agile manifesto values. These principles have resulted in a number of widely accepted best practices that are used in many agile approaches such as extreme programming, scrum, DSDM, Adaptive Software Development, Crystal, and Feature-Driven Development.

The Agile Manifesto has spread like wildfire since 2001, forever altering the way we develop custom solutions and making impacts far beyond its original software origin.

Visual Representation of Agile Manifesto

Agile Manifesto Read Post »

Agile/Scrum Ceremonies and Metrics Useful in EVMS Variance Analysis and Corrective Action

By / / , , , , , , , , , , , , , , , , , , , , , , , , , ,
, , , , , , , , , , , , , , , , , , , , , , , , , ,
Agile Scrum EVMS

P. Bolinger, CSM October 2016
Humphreys & Associates

How can Agile/ Scrum be used to support EVMS Variance Analysis and Forecasting in a way that provides program managers with cost and schedule integrated information for no extra effort?

The discipline of EVMS and the Agile/Scrum practices have several touch-points that are covered in two major documents: NDIA IPMD Agile Guide March 2016, and PARCA Agile and EVM PM Desk Guide. Neither of these documents, as yet, drives to the level of specifics when it comes to best practices for use of Agile to support EVM Variance Analysis and EVM Forecasting.

Looking at the literature for Agile/Scrum, we know that there are recommended ceremonies that are conducted at various levels of the product structure and at different times during the project life cycle. These ceremonies are supported by many discussions of the metrics that can be collected at each ceremony and their potential use in managing the technical work of development within the Agile/Scrum framework. But where do these ceremonies potentially support EVMS Variance Analysis and Forecasting?

Now suppose we are presented with a control account that has exceeded the EVMS thresholds for cumulative cost and schedule variances. Wouldn’t it be great to have at our fingertips the underlying data from the process? In this case we might find, for example, that the Velocity is less than that needed to meet the end goal, the story cycle time is longer than desired, the pass/fail ratio is not favorable, too many team members have been absent in the last Sprint, the number of disruptions has been excessive, and the work to accomplish the stories is higher than estimated. It is not difficult to surmise the outcome would likely be a behind schedule and overrun condition in the EVMS. These data measures would provide the fodder for deep diving to the root cause and impact statements.

Where would we get that information?

Let’s start with the Agile/Scrum ceremonies. The particular Agile/Scrum ceremonies that we find conducted during the project are:

Backlog Refinement. This ceremony can be nearly continuous. It involves redefining the backlog of development work (scope), the prioritization or re-prioritization of that work, and potentially the assignment of responsibilities for the backlog.

Release Planning. This is a recurring ceremony aligned with the release cadence for the project. It involves establishing the capabilities and features of the product and when they will be released.

Sprint. This is a short time-defined effort to accomplish the design, code, and test of some subset of the product. The Sprints are controlled by the self-managing teams.

Daily Scrum or Standup. This is a daily (recommended) team meeting to discuss what has happened, what roadblocks exist, what is planned for the day, and other necessary items.

Sprint Review. The session in which the team products are demonstrated to the owner and self-off is accomplished.

Sprint Retrospective. A meeting of the stakeholders to discuss what went right (or wrong) during the Sprint and to define improvement actions that are needed.

The relationship of these Agile ceremonies with EVMS might look like this:

Agile/Scrum Ceremony

Agile Purpose

Relationship to EVM Variance Analysis, Root Cause Analysis, Corrective Action Planning & Follow-up and Forecasting

 

 

 
Backlog Refinement Manage, estimate, prioritize, and organize the product backlog in an on-going routine. Estimating – impacts the EVMS ETC and EAC as well as durations of efforts.
    Prioritizing – in response to issues is corrective action management.
    Organizing the backlog could be a form of corrective action effort.
     
Release Planning Establishing the contents and timing for releases of product. Updates could be part of corrective action planning in response to issues. Creation of new work packages. Changes to planning packages and SLPPs would also.
     
Sprint Short time-box performance unit. Work is done in Sprints. Below the work package. Short span measurement period is possible.
     
Daily Scrum and Stand-up Make short term plan, adjust to issues, discuss problems, clear roadblocks. Much of the daily action would relate to root cause analysis and corrective action planning although the time-frame is very short and the issues may be too small to individually impact feature work package or the Epic control account.
     
Sprint Review Demonstrate the product, update released work, make changes to product. This would relate to corrective action planning and follow-up. Issues would be found here that would impact risks, ETCs, corrective actions, performance metrics.
     
Sprint Retrospective Reflect on the project, progress, people processes, what was good, what was bad, and take actions to improve. This should be the richest source of supporting information for EVMS root cause & corrective action within the VAR realm. Very timely for variance analysis as it happens potentially many times during work package (feature) duration.
Feature Retrospective (not one of the basic ceremonies) Review situation regarding technical scope deficit. Reflect on the project, progress, people processes, what was good, what was bad, and take actions to improve. Because this only happens at the end of the feature it is limited in value for variance analysis timeliness. Any lessons learned can only be applied to future feature work.

But where is the meat? Where do we get actionable data or at least data we can analyze to decide what management efforts are required?

There are numerous potential metrics that can be collected during these ceremonies. These metrics can form the basic data set that could be analyzed to define the root cause of cost and schedule variances. In addition to isolating the cause of issues, within some of these ceremonies the impact of the issue on the Sprint, or Feature, or team may be made. Certainly, these metrics can be used as the basis for projecting future workload and performance.

The total number of potential metrics is not known. In this paper, we looked at 17 metrics and considered what the data might mean. The results of this review are contained in this matrix:

Metric

Type of Measure

Discussion
Sprint Burn up/Burn Down Backlog Value of backlog remaining for Sprint. Decrease is expected when work is done. Increase means work increased. Burn up can include total completed plus remaining; a great metric.
Feature Burn Up/Burn Down Backlog Value of backlog remaining for Feature. Decrease is expected when work is done. Increase means work increased or shifted.
Customer support requests received. Disruption Number of instances. Unplanned interruptions by customer can lower the output of the team if excessive.
Disruption measures Disruption How many and what type (except customer support requests). Higher disruptions impact team efficiency.
Estimate Accuracy (Sprint or Feature) Estimating Measure of budgeted value (estimated value) for the Stories in the Sprint or Feature versus the actual cost (calculated cost) of the Stories when done. Related to team size.
Discovered work Estimating Emerging work discovered during the Sprint. Will translate to extra effort in future if adopted into backlog.
Exceeds WIP Limits Management If WIP limits are set on team or individuals then exceeding set limits will impact efficiency and output.
Retrospective Action Log Management Count of improvement actions listed in Retrospective. Increasing count means issues are not being resolved.
Attendance Management Comparison of actual hours worked by team compared to baseline expectations in plan.
WIP Productivity Measure of the number of stories or points in WIP at any time. WIP growth can indicate bottlenecks and inefficiencies.
Velocity Productivity Measure of the amount of work (Stories or Points) accomplished during a time period. Higher velocity means greater throughput per person/team.
Stability measures Productivity Comparing Sprint by Sprint basic measures from this list. If there is high variability between Sprints in measures, then future is unpredictable.
% Tests Automated Productivity Higher automated testing should increase efficiency & decrease cycle time.
Defects found by team Quality Number of bugs reported during team effort. Measures quality of work. Higher bug incidence translates to lower output and higher costs.
Defects found by customer Quality Number of bugs reported by user/customer. Measures quality of work delivered. Higher bug incidence translates to lower customer satisfaction and higher rework costs.
Pass/Fail (Re-do) measures Quality How does the rate of success in testing compare to the number of attempts? High success rate should mean greater output and efficiency.
Cycle Time Schedule Time from start of a story to complete. Short cycle time is desired.

Let us continue the theme of the behind schedule and overrun control account and look at what information would be available for support to developing the estimate-to-complete. An updated and refined backlog would have the scope of work remaining for the control account. The updated release plan would have the timing for the deliveries to be made in the control account. The metrics collected about the effort expended per accomplished story or story point would provide a factor for projecting future real-work hours. Planned corrective actions and improvements would tell us how we might expect improvement to the quality and improvement in the speed or cost of work. The insights available from a full set of metrics are impressive.

Does a project have to collect all of these metrics? If not all then which ones would be the right ones? Questions like that would be answered by the project management team analyzing their prior experience and the particular challenges of the project. The team would establish a data collection plan, likely described in their Software Development Plan or Program Management Plan that would explain the metrics, meaning, and frequency along with their purpose. With a clear understanding of the technical data to be collected and analyzed, the Control Account Managers would not have a difficult task to define how they would use that data in developing Variance Analyses and generating well-considered Forecasts. In fact, these tasks should be much simpler with the data in hand.

Agile/Scrum Ceremonies and Metrics Useful in EVMS Variance Analysis and Corrective Action Read Post »

Scroll to Top