According to the Department of Defense (DoD) Earned Value Management (EVM) Implementation Guide [EVMIG 2006] EVM is defined as follows: “EVM is a program management tool that integrates the work scope, schedule, and cost parameters of a program, in a manner providing objective performance measurement and management. As work is performed, the corresponding budget value is ‘earned’.” Currently, an EVM System (EVMS) is required for DoD contracts of more than $20 million and, for most DoD contracts of $50 million or more, the company’s EVMS must show validation, which is a formal recognition of certification by an independent party that a company’s EVMS meets certain guidelines. However, EVM can (and in many cases, should) be used on other DoD contracts and software projects in industry. As the articles in this issue will demonstrate, there is a lot of “value” in using EVM on projects.
Basically, EVM uses three parameters to assess the health of a project at any point in time: Planned Value (PV), Actual Cost (AC), and Earned Value (EV). Figure 1 illustrates two of these parameters: PV and AC. PV is the budget assigned to the work scheduled to be accomplished. In EVM, PV is usually estimated in the beginning of the program or project, and is assessed at various points in time, such as at project milestones. For example, in Figure 1, the three month milestone could be a requirements review for a software project, and the six month milestone could be a design review. At the scheduled end of the project (12 months, in this case), the work will be finished, and PV will be the Budget at Completion (BAC), the budget for the entire project ($1 million, in this case). At selected points in time, where PV is identified, the Actual Cost (AC), the amount actually spent, can be compared to PV. At the end of Month 3 in this case, PV is $200K and AC is $230K.
Figure 1: PV and AC Only
What often occurs now is that program managers and other stakeholders draw conclusions from comparing these two values. It would appear in this case that the project is overrunning by $30K, and we may be in trouble. If the AC was lower than the PV, it could appear that money is not being spent fast enough. In fact, some projects have been cancelled because of “insufficient spend rates” where AC was lower than PV, and a conclusion was (wrongly) drawn that the programs were not managed well. Such isolated comparisons of AC and PV are misleading, however, since we do not know how much work was actually accomplished. The value of the work accomplished to date is the third parameter, Earned Value (EV). Figure 2 shows our project chart with EV added for a third reference point. It shows we not only accomplished more than we planned to, but more than we actually spent to date!
Figure 2: EV, PV, and AC
The frequency of assessing project health depends on factors such as the size and complexity of the project, project risks, the development environment and project duration. For smaller projects of short duration, one might choose to monitor EV bi-weekly or monthly. The goal is to select a frequency that will provide visibility into cost and schedule issues while there is still time to resolve potential problems.
Once the three parameters, PV, AC, and EV, have been determined, quite early in the project schedule, some computations can be made to assess the health of the project. Two variances, cost and schedule variance, are first computed as follows:
Cost Variance (CV) = EV – AC
Schedule Variance (SV) = EV – PV.
For the example above, CV is +$20K, and SV is +$50K. Positive variances usually mean a project is doing better than expected, and negative variances indicate a worse-than-expected situation. To determine how good or bad the project will be if current trends continue, two other computations can be made:
(Cost) Estimate at Completion = Budget at Completion x (AC/EV)
(Schedule) Estimate at Completion = Planned Schedule x (PV/EV)
Here, the Cost Estimate at Completion is $920,000, or an $80,000 under-run. Schedule Estimate at Completion is 9.6 months, or 2.4 months ahead of schedule.
Of course, the assumption is “if current trends continue”; there is no guarantee of continued success. More often, one or both variances are negative, and the project manager may propose a “get well plan” for maintaining control of costs and schedule while accomplishing the work. The real significance of this EVM approach lies in the fact that these estimates can be obtained very early on in the project. It is well established that at 20% into the project these estimates are quite accurate because project productivity seldom increases and typically remains stable over the life of a project.
It should be noted that, as explained by Flemming and Koppelman, EVM provides an early warning signal to project management that a project may be in trouble; however, use of EVM will not in itself prevent problems caused by poor management, inadequate funding, or scope creep [Flemming and Koppelman 2005]. Also, use of EVM does not tell a project manager how to resolve variances; EVM only indicates that the variances exist.
One challenge to software project managers is that a lot of earned value examples are from hardware projects, and involve measures like units produced versus units planned. It is easy to track EV because each unit can be assigned a value and then it becomes a simple task of counting to derive the EV at any point in time. The challenge for software is to break down the scope of work into elements that can then be allocated an appropriate portion of the total budget. The accuracy in doing this determines how successful our EVMS will be in monitoring the state of the project and alerting us to potential cost and schedule problems. One must also define the earning rules up front so that the EV is based on objective criteria, rather than subjective judgment. For example, if a portion of the budget is allocated to requirements analysis, perhaps the criteria for crediting the EV would be based on the number (percentage) of requirements traceable to design, as opposed to a subjective judgment that requirements analysis is, say, 40% complete. There are various schools of thought regarding when EV can be taken. In agile development EV can only be taken when the software feature or functionality is actually implemented in the software and fully tested. There is no partical credit given for design. The article by John Rusk discusses the rationale for this rule. Since the 1990s, some viable measures of EV have been suggested for software. Some of these measures were suggested from an Air Force Institute of Technology thesis effort [Ayres and Rock 1992], and later published as part of a journal article [Christensen and Ferens 1995].
They include:
1. Requirements and Design Progress: The number of planned versus actual requirements completed and, later, the number of planned versus actual component designs completed. This could be useful in assessing PV and EV.
2. Code and Test Progress: The number of planned versus actual units coded and, later, tested. This also could be useful in assessing PV and EV, albeit later in the program.
3. Person-months of Effort Expended. This is useful in assessing AC.
Let’s say we use a cost estimating model to estimate the cost of a software project. (This could be a single project, the first iteration or spiral of an evolutionary development project, or a major task within a larger project), and the estimates from a cost model are as follows:
Requirements Analysis: 3 Months, $200K
Design: 3 Months, $300K
Code and Test: 6 Months, $500K
Total: 12 Months, $1M
These estimates are plotted in Figure 1 as PV at each schedule milestone. During planning for this project, we also determine we should have 40 requirements defined by the end of Month 3, and 60 components designed by the end of Month 6. For this example let’s assume that all requirements and components are roughly equal and we will assign a value of $5K for each requirement defined and, later, $5K for each component designed. In a real world project this process of assigning value is much more complex. Our plan indicates that at the end of Month 3, we can expect to spend $200K and have all 40 requirements defined. Suppose we then use the “Person-months of Effort Expended” measure to assess AC which, in this example, is $230K.
Let’s say that, at the end of Month 3, we have all our requirements defined ($5k x 40 = $200k), and we have designed 10 components ($5k x 10 = $50k). As shown in Figure 2, the total EV for the project at that time is $250K. We have done $50K more work than we had planned, even though we have spent $30K more than planned. The results are positive variances in both cost and schedule. If we want to find out why we are doing so well, there are various measures we can consider. In reality, the project manager would investigate the reasons behind the variances and be able to explain the causes; however, since positive variances indicate a healthy project, the manager would probably not spend a lot of time investigating unless one or more of the variances drops to a negative value.
Today, there are many resources available for software measures that can be useful in EVM, either as direct measures of PV, AC, and EV, or indicators that can be used to help explain cost and schedule variances. The Practical Software and Systems Measurement (PSM) Guide [PSM 2003] includes more than 50 measures that may be useful (including EV itself), and Capers Jones [Jones 2008] describes a multitude of potentially useful measures. Practitioners [Christensen and Ferens 1995] also describe key measures for explaining variances.
This is not to say that Earned Value for software is an easy undertaking. An organization must be committed to tracking EV measurements, as well as AC. Also, PV for software requires accurate software size, effort, and schedule estimating, which is not always easy to do (See STN Volume 11, Issue 3). A good work breakdown structure (or some other work decomposition method) is needed to determine what each separate software project task entails and how much for the overall time and effort will be needed for it.
The articles in this issue will help motivate readers to use EVM, explain how EVM can be used for software projects, and provide methods to use EVM. The first article, by Wayne Abba, one of the world’s best known experts in EVM, examines Government Policy for EVM, from its inception in the 1960s to the present. He comments on how government policy shifts affect the value of EVM as a management tool. The next two articles address use of EVM for software projects. The article by Hunt, Solomon, and Galorath shows how to specifically address EVM for software projects, including using Solomon’s widely-published Performance-Based Earned Value methods. The article by John Rusk, a software architect from New Zealand, shows how EVM can be applied to agile software development efforts. In the next article, David Bachman, the EVM curriculum director for Defense Acquisition University (DAU), explains how EVM artifacts and estimates are used in DAU’s 12-Step Integrated Program Management Model for decision making, action planning and addressing program risks. This article includes a reference to an example for using the model on an actual program.
In addition to these EVM theme based articles, DACS presents an article (by multiple authors) that describes the SPRUCE initiative. SPRUCE, which stands for Systems and Software Producibility Collaboration and Experimentation Environment, is funded by the Office of Secretary of Defense (OSD) and is technically guided by the Air Force Research Laboratory (AFRL) at Rome, NY. It is an open web portal to bring together DoD software developers, users, and software engineering researchers virtually by enabling their collaboration on specifying and solving software producibility challenge problems from real-world scenarios.
References
[Ayres and Rock 1992] Ayres, Bradley J., and William M. Rock, “Development of a Standard Set of Software Indicators for Aeronautical Systems Center” (AFIT Thesis GSS/ENC/92D-1), Air Force Institute of Technology, Dayton, OH, September, 1992.
[Christensen and Ferens 1995] Christensen, David S. and Daniel Ferens, “Using Earned Value on Software Development Projects”. Acquisition Review Quarterly 2: 155-171.
[EVMIG 2006] “Department of Defense (DoD) Earned Value Management Implementation Guide”, Defense Contract Management Agency, October, 2006.
[Flemming and Koppelman 2005] Flemming, Quentin W., and Joel M. Koppleman, “Earned Value Project Management: Third Edition”, Newton Square, PA, Project Management Institute, 2005.
[Jones 2008] Jones, Capers, “Applied Software Measurement, Third Edition”, New York, McGraw-Hill Osborne Media, May, 2008.
[PSM 2003] “Practical Software and Systems Measurement”, Version 4.0C, Department of Defense and U.S. Army, March, 2003.
About the Authors
Dan Ferens works for the ITT DACS as an analyst and as an instructor for a 12-part series in software affordability which has been taught mainly to Air Force Research Laboratory (AFRL) scientists, engineers, and managers in Rome, NY. Dan retired from AFRL in early 2007 after more than 35 years of service to the Air Force as a military and civilian employee. Dan has been involved in software estimating since he became a civilian in 1978, both as an AFRL analyst and program manager, and as a Professor at Air Force Institute of Technology where he taught classes on software estimation and other software engineering and management topics for 13 years. He is currently an Adjunct Instructor at SUNY Institute of Technology in Utica, New York where he teaches a class in information technology project management. He is a life member of the International Society of Parametric Analysts (ISPA), where, in 1999, he received the prestigious Freiman award for lifetime achievements in parametric estimating. He is also a member of Toastmasters, International where he holds the rank of Distinguished Toastmaster. Mr. Ferens has a Masters degree in Electrical Engineering from Rensselaer Polytechnic Institute, and a Masters Degree in Business Administration from the
University of Northern Colorado. He and his wife, Marcie, currently reside in Fulton, New York.
Ellen Walker, is a DACS Analyst, currently developing a series of publications on software “best practices” as part of the DACS Gold Practice Initiative. She has spent the past 25 years as a software developer in various roles spanning the entire software life cycle including project management of multiple business process re-engineering efforts within the DoD community. She is also experienced with assessment initiatives such as the Capability Maturity Model for Software (CMM-SW) and the quality management practices of the New York State Quality Award program. Ellen has an MS in Management Science (State University of New York (SUNY) at Binghamton), and bachelor degrees in both Computer Science (SUNY – Utica/Rome) and Mathematics (LeMoyne College).
Author Contact Information
Email: Daniel Ferens [daniel.ferens@itt.com]
Email: Ellen Walker [ellen.walker@itt.com]
DACS Gold Practice Initiative
ROI Dashboard
