Estimation and Measurement Services Version 1.0 (EMS 1.0) Part 0: Primer

Abstract

This document describes the Open Services for Lifecycle Collaboration (OSLC) Estimation and Measurement Services Version 1.0 (EMS 1.0), a set of REST services and resource representations that enable the integration of software estimation tools with other tools used in the software development lifecycle. This document is a non-normative primer which is intended to serve as an easily read introduction to the concepts and formats that form the normative specification. For full details see EMS 1.0 Part 1: Resources and Services.?

Status of this Document

This document is a working draft. Please send comments about this document to the public community@open-servicesNOSPAM.net mailing list or add comments to the EMS 1.0 Part 0: Primer wiki page.

Table of Contents

• Introduction
• Initiating a Project

Introduction

The goal of the EMS 1.0 specification is to enable integration between software estimation tools and other tools used in the software development lifecycle, especially project management tools. In this context, a project is a time-bounded work effort that produces some unique result. The development of a major or minor release of a product or application is a good example of a project. The job of the software development manager is to steer the project towards completion on time, within budget, with good quality, and delivering the required capabilities.

It is common knowledge within the software industry that managing software projects is difficult. Each project is unique and is often complex. The underlying technology changes rapidly, the business environment is dynamic, and requirements alter during the course of the project. Project managers need to be able to accurately assess the progress and health of projects. Simply asking developers for their status is often insufficient to accurately judge the true state of the project. This is where estimation and measurements come in. Project managers augment the verbal status reports from the development team with quantitative measurements on the project and the artifacts being produced. These measurements are then compared to the estimates to determine if the project is progressing as planned. If the measurements disagree with the estimate, the project manager diagnoses the root cause and takes corrective action to steer the project back on course, or, in the absence of a feasible corrective action, informs the project sponsors that the project is off course and then replans the project.

The remainder of this primer illustrates the features of EMS 1.0 using project management scenarios. The examples used are highly simplified in order to convey the underlying concepts as clearly as possible. The full details of the resources and services are described in the normative specification, EMS 1.0 Resources and Services.?

Initiating a Project

Project initiation is the first phase in the lifecycle of a project. This is where the project comes into existence in order to satisfy some set of business requirements. A project might create a new product, enhance an existing product, or perform maintenance on it. The project requirements define the desired capability of the software to be developed. Let us assume that the business benefits associated with the requirements is understood. The first task of the project manager is to establish the project plan, including the resource requirements, cost, schedule, and risk. The plan is combined with the business value to form a business case. If the benefits outweight the costs, and the risk is acceptable, then the project is feasible and is likely to get approved.

Software estimation tools enter here by taking the project assumptions as input, and producing a plan as output. Consider the following highly simplified model. In this model the project inputs are size and duration. The output is effort.

Size Assumption

The size of the project measures how much capability is produced by it. We therefore need a way to convert the business requirements into a size. We also need a way to measure the size as embodied by the resulting software. In this model we'll use the count of source lines of code created or modified in the project. This metric is referred to as effective source lines of code (ESLOC). ESLOC is certainly measureable, but it's relation to business requirements is less clear. A much more plausible measure of size is function points. Fortunately, experience has shown that there is a reasonable correlation between functions points and lines of code for a given programming language. We can therefore translate the business requirements into function points and then apply a suitable gearing factor to produce an ESLOC value as the project size.

Rather than continuing to talk in the purely abstract, we'll use a fictional project, Tsunami 1.0, being planned by BrainTwistors? Corp. Tsunami is a Japanese logic puzzle and BrainTwistors? is a US-based game website that generates revenue by selling advertising for game stores. BrainTwistors? Corp. is planning to create a web version of Tsunami since this will drive traffic to the website and increase advertising revenue. Future plans include developing free iPhone, BlackBerry? , and Nintendo DS versions, and charging for puzzle downloads. The BrainTwistors? Corp. game designers have come up with a design for Tsunami 1.0 and have worked with the software architects to translate it into function points and then ESLOC. The size assumption for Tsunami 1.0 is:

size = 50,000 ESLOC

Duration Assumption

Duration is the elapsed calendar time of the project. In general, duration may be either an input assumption, or an output of the estimation process. Duration is certainly measurable and may be measured in days, weeks, months, or years.

For Tsunami 1.0, time to market is critical since BrainTwistor? Corp. needs to drive more traffic to its website in order to achieve its advertising revenue target for the coming fiscal year. The project must be completed within six months. The duration assumption is therefore:

duration = 6 months

Project Portfolio Management Tools

Since businesses always have limited financial and human resources, it is a best practice to apply portfolio management techniques to select the best ways to invest these resources in new projects. Project portfolio management tools enables proposals for new projects to be prioritized based on their business value, alignment with strategic goals, and risk.

BrainTwistors? Corp. uses the web-based PfGenius? project portfolio management tool. PfGenius? lets businesses create and prioritize projects proposals, and review the execution status of approved projects. PfGenius? has built-in workflow capability to enact the project portfolio management processes. A BrainTwistors? Corp. product manager, Pedro Mendelzohn, has created the proposal for the Tsunami 1.0 project in PfGenius? , quantified the business benefits, and added its size and duration assumptions. Pedro predicts that Tsunami 1.0 will generate 500,000 USD of additional advertising revenue if shipped in six months. The business benefits are therfore:

benefit = 500,000 USD

The next step in the project initiation process is to estimate the effort and cost. In PfGenius? , Pedro changes the state of the Tsunami 1.0 project proposal to Requires Software Estimate. This state transition sends a notification to the person in the software estimator role, informing them that they need to work on a proposal. Fortunately, PfGenius? implements the EMS 1.0 specification so it can easily integrate with software estimation tools, making the job of the software estimator very simple.

Software Estimation Tools

The most reliable way to estimate new projects is to base the estimate on similar past projects. Software estimation tools use predictor models that are calibrated on a set of past products, either from a standard industry pool or from the projects performed by the business itself.

BrainTwistors? Corp. uses a very simple software estimation tool, Guestimator 101, which also implements EMS 1.0. Guestimator 101 is a desktop application intended for software development organizations that use a single implementation technology, that use teams composed of a uniform combination of experienced and novice developers, and that execute projects within a relatively small range of sizes and durations. This makes project execution fairly predictable. In fact, Guestimator 101 simply maintains a database of past projects with their size and effort measurements, and models the relation between them as:

size = P * effort

P is the productivity constant for the organization. Guestimator 101 does a statistical analysis of the historial project database and computes the value of P that fits the data the best.

Estimating Effort

In EMS 1.0, the tool that contains the project assumptions is the service provider and the tool that computes the estimate is the service consumer. The consumer sends a GET request to the provider to retrieve the project assumptions. The consumer then computes the estimate, typically under the interactive control of a person skilled in software estimation and the use of the software estimation tools. When the estimate is complete, the consumer sends the estimate in a POST request to the provider.

At BrainTwistors? Corp., Syd Ethan is a veteran software estimator and a power user of Guestimator 101. He receives the workflow notification from PfGenius? saying that the Tsunami 1.0 project proposal is ready to be estimated. Syd copies the Tsunami 1.0 project proposal URI from the notification, launches Guestimator 101 on his desktop, invokes the Open Proposal command and pastes in the URI when prompted by the dialog box. Guestimator 101 then GETs the URI and displays the project assumptions to Syd. Syd then estimates the project. The Guestimator 101 historical project database yields a productivity of 2,000 ESLOC/person-month:

P = 2,000 ESLOC/person-month

Guestimator 101 uses the value of P calibrated from its historical database, the project size assumption obtained from the Tsunami 1.0 project proposal, and its model equation to calculate the effort for the project:

effort = size / P
= (50,000 ESLOC) / (2,000 ESLOC/person-month)
= 25,000 person-month

Syd is satisified with this result and invokes the Send command in Guestimator 101 to send the estimate back to PfGenius? . Guestimator POSTs the result to an estimate URI obtained from the previous GET response. Syd launches PfGenius? in his web browser and verifies that the estimate was received and looks correct. He then changes the Tsunami 1.0 project proposal into the Software Estimate Done state. This state transition causes a notification to be sent back to Pedro so he can complete the business case.

Completing the Project Initiation Process

After the software estimate is done, the development cost can be computed, e.g. by applying a labour rate to the effort estimate. The business value of the project is computed by subtracting the cost from the benefit. The business value is used as one criterion in the project priorization process. Strategic alignment is also considered when prioritizing.

BrainTwistors? Corp. uses a standard labour rate of 150,000 USD/person-year:

rate = 150,000 USD/person-year

The development cost is therefore:

cost = effort * rate
= (25 person-month) * (150,000 USD/person-year) / (12 month/year)
= 312,500 USD

The business value is the difference between the benefit and the cost:

value = benefit - cost
= (500,000 USD) - (312,500 USD)
= 187,500 USD

The business value is well into the positive range. Furthermore, the proposal aligns with the BrainTwistors? Inc. strategic goal of growing the business. During the prioritization process, the Tsunami 1.0 proposal ranks high and gets approved. The proposal is then sent into a software project management system for execution. Project execution will be covered in another scenario.

Summary of Resources and Services

The following resource types and REST services are used in this scenario. The format of the resources will be discussed later:

ProjectProposal?

this resource type describes the Tsunami 1.0 project proposal. The software estimation tool GETs this resource from the project portfolio management tool. It contains the project assumptions and a SoftwareEstimateCollection? URI where software estimates can be POSTed.

SoftwareEstimateCollection?

this is a collection resource type which is a property of the ProjectProposal? resource type. The software estimation tool POSTs SoftwareEstimate? resources to this URI.

SoftwareEstimate?

this resource type contains the software estimate for the proposal. The software estimation tool POSTs this resource to the SoftwareEstimateCollection? URI that was obtained from the ProjectProposal? URI.

Risk

Aside from using a highly simplified predictor model for effort, the above scenario omitted the important aspect of risk which we'll now discuss. Although the business value of the Tsunami 1.0 proposal looked good, it was based on several assumptions which were themselves predictions or estimates. Investment risk is a measure of the uncertainty in realizing business value. Since the assumptions were uncertain, so is the estimated business value. In order to make sound investment decisions we need to quantify the risk and demand a higher rate of return for risky investments.

The approach to quantifying risk is to view the estimated business value not as a single number, but as a probability distribution. A probability distribution gives us the probability that a measurement will have a value in any given range. For example, the size estimate might have a probability distribution that gives positive values for measurements that range from from 30,000 ESLOC to 100,000 ESLOC. This would imply that the actual size will be greater than 30,000 ESLOC but less than 100,000 ESLOC.

Probability distributions are also referred to as random variables. When a random variable is repeatedly measured its values define a probability distribution. The risk of a probability distribution is its variance which is a statistical measure that quantifies how spread out the values are. If the variance is zero then there is no uncertainty or risk. The measurement always has a definite value.

The above scenario must therefore be modified to include probability distributions for the size, effort, and benefit. These then combine to produce a probability distribution for the business value.