The Coad Letter: Process Edition , Issue 104, Goal-centric Process and Project Management, by Jon Kern

By: Coad Letter Process Editor

Abstract: In this issue, we'll look at a process paradigm that I coined a year or so ago.

Promises, Promises...

Most development projects have a sense of what needs to be built, and the time frame in which the finished product is desired. Pretty simple, eh?

Most development projects also miss both of these goals. But, we all knew that line was coming. (I "set you up" on that one.)

Coming from the Department of Defense (DoD) professional engineering services world (1986-1995), I had been an early user of project planning tools (DOS-based Harvard Project Planner might have been my first "real" tool). I learned about PERT charts, and Work Breakdown Structures (WBS), and Gantt charts (always hard to spell!). I also became quite adept at recognizing the underlying subtlety: there is actually an algorithm running behind the scenes of these graphical tools!

To make customers and managers happy, I would back up my engineering project estimates with carefully laid out project plans. From the end goal, I would work backwards with an assortment of tasks, breaking down the various work steps (hence, the term WBS). Turning these activities into a network of dependencies (that is, Task 27 must preceed Task 42), applying estimated durations, and adding any desired milestones (for example, preliminary design review), I formed the PERT chart (also known as, Network Diagrams, Precedence Diagrams). From a point of view of Critical Path Management (CPM), most tools were able to adroitly produce the ever-popular Gantt chart, making some tasks serial while others were able to be run in parallel. The Gantt chart resulted in showing the tasks as a function of calendar (usually "Days from Start" for estimates, and actual dates for live plans). I even began to get fancy with lead-lags, ALAP (versus the default of ASAP), resource leveling, and baselining.

An interesting lesson I learned was that many people wanted a Gantt chart for their project plan, but did not want to adhere to the rules of the PERT/CPM algorithms. Folks just wanted a "pretty picture." This happened so often that I found a simple Gantt tool that these folks could use to just draw the picture.

These always looked great in a proposal! Multi-year wall charts. Amazing to be able to predict out the future to such a degree! Whoo-hoo!

Goals and Process

In virtually all business pursuits--including application development--there are desired goals. These are usually higher-level abstractions, and are of more interest to the stakeholders than to the average project team member. Also, in virtually all business pursuits--including application development--there are (or at least should be) desired best practices (processes) to achieving the goals.

Goals--which play a crucial role in business processes--can be thought of as follows:

  • Use Case
  • User Story
  • Feature
  • Enhancement Request
  • Defect Report
  • Expense Report
  • Hiring a new employee

Processes can be things like:

  • Unified Process uses Use Cases as goals
  • Extreme Programming (XP) uses User Stories
  • Feature-driven Development (FDD) uses Features
  • Software Change Request (SCR) process uses Feature Enhancement Requests
  • Defect Tracking process uses submitted defects as a goal (to fix <g>)
  • Expense Report Process handles Expense Report submissions
  • New Employee Process kicks in whenever a job offer has been accepted.

In these types of goal-driven processes, the goal is the leading entity around which resources and tasks and plans are set into motion. It is also the basic entity by which progress can be measured easily. In FDD, a list of prioritized features are developed early on and used to guide the tasking and scheduling of the team. In the SCR process, the Change Request is either a new feature or a bug, or is for current release or a future release--all of which combine to determine when the team will be working on the request. In XP (like FDD), the User Stories are prioritized and used to closely steer the team's efforts in small time blocks.

Estimating Software Projects

For some reason (likely my familiarity with a given tool and process), I began my first software projects in the same manner that I had learned to do engineering projects: attempting to predict the tasks that I would do to build out the application. I would assign costs to resources, and resources to tasks, providing a cost roll-up as a byproduct of the project plan. Unfortunately, as software grew more and more complex, this technique began to be less and less effective. That is, the classic dilemmas that all software projects face these days--for example, ill-defined requirements, changing requirements, dynamic business environment--began to make the relatively static project plan difficult to maintain, and woefully inaccurate.

In a typical software development Gantt chart, the large majority of the tasks are of no interest to the client, but the milestone dates are important. Conversely, the detailed tasks are likely of great interest to the development team. I began to notice this dichotomy, or duality of purpose, in the project plan's usage. I also did not like the non-adaptive nature of a project plan. That is, it was not easy to provide the following:

  • Stochastic variabilities
  • Branches, or different paths
  • Automatic project updates by the "doers"

The first element applies to basic project estimation rules. One often has a best case, most likely case, and a worse case scenario--some type of statistical variation on the basic plan. This can even be applied at the low-level task, as needed, and the results reflected in the higher-level roll-ups of tasks. Most Gantts that I have seen guiding projects just stick with a single set of deliverable dates.

The ability for a project to have "if-then-else" logic is typically not available in the form of a Gantt, yet almost all processes have decision points with 2 or more branches.

The lack of "auto update" of task status requires manual labor to update the project. MS Project Server now offers a means for a team to keep the project and tasks up to date. (I have not tried it, only read about it.) Naturally, if you want to use a Gantt to show status, you can likely do a simple ROI calculation on the cost of maintaining the project's accuracy versus the cost of the Project Server system.

The following figure shows a sample process with some decision blocks:

Activity diagram with activities and a decision block and 2 branches.

Figure 1: A sub-process for determining if a requirement (user story) meets the needs of the process.

If your process contains branches (such as in Figure 1), how would you provide an estimate or use a Gantt representation? If your development process does not contain branches, I'd love to hear about it!

Gantt-based Estimation

Have you ever done a +/- 30% estimate? Ever apply it to a Gantt (fuzzy task durations)?

How accurate are the far-term tasks on any Gantt you have seen?

In general, if you have to make a Gantt for estimation purposes, I recommend the following:

  • Remind the recipients that:
    • It is only an estimate
    • It is not fixed in stone
    • It is subject to change
  • Remind the team that:
    • It is only an estimate
    • It is not fixed in stone
    • It is subject to change
    • It is probably wrong the day after it is published
  • The further out in time that you must plan, the less detailed you should be. That is, you can be more confident planning near-term tasks (this month) than tasks that you think you will do in one year.
  • Don't get too hung up on massively intertwined dependencies unless they are absolutely irresolute. (Of course, if you think there are lots of dependencies--and you are doing software development, maybe this is a sign that something is wrong in the architecture or general approach?)
  • Try to show only a high-level Gantt with major milestones and phases. Try to get the stakeholders to agree to a more agile process for managing the application development project.

Simplified, Goal-centric Estimation

Using a goal-centric approach for software development, you can follow XP and FDD style of project planning. First determine the basic high-level schedule goals (usually the stakeholders have an initial desired goal--it may or may not be realistic, time will tell). Then determine how long your iterations will be--for example, say you decided to build a usable version of the product every 3 weeks, and deliver a beta release in 6 months, with a version 1.0 release one month after that. Then, you can do the following with each feature request in your list of features (also, use this process as new feature requests arrive):

  • Create a rough-order-of-magnitude estimate of duration.
  • Ensure the estimate is within reasonable bounds (see Figure 1).
  • Work with the customer to determine priority for the feature.
  • Drop the feature into the proper release "bucket." (Some features go into the "way out in a future release" bucket.) See Figure 2. Basically, you may want to consider "Must Have" features that will make up a Release 1.0 product release (and you may have a beta milestone and some alpha milestones). Then, you can add "Nice to Have" features for Release 2.0, and so on.
  • Create a simple spreadsheet to record the feature, release, iteration, and time estimate.
  • Assign a cost per unit time and do the math for the first major milestone or release. You can plan more than one release in advance, but you must remember the accuracy drops off as things may change.
  • You need to have an idea of your project team at this point. You will need to be able to think in terms of how many units of development time your team can deliver each day.
  • Estimating a calendar target date might be a bit more difficult using this method at first. You should map out any initial startup efforts, and you should do some rough dependency mapping. That is, if strong dependencies exist--either software dependencies, or resource dependencies--this will force these tasks to be done in serial. You should take into account your team size to determine approximately how long it will take to complete your first release.
  • For iteration planning (for example, a three-week length), you work in reverse--that is, you keep the time fixed, and attempt to manage to deliver a specific number of features. As I have maintained for the past 10 years, it is better to slip features than miss dates.
  • Your near-term project plans look simply like a list of Goals (features, in this case).

Handling a new story.

Figure 2: High-level view of initial handling of a new requirement.

In general, paying attention to only the scheduling of the "Goal" (feature, user story, use case) per unit of development time, you can simplify your project estimation tasks.

Tracking the Project

If you haven't had the joy of working with a Gantt-style project plan that has thousands of tasks, well... consider yourself fortunate. The challenge of assigning each task is tedious (as it is often a manual process to map a project team member to the task). Once the project tasks are started, updating the status of each task is yet another highly tedious task. What about working on a task that needs to be done yet was not thought of when the plan was laid out? A progressive project manager will allow the project to be amended. However, sometimes, when the cost of making such a project change is too huge, the "unforeseen" tasks often just get done and not recorded--to the detriment of all:

  • Other tasks don't get the resource on time (but it isn't clear why),
  • Actual effort that went into the project goes unrecorded (so project-based reporting is inaccurate),
  • It is unlikely you will remember to apply this "missing" step in a future project that uses the same process.

Of course, to a large degree in a software project, the tracking of fine-grained tasks is not nearly as important as the completion of tangible results. For example, the completion of a feature or a user story such that customer sign-off is now required. The completion of each "goal" is what is important.

Reporting Progress

The Gantt chart is a very visual reporting tool loved by many. Indeed, it has an excellent reputation for being an easy-to-read view of the project's timeline. Gantt charts are an excellent means to show task planned versus actual, actual time spent versus estimated duration, and so on.

However, in the spirit of eXtreme Programming and Feature-driven Development (et al), it is sufficient to anchor the reports to the status of the stories or features. From the stakeholder's perspective, a simple tabular listing of features and their current state, plus a trend chart showing planned application feature growth versus actual, will provide an instant view of the status of the overall project.

In Goal-centric Project Management, project status is comprised of a simple list of goals and their status. For example:

  • Feature,
  • scheduled,
  • status,
  • est. duration,
  • actual duration
  • actual start & end (optional)

Simple!

Naturally, it is also pretty clear that one could take this information and present it in the form of a Gantt chart. It just so happens that--unless you record actual start and stop times on a given feature--the Gantt chart won't be too complex in terms of serial tasks.

Summary

If you are struggling with driving your development process with a traditional MS Project/Gantt-style planning mindset, why not try Goal-centric Process and Project Management on your next iteration?

As always, your comments are appreciated.

In the next issue, I'll describe how you can run with a repeatable process and incorporate a Goal-centric approach to Project Management.

-- jon


Server Response from: ETNASC01