The Coad Letter: Modeling and Design Edition, Issue 84, Communication, Communication, Communication

By: Coad Letter Modeling Editor

Abstract: In this issue, we look at software development process and communication.

Communication, Communication and Communication

When we examine two developers writing a software application in their spare time we see little that we would call a formal process. However, when we examine a project with hundreds of developers distributed across multiple locations working to develop a large software system, process often seems to be all we can see.  Both examples do have process, but the first is much simpler and very informal.  It is most likely maintained wholly within the minds and interactions of the two developers who, over time, have learnt to communicate very effectively with each other.  In larger projects, the processes tend to be both much more visible and much more formal, although you will still find many small ‘processes’ in a large organization that are hidden, ‘understood’, or part of the tribal knowledge and not recorded anywhere. In fact the processes (not necessarily productive processes) that last the longest are those that become habit; ‘it’s just the way we do things here’. One goal for a process designer is to make their process straightforward to learn and remember so that following it becomes habit as quickly as possible.

So what are the fundamental differences between two developers writing software in a garage and thousand-person, multi-million dollar software development efforts?

Those who work in the real estate industry tell us that the three most important aspects of real estate are location, location and location. The software development process equivalent is communication, communication and communication. Communication is taking place constantly within a software development process at every level.  In fact, no process (work) can occur without it!  In failed projects, communication, or failure of it at some level is usually a major contributor to the project's downfall.

If we consider developers as nodes in a communication network all potentially linked to each other by communications channels, then there is only one channel between our two developers in their garage.

However, as we add more developers, the number of potential communications channels grows geometrically.

Between four developers there are six potential communication links. Between ten there are forty-five potential links and there are 4950 potential communication links between a hundred individuals on a team.

If not managed, there will either be too much time spent communicating and nothing will get done or there will be too little communication and results will not integrate and work together.

As a team grows larger, managing communication becomes increasingly difficult.
 

Language

Communication requires language. Within a development team different sub-teams use different languages. Some of these languages are textual, others are graphical and others are more mathematical in nature. There are languages for expressing requirements, defining interfaces, defining database tables, for communicating analysis and design models, for describing algorithms, for describing patterns, and a myriad of different programming languages for communicating instructions to a computer. Much of the work of a development team involves translating from one language into another.

… and then of course, to add to the mix, there are communication channels and languages required to communicate with customers, users and sponsors of the project.

Each time we translate from one language to another there is a potential to lose information or communicate the wrong information. Errors can occur when converting statements made during verbal interviews to structured requirements and diagrammatic models, adding documentation to the models, developing persistent storage models, and even creating the code. If information is not accurately transferred at each of these ‘interfaces’, we have the potential for building the wrong system. Therefore, we want to keep the number of translations of information to a minimum and to automate as many of the translations as possible to reduce the chance information is not lost.

To further complicate matters, at each step along the way, the people involved in transferring the information may not know what they need to communicate.  For example, the user defining the system may not know what it is that they really need.  They may end up describing task steps or symptoms of a larger problem without actually communicating the needed functionality.  So not only can we make errors in translating the information from one form or medium to another, we may not even be translating the right information! Therefore a good process uses numerous feedback loops to provide validation and verification as often and as early as possible.

Another big problem preventing communication is the fear of being wrong. If a user, manager, analyst, or developer is so scared of being seen to have made a mistake that they withhold information, clear communication is obviously compromised and the error is often compounded over time. It is right that we should have accountability within a project but our software development processes should encourage and support an environment of trust and responsibility rather than one of suspicion and fear of recrimination.

A software development process describes what to communicate, when to communicate, how to communicate and to whom to communicate (most process descriptions will not tell you why; this is normally left for someone to write a book about!).
 

Complexity

There is a limit to the amount of complexity mere humans can hold within their heads. As the size of a software system grows, the complexity of that software system grows ‘at least as fast as the square of the size of the program’ [Weinberg] and quickly outstrips the relatively fixed capacity of a human brain. Gerald Weinberg calls this law of software development the size/complexity dynamic.

The best strategy we have for dealing with such large problems is decomposition. We break the problem down into smaller and smaller problems until we have reduced the size of the problems to something we can manage. We then go about solving each small problem, integrating the solutions to form the solution for the bigger problem.

Given enough time our two friends in their garage could produce very large and complex software systems by solving one small problem after another. Adding more people to the team allows us to solve many of the small problems in parallel and deliver large systems quicker. The more people we add, the more we can do in parallel.

Here's the catch: the more people we have working in parallel, the more we are likely to bump into communication and coordination problems. If there are significant dependencies between the problems being solved, then we have to work hard at putting the right communication channels in place so everyone has the information they need at the right time.

So we are back to managing communication again.
 

Quality

According to Philip Crosby’s definition [Crosby], quality can be simply defined as conformance to requirements; how often the software behaves as required. The question as, Gerald Weinberg [Weinberg], points out is whose requirements?

We can talk of quality in software development having both an external and internal face. A user talking about the quality of system talks about the user interface, the response time, the reliability, and the ease of use of the system. A developer talking about quality talks about elegance of design, ease of maintenance and enhancement, compliance to standards, patterns and conventions.

These two faces are of course related. Low internal quality makes it much, much harder to maintain high external quality. This is especially true for incremental or iterative development where after the first iteration or increment we are building on what we built before.

Recognizing when a design for an iteration is good enough is a very valuable skill. If we make the design too simple we add to the re-work that we must do in later iterations when we need to extend that functionality and when time pressure may be more intense. If we select a more general and complex design we could be spending time building in flexibility that will never be used.

It often seems that developers have a natural complexity level; some developers always seem to make their designs too complex and other always seem to oversimplify. Recognizing these tendencies in themselves helps developer make the appropriate trade-off decisions, as does a good knowledge of the problem domain. Whatever the trade-off decisions, it is important that they are made visible to others on the team – that communication word again.
 

The Quality Spectrum

Simply splitting quality into external and internal views is really too simple a model; users and developers are not the only people with opinions about a system's quality. A software manager, will look at quality in terms of ease of maintenance and enhancement, compliance to standards and conventions, ability to deliver it on time

A project sponsor will look at how the well the system meets their business requirements.  Does it allow them to meet a constantly changing business environment and be proactive in meeting the challenges that are ever present in the marketplace?

Add to this the views of testers, technical writers, technical support personnel, and so on … and we need to look at quality is as a spectrum, with internal quality at one end, and external quality at the other.  Developers and software managers view the system more in terms of internal quality.  System users and business sponsors tend to view the system more in terms of external quality.
 

Building in Quality

A naïve view associates quality with testing of code and testing with a separate test team that goes to work once coding is complete. This model has severe problems.

It is a reminiscent of waterfall thinking. A waterfall approach to software development says do all the analysis first. Then do all the design. Then do all the coding. Finally test the system.

One of the most documented problems with a waterfall approach over the last couple of decades is that mistakes made early in the process are often not found until very late in the process. The cost in finding an error later rather than sooner varies from study to study but is always much higher.

“Observe that it is perhaps 100 times as costly to make a change to the requirements during system testing, as it is to make the change during requirements definition.” --Richard Fairley, Software Engineering Concepts, McGraw Hill, 1985
One way to ease this problem is to split the project into iterations or increments so that the distance in time between analysis and test is reduced. If we test something earlier we will find the problem earlier and the cost of fixing it is reduced.

A complimentary approach is to use practices that increase quality throughout the four activities. In other words, broaden the concept of quality so that it is more than just the testing of running code. Inspections are one such technique. Audits and metrics tools are another.

People typically want to do ‘quality’ work.  Each individual developer has his or her own idea about the acceptable level of quality of a system and that is usually approximately the best quality that they have achieved in the past.
It is better to ask those developers who have a lower idea of quality to reach for a higher standard than to ask those with higher standards to reduce their ideas of acceptable quality. Our self-esteem is linked to the quality of what we produce. If we are consistently forced to produce what we consider to be low quality work, we will lose self-esteem, low morale, and as a result, lower productivity. On the other hand, if recognized as such, asking developers to produce a higher quality product than they would naturally do actually enhances their self-esteem … if they can do it.

Even if an organization standardizes on a level of acceptable internal quality, it may well be lower than that of the individual developers’ ideas of acceptable quality. So at the beginning of a project, the development team needs to agree on what is an acceptable level of internal quality and what is not. It, obviously, cannot be lower than that of the organization (except in unusual circumstances) but may be set higher. This level of quality is made public in published design and coding standards and enforced primarily by some sort of design and code inspection (automated source code audits and source code formatters help enforce code naming and layout standards).

In iterative, object oriented software development where we want to reuse results, we need to ensure internal quality is built in early so that we have quality to build on later. If we allow low quality results at the start we will find ourselves in a vicious cycle of low quality resulting in more low quality.

A word of caution: Be careful not to confuse quality with optimization. Great results can be made from combinations of sub-optimal parts. Optimizing a small part may make no significant difference to the whole.  An over complicated, bug-ridden part, or and incorrect implementation of a requirement, on the other hand, can be the cause of many problems in other parts.
 

Process Pain and Relief

In January 2000, a number of people including Peter Coad and Jeff De Luca were sitting in a room discussing the purpose of a software development process. The result of that discussion was a statement similar to the following:
“The purpose of a software development process is to enable and enforce the repeatable delivery of working software in a timely and cost effective manner supplying accurate and meaningful information to all key roles inside and outside a project with the minimum of disruption to developers.”
When a process emphasizes only enforcement, it stifles creativity and innovation and causes frustration for the developers who feel their ideas are considered of no worth. On the other side of the coin, when a process emphasizes only enabling, creativity and innovation are allowed to reign unbridled causing frustration for managers and team leaders because everyone is doing their own thing and results do not integrate.  This is a constant balancing act. Any process should be monitored and reviewed for effectiveness and value, and should be modified when necessary to maintain its value to the organization.

Similarly, having hundreds of pages of steps to execute in an over specified process, demoralizes the team members, to the point that they switch off the minds and blindly follow the steps. At the other extreme, process descriptions that are too vague, abstract or general cause each one of us to make up our own way of doing things and work harder than necessary to achieve the desired results.

The main mechanism for communication within the development team and between a development team and its client is often large documents. In a process that emphasizes the production of documents, care must be taken that the focus is not put on recording results when it should be on getting the results right; that emphasis is put on achieving quality results instead of quality formatting of results.

If the overhead of producing progress and status reports is too high it will not be done well and the information they contain is more likely to be inaccurate or irrelevant. Without accurate and meaningful information, managers either have no useful feedback loop from the software process or even worse, late or inaccurate feedback. Managers cannot take appropriate action if they do not know what is actually happening on a project. Worse still, they may take inappropriate action that increases rather than reduces the pain.
 

Summary

A good process is well bounded; it provides enough structure to guide innovation and creativity, constraining it to appropriate times and places.

A good process clearly defines tasks.  The tasks are focused on results without specifying minutia so that we progress efficiently and yet still have the freedom to adapt to unusual events or changes in circumstances.

A good process produces accurate progress and status information for team leaders, project managers, upper management and the client, while minimizing the impact on the developer's time.

A good process quickly becomes a matter of habit rather than a set of conscious acts. Developers do not want to have to consult a 3000-page process manual every time they reach a new step in an activity.

A good process helps a team maintain quality and manage complexity .

A good process optimizes communication within and outside the team.



[Crosby] Philip B. Crosby, Quality is Free, The Art of Making Quality Certain, Penguin Books U.S.A 1980
[Fairley] Richard Fairley, Software Engineering Concepts, McGraw Hill, 1985
[Weinberg] Weinberg, Quality Software Management vols. 1-4, Dorset House 1992-97

©2001 Giraffe Productions


Published on: 3/13/2002 12:00:00 AM

Server Response from: ETNASC04

Copyright© 1994 - 2013 Embarcadero Technologies, Inc. All rights reserved.