You are hereA New Chore for Sisyphus / Introduction
The following quote was taken from a book review of The Mythical Man-Month1 that was published on the rather edgy Internet site known as Slashdot:
What's bad about this book is that people don't read it. There's no particularly good reason, they just are too busy reading the latest book on today's fad. The irony, of course, is that today's fad will be laughed at next year, while MMM will still be around a decade from now...2This is by no means the most recent review of The Mythical Man-Month and that, in itself, is an interesting phenomenon. In an industry that considers something from last year to be hopelessly obsolete, a book that was published over thirty years ago still has much to offer. As the quoted review states though, unfortunately, not enough people have read The Mythical Man-Month. Based on my experience in various software development roles, I have to agree. My career as a software development professional started in 1980 at what was then TRW Defense and Space Systems Group (DSSG)3. TRW DSSG was then at the forefront of large scale software development such that projects spanning several years with hundreds of developers were considered normal. Although I don't have a source other than a hazy memory of a conversation, at the time, TRW DSSG claimed to be the third largest producer of computer software (after the U.S. Government and IBM). Dr. Barry Boehm was the company's Chief Software Engineer and it was during this period that he wrote Software Engineering Economics4 and created the Constructive Cost Model (COCOMO). Given this background, I professionally “grew up” in an environment where The Mythical Man-Month was considered gospel. After leaving TRW in 1992, I worked at several different non-defense companies in a variety of software development roles. I initially found release from the high-ceremony, highly regulated development methodologies of the aerospace and defense industries refreshing. Over time, I noticed that only very small projects seemed to be successful without the structure of a well-defined software development methodology. As project size and complexity grew, the same freedom that allowed a small project to be completed quickly became just an excuse for flailing and failure. Attempts to point out that methodologies existed for engineering larger efforts were met with the derisive response, “There isn't time for that.” My experience with “non-methodology” software development culminated in the “death march” described in Chapter 1. This experience ultimately led me to write this book. Behind this simplistic, “there isn't time for that,” attitude, there seemed to be a common set of fallacious assumptions. The scariest part was these assumptions had been debunked in The Mythical Man-Month over thirty years ago. Attempts to point out this out weren't met with any sort of well-reasoned response. Instead, my suggestions were “refuted” by pointing out that The Mythical Man-Month was over thirty years old and, therefore, hopelessly obsolete (see the quoted review, above). Pushing the issue would only elicit an ad hominem attack that usually included comments about “mainframes” and “punch cards” or worse. Pointing out that the principles in The Mythical Man-Month fully explained the most recent development failure was no more successful. These efforts, at best, transferred the ad hominem attacks to whoever was unlucky enough to have been chosen as the designated scapegoat for the last software development disaster. What seemed to be lacking was a chain of causality that directly linked the practices of the previous effort to either its inglorious outcome or the current problems. This book attempts to fill this gap by providing this chain of causality. In order to do this, I will concentrate on the central themes of both the eponymous essay of The Mythical Man-Month and the two essays added for the twentieth anniversary edition, “No Silver Bullet – Essence and Accident” and “No Silver Bullet Refired.” In addition to ideas raised in these essays and the remaining essays of The Mythical Man-Month, this book draws heavily from Peopleware by Tom DeMarco and Timothy Lister5 and Death March by Ed Yourdon6 for supporting arguments. My first goal is to explain how software development projects are governed by a set of fundamental constraints rooted in both human nature and the nature of collaborative problem solving. Primary among these is that, for projects above a certain level of size and difficulty, the nature of the effort required changes from one of simply implementing a computer program to one of managing the implementation of the project. This transition occurs when the size of the program outgrows what a small homogeneous team can successfully implement. Both the nature of the problem (e.g., the need for highly specialized groups within the development organization in order to solve specific aspects of the problem) and the capabilities of the individual team members have a huge impact in determining when this threshold is crossed. Because of the profound effect individual abilities can have on projects, there is no single, simplistic answer as to whether a particular task can be accomplished by a certain organization within a specified period of time. Although what can be said is that, as the size of effort grows, the more likely it is that organizational and industry productivity norms will win out over individual ability. For larger projects, the cost and achievable schedule for a particular development effort will, at best, be determined strictly by the size and nature of the program to be developed. My second goal is by far the most important to both the organization performing the work and to those who will be using the program that results. I will show how attempts to ignore or circumvent this change in the nature of the effort required have a profound impact on the fundamental quality of the product. Forcing a project into an abbreviated schedule, even with supposedly sufficient developer and technological resources, will rarely result in the specified product being completed within the desired schedule. Forcing projects onto too short of a schedule will have long term quality repercussions that go far beyond just causing burn-out and stress for the development team. As the team attempts the impossible, accepted software-engineering practices are discarded since they generally require more of the one thing that isn’t available: time. This lack of a disciplined development effort initially shows up in a lengthy integration effort but the repercussions continue to haunt the development team and their product. It is this relatively hidden but lasting effect of attempting to do too much in too little time that is the ultimate cost of the folly. Thus, the size and complexity of the problem determines the minimum feasible schedule required for solving the problem. Attempts to force a project to completion in less than this minimum feasible schedule will have serious consequences for the ultimate quality of the product that results. These consequences will go far beyond superficial coding errors and will also limit the ability of the development organization to provide future product revisions. This causal relationship can best be exposed by looking at the practices that become common in such an effort, the resulting software development process and the consequences to the product of the development organization using such a “process.” Thus, this book starts with a study of the “death march” or “constant crisis” (think death march but on a continuing basis) as software development processes and the implications that using either process has for the final product. Finally, this book will look at what an organization can do since the schedule pressures for developing new capabilities are typically not within an organization’s control. Here I will look at both existing attempts to solve the problem and propose an alternative approach. In particular, the applicability of “agile development” methodologies (See the discussion in Chapter 6 for a better characterization of agile development methodologies) will be considered. This will include specific attention to how such methodologies differ from the so-called death march and the types of software development problems these methodologies can and, more importantly, cannot solve. The non-linear nature of effort and schedule required for larger projects will frequently not be understood by the technical staff tasked with carrying out the effort; let alone by their non-technical counterparts in sales, marketing or upper management. No rational software project manager will cut resources and especially schedule duration for a project beyond a certain point. This is exactly what market pressures as expressed through management edict force the development manager to do. It is my hope that by showing how such an effort has the effect of “eating their seed corn” that a more rational alternative will be examined and tried. Such fits of desperation indicate a deep lack of competence in any manager who demands such an endeavor. I’ll somewhat excuse the poor soul who is the recipient of such a demand as long as they now point out the consequences of attempting the impossible. The ideas and concepts presented in this book can be summarized as:
- Big, complex problems are difficult to solve and, thus, take a long time to solve.
- Attempts to force an early solution to such a complex problem will not achieve the desired schedule and will have continuing consequences in the form of deep rooted bugs and instabilities.
- While it is not possible to significantly accelerate the process of solving big, complex problems, some projects are just a collection of small problems that individually aren’t hard to solve. Such projects can usually be developed relatively quickly if attacked with an appropriate methodology.
- Many software projects or products consist of a few, difficult pieces plus a collection of easier pieces.
- The optimal development strategy is to recognize the ease or difficulty of each piece to be developed and then apply the appropriate development methodology given this understanding of the nature of the effort required for each piece. This approach will not make the solution to the hard pieces available any sooner. However, this solution will not be plagued with the bugs and instabilities that result from attempting to develop such a solution under too short of a schedule.
1Brooks, F. P., The Mythical Man-Month: Essays on Software Engineering. Reading, MA.: Addison-Wesley 1995 (Anniversary Edition). Reference as TMMM.
3TRW's defense and space related business sectors went through a number of reorganizations while I was employed there. The company was eventually acquired by Northrop-Grumman and the non-automotive businesses became Northrop-Grumman Mission Systems. The automotive related sectors were then “spun off” by Northrop-Grumman and continue to do business as TRW, Inc.
4Bpehm, B. W., Software Engineering Economics. Englewood Cliffs, NJ.: Prentice-Hall, 1981.
5DeMarco, T. and Lister, T., Peopleware: Productive Projects and Teams. New York, NY. Dorset House Publishing 1000 (2nd Edition). Reference as Peopleware.
6Yourdon, E., Death March. Upper Sadle River, NJ.: Prentice Hall Professional Reference. 2004 (2nd Edition). Reference as Death March.
7W. W. Royce described the classic waterfall development methodology was in “Managing the development of large software systems” Proceedings of IEEE WESCON, August 1970.
8See Jacobson, Booch, and Rumbaugh, “The Unified Software Development Process”, 1999
This work is copyrighted by David G. Miller, and is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.