Agile Values, Principles, and Practices

Applies to: Agile

Authors: Greg Smith and Ahmed Sidky

Get this book in Print, PDF, ePub and Kindle at manning.com. Use code “MSDN37a” to save 37%.

This topic contains the following sections.

• Is Agile Just Another Process?
• A Paradigm Shift from a Plan-driven Mentality
• Copyright

The tragedy started when the crew accidentally bored into an adjacent, abandoned mine that was flooded with water. The miners’ map told them, incorrectly, that the abandoned mine was hundreds of yards away. The men scrambled to reach the exit, but the rising water blocked the way out. Their only option was to seek out the highest point in the mine.

Word of the accident spread above ground and a rescue team was formed. The rescue team estimated where the crew was located in the mine and picked a spot to drill. Maps revealed a gas line that ran close to the target drilling point; and if their coordinates were incorrect, they might rupture the line and create an explosion. Being careful to avoid the gas line, the team began drilling a small, exploratory hole. After 90 minutes the drill broke through the wall of the tunnel, and the rescuers listened anxiously for any sounds from the miners. After minutes of sobering silence, the rescuers could hear the trapped men pounding on the drill bit with their hammers. The miners had been located. Now the challenge was to get them out of the mine before hypothermia set in.

The rescue team outlined a two-part plan. First, they would drill additional holes to help pump water from the mine. Second, they would use a “super drill” to create a 2-foot-wide escape tunnel for the miners. The drilling work began without a hitch, but then the super drill bit broke 105 feet below the surface. A special “fishing” tool was needed to extract the bit. In the past it had taken 3 days to build such a tool. The rescuers knew they did not have 3 days to get to the miners out.

Rescue workers contacted Frank Stockdale, the plant manager at Star Iron Works, and asked him to build the tool they needed. They faxed engineering prints to Stockdale and explained the dire situation to him. Using his 95-member machine shop, Stockdale was able to reduce a 3-day job to 3 hours. The rescue team then removed the broken bit and resumed drilling the rescue tunnel.

Finally, 78 hours after the tragedy began, the drill penetrated the shaft and the drill operator shouted with joy. The last miner was pulled to safety 5 hours later from the Quecreek Mine in Somerset County, Pennsylvania on July 28, 2002. After being trapped 240 feet below the surface, and with body temperatures as low as 92.5 Fahrenheit, they would all make full recoveries (see Figure 1.1).

Figure 1.1 A Quecreek miner is rescued from the flooded mine after spending 3 days crouched in waist-deep water. (Photo courtesy of the U.S. Department of Labor.)

You may wonder why a software development book starts with a story about a mining rescue. If you’ve performed Agile software development previously, you’ve probably identified the parallels. Let’s look at a few.

All software projects have constraints. Similar to the situation during the Quecreek rescue, the number-one constraint is frequently time. The Quecreek rescue team had a few days to reach the miners. Software projects are often limited to a few days, weeks, or months, after which they’re of no value. Like a Sunday newspaper delivered on Monday, all the quality work and effort invested in the project are worthless if you don’t meet your most critical priority.

The rescue project also had a clear vision of the primary project priority: to reach the miners while they were still alive. A secondary priority was to reach them before they got hypothermia, and a third priority was to reach them before they started losing consciousness due to hunger. The team focused on delivering the number-one priority first.

When you perform software projects, you can lose track of your priorities, things can get muddy, and low-value work can hold up project delivery. Agile software development asks you to follow the Quecreek model by identifying what is critical and focusing on delivering to meet the critical need as soon as possible.

Quecreek also reinforces another Agile tenet: you should expect change, you should embrace change, and you should be ready to plan and adapt frequently. The Quecreek rescuers adapted to broken drill bits, gas lines blocking their path, and the need to reduce the time required to create a fishing device. In software development, you encounter similar situations. You discover a missing requirement, you identify a technical constraint that prevents you from following your initial design, or a third party delivers their part of the project later than expected. These types of issues happen on every software project; and to ensure success, Agile asks you not to be surprised but to continue to perform by adapting to the reality of the situation.

Finally, the Quecreek rescue demonstrated goodwill and collaborative team work. Ideas came from all team members, such as the suggestion to try positive air pressure to keep the water at bay. Goodwill and collaboration were also demonstrated when the rescue team approached Frank Stockdale and asked if he could create the fishing tool. Stockdale didn’t ask the rescuers to spend days creating a contract and going through legal papers; instead, he trusted the rescue team and quickly delivered the fishing device.

Agile development depends on this type of relationship with customers and vendors. You want a vendor who is a partner, not a vendor who is considered the enemy because you spend more time talking about contracts than ensuring the delivery of value.

In this chapter, we’ll help you understand the need for Agile practices, what Agile really is, and how Agile contrasts to plan-driven development practices. We’ll conclude the chapter by discussing the most important consideration when pursuing a development process: how does Agile correlate to the most common corporate goal of increasing revenue and reducing expenses?

Is Agile Just Another Process?

Many people may think that Agile is just another software development process. Although that is true to a degree, there is a lot more to Agile than just a process or just a set of practices. Agile (or agility) is more of a mindset—a way of thinking about software development. This Agile mindset can be applied to any process using any set of practices. The best way to illustrate our understanding of Agile is through Figure 1.2.

Today the market is moving quickly, and as a result, the software development lifecycle needs to be flexible enough to enable organizations to seize new and emerging market opportunities before their competitors do. To reach the desired ability to respond to constant change, your software process needs to focus on what is truly important.

Similar to the way you pack light when you’re going to backpack around Europe, your process needs to be light. You need to increase everything in the process that adds value to the end goal and decrease everything that doesn’t add value. Agile values attempt to highlight what adds value in a software development process.

Figure 1.2 The relationship between Agile values, principles, and practices

The Agile Manifesto and Related Values

In 2001, a group of authors wrote a document called the Manifesto for Agile Software Development, with a goal of identifying the values that yield the most benefit to a software development process. Let’s look at the manifesto, which is available online at http://agilemanifesto.org:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

When people first read the manifesto they immediately agree with the stated values or they hesitate. The hesitation usually comes from the perception that an Agile methodology throws away the items on the right (processes, tools, documentation, contracts, and planning). This is completely false. The manifesto is saying that the items on the right do add value to the development process but the items on the left (interaction between individuals, developing working software, and so on) provide more value to the process. The manifesto is trying to point out that organizations traditionally put a huge emphasis on the items on the right, such as processes and tools, and neglect the items on the left, such as the interaction between individuals. An Agile mindset promotes the re-emphasize that an Agile process can and sometime should contain some of the items on the right; but you need to make sure that each of those items adds indispensible value to the project.

The Agile Principles

Moving to the layer that surrounds Agile values in Figure 1.2, let’s consider the Agile principles. The Manifesto for Agile Software Development defines a set of 12 principles that represent the characteristics or inherent traits of an Agile process:

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. As obvious as this principle may seem, it’s often violated in traditional software development. It’s important to remember that customers are asking you to deliver working software that adds value; they don’t want a prototype or a set of documents. The earlier you can start delivering working software, the earlier you can begin satisfying your customer.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage. Your customers are competing in a dynamic market, and therefore they may have to change the requirements for their software in order to gain a competitive advantage. It is important to note that you should welcome changing requirements, but no one said this change is free.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Have you ever shown your customer software for the first time and received no feedback? In most cases, you receive feedback—sometimes minor, but usually major. The trick is to deliver software early so that you can get feedback early. This early feedback can save you re-work down the road.

4. Business people and developers must work together daily throughout the project. This principle is careful to say business people and not the customer. In most cases, it would be impractical to work with the customer on a daily basis; but generally there are multiple business proxies. These proxies may not know everything about the customer’s wants and needs, but they usually know more about the business needs than the developers do. These proxies may be analysts, product managers, or program managers. The key is to maintain constant communication between the developers and the business people to ensure that the project never goes off track—not even for a day.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. Remember, people aren’t resources. Software development is different from manufacturing. Software development is more of an art. Project teams need to be motivated and trusted. If you have motivated team members they will find a way to give you their best; and that’s what an Agile process needs—everyone’s best.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Instant messaging or the telephone should never replace face-to-face communication. A lot of context is lost in communication over email and instant messaging—not to mention the fact that ambiguity increases with nonverbal communication. Face-to-face communication also lets you run with less formal documentation.

7. Working software is the primary measure of progress. If you recall, the customer is primarily interested in working software. So why would you measure progress in terms of anything else? Today, the progress of most software development efforts is measured in terms of their plan. When requirements are complete, the managers say the project is 30 percent complete. In a plan-driven world, this may be correct; but in a value-driven world, where the value is the working software, the project is 30 percent complete when 30 percent of the required functionality is coded, integrated, tested, and deployed. This is a fundamental difference between the Agile value-driven world and the traditional plan-driven world.

8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. In traditional development, the team often has to work late toward the end of a project, although at the beginning of the project they may have taken 2-hour lunch breaks. This is primarily due to the way project activities are distributed across the project’s lifecycle. There isn’t much for developers to do at the beginning of the project, but at the end everything is put on their shoulders because of tight delivery schedules. With Agile development, you deliver every two weeks or so, and development begins with the first iteration. Efforts are distributed more consistently throughout the project lifecycle, which leads to a constant development pace for the team.

9. Continuous attention to technical excellence and good design enhances agility. A successful gymnast needs strong muscles. Similarly, technical excellence is an essential enabler for a truly Agile software development process. For example, extensible designs and architectures make it much easier to build the product in an evolutionary manner. Automated testing frameworks are needed to ensure that refactoring one part of the system doesn’t affect other parts. Continuous integration is essential if you want assurance that your software is working after every change.

10. Simplicity—the art of maximizing the amount of work not done—is essential. No code means no bugs. The more code you write, the more bugs your code may have. If something isn’t essential to the product, then don’t build it. Some developers tend to develop massive underlying frameworks and infrastructures in the system under the assumption that those elements may be needed in the future. The key is simplicity: try not to develop anything that isn’t essential to the features you’re developing now. Remember, the more time you invest in anything, fact that you don’t need a piece of code or that you need to change it.

11. The best architectures, requirements, and designs emerge from self-organizing teams. In traditional software development, analysts write requirements, and architects lay out the architecture of the system. Then the requirements and architectures are communicated to the team in a document. In the Agile world, we encourage teams to self-organize. True self-organization involves giving the whole team the task and asking them, as a team, to complete the task without specifying who should do what—they’re left to self-organize. It will naturally occur that architects will lead the discussion when it comes to architecture, but now everyone is free to challenge them and suggest new ideas that may enhance the architecture the architects would have come up with on their own. This form of collaboration also increases the knowledge transfer within the team.

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. We believe this is probably the most important principle of agility. The idea of always reflecting on what you’re doing and trying to Figure out better ways to do things is the essence of continuous improvement. Without continuous improvement, people and organizations remain at a status quo. If you adopt only one thing that will make your process better, regularly reflect on your process as a team. You need to identify what you’re doing well and continue doing it, and you need to identify what you’re doing poorly and improve it.

The Agile Practices

The last layer in Figure 1.2 represents the Agile practices. These are activities that are used to manifest or implement the Agile principles and values. There are numerous Agile practices, such as user stories, test-driven development, pair programming, daily stand-up meetings, and so forth. But no specific set of Agile practices is defined—it’s anti-Agile to say that there is a defined set of practices and that no new practices can be created. Organizations create different Agile practices or tailor existing Agile practices to address specific organizational or team needs. Teams may also need to be creative and come up with new Agile practices to achieve agility while adhering to organizational constraints.

Known Agile development methodologies like Extreme Programming (XP), Scrum, Lean, and Feature Driven Development (FDD) consist of a set of Agile practices that have a certain synergy. Some methodologies, like Scrum, focus more on Agile practices related to project management; others, like XP, focus more on technical Agile practices. No methodology is better than the other; it all depends which works best in your environment and within your constraints. Better yet, in this book, we won’t talk about a certain methodology: we’ll talk about a generic set of Agile practices and then show how you can customize the practices to fit your organization.

A Paradigm Shift from a Plan-driven Mentality

Traditionally, once a project starts, a requirements package is created and then is “signed off.” The project manager assumes that this sign-off results in a fixed set of requirements and that now planning can begin. The project manager estimates how long it will take to complete the requirements and creates the project plan. The plan predicts that the project will be finished by a certain date, and that date is communicated back to the customer.

The fundamental flaw in this approach is that the plan, which drives everything, is based on an assumption that the requirements are fixed and won’t change. Experience has shown us that this is never the case; requirements are never fixed—they always change. When the requirements change, the plan is affected; and as a result, the completion date needs to change too. Unfortunately, in many cases, that is impossible, and the team has to deliver by the date they committed to. This is when a major crisis occurs and the project starts to go out of control.

The value-driven Agile approach switches the whole mindset. It assumes from the start that whatever requirements exist up front are not fixed and that they will change. The Agile mindset also assumes that you have to deliver by a certain date. This approach fixes the time and resources and leaves the requirements undetermined. To us, this approach more closely resembles the reality of creating software. Now the whole notion of value-driven makes perfect sense. When you have a fixed amount of time in which you aren’t sure whether you can deliver all the requirements (because they will change and hence the time needed to finish them will change), the natural reaction is to prioritize the requirements and finish first those that add the most value to the customer.

You may be thinking, “What about the requirements that aren’t finished by the delivery date?” That is the reason you use the value-driven approach. You acknowledge the fact that not all of the requirements will be completed by the delivery date. The important question to ask is whether you have delivered enough features to support a system that provides value to the customer.

Figure 1.3 shows an interesting finding from a study by the Standish Group. Only 20 percent of the features in a system are often or always used; 45 percent of the features are never used. Another study showed that when a new system was installed at DuPont, only 25 percent of the system’s features were really needed. The important point we’re trying to emphasize is that if you can deliver, say, 35 percent of the features by the delivery date, you may be giving the customer all the value they’re looking to attain from the system.

Figure 1.3 A study by the Standish Group indicates how often features are used in a typical application.

The traditional plan-based approach isn’t flawed in and of itself; it just isn’t suitable for today’s software industry. The plan-based approach was originally based on traditional project management concepts, which originated from the construction industry. In the construction industry, the plan-based approach is suitable: the blueprints, which are the requirements, are fixed and probably won’t change while the building is being built. You can estimate how long it will take to build the steel pillars, pour the concrete, and so forth.

The reason why the traditional plan-based approach is suitable for the construction industry but not for the software industry comes back to the difference in the way we control empirical systems (like software development) and the way we control defined systems (like construction or manufacturing). Table 1.1 shows the differences between the characteristics of a defined process and those of an empirical process.

Table 1.1 - Comparison between a defined process and an empirical process

Predictable manufacturing (defined process)	New product development (empirical process)
It’s possible to first complete specifications and then build.	It’s rarely possible to create up-front, unchanging, detailed specs.
Near the beginning, you can reliably estimate effort and cost.	Near the beginning, it isn’t possible to reliably estimate effort and cost. As empirical data emerge, it becomes increasingly possible to plan and estimate.
It’s possible to identify, define, schedule, and order all the detailed activities at the start of the project.	Near the beginning, it isn’t possible to identify, define, schedule, and order activities. Adaptive steps driven by build-feedback cycles are required.
Adaptation to unpredictable change isn’t the norm, and change rates are relatively low.	Creative adaptation to unpredictable change is the norm. Change rates are high.

After reading the Table, it’s easy to see that software development is definitely an empirical process, not a defined process. The problem is that we’ve been approaching software development for years as a defined process—and that approach doesn’t work.

Previous article: Moving to Agile

Continue on to the next article: Agile and the Bottom Line

Copyright

©2009 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.