Insight No.36 : Apr 2012 : What's Coming Around the Corner?Agile and Project Management: Better Togetherby Ned Schoenfeld
While Agile software development is incredibly useful in certain business scenarios, it also poses project management challenges. For example, how do you get Agile to fit well within the corporate environment? In this article, I'll provide the answer.
In most traditional organizations typical technology development follows the Waterfall methodology. That same model is also common in program and project management. In each of those disciplines you know exactly where you are today, and you know exactly where you want to end up. What you may not know yet is how to connect the dots from today to the future, which requires you to figure out how much time, how many resources, and what kinds of special tools or materials you'll need. That approach works as well for an IT project as it does for building an office building.
In the Waterfall model, everyone knows what you're aiming for, and so the goals are clear. Waterfall works best when you're trying to deliver an end-to-end, complex process. At the end of the project, presumably you'll have some kind of review to make sure that you've captured the intended benefits set out at the beginning of the project.
But what often goes wrong with Waterfall is that because projects tend to be very long and linear, you sometimes find out somewhere in the middle that you misjudged the complexity or resources required or didn't fully understand the nature of the entire problem. Therefore the solution often changes from what you originally envisioned or the realization of the benefits somehow falls apart late in the project. Somewhere in the middle of the work, the project has gone off the rails, and you have to spend time doing triage, trying to figure out how to salvage the project or even shut it down before it costs a bundle and doesn't deliver what the organization expected.
The Quick Impact of Agile
Agile promises a potential solution to that scenario. The Agile concept is this: You are no longer aiming for a specific end state - you are now aiming for a minimum return on investment (ROI) You don't really need to worry so much about exactly where you're going to end up as long as each iteration you go through with this program delivers incremental ROI. If it turns out you end up in city B instead of in city A, that's just fine, because that just means that city B's ROI started to look more achievable along the way.
Sounds simple, right? Agile is effective when it's used wisely in an organization. But this development approach also runs the risk of making everyone busy, constantly seeking a maximized ROI. They may never find it, and before long you've got a small disaster on your hands. You've gone through 10 or 15 iterations with Agile and found yourself in just as bad a shape as you would have been in Waterfall. So you've wandered aimlessly through these iterations, not achieving the ROI you wanted. Even worse, because you didn't really have many goals outlined at the beginning of the initiative, it's going to be less clear about where you are or what to do for triage to fix it.
The Missing Ingredient: Agile Project Management
The antidote is to apply some Agile project management. But first, let me lay the scene.
The success of any Agile development initiative (Waterfall, too!) has to start off with a clear business case. While that business case will be somewhat different from one developed for a Waterfall methodology, it still needs to lay out principles and goals the team is looking to achieve. It also needs to limit the areas in which they'll do exploration during each iteration to make sure they're granular and focused.
An "iteration" in Agile is a complete development project - planning, requirements definition, design, coding, unit testing, and acceptance testing - that can be performed in a short timeframe; because an iteration typically lasts only three to four weeks, the scope is quite small.
For example, perhaps you're looking for a new algorithm for equity trading or a new truck routing solution for your organization's shipping process. Those are narrowly defined, highly specific problems that exclude a lot of possible dead ends. They're also prime targets for Agile development.
So you'd lay out in the business case who the key stakeholders are, how the decision making around each iteration will work, and what the expected ROI is that you're going to try to achieve with your development work over the next few weeks.
Then you'd use traditional project management tools to assess how you're doing within that four-week timeframe at achieving your ROI target. With Agile, it becomes pretty easy for the team, which is collaborating closely already, to figure out after three or four days whether they're heading in the right direction or running off the rails.
Now the Agile benefits start to surface, since you're going to waste very little money getting sidetracked, because everyone's monitoring something that's relatively simple and discrete and much less complex than a full Waterfall engagement.
You adjust the direction of your development "ship" every few days. With a Waterfall program, you might meet once a week or even once a month to collect status. You'd have a team meeting maybe every two or four weeks. But in Agile it's feasible to have a brief team meeting of all stakeholders - all of those people involved in the process - every two, three, or four days to make sure you're staying on track and still heading toward all of the clearly defined goals laid out in your business case.
The role of the project manager in this environment is to be part of the Agile development team and to provide rapid and insightful dashboarding as well as encourage communication among stakeholders. That project manager will follow up with each stakeholder to find out whether they're working on what they expected to be working on (based on the business case), where they are in that process, and how close they are to completion. Perhaps more importantly, the potential of ROI realization for this iteration can be discussed with each stakeholder, too.
From these responses the project manager is in an excellent position to decide whether the project is on target and on budget and to share those findings with the participants and stakeholders. If too many concerns are being raised by the stakeholders, the project manager has the opportunity to call an emergency meeting for the team and stop the iteration in its tracks until the stakeholders have had an opportunity to communicate and resolve their concerns.
Because Agile is built on collaboration, team members are going to be more than eager to jump in and participate with each other, to try to work through a bottleneck, and come up with a new plan. That allows the project plan milestones to remain constant, while the work continues dynamically to meet the challenges as they're uncovered.
Applying Project Management to Agile
Compared to the Waterfall model, Agile is lightweight and rapid. So while much of the same project management expertise is going to be brought to bear in a development project, they're going to need to follow a rapid cycle time.
Dashboard updating will occur at least once a day so everyone knows where everything stands. Project meetings will take place as and when necessary - certainly in cases of emergency, but in any case at least twice a week, to ensure that the iteration always starts and ends on time and has an ROI benefit. Adherence to these practices ensures that you won't "lose" an iteration since it's still being tracked.
If your company has a program management office (PMO), it can look at the artifacts being produced and the results being achieved by this Agile team, which is following a business case and is using its dashboard and meetings to guide its multiple iterations. The PMO can tie its work into a successful program management framework that's connected to the rest of the organization that might still be working with a Waterfall development methodology.
Agile and Waterfall, Working Together
Yes, the types of milestones and cost benefit analyses are a little different in Agile from Waterfall. But each Agile iteration becomes a deliverable. You'll be able to measure ROI as each iteration completes, much the same as you can measure cost and benefit along the phases or tasks of a Waterfall methodology. But the result is that those two approaches can come together and sit within an overall project management office or PMO function in the larger organization.
The organization then consists of a portfolio of multiple Agile development teams and multiple Waterfall development teams, both of whose output can be pulled together into a common PMO function in order to give the organization control over all of its development resources and all of its deliverables.
What does this look like on the ground? Let's say you work for a large supply chain operation that sells multiple items through a popular website. The overall transactional process includes advertising products that people click on to buy, which brings up a payment screen and a shipping screen. Once the purchase is complete, the order is transmitted to a warehouse for fulfillment. Most of the work involved in getting that process up and running would probably fall within a Waterfall development methodology.
Agile could come into play by allowing you to take any one of those little pieces out and brainstorm better ways for solving the specific problem through an Agile development methodology. An example might be development of a new way of handling purchasing by existing customers so that it can be done with a single click. The Agile team might end up creating eight or nine possible solutions to the puzzle - each created in a short duration.
Senior leaders would have the chance to look at the potential solutions, select one, and then build out around that. Future iterations would focus on improving the new idea. All of those results deliver good organizational value and avoid tackling that big end-to-end transactional process puzzle in the relatively expensive Waterfall approach.
Keeping Agile in the Organizational Fold
A failing in some organizations is that they allow the Agile teams to go their own way, operating separately from the PMO process and from other Waterfall developers. In these cases Agile-developed projects are separated from any kind of formal organization-level program management and removed from portfolio visibility. Executives can no longer make portfolio decisions that encompass Agile development, which is, of course, a tremendous disadvantage since Agile often delivers the fastest results to an organization.
By making sure that Agile projects are part of the portfolio, company leaders have an easier time balancing those few long-term, big-risk Waterfall projects with a collection of short-term Agile projects. Your portfolio return can look fantastic from both a financial and a delivery perspective.
Interestingly, such an outcome can give an organization a lot of confidence. People see a lot of numbers being put on the board all through the year on a regular basis, which can be highly motivating. Staff who are working on the short-term work are seeing quick results; and they know those long-term things they're relying on through the Waterfall methodology are also moving along, and everyone's working towards a common goal, which, of course, is what any organization wants.
Agile Isn't Limited to Software Development
Agile projects aren't exclusive to software. Any endeavor involving a complex process that includes highly focused problems could apply Agile magic. I've seen advertising firms use traditional program and project management to stay on top of proposals for new clients. If there's some aspect of that proposal that demands new thinking - how to do analysis of social media results or exploit some other unique media opportunity - an Agile team could be put in place to come up with potential solutions.
The same is true for manufacturing to address a discrete aspect of prototype development for a new product - such as the user interface for a new console. The multiple iterations produced by Agile at the beginning of the process could be useful to put in front of focus groups, so they can help the company select which panel is going to be most effective and most likely to encourage purchasing.
Pcubed has found that integrating the use of Agile methods into any existing project infrastructure requires a few essentials: First, you need to make sure you have a strong business case up front that clearly defines the goals of the Agile team. Second, your program and project management infrastructure needs to be sturdy enough to absorb the unique on-the-run reporting demands of Agile. Third, to jumpstart your efforts, we advise you to seek out a trusted partner - like Pcubed - with experience in both types of projects to coach your people in making the transition into these new ways of working. Agile can be a powerful addition to the toolset you use for delivering your projects - no matter what realm they fall in.
Ned Schoenfeld is a Managing Director at Pcubed based in the New York office. He has held executive positions at UBS, Citibank, and other financial services and healthcare organizations. His personal interests include new technologies, military history, photography, and leadership. Contact Ned at ned.schoenfeld@pcubed.com.