Over the last number of years the phrase “Agile Project Management” has become quite popular. You can do a search on the phrase and find many definitions. All of them seem to be a sort of mashup of Agile practices and existing PM methods.
Perhaps sounding like a great idea, how well do these approaches mesh? Do they form a product greater than the sum of its parts? Read on to see why I believe this is detrimental for software development.
Incompatible Core Beliefs
In decades of work in the software development domain, from developer to management, I’ve worked with many Project Managers and PMOs.
During that time I’ve come to believe that the primary, core belief of Project Management is that, with enough research and analysis, the future becomes largely knowable. In other words, predictable.
Thus Project Management approaches include long, in-depth phases for upfront work such as research, analysis and design. These will almost certainly include formal reviews and sign-off activities. Multiple levels of approval may be required to move from phase to phase.
Risk reduction is a primary goal of these activities.
Detailed plans and schedules are created to bound the scope of the project. Elaborate change management processes are developed to discourage change to these plans. Protecting the plan, to include time, cost and scope, is the key goal throughout the project.
Over the last 15+ years I have become more and more involved in Agile software development. As a developer in the 1990s we were also using practices and techniques that became formalized as XP. They just didn’t have a name yet, nor was the Agile Manifesto written.
Over that time, based on my experience and reading & talking with many others, I’ve come to believe that a core belief of Agile is that the future remains largely unknowable. That unknowns and unforeseeable events can occur at any time in the world of software development. And with ever increasing complexity, using experimentation is the best way to tackle Complex problems (see: Cynefin).
To limit the negative effects of these unpredictable events, short iterations are used to incrementally develop software. Multiple feedback loops are used to reduce risk. Feedback loops are designed to ensure we are building the right thing, and building that thing right.
Rather than detailed plans and schedules, forecasts are created based on empirical data analysis. Change, including scope, is not discouraged. Change is leveraged for competitive advantage. Even late within the creation of the product.
Cost to Build and Change
The next factor to consider is costs:
Cost of Materials
For the effort being undertaken, examine the cost of the materials. As an example, to develop a software product the costs would be some computers, perhaps some cloud & hosting, some software tools and people to develop the product. These costs are well understood and likely to be relatively low compared to the ROI projections.
Contrast that with the cost of materials to build an oil refinery or a hospital. You will need a lot of steel, concrete, electrical wiring, plumbing, heating and cooling, etc. These initial costs will be incredibly high when compared to the upfront costs to develop a software product.
Costs to Build
Next, think about the costs to build the final product.
For the refinery or hospital, you will need in-depth research & plans to ensure all of the systems will work together. The foundation must be strong. There needs to be space enough for all the plumbing, electrical, network, etc systems to service all areas. Sizing all systems needs to be researched and addressed. Legal compliance must also be considered and planned.
For the software product, the costs to build are minuscule in comparison. Compilers and storage have become incredibly cheap. Decades ago, when I started programming, it could take hours to complete compiling your code. So it made sense to spend more time planning and designing before writing the actual code and sending it off to compile. Now it literally takes seconds and is very cheap comparatively. Entire design, write, compile, test loops are minutes rather than days.
Cost of Change
Finally, let’s look at the cost of change. The further along in the creation of the product, how much cost is incurred to change the plans or correct errors discovered?
For the construction projects, the later a problem is discovered the more it costs to correct. If you’ve finished building all the walls in a hospital, how costly would it be to change the plumbing system if the initial design proves to be out of code compliance? How easy is it to correct foundation issues when the refinery is nearing completion?
For software products, the cost of making changes to the codebase, or changing the design of the application, is minor in comparison. Modern development practices attempt to ensure code rework is relatively low cost. Refactoring the system, as it’s being developed, is encouraged.
Frequent feedback from stakeholders help ensure the team is on track and enables low cost course corrections. Change, when needed, is encouraged to improve the value of the final product.
When the costs of materials, construction and change are high, it makes complete sense to attempt to reduce the risk by doing in-depth analysis and design before starting construction.
Project Management approaches include many gated phases to reduce these risks. Long phases of research, analysis and design are done before purchasing anything. This work would be considered “upstream” or the “up front” work of the project.
When these costs are relatively low, elaborate analysis and design phases become unnecessary. The Agile approach is to begin construction as soon as reasonable and use feedback loops to help define and guide the effort. Understanding of the problem and solution space will emerge as the effort progresses.
An iterative and incremental approach reduces the risk of building the wrong thing, or building the thing wrong. There is little “up front” work before construction. Analysis, design and construction are combined and undertaken by a team, or team of teams, comprised of all the skills necessary to define and create the product.
Batch Size
Let’s talk batch size!
A Project Management approach creates very large batches of work for the entire project. Perhaps these batches would be called “phases”. These large batches progress through a number of stage gates as a single entity. Typical would be research to kick off the large project. Then analysis and perhaps high-level design. Depending on the type of project different stages will be utilized.
Each batch is a very large “bet” that it will prove to deliver value once completed.
Agile approaches value reducing batch size from an entire project, or phase of a project, to something far smaller. For teams using Scrum the Sprint is a limiting timebox to reduce batch size to typically a 2 week effort. Kanban encourages smaller batch, even down to a single work item.
Smaller batches allow for more frequent feedback points. Smaller “bets” are made to reduce risk. Course correction happens far more frequently.
From Project to Product
Project Management is, of course, focused on projects. A few characteristics of a project:
- They have a start and an end.
- Teams are typically temporary and span only the life of the project, or their effort within the project. Once the project completes maintaining the effort is assigned to a different team.
- Large batches are passed from skill specialty (silo) to the next skill specialty (silo). Various forms of comprehensive documentation is the primary way of conveying information from silo to silo.
Agile approaches favor a Product perspective:
- The entire product lifecycle is considered.
- Teams are typically long-lived and support the product cradle to grave. “You built it, you own it”. Teams do not hand off work to a separate “maintenance” group.
- Fully cross-functional teams work using small to very small batches. Shared understanding is favored over passing large documents between skill silos – all the necessary people are in the same team, or team of teams.
Hybrid – the Worst of Both Worlds
Project Management and Agile approaches have a number of competing values and practices as described above. A quick summary by item:
Project Management:
- With proper analysis and research the future is largely predictable. This “up front” work is performed to create detailed plans to reduce risk.
- Effective when cost of materials, cost to build and cost of change are high.
- Favors passing large batches of work from skill specialty to skill specialty. Feedback on the value of the project is only possible when the construction phase of the project completes. This is typically many months, to over a year, from inception.
- Projects are temporary, as are likely the teams. When work is completed it’s passed on to a maintenance group.
Agile approaches:
- The fundamental belief is that the future is largely unknowable. Change happens too fast, with unforeseeable unknowns, to predict very far into the future. Forecasts are preferred to detailed estimates and long range plans.
- Effective when cost of materials, cost to build and cost of change are low.
- Favors small batches of work performed by fully cross-functional teams, or team of teams. Frequent feedback loops continually course correct as needed.
- Teams support the entire product lifecycle.
When attempting to combine the 2 differing approaches, how do you proceed? This is the challenge of “Agile Project Management”.
How do you reconcile the value difference? The future can’t be knowable and unknowable at the same time. Different mechanisms are used to lower risk in the competing approaches. Batch size is vastly different.
The end result is typically a franken-process. Larger batch project phases with work done by Scrum teams. Infrequent releases and feedback loops. Detailed timelines created and teams held accountable to delivery, often without their input.
Dual track Scrum is often attempted which just compounds the dysfunction. It sounds “agile-ish” but it’s a disguise for project/silo thinking using an Agile framework.
An example of dual track: the Design team works “2 sprints ahead” of the Development team.
Some problems with this:
- It’s nearly impossible to have 2 teams work at exactly the same pace where work would flow at the correct rate between them.
- The designs may not be implementable, you won’t discover this for 2+ sprints.
- Whenever there is a handoff (2 sprints later) there is a queue and information loss.
Actual Agile approaches would combine the 2 skillsets into one team, removing the delays, handoffs and information loss.
At the end of the day, the 2 approaches, Agile and Project Management, are simply too diverse to combine. Mash-ups typically create many more problems than they solve. People and teams will be confused. Delays will likely be introduced.
Both Useful, Depending on Domain
I am absolutely not saying Project Management isn’t valuable. It is.
A number of years ago I was a Development Manager for a construction company. They built things like refineries and chemical plants. Very complicated stuff. And the Project Managers were the stars of the company. They kept the costs down and money flowing. They were well rewarded and very much earned it.
Project Management is just not effective in the software development domain.
I became a professional software developer in the mid 1980s. Project Management was utilized pretty much everywhere I went. It was very ineffective as a mechanism for managing software development. The Iron Triangle was used as a weapon against teams. Estimates were turned into commitments chiseled in stone. Work Breakdown Structures and Gantt charts were used to chastise and penalize teams when estimates proved to be incorrect, even as padded as they were. (there’s that problem with unforeseeable unknowns and detailed planning!)
Project Management was a primary driver for the creation of the Manifesto for Agile Software Development.
Since that time, Agile approaches have proven to be far more effective for software development.
Wrapping Up
Your domain will pretty much dictate the preferred approach. If your problem space is well understood, material, build and change costs are high, Project Management could serve you well.
If your problem space is the Complex domain, where software development mainly resides, favor Agile over Project Management, or any form of mash-up of the two.
Avoid the headaches of “Agile Project Management”. It’s a marriage that never should have happened.
If you liked this post and want to learn more about Agile, Lean or Leadership, please consider purchasing a book from my Recommended Reading page.
Until next time!
