“If teams miss their commitments, or promises, it should be painful.”
I’ve heard this over the years when I was a developer, as a front-line manager and also in my career as an Agilist. It’s a sentiment widely held by many in senior management roles; especially among executives.
I thought I’d dive into that a bit – how that perspective might affect people and teams. How that culture might affect delivery performance.
When developing software products teams rarely operate independently. They are struggling with a web of dependencies, inherent uncertainty & complexity, and shifting priorities. Within such an environment, the expectation of predictability from teams, in which they can make firm commitments, becomes not only unrealistic but actually harmful.
As an Agile Coach with a developer and Lean background, I’ve experienced firsthand how this expectation can demoralize teams, crush innovation, and reduce delivery performance.
In this article I will describe the reasons why expecting high levels of predictability in such environments misses the mark, how these constraints impact teams, and why penalizing teams for unpredictability is counterproductive.
The Illusion of Control: Identifying the Challenges
The desire for predictability is understandable. Stakeholders want to know when something will be done, what features it will include, and how much it will cost. This is Iron Triangle Project Management Thinking. Agile was created as a replacement of this thinking, not to improve it.
The reality of complex product and services development complicates creating precise estimates. Several factors contribute to this unpredictability:
- Dependencies: Software development teams rarely exist in isolation. Teams often rely on other teams, external vendors, or shared resources. A delay or change in one area can ripple through the entire effort, impacting timelines and deliverables. These dependencies are complex and interconnected, making it difficult to accurately predict their impact. For example, in an effort with 3 dependent teams there is mathematically only a 12.5% chance of on-time delivery. The chances drop to 6.25% if you add a 4th dependency.
- Uncertainty: Software development, by its very nature, involves a degree of uncertainty (See Complex Domain, Cynefin). Customer needs and goals may be unclear or evolve during the product lifecycle. Technical challenges may arise that were unforeseen. Market conditions may shift, necessitating changes in direction. This inherent uncertainty makes precise planning and prediction an exercise in wishful thinking.
- Changing Priorities: Business needs change, customer feedback emerges, and new opportunities arise. As a result, priorities often shift throughout the effort. These changes can disrupt existing plans and require teams to adapt quickly. While Agile methods embrace change, constant shifts will impact predictability.
- Emergent Complexity: As an effort progresses, new insights and information emerge, leading to a deeper understanding of the problem and potential solutions. This emergent complexity can lead to changes in design, architecture, or even scope. While these changes are often beneficial, they can also make initial predictions inaccurate.
How These Factors Affect Predictability
These factors create an environment that makes accurate prediction unrealistic:
- Unreliable Estimates: When dependencies, uncertainty, and changing priorities are prevalent, initial estimates become outdated almost immediately. Teams base their estimates on incomplete information or assumptions that later prove to be incorrect. As the effort progresses and new information emerges, these initial estimates become less and less reliable. A major difficulty is that you cannot know ahead of time which of these decisions or assumptions will later prove to be wrong.
- Reduced Agility: The pressure to be predictable can reduce a team’s agility. Sticking to the plan & making their commitments reduces the teams desire to adapt to new information. When early assumptions and decisions prove to be incorrect they will push on. Rather than pivoting when they should they will persevere against the original plan to avoid retribution.
- Reduced Innovation: Teams will become hesitant to experiment with new approaches for fear of deviating from the plan. They won’t “take a chance” on something new or different. They will stick to what they know as that carries less risk of unfavorable results.
- Demotivation and Burnout: Constantly being held accountable for inaccurate predictions in demoralizing. People may feel like they are constantly failing, even when they are working hard and doing their best. This can lead to burnout and decreased productivity. “Quiet quitting” is an example of this behavior.
The Unfairness of Penalizing Unpredictable Teams
Given the challenges of working in environments with high dependencies, uncertainty, and changing priorities, penalizing teams for unpredictability is not only unfair but also counterproductive. A few reasons why:
- Focuses on the Wrong Metric: Predictability is not the correct measure of a team’s success. The true measure is the value they deliver to the customer and the business. Focusing on predictability can incentivize teams to prioritize adherence to a plan over delivering higher value.
- Discourages Transparency: When teams fear punishment for being unpredictable, they may become less transparent about their progress and challenges. This can lead to hidden problems which make the effort even less predictable. We’ve all experienced “watermelon status reports” where problems and lack of progress are hidden until it’s impossible to continue doing so.
- Discourages Learning and Improvement: If teams are penalized for inaccurate predictions, they are less likely to experiment with new approaches. This can hinder continuous improvement and prevent the team from becoming more effective over time.
- Reduces Value Delivery: Teams will quickly learn to “sandbag” their estimates and commitments to reduce the chances of punishment. It’s human nature. Ironically, this will reduce the pace of value delivery as teams learn to “play it safe”.
- Undermines Trust: Penalizing teams for unpredictability erodes trust between the team, management and stakeholders. This can create a toxic environment and make it difficult to collaborate effectively.
A More Effective Approach: Embracing Uncertainty
Instead of demanding unrealistic predictability, a more effective approach is to embrace uncertainty. Efforts should involve reducing the risks and effects of high levels of uncertainty and unpredictability. Focus on building resilient and adaptable teams. This involves:
- Accepting Uncertainty: Acknowledge that uncertainty is a natural part of software development. Don’t try to eliminate it, but rather learn to manage it effectively.
- Focusing on Value Delivery: Prioritize delivering value to the customer and the business over adhering to a rigid plan.
- Embracing Agile Principles: Use Agile practices to break down large efforts into smaller, manageable items of value. Use iteration and incremental delivery to reduce risk. Shorten all feedback loops. This enables teams to adapt to changing circumstances and incorporate feedback in an emergent way.
- Promoting Transparency and Communication: Generate open communication and transparency between the team and stakeholders. This allows for early identification of potential problems and facilitates collaborative problem-solving.
- Building Resilient Teams: Invest in training and development to help teams become more adaptable and resilient. Encourage experimentation and learning from mistakes. People and teams will naturally go the extra mile when they are not continually penalized for missing “commitments” for reasons beyond their control.
- Simplifying the Environment: Examine the dependencies across and between teams. Use techniques such as Blocker Clustering to measure the impacts of dependencies and create experiments to reduce and eliminate dependencies. As management this is an area where you can have a huge positive impact on value delivery – untangling the web.
- Embracing Rolling Planning: Instead of creating a detailed plan upfront for the entire effort, focus on planning in much shorter iterations. Frequently update and re-communicate the plan as the effort progresses and more information becomes available. Let the plan emerge as more becomes known and learned.
- Measuring Progress Based on Value Delivered: Instead of rewarding adherence to a plan, measure progress based on the value delivered to the customer and the business. If the pace is considered too slow, leverage System Thinking to reduce waste and non-value add activity. Consider adding another team as needed.
- Forecasting Using Actual Data: Use Lean Metrics such as Cycle Time and Throughput with techniques such as Scatter Plots and Monte Carlo simulations. Augment with probability ranges rather than exact dates, promises and commitments. Identify and list all the assumptions used to generate the forecast. Reduce risk by validating those assumptions early. Using actual data is far superior than using estimates for the many reasons identified above.
Shifting the Mindset
The expectation of high predictability in complex efforts with multiple dependencies, inherent uncertainty, and ever-shifting priorities is wishful thinking. Instead of penalizing teams for being unpredictable, we should embrace uncertainty and focus on building resilient and adaptable teams that can deliver value in an emergent environment.
By shifting our mindset from a focus on predictability to a focus on adaptation and value delivery, we can create a more positive and productive environment for teams and achieve greater delivery success. This requires a fundamental shift, one that recognizes the limitations of prediction and embraces the power of adaptation. Only then can we empower our teams to thrive in the face of complexity.
Until next time!
