If you’re in the software development space I’m certain you’ve been involved with some form of a management directive for teams to “go faster”.
We all know that directly chasing speed is reckless and will likely lead to lower quality, much higher stress and perhaps even gaming of metrics.
Increases in predictability, flow and quality should be the side effects of continuous improvement efforts. Inspect & Adapt is a foundation of Agile and Lean.
By using the information in this article you will discover one method of workflow improvement – finding, measuring and reducing the negative effects of hidden queues.
OOTB
Most ALM (Agile Lifecycle Management) tools will have a default workflow modeled within the tool. It’s going to be very simplistic and likely some form of To Do -> Doing -> Done.
Unless you are doing Ensemble Development, meaning you have a team WIP (Work-In-Progress) limit of 1, where everyone works on a single item together, this default workflow is doing you no favors. In fact, it’s making analysis of your development system much harder.
Pros of this model:
- Makes work visible
- Great for teams doing Ensemble Development
Cons of this model:
- Cannot easily understand the team process
- Cannot identify any handoffs which might exist
- Difficult to identify the constraint
- Nearly impossible to calculate process efficiency (using value add and non value add measures)
Model Your Activities
Depending on your organizational structure and management style, you may or may not have imposed process steps. Common silos of skills include analysis, coding and testing, each of which represent distinct management hierarchies.
If this is the case, or if the team passes work from person to person as it is being done, model those process steps/activities. Be honest about what is really happening. It’s OK. Being honest will make improvement opportunities easier to identify.
Examine the current process and model the various ways work is being transformed. A simple workflow is modeled in this image, yours may be similar or include additional workflow states.
Pros of this model:
- More accurately assess progress on work items
- Can calculate time spent per activity to use as a metric for improvement efforts
- Can swarm to activities where work slows promoting cross-skilling
Cons of this model:
- Cannot tell if a work item is complete or not within an activity
- Cannot easily measure time work is at rest waiting for the next activity to begin
Expose Hidden Queues
Modeling the activities of process is a great first step. But here’s the thing, there are handoffs happening within the process. And one thing we know for certain, all handoffs imply a queue.
The downstream/receiving team member(s) are likely not idle and waiting for you to complete something. They are already busy with work which previously came to them.
So the new work moving to them will go into a queue even though you cannot see it on the board which only shows activities. The queues are invisible, they are hidden within the states.
Queues cause delay, which is another thing we know for certain. While work is at rest, meaning sitting in a queue, no value is being added to it.
In order to assess the impact of these queues, we need to measure how long work items are sitting idle waiting for availability.
The best way I’ve found to do this is to model the handoffs/queues as “buffer” states. It’s far simpler to measure time spent in a buffer state than to analyze timestamps.
Make the invisible visible.
Here is an example of modeling those buffer states.
Pros of this model:
- Easy to measure the amount of time work is at rest (non value add time, aka waste)
- Adding WIP limits to buffer states helps improve flow
- Smoother flow leads to higher predictability
- Improves constraint identification – where is work slowing down?
- Promotes conversations of how to work more collaboratively than in individual skill silos
Cons of this model:
- Might limit the team to continue working in silos if they don’t experiment with removing buffer states by eliminating handoffs
An Example – Progressive Elaboration
In this example I have added 10 work items to the 3 different workflows.
In the default workflow above it’s very difficult, without deep analysis of tasks and their start/done timestamps, to understand if there are any roadblocks. Many teams do not use tasks, nor do I recommend them unless you really need them, which makes analysis even more difficult.
All systems have a constraint, where is it in this workflow? Where should you target improvement efforts?
Do you suspect there is a delay in Analysis? How would you know for sure?
If a stakeholder called for an update on their request, could you quickly answer their question? Is it possible for them to self-serve so the do not need to call?
Review this image of the same work items in a workflow modeled to show the activities.
Does your understanding of the system still hold? Does it still seem like there might be a blockage in Analysis?
Now you might suspect there is a problem in Development. Perhaps we need more people or they need additional training.
However, once we model the handoffs, to make the hidden queues visible, we finally get a proper understanding of where work is piling up. We have discovered a constraint.
Now we can measure the impact of the delays and work to reduce the effects of the constraint. Use your Lean metrics such as cycle time to evaluate the effects on flow of improvement efforts.
Reviewing a CFD (Cumulative Flow Diagram) can also be useful to detect patterns in the system.
Next Steps
In 40+ years in IT I’ve learned a few things. And one of them is that doing the work still takes considerable time. Improvements have been made for sure but the domain still remains complex, people still need time to think, and the need for agility with high quality remains very high.
The majority of improvement efforts I have seen over the years target teams “going faster”. Directly targeting speed is risky. Quality might drop as teams take short cuts to make their dates (often imposed on them). That hasn’t changed since I began coding in the mid 1980s.
This technique is based on the understanding that doing the work still takes time and that the biggest delays in the system are work sitting at rest (non value add time). There are many causes of this, such as too much WIP (Work In Progress) and hidden queues as discussed here.
The average development system runs at less than 10% efficiency (value add time/non value add time). So if the team has a lead time of 100 days, at 10% efficiency that means value add time is 10 days. Which overall is good for a normal, unexamined system.
In that system, using traditional thinking, if you double the speed of the team the development will take 5 days rather than 10. Lead time will drop from 100 days to 95 days. Will your customers even notice?
Turn that around. Reduce the wait time by half. Lead times will go from 100 days to around 55 days.
No changes to how work is performed. No risk to quality. No added stress to “go faster”.
Just increased flow. Your customers will certainly notice that.
Are you willing to do what it takes to find, measure and do something about the effects of hidden queues?
Until next time!
