Using Flow To Manage Testing Bottlenecks

In today’s fast-paced world, organizations are eager to become more agile. But, as we often see, the transition isn’t always smooth. Many enterprises encounter a significant “stabilization” or “hardening” phase, which can consume around 30% of the entire release effort and schedule. This often results from high transaction costs associated with testing cycles and a desire to optimize testing efficiency by reducing cycles and pursuing scale.

The problems are clear: waste, ineffective handoffs, and rework lead to low predictability and raise serious questions about meeting release-level quality standards on time. Big-batch thinking inherent in traditional software development fuels this cycle, leading to increasingly longer stabilization phases and rushed development cycles.

When it comes to agile transitions and testing, several scenarios emerge:

• Ideal Agile: Agile teams take on all stabilization work, delivering release-ready software within each increment. Massive test automation minimizes regression and stabilization efforts.

• Complex Environments: High regression effort and extensive testing coverage make it impossible for agile teams to complete all testing within a sprint. Enterprise readiness, performance, and security aspects require specialized approaches and considerable time.

• Evolutionary Agility: Organizations take a more evolutionary approach without necessarily having fully-fledged agile teams, resulting in lower “doneness” of work.

• Focus on Release Costs: Some organizations are primarily focused on reducing release costs and improving due date performance, with agility as a secondary benefit.

For complex scenarios, a flow-based recipe can significantly reduce stabilization costs and improve predictability, even without full team-level agility and engineering practices.

missing-image-placeholder

Late feedback on decisions is a major culprit. Late quality is always more expensive than timely quality. We often work in big batches due to perceived efficiency, but this delays feedback. Multitasking, even at the organizational level, further exacerbates the problem, as it increases the time required for features to be ready for feedback.

Ultimately, we end up “adding” quality after the fact, leading to a choice between compromising on due dates or compromising on quality. This often results in teams being driven beyond sustainable pace, ignoring bugs, and applying quick fixes, increasing unpredictability.

1. “Done is Done”: A feature is “Done” only when it’s completely finished, passing release-grade criteria, including all test coverage and defect fixes.

2. Limit Features in Progress: Allow a new feature to start only when a previous one is truly finished and release-ready with a low number of open defects.

3. Optimize Features to “Done”: Developers should focus on getting features to “Done” rather than just “Coded.” This may involve investing more in quality at the coder level, such as automated unit tests, code reviews, and pair programming.

4. Deal with the Testing Bottleneck: Identify work types that don’t need to go through the testing bottleneck or require less capacity from it. Choose work that is not testing-heavy or can be tested without involving the bottleneck.

5. Invest to Elevate Bottlenecks: Create a backlog of engineering investments to improve testing capacity, such as test automation. Leverage the “Test Automation Pyramid” (unit testing, service/API testing, UI testing) to reduce the cost of test automation and benefit from developer involvement.

6. Dynamic Stabilization Decisions: Choose when to go into stabilization based on metrics like the number of open defects and the amount of feature work in progress.

Escalating stabilization costs are still a common pain point. Lean/agile approaches can significantly help. By applying an end-to-end, pragmatic approach and tying all the parts together, we can drive evolutionary improvements and global optimization. This solution can be used as part of agile implementation projects, on a standalone basis, or as a precursor to adopting a more agile approach.

Ultimately, this method helps us identify bottlenecks, focus improvement efforts on these bottlenecks, improve flow, and achieve lower stabilization costs and better release flow that enables tighter feedback loops, improved time to market and convergence to valuable outcomes. It’s about working smarter, not just harder, to deliver high-quality software predictably, which is a crucial element of providing awesome products that delight your customers.

Notes

This blog post is based on a paper included in the proceedings for the Lean Systems Conference in 2011 where I presented this topic. I refreshed and reshared it on substack recently.

PS interested to see the slides from the 2011 conference talk?

Using flow approaches to effectively manage agile testing at the enterprise level from Yuval Yeret

Improving flow means fewer bottlenecks, faster feedback, and less wasted capacity. Explore flow advisory or let’s talk.