Finding the right Dev to Test Ratio when working in Kanban

In a previous post I started talking about the ratio between Dev and Test, and promised to revisit how it looks like in an Agile/Kanban environment. Whenever I talk to teams/managers about Kanban, whether as part of a workshop, or with a team actually practicing Kanban, the issue of testing as the bottleneck surfaces quite quickly. A typical situation looks like this: [caption id=“attachment_134” align=“aligncenter” width=“614” caption=“Testing Bottleneck”][/caption] What we see here is a classic bottleneck in a kanban system. Testing are at their work in progress limit, meaning they cannot take on more work. Acceptance has no work in progress, what we call a “bubble”, and development are at their limit as well. When we take a closer look, we see even more indications this is a bottleneck. We see nothing from Testing is DONE waiting to be pulled, which explains why Acceptance has a bubble. In development, a significant part of the work in progress is DONE waiting to be pulled into testing. The implications of this situation is that Development will not be able to pull in new work and will have to look what else they can do to help the flow of work. In theoretical discussions workshop participants are quick to grasp that development now need to go and see how they can help the testing. In real life what you usually see at first is the developers seeming oblivious to this simple conclusion, and the testers starting to get defensive about blocking the flow. All of it quite natural… I try to get teams to use the five focusing steps from TOC at this point. IDENTIFY the system’s constraint - Our Kanban board found the bottleneck/constraint for us. Decide how to best EXPLOIT the constraint - Here I ask teams to think about whether testing is the most efficient it can be, and whether there are ways the team can help them be more efficient. Some practical ways to do that are:

• Break down the work of testing into smaller tasks. This will help the team identify tasks which can be offloaded from testing to other members of the team, or ideally automated. It provides better visibility to where the majority of the bottlenecks time is spent.
• Go see how testing spend their time and how much of the time they’re actually testing versus doing other things. You can get some nice ideas from the “TOC Blue Light” story. The idea here is that testing should be able o spend most of their time actually testing. If they’re waiting for code, for developers to come by, for setups to happen, for data to populate, etc. then there are probably ways to help them be more efficient. As a manager you might need to ask your team questions to try and direct them towards exploring this issue.
• Explore how much rework testing has to deal with. Rework comes in the form of Defects they need to open, wait for resolution, and verify. Repeated testing due to repeated problems. Changes to implementation that come in late after a round of testing, since the implementation and acceptance criteria / test plan are not aligned. Reduce rework to help exploit the testing constraint.
• Practices such as ATDD help align requirements, acceptance criteria and implementation. Other practices from the XP world such as TDD, Pair Programming/Code Reviews, Coding Standards, Continuous Integration, help increase the actual quality of code that comes into testing.
• Discussing and defining what it means for a story to be Ready for Testing (or alternatively the Definition of Done of Development) is a very good way to reduce rework as well.

SUBORDINATE everything else to the above decision - Kanban limited WIP inherently subordinates everyone to the constraint, Providing them with slack time that can be used to help exploit the constraint after identifying some potenial areas for improvement. Other ways to help may be to take on some testing work that they can help with. This is a short term solution though, both since developers don’t really like to do testing for a long time, as well as since its not their strong suit. ELEVATE the system’s constraint - Sometimes Exploiting will be enough. In other cases, the constraint is so strong that you will want to elevate the constraint in a more strategic way. This is where actual investment and changes in structure come in. One alternative is to shift or share responsibilities - e.g. make test automation the baby of the entire team, not just the testers (see “Why test automation costs too much” for some related ideas ) Sometimes elevating via changed responsibilities will not be enough. One other thing to look at before coming to the conclusion that you have the wrong ratio, is the strength of the developers and testers. I’ve seen many cases where the team has real stars in the dev team, but the engineers in testing are not up to their level. Especially in an Agile environment, its quite important to have strong testers that are able to keep up the pace of the team. Its also important that that they have the respect of the developers, especially if you are going for something like ATDD where the testers actually participate in guiding the way of the team. If you have strong testers and they’re still the constraint/bottleneck, maybe indeed your ratio is not a good one, and you need to consider what to do. But if you went thru the focusing steps, tried to exploit the constraint, have a kanban system where this process is visible and there is concrete data that shows the delivery implications of the constraints (ideally the economic cost of the constraint as well), it will be easier to make and get buyin for decisions that have economic outcomes (increasing  headcount, moving headcount) Summary Your kanban system will surface any constraints that are related to different throughput of Dev and Test. Kanban together with TOC 5 Focusing Steps helps the team make the best out of what they currently have and improve their processes and tools focusing on the areas which require the most improvement. If all of this still is not enough, you have a good story and substantiation to make farther reaching decisions about ratio with. So what is the right ratio? The best answer would be to try using an Agile/Kanban system and find out. For those looking for specific numbers, I can say based on what we’ve seen so far at Agilesparks that a ratio of 2:1 has a good chance of working for a team that is willing to adopt Agile engineering practices including test automation by the team and ATDD, and where the testers strength is comparable to the developers.

Improving flow means fewer bottlenecks, faster feedback, and less wasted capacity. Explore flow advisory or let’s talk.