Category Archives: Agile

Finding the right Dev to Test Ratio when working in Kanban

Estimated Reading Time: 4 minutes

In a previous post I started talking about the ratio between Dev and Test, and promised to revisit how it looks like in an Agile/Kanban environment. Whenever I talk to teams/managers about Kanban, whether as part of a workshop, or with a team actually practicing Kanban, the issue of testing as the bottleneck surfaces quite quickly. A typical situation looks like this: What we see here is a classic bottleneck in a kanban system. Testing are at their work in progress limit, meaning they cannot take on more work. Acceptance has no work in progress, what we call a "bubble", and development are at their limit as well. When we take a closer look, we see even more indications this is a bottleneck. We see nothing from Testing is DONE waiting to be pulled, which explains why Acceptance has a bubble. In development, a significant part of the work in progress is DONE waiting to be pulled into testing. The implications of this situation is that Development will not be able to pull in new work and will have to look what else they can do to help the flow of work. In theoretical discussions workshop participants are quick to grasp that development now need to go and see how they can help the testing. In real life what you usually see at first is the developers seeming oblivious to this simple conclusion, and the testers starting to get defensive about blocking the flow. All of it quite natural… I try to get teams to use the five focusing steps from TOC at this point. IDENTIFY the system's constraint – Our Kanban board found the bottleneck/constraint for us. Decide how to best EXPLOIT the constraint – Here I ask teams to think about whether testing is the most efficient it can be, and whether there are ways the team can help them be more efficient. Some practical ways to do that are:

  • Break down the work of testing into smaller tasks. This will help the team identify tasks which can be offloaded from testing to other members of the team, or ideally automated. It provides better visibility to where the majority of the bottlenecks time is spent.
  • Go see how testing spend their time and how much of the time they're actually testing versus doing other things. You can get some nice ideas from the "TOC Blue Light" story. The idea here is that testing should be able o spend most of their time actually testing. If they're waiting for code, for developers to come by, for setups to happen, for data to populate, etc. then there are probably ways to help them be more efficient. As a manager you might need to ask your team questions to try and direct them towards exploring this issue.
  • Explore how much rework testing has to deal with. Rework comes in the form of Defects they need to open, wait for resolution, and verify. Repeated testing due to repeated problems. Changes to implementation that come in late after a round of testing, since the implementation and acceptance criteria / test plan are not aligned. Reduce rework to help exploit the testing constraint.
  • Practices such as ATDD help align requirements, acceptance criteria and implementation. Other practices from the XP world such as TDD, Pair Programming/Code Reviews, Coding Standards, Continuous Integration, help increase the actual quality of code that comes into testing.
  • Discussing and defining what it means for a story to be Ready for Testing (or alternatively the Definition of Done of Development) is a very good way to reduce rework as well.

SUBORDINATE everything else to the above decision – Kanban limited WIP inherently subordinates everyone to the constraint, Providing them with slack time that can be used to help exploit the constraint after identifying some potenial areas for improvement. Other ways to help may be to take on some testing work that they can help with. This is a short term solution though, both since developers don't really like to do testing for a long time, as well as since its not their strong suit. ELEVATE the system's constraint – Sometimes Exploiting will be enough. In other cases, the constraint is so strong that you will want to elevate the constraint in a more strategic way. This is where actual investment and changes in structure come in. One alternative is to shift or share responsibilities – e.g. make test automation the baby of the entire team, not just the testers (see "Why test automation costs too much" for some related ideas ) Sometimes elevating via changed responsibilities will not be enough. One other thing to look at before coming to the conclusion that you have the wrong ratio, is the strength of the developers and testers. I've seen many cases where the team has real stars in the dev team, but the engineers in testing are not up to their level. Especially in an Agile environment, its quite important to have strong testers that are able to keep up the pace of the team. Its also important that that they have the respect of the developers, especially if you are going for something like ATDD where the testers actually participate in guiding the way of the team. If you have strong testers and they're still the constraint/bottleneck, maybe indeed your ratio is not a good one, and you need to consider what to do. But if you went thru the focusing steps, tried to exploit the constraint, have a kanban system where this process is visible and there is concrete data that shows the delivery implications of the constraints (ideally the economic cost of the constraint as well), it will be easier to make and get buyin for decisions that have economic outcomes (increasing  headcount, moving headcount) Summary Your kanban system will surface any constraints that are related to different throughput of Dev and Test. Kanban together with TOC 5 Focusing Steps helps the team make the best out of what they currently have and improve their processes and tools focusing on the areas which require the most improvement. If all of this still is not enough, you have a good story and substantiation to make farther reaching decisions about ratio with. So what is the right ratio? The best answer would be to try using an Agile/Kanban system and find out. For those looking for specific numbers, I can say based on what we've seen so far at Agilesparks that a ratio of 2:1 has a good chance of working for a team that is willing to adopt Agile engineering practices including test automation by the team and ATDD, and where the testers strength is comparable to the developers.

So what is the right ratio between developers and testers?

Estimated Reading Time: 2 minutes

One of the questions I’m asked quite frequently is what is the right ratio between developers and testers.

A variant on that question is what I typically see in other organizations as the ratio.

Well lets answer the second variant and then try to deal with the first.

Typically what we see in agilesparks customers is 2:1 or 3:1. There are some exceptions of 3:2 or 5:1 but they are quite rare.

What is right? Well as u might expect it depends:
– what are the skills/strength of developers and testers. Not all engineers are made equal…
– how are the responsibilities spread between roles? Eg test automation etc.
– what is the development life cycle style used by the group
– what is the quality of the code when delivered to testers
– technology and character of the system. Usually the lower the product is in the stack the more testing it needs per code developed. This is due to the amount of impacted areas, the complexity to test, and the breadth of the configuration/platform matrix.
– how much test automation exists

One of the typical dynamics of a phased waterfall-like development lifecycle is that QA phase starts late and pressured and compromises in quality are part of life

This means very tough weeks/months for the poor testers which get a big blob of code to work on, they are outnumbered, usually outskilled and the following happens:
– the perception is that test is the bottleneck. ( well I would say that in waterfall each phase is a bottleneck. Those that tend to be the last ones just get the short end of the straw 😉
– either test phase is prolonged or developers are asked to help do testing. Or both.

But what also happens is that organizations get used to this. It’s a fact of life. For some reason the need to strengthen the test organization is usually outweighed by ability to develop more features each version.

Another thing is that developers do end up helping testers they usually do it in crisis mode when their only option is to help do the work, not help testers have an easier job the next time

Lets see what happens when such an organization moves to agile/kanban

In kanban or any kind of lifecycle that is feature driven developers and testers are expected to have more
Or less the same pace / feature velocity. So our typical outskilled outnumbered testers usually quickly appear as a bottleneck early in the transition to agile.

Limiting the WIP in test and Dev causes Dev to stall, leading to one of the classic discussions in kanban workshops and even some kanban games I wouldn’t mention to avoid spoiling the fun ( Carlos – we know you’re out there!)

In an upcoming post I will try to cover what can be done when this typical scenario happens, using practices from Agile Testing, the TOC five focusing steps, and agile engineering practices I general.

And we will try to think what is the right ratio for an agile environment.