Recently I participated in a steering discussion for one of the large-scale agile marketing implementations I’m consulting. We’re using a marketing variant of the Scaled Agile Framework (SAFe) there – including planning and executing on a quarterly cadence using Program Increments (PIs).
A key struggle that surfaced was: “Planning the quarter just a week or two before it starts is way too late for us since we have so many long-lead-time activities that support us – things like media buys & field event logistics. Can we plan the quarter earlier in the quarter? Should we consider planning the quarter a quarter in advance? “
My take on this is that when we plan the Program Increment we plan whatever work we need to do in that time period. Some of that work will be delivered throughout the quarter & some would be delivered in the next quarter or even later (e.g. when working on the huge annual customer event). The key question to ask is from a Cost of Delay perspective when will we reach the last reponsible moment to start developing the campaign/program and if that moment is in the upcoming quarter it needs to be considered as part of the planning.
Another way to look at some of these activities is as Enablers (Another SAFe concept) – we do them now to enable us to deliver later. At each point in time some of our capacity would be dedicated to the “Runway” which is work that enables later delivery. (This is called Architectural Runway in canonical SAFe but that name is not very appropriate to the Marketing world I guess…).
NOTE – If we find too much of our capacity is dedicated to “Runway” activities it is an indication that our time to market is probably quite long since most deliveries require multiple quarters to mature. We should look at the main reasons we need to use the “Runway” and start to think about ways to minimize the lead time/overheads associated with them.
An example – Purchasing Media for the whole year due to “Economies of Scale”. Does that mean we need to plan media usage for all campaigns almost a year in advance? Or is there an effective way to purchase the media and figure out the most effective use once we actually start to plan out the details for each campaign/activity throughout the year?
This is just one example of the “Lost In Translation” effect when applying Agile in Marketing. What I find helps is remembering the principles/models (That’s part of my role here – making sure people understand lean/agile deeper – beyond the superficial Scrum Master/Sprint/Agile Team level).
Agile Meets Research / Data Scientists
As agile spreads wider and wider I often get to work with researchers (a.k.a Data Scientists) working closely with product development. When going agile these people struggle to figure out how it fits their unique style of work. One of those researchers I encountered on an Advanced Analytics group in Intel was Shahar. We had a chat recently and I asked him if he would be so kind to write a guest post describing his perspective. And he delivered! If you’re a researcher trying to make agile work or you’re implementing agile and you’re trying to help your researchers figure it out, this should be interesting!
The Kanban for Simple problems Myth
Depending on who you listen to, you might get the idea that a Kanban system might be great for simple problems, or even complicated ones, but when the going gets tough and a complex problem needs to be solved, you need something like Scrum (See Ken Schwaber’s post, and don’t skip the great comment thread…).
I don’t know if those making these statements really mean them or are using them as FUD to try to protect the Scrum brand. I am trying to figure this out for myself and thought I would share my thoughts. I will of course try to stand on the shoulders of giants in the Kanban and Complexity world like David J Anderson and Dave Snowden, and mainly summarize my understanding.
My approach here will be to lay out my understanding of the requirements from an approach that embraces Complexity, and then how I think Kanban maps to those requirements.
So what should be the behavior of an approach that embraces Complexity?
It seems that simple, non-prescriptive frameworks is what you need when you are working on a complex domain. These allow emergence and empirical learning. In order to learn you need feedback. In order to have effective feedback, you need to output from the system. In order to have fast learning you need fast feedback, and for fast feedback you need early and ongoing output. When I explain the Agile Manifesto to people I lay this as the main rationale for “Working Software”. You want to learn fast both about the fit of the product to the needs of the business/users, as well as the fit of the development process to the task at hand. You “Inspect and Adapt” both the Product and the Process.
This emergence seems to work best when the work is constrained in several ways. Takeuchi and Nonaka in a 1986 Harvard Business Review study entitled “The New New Product Development Game” describe timeboxing as one constraint. When coupled with a higher purpose/energizing vision this leads to faster time to market and product innovations/creativity.
A Scrum and Complexity post at AgileEvolution.com talks about punctuated equilibrium – the equilibrium being the “Safety Zone” of working in a stable system for a while (e.g. during a Scrum Sprint when the Sprint Backlog does not shift within the sprint) punctuated by events that allow the chaos/shifting world outside to affect the system, and then return to the “Safety Zone” to have an opportunity for behaviour that fits the new reality to emerge. This has been observed in nature as well as contributing to effective evolution.
Ken Schwaber talks about the container –
“A container is a closed space where things can get done, regardless of the overall complexity of the problem. In the case of Scrum, a container is a Sprint, an iteration. We put people with all the skills needed to solve the problem in the container. We put the highest value problems to be solved into the container. Then we protect the container from any outside disturbances while the people attempt to bring the problem to a solution. We control the container by time-boxing the length of time that we allow the problem to be worked on. We let the people select problems of a size that can be brought to fruition during the time-box. At the end of the time-box, we open the container and inspect the results. We then reset the container (adaptation) for the next time-box.”
To sum up – we need Containers, Punctuated Equilibrium, and freedom for emergent behavior within these containers.
Well, does Kanban embrace Complexity?
If we start from Punctuated Equilibrium – You might think Kanban doesn’t provide it since it doesn’t provide the safety zone of the sprint. But actually, if your policy is that you don’t touch what is currently in progress, you get the environment you need. There is the stability of the work in progress, and opening the hatch to the chaos on the outside whenever work is completed and we pull a new item to work on. The next items to work on can change as many times as we want. We just care about the snapshot state of the items ready to be pulled in when we open the hatch. Yes, you can say expedited items can interrupt that equilibrium. But Scrum teams deal with expedited and support items as well. Both approaches will tell you to try to shape the demand such that you reduce the amount of these interruptions, and try to create teams that can focus and achieve effective flow.
Emerging Process in Kanban
Beyond that, remember that the Kanban Method starts from your current reality and helps you see the dysfunctions, wastes and inefficiencies and provides the vision of a better reality. The specific steps that you need to take from where you are towards the effective flow vision will differ depending on your context. That is by the way part of how Kanban embraces complexity. It doesn’t prescribe a lot of practices or roles. It acknowledges your context is your context, and while there might be some good practices that might fit some situations, in complex systems you cannot even get a playbook saying this practice will get you out of this mess. You only get a catalog of practices/ideas that you might want to try out and see if they are a step in the right direction. If they are, reinforce them. If they are not, throw them away and choose something new. Evolve your system of work towards a more productive and valuable state.
The Kanban Container for Punctuated Equilibrium
If we dive into the details, one might ask what is the unit of work that is the container in Kanban. What is the equivalent of the Scrum Sprint Container? I see several options. One is to look at the level of the Business Value Increment (BVI) (or Minimally Marketable Feature, MVF, whatever you prefer to call those). When you pull in a BVI, you set a clear boundary and create a container for people to play in. They now need to deliver smaller slices of functionality until they reach a state where they can output the BVI. Within that container the functionality requirements might change adding and removing stories/tasks and the implementation will emerge.
There is nothing in Kanban that tells you what it means to be ready to start working on a BVI and what it means for a BVI to be done. You start with your current process policies, and hopefully with time you adjust your policies to get better results from the container. If you see that you waste too much time hunting for details after starting, you might try tightening up your definition of READY. If you see you are spending too much effort upstream of the work, you might want to try relaxing your definition of READY. If you get too much failure demand, try a tighter definition of DONE. You get the opportunities to affect the constraints of the container.
Another option is to look at a lower level as the container. Maybe a User Story is a better container? have a safety zone while working on the story, and look at the story boundary as the time you look outside? The BVI resonates better with me personally, but I’m not sure about it, and would love to hear what others think.
But aren’t we missing the Timeboxing?
One problem with a Feature/BVI is that it’s missing the timebox. The timebox is another constraint/aspect of the container. Without it we are missing a certain pressure to be creative and innovate. On the other hand, with it we might be pressured too much and sacrifice quality or value for the sake of time. In a sterile world, the Scrum Timebox provides the pressure while allowing continued work to deliver value if the direction that emerges at the end of the timebox is seemed useful. In reality, the timebox itself sometimes provides too much pressure, leading to lower quality, unsustainable pace, or losing opportunities for value innovation.
Don Reinertsen recommends we look at Networks and Operating Systems for ideas. Modern operating systems need to deal with processes/jobs that have unpredictable duration, and still provide responsive multi-tasking. They simply cannot “trust” a process to return control to the operating system to allow another process to get some CPU time. So they use pre-emption. This means that after a time-slice called quantum, the operating system wakes up and has a chance to decide what to do. Do we keep running the current process, or is it better value for the overall system to evict it and run another process? We can use this quantum approach with Kanban as well. We can set a quantum time for each work item in progress. When that time expires we decide whether we get more value from continuing to work on it, or from finishing it up and evicting it. Applied to BVIs, it means that after a certain time, we wake up and run a semi “steering committee” for that BVI and decide whether to continue developing it, throw it away, trim the tail, or whatever else we want to do. This can add timeboxing that is BVI specific.
There is more to be said about how this BVI as the container might work, but let’s leave it to a future date. This post is running long anyhow. I also think FDD Design/Build by Feature might provide some practical examples of how this might look like.
How can we improve Kanban for Complexity?
Well, assuming you are convinced that the right Kanban system can embrace complexity, there is only one issue I’m thinking about – Not all Kanban systems will embrace complexity effectively enough. If your process has too many prescriptions and hand-offs, not enough protection of the work in process, not enough punctuation opportunities to invite chaos to pay a visit, then your Kanban system might do you a disservice.
Kanban talks about starting with what you have. Assuming what you have is not a good system for embracing complexity, what do we do to ensure Continuous Improvement / Evolution of the system is towards a direction that better embraces complexity?
Is it enough to set the compass to the “Improved Flow” North Star? Do we need to give more guidance? I’m still thinking about this, hopefully you are now too… Leave a comment and let me know what you think…
Feature Injection - http://www.infoq.com/news/2009/05/feature-injection-comics