I recently worked on some measures/metrics that help answer the question "How effective is our product management process, and how is it contributing/affecting the overall performance of our product development line"
Here are some slides from slideshare about the topic. I might write it up in a more elaborate form in the future.
If there are specific questions about concepts I mentioned, mention it in a comment (or on twitter or something) and I might write up a specific area.
If you have examples of dashboards showing similar measures in practice, I'd love to see!
An interesting question was raised to one of our coaches at Agilesparks.
One of us answered that same like you probably prefer to answer emails throughout the day rather than at the end, you want to deal with bugs during the sprints.
I of course fully agree that you need to deal with bugs during the sprint/release, or in other words keep the Bugs In Progress number limited. (Anyone for Limited BIP society?)
But this answer doesn't convince me.
One of the key questions project managers and senior management in general ask themselves and their teams on an ongoing basis is – "Are we on track to deliver the scope we committed to, on time". In some environments "on budget" is added to the question.
If you are talking about a Release Scope, the answers are quite similar whether you're doing Scrum or Kanban. If you don't care too much about the budget aspects, a Release Burnup can show you the commited scope, the committed date, and the actual progress in working software towards that goal – Plan versus Actual. If you ARE interested in the budget picture – committed budget versus actual, and are we on track to finishing the release with the budget we committed to – use AgileEVM on top of that. (http://www.infoq.com/articles/agile-evm is a good place to start)
Basically for all of this – you are measuring the amount of done features work compared to the amount of features work originally planned for. Whether sized using effort days, story points, function points, the idea is the same.
In a conference a couple of months ago I talked about Agile Release Management and covered this subject somewhat. You can check out the slides at http://www.slideshare.net/yyeret/managing-projectsreleases-using-
I would add that this expectation of management is what we call Predictability in the Kanban world, and based on some encounters I've had with senior management, we as the Agile community have not been doing a great job at connecting to the expectation of Predictability. In many cases its the opposite – we create the impression that Predictability is a lost cause because everything is Agile.
In Kanban we try to better connect to this expectation of Predictability/Commitment to the important things. Senior management doesn't care about committing to a sprint goal and meeting it. They care about meeting commitments to deliver a release on time and with feature highlights communicated to the stakeholder community. They care about meeting commitments to deliver certain features on time to internal and external parties that count on this feature to continue and do something else.
Predictability will continue to be important. The way its measured might change. For now, most teams/projects are indeed evaluated based on the answer to "Are we on track to meeting the release goal on time". We should support those teams with an approach that complements their kanban flow-based workflow. The methodology is all there if you connect the dots.
The room for improvement is mainly in connecting the dots and providing a structured methodology that can be applied as a framework, as well as better tool support.
What are the gaps?
First, The thinking around CFD needs to switch from history to also a forward-looking predictive chart. What do I mean?
Most CFDs you see today focus just on an operational view CFD – what is the current state, as well as history, which can help you improve your process, operation.
I'm Missing a view of the work needed by a certain date, and whether we are on track to achieve our commitments/goals. Tools that extend the CFD to a view that includes current trend, required trend to meet the goal, and trend of requirements churn can answer this question – you see whether the DONE trending towards the overall committed scope is on time or not.
One more complication is that of course you sometimes want your board to reflect many releases, not just one. You're working to finish one release, and then you move to another.
In this case, You probably want this view per-Release on the board.
So we need visibility charts that can aggregate the status of several cards e.g. Feature, Release, MMR, MMF whatever you want to call it. In FDD Parking Lot diagrams are a popular way to convey the status of various features/aspects in a Project/Release. An extension of a Parking lot diagram can be to have a mini-burnup of that entity. So beyond just the status (which is basically the current point of a burnup), you can have a mini-graph showing the status of entities comprising this feature. See below for a sketch of how this can look. ( Note that the Warning Indicators box are taken straight from the organizational dashboard page of LeankitKanban. I recently started to explore the capabilities in this dashboard and find them quite useful to help bring a process under control, and the sort of stuff you might want to look at in an operational review).
The color of each parking lot / feature can easily be derived from where the actual progress is compared to the expected progress curve. The expected curve can be defined to be Linear (yeah right), S-curve based as David Anderson is fond of, or whatever you think the profile should look like. Once you are below the curve, you start to gain reddish colors. Above it – you are green. With Agile approaches relying on Working Software as a measure of progress, you can really trust those colors… Its no more a watermelon (green outside, red inside – courtesy Kent Beck)
For those interested in the details, here is one way a CFD can be extended to provide burnup capabilities.
With this in mind, the mini-burnup in the parking lot can be upgraded to a mini-CFD
Now, with a CFD, some more intelligence can be applied to help determine the color/state of the Feature. High level of WIP can be a leading indicator of problem (but knowing about Little's law and how a CFD looks like you probably know that it will be apparent in the burnup being quite flat as well…). I'm guessing that with time, we will learn to study and identify progress risks using a CFD, beyond the basics we currently use.
Bottom line – my feeling is that in order for Kanban to cross the chasm into the majority of projects/development groups, who are quite concerned with delivering Releases and Features on schedule, not just with trusting the Flow, we will need to provide more and more tools and focus to support this use case. The core thinking is there, the hunger on the part of the IT world is there as well it seems, so lets go out there and make it happen. my 2c…