The foundations of the Agile Manifesto

There have been many occasions where the Agile Manifesto has been pulled out in a presentation. Makes sense, it has been around for a while and started a movement which is still growing today.

We usually get caught up with the Agile Values and Principles but have you ever thought about the very first line?

"We are uncovering better ways of developing

software by doing it and helping others do it."

Recently, I had cause to realise the how this is the foundation of the Agile Manifesto by making and acknowledging a mistake.

I had been recording leadtimes for a while. The information was useful but there was something troubling - 13 point stories just didn't fit. No matter how we prepped or planned they always took longer than our sprint window, sometimes MUCH longer.

So I wielded my power as an SM and banned them. Poof! Gone! Good riddance.

Thing is, I had just moved the problem. Over time, I noticed the same issues we had observed with 13 point stories start to affect 8 point stories. I don't think this was conscious but I do think it was due my 'ban'.

It struck me that we had broken one of the foundations that are baked into the Agile Manifesto and everything it contains - by no longer doing it, we could not learn from our mistakes or successes.

It seems kind of obvious but I had never thought about it before. 

The manifesto was born out of 'doing' and 'sharing' - the people who some up with it did not hypothesise! You can only learn from doing - by banning my 13 point stories, I had stopped all feedback from experience and therefore any learning too. We accelerate our collective learning through sharing our experiences, both good and bad.

So I came clean with my team, said I goofed and the team took in not only 13 point stories but much larger ones too. Each time we tackled one, we learnt a little bit more and the next one was just a little bit smoother.

(But they still don't fit!)

Like photos, sprint composition is an art

A while back I started looking at metrics used in Kanban to better understand some of the patterns I was seeing in my sprints. The change from looking at a sprint velocity to looking at leadtimes for individual stories has been an eye opener.

In particular, I started to look at the composition of sprints rather than just trusting a velocity. The problem I have with velocity is the perceived safety it provides. Estimate stories, add the totals and when you reach something near your velocity you should be good to go.

Imagine this: we have a velocity of about 40 so we should be good to take in four 8 point stories and still have a little room right? This is where leadtimes start to give us a little more information.

My team is quite large, having 8 developers and 2 testers. We generally work in pairs so I see this as 4 parallel development tasks with an additional 2 testing streams. The team work on development and testing in parallel as much as possible - joining up is still a little fun but we get by.

I know from looking at my leadtime data that the average leadtime for an 8 point story is about 6 days. The worst we have turned a 8 point story around in is 9 days, with 4 days being the best. We work in a 2 week sprint cycle, which is realistically 9 days once we take out planning and other ceremonies.

Working with the averages alone, to have a 50% chance of completing all the 8 point stories in the sprint, we would need to start them all by the 4th day of the sprint. If we wanted this to increase to a 85% chance, the leadtime increases to 8 days meaning we would need to start them all on the second day of the sprint.

Here are some of the problems this could introduce:

Higher WIP: With 4 stories that all need to be started within a few days of each other, we force a higher WIP. Higher WIP means longer lead times.

Loss of fluidity/options: With 4 parallel work streams each using a pair of developers, we might find it hard to move people between tasks. We basically end up with a static pairing from the beginning of the sprint to the end. This limits our options or introduces waste where we have to move people around e.g. illness or a specialism that we have not shared yet.

QA 'tsunami': There is a risk that all 4 stories will need QA at around the same time. This might create a bottleneck in testing and slow throughput. If you had some flex in the team, you could get the developers to help with the testing - in this scenario that is probably not an option.

The composition of how that 40 points is reached is as important as the number itself.

Having a mix of smaller and larger stories creates more opportunities for the team to collaborate on delivery. Having stories finish regularly throughout the sprint allows the team to find points at which they can mix up their pairing or help out with other stories in progress. Having this flex allows you to shift responsibilities around the team to help keep stories moving across your board.

If you do have larger stories then look to create tasks during planning that allow a larger number of people to work on the story at the same time. This helps lower your WIP and decrease the leadtime for the story, which will give you more choices as to when you can start the story during the sprint.

Be careful not to overload a story - just putting more people on a story will not make it progress faster, it needs to be a part of the plan.

This might go back as a far as preparation where you could choose to break up stories a little further to allow them to be worked on in parallel during the sprint. You might need to negotiate on the independence of the stories you have broken down but it might save you from negotiating on what you can deliver.

Size (sort of) matters

Recently, I have been keeping a small set of metrics on the stories from my team. I started with simple dates so I could look at leadtimes - the time taken from when we start a story to when we deliver it.

With just these 2 dates you can do a lot! A lot of this came from looking over what was presented by Mattia Battiston at a meetup not so long ago. In particular, I was interested in his observation that there is very little correlation between story sizes and leadtime. This is something I have battled before so I was interested when my own data showed a very similar pattern:

Looking at this you could infer that a 2 point story might take as long as n 8 point story, which is hardly useful. What I missed for a while is the upper bounds across all the story sizes. We have a couple of anomalies at the top but the vast majority are within 10 days - which you should know is the length of our sprints.

That caught my interest. Although we cannot say with any sort of accuracy how long a 5 point story will take - in this example the longest was 26 days and the lowest was 4 days - we can say that 95% of 5 point stories can be delivered within 10 days or less. You can repeat a similar statement for story sizes 8 and below.

So maybe we should revise what we are expecting from our estimation by the team. What we are very good at doing is working out what can deliver within our sprint window. What we actually want to know is how many stories we can expect to be delivered if we want to use this to forecast, which is our throughput.

For me, looking at leadtimes influenced how we were working on and preparing stories. Littles Law describes how we should see a lower leadtime if we lower our WIP. We think more about how more of us can work on a single story, this often starts in prep. We open less and this lowers WIP, decreasing leadtimes.

Having a leadtime which only just fits in a sprint window means we have limited options on how to plan the work in the sprint - we have to start everything on day 1. This would mean loosing fluidity - everyone would be one a single story from beginning to end. Not many options for us at all.

Lowering leadtimes by a few days gives us some flexibility in when we can start stories. We use our story points to influence when we start on things. Larger stories might need to go first but smaller stories run in parallel so that we can deliver throughout the sprint rather than everything at the last minute, which would probably create a bottleneck with testing.

For me the sizes of stories begin to matter less and less. They serve their purpose during prep and planning and help us get to the answer: "Will this fit in our sprint?". When we see a 13 we look to break it down further since we can see it is rarely delivered in less than 10 days, which does not give us enough options. 

If we bring something in mid-sprint, we can look at the previous leadtimes and give a percentage "75% of previous 5 point stories have been completed in the time we have left" allowing us to make a commitment with some accuracy on success.

There are many more interesting things you can gleen from leadtimes and some basic spreadsheet fun, Mattia's presentation is a great place to start.