Why You Should Measure Your Development Process 📈

And some practical, battle-tested advice to get started.

Hey 👋 this is Luca! Welcome to a ✨ monthly free edition✨ of Refactoring.

Every week I write advice on how to become a better engineering leader, backed by my own experience, research and case studies.

You can learn more about Refactoring here.

To receive all the full articles and support Refactoring, consider subscribing 👇

---

In 2004, when I was 15, I convinced my parents to drive 600km to make me meet my internet friends.

In the previous two years I had spent hundreds of hours playing Super Smash Bros Melee, a fighting game on the Nintendo Gamecube, and now the first national tournament would be held in a small city in northern Italy.

I showed up and closed in the top 10 🏆 but most of all I made a lot of new friends, many of which I am still in touch with today.

A Melee match between two of the top European players ☝️ Peach has some sick combos.

I was thinking at this after speaking with Dan Lines, and discovering he was a fellow Smash player too.

Dan is COO of LinearB, and last week I joined him on his podcast, Dev Interrupted. He had read my take on creating a measurable Pull Requests process, and wanted to expand the conversation on measuring engineering teams.

It was a great chat, and if you want to listen to the full episode, you can go straight here on Spotify:

The main theme is that today we have plenty of tools to setup a data-driven approach to the dev process. However, this is still uncharted territory for many companies, because of two big blockers:

• 🗣️ Culture: any new initiative that involves measuring people performances is bound to be controversial. And the cultural groundwork to make it happen and bring everyone on board is hard.

• 📚 Skills / Experience: most people haven’t dealt with any of this before, and genuinely don't know which goals they should set for this process, and how to build it from scratch.

We discussed these problems and ended up with some sort of FAQs for data-driven engineering. Check them below 👇

🙅🏼‍♀️ Engineers don't usually like to be measured

Many people have a gut reaction to having their work measured. They think metrics don't tell all the truth, and are not able to fully represent their contribution.

I agree with them!

Metrics provide an incomplete picture, but they also give clues that we would miss otherwise. In the end they are just a tool — they can be used for good or for bad.

The "rejection" feeling, in particular, usually comes from the fear that managers will take these numbers as-is and judge people upon them.

This is the dystopian future we don't want to get to. But I may also add that if managers are bad at evaluating dev work, they will probably stay so with or without numbers.

To win people over measuring, we—the managers—have to demonstrate we can put a process in place in which metrics are used properly, and for good.

❓ Why measure at all?

I start with a basic observation: it's hard to improve something you don't measure.

Think about losing weight without weighting yourself. It's not that you can't do that, it's just undeniably harder.

That's because measuring lays the groundwork for so many good habits on top — like setting goals, having reviews, retrospectives, and rally your team (or yourself) towards such goals. These are all things you can't have properly without a quantitative approach to your process.

Finally, measuring something signals that you care, and helps creating an improvement mindset in your team.

⚖️ How do we understand what is worthy to be measured? Or should we measure everything?

It's a good question because measuring doesn't come for free — to put numbers at work you should take the time to analyze them, create targets for your team and review things regularly.

Improvement doesn't come from number themselves, but from discussing them with your people and finding solutions to get better.

It's a lot of work!

So you should really be aggressive in tracking only what's valuable, and discard what's not. To me, it comes down to what's actionable. Ask yourself: will my decisions change because of the value of such KPI? Most of the times, to be honest, the answer is no. And in these cases you make yourself a favour not to measure it.

🔨 So, what would you measure for an engineering team?

There are multiple levels of metrics.

Some are useful at a function level, and are those that you would put, for example, in your quarterly Engineering OKRs. Then you have more granular ones, that you just discuss at team level — and possibly individual level.

For high level metrics you could track things such as Velocity, Testing Coverage, and overall Cycle Time. These are good KPIs that can be understood by other non-engineering teams.

At Team level, you can track more granular things, like metrics from the Pull Requests process, code churn, how much time is spent on new features vs refactoring — you can really get down to things that tell you how people work and where the bottlenecks are.

Examples of Engineering metrics to be used in different contexts 👆 the company / team / individual distinction is often blurred — but some metrics are definitely better in some contexts than others.

🎽 Are these team numbers? Individual numbers? Both?

We track both team numbers and individual numbers, but always with the goal of improving as a team. This is important to make sure people feel safe. They should not feel judged upon numbers alone, as we were saying at the beginning.

I like to think we behave like a professional sports team: we measure a lot of stuff, even for single players, but we win and we lose as a team.

So some individual numbers might be discussed in 1:1s, but never in team meetings.

🏁 How do I start?

If this is something totally new for your team, just start small.

You can do some research first, about which tools exist and what you can measure, then mix that with bottom-up exploration together with your team.

People's buy-in is crucial, so make sure to involve at least your tech leads / senior devs early on. Integrate an analytics tool and make them play with it.

You should tell them what you want to achieve, and they should challenge it, improve it and make you understand what's the best way to reach it — tracking which KPIs, and so forth.

And when choosing what to measure first, it's important to start with something people can relate with. I think Pull Requests are a good candidate — everyone has had the experience of waiting days for a PR, or having done a poor review of something that went to production and made everything explode.

People understand what a good PR looks like: it should be accurate, and it should be fast. So for us it was quite low friction to attach a few numbers to that.

📚 Resources

Finally, I list below a few tools you can use to measure your engineering process:

• LinearB

• Waydev

• Code Climate Velocity

• Pluralsight Flow

These are all excellent tools that provide you with stats about your team, and guidance on what to do with these numbers.

That’s it for this week! Do you track engineering metrics in your team? If yes, how do you do it? If not, are you planning to? Let me know in the comments! 👇

---

Hey, I am Luca 👋 thank you for reading this through!

Every week I publish some advice about product and engineering management. If you liked this post and you haven’t already, you can subscribe below and receive weekly updates in your inbox!