There is always a danger when being driven with early metrics, especially leading one, and the developers know this one very well: early optimization is the root of all evil. It's especially severe in subjective metrics like survey. Like Continuous Delivery told us, "if it hurts, do it more often". A good coach will know the early results are likely invalid, that when adopting a new approach, you will see a decline in performance, that people usually resist changes and feel uncomfortable before adopting them and ripe their benefits. And let not forget Goodhart's law: any metric that become a target is no longer a good metric.
This for me is a big warning sign I would have on any DX approach too focus on metrics. And don't get me wrong: metrics are a good thing, but only if used at the end of a good design process. Metrics should always be the last part: set up a goal, ask yourself a good question about that goal, and then measure to answer your question. A metric should always stay a tool for learning, not a tool for leading (although good leaders should always learn using metrics).
I agree about the caution! However, many leaders take these limitations as a reason not to measure stuff, which is wrong. We need to keep working on metrics to better understand them and figure out how to use them for good :)
Thanks for this article! Developer experience is something that is very high on my list of concerns. I have a question - do you know of any existing tools that can be used for experience sampling? I'd love to have this pop up on GitHub after merging a PR or sth like that
There is always a danger when being driven with early metrics, especially leading one, and the developers know this one very well: early optimization is the root of all evil. It's especially severe in subjective metrics like survey. Like Continuous Delivery told us, "if it hurts, do it more often". A good coach will know the early results are likely invalid, that when adopting a new approach, you will see a decline in performance, that people usually resist changes and feel uncomfortable before adopting them and ripe their benefits. And let not forget Goodhart's law: any metric that become a target is no longer a good metric.
This for me is a big warning sign I would have on any DX approach too focus on metrics. And don't get me wrong: metrics are a good thing, but only if used at the end of a good design process. Metrics should always be the last part: set up a goal, ask yourself a good question about that goal, and then measure to answer your question. A metric should always stay a tool for learning, not a tool for leading (although good leaders should always learn using metrics).
I agree about the caution! However, many leaders take these limitations as a reason not to measure stuff, which is wrong. We need to keep working on metrics to better understand them and figure out how to use them for good :)
Thanks for this article! Developer experience is something that is very high on my list of concerns. I have a question - do you know of any existing tools that can be used for experience sampling? I'd love to have this pop up on GitHub after merging a PR or sth like that
Thank you Kacper! Experience sampling was news to me and AFAIK Abi and DX are the first to experiment with it