As many of you already know, in the past few months we have been working on a comprehensive survey about the State of Engineering Productivity.
The objective was to identify what makes engineering teams productive (and unproductive), and how teams perceive and define productivity itself.
For this whole work, I was glad to partner with my friends at Hatica — we put together our respective communities to create the very best research we were capable of.
Hatica is an engineering management platform that aggregates work activity from apps in the tech stack of engineering teams, produces actionable insights into user-friendly dashboards, and helps improve the overall developer experience.
We followed these steps:
We ran some initial 1:1 interviews with selected members of our communities.
We used those insights to create a wide survey about personal & team productivity, collaboration, dev process, and the usage of metrics.
We distributed the survey and got 200 thorough answers from developers, managers, and tech leaders across the world.
We cleaned up the results, gathered insights, and wrote this commentary.
We are now publishing the first edition of this report, exclusively to Refactoring subscribers!
Here is what we will cover:
🙋 Demographics — who we reached out to: roles, company size, and work setup.
🎽 Team vs Personal productivity — how engineers and managers rank themselves and their team, based on their role and work setup.
🚀 Productivity offenders & boosters — broken down by several dimensions.
📊 Engineering metrics — which data teams use to understand productivity, and how.
Let’s dive in!
🙋♂️ Demographics
We got 204 answers to the survey.
We intentionally went for quality over quantity, which means the survey was substantial, with plenty of free-form questions that took a while to answer.
We took this route because productivity is a nuanced topic, and we didn’t want to pidgeon-hole answers by making people choose from predefined lists for things like bottlenecks, processes, and stuff they would like to do better.
A lot of free-form answers also mean a lot of manual work to clean up, categorize, and collect insights from such data. We reviewed every single answer, attached tags to it, and drew correlations.
Here is more about the people who joined this:
💼 Role
Respondents are split almost evenly between ICs (from regular Software Engineers up to Staff+ Engineers) and Managers (EMs, Heads, Directors, VPs, up to C-level).
👥 Company Size
One out of three comes from small teams with less than 25 engineers, and one out of four from big tech with 500+ of them.
🏢 Work setup
The most popular work setup is hybrid, by far.
77% of respondents work in companies that combine remote & IRL work, but 29% only work remotely in those companies.
There is also a 17% from remote-first, no-office teams, and a small 7% from strictly co-located ones.
🎽 Team vs Personal Productivity
We asked people how they would rate their personal productivity from 1 to 10, and the one of their team.
For various psychological reasons, broad questions like this always average around 7, so no big insight here — but it is interesting to break answers down by role and work setup:
1) Hybrid is the most common, but also the worst
Extreme setups, that is, either remote-first or 100% co-located, get the highest scores both in personal and team productivity.
This confirms what we have frequently heard: hybrid setups are the trickiest — even though they are the most common.
It is tough to create an environment where remote folks are not at a disadvantage with respect to those who meet IRL more frequently. From handling impromptu meetings, to getting face-time with managers, keeping the playing field level is hard.
Hybrid setups often create a sense of disparity, fueled by a lack of visibility into information and progress among team members.
2) Your manager feels more productive than you
Another interesting result, which probably won’t help with managers’ reputation among engineers, is that the higher people rise through the ranks, the more productive they feel — to the detriment of the rest of their team.
While software engineers feel on average less productive than their team at large — making the case for widespread impostor syndrome — the same isn’t true for managers and Staff+ ICs, who generally feel significantly better about themselves than how they feel about their teams.
As we will see, factors reported as impacting personal productivity (negatively or positively) significantly differ based on people’s roles, which, combined with the above, points at a general disconnect between the needs of regular engineers and that of their higher-ups.
🚀 Productivity offenders and boosters
We gave respondents two free-form questions about productivity:
⬇️ What is the most common offender to your productivity
⬆️ What is something that makes you more productive
Here are the results, based on the tags we put on individual answers:
So, when it comes to what’s the most annoying during work days, meetings and context switch turn out to be the worst offenders, with lack of clarity being a distant third.
It is worth noting, however, that these items are not orthogonal. Too many meetings cause more context switch, just like too much context switch generally points to a lack of clarity.
In fact, this gets much clearer when you look at answers about productivity boosters.
What people truly need is simple — it’s two things:
🔭 Clarity on what to do
🧘 Time and space to do it
While we found this to be universally true across the survey, we could isolate different flavors based on roles and work setups.
1) Productivity for Software Engineers
Engineers suffer less from meetings than the other cohorts (managers, tech leads, directors, …) likely because they have to do less of them. Conversely, they are more impacted by bad dev processes and cumbersome releases.
Being blocked by others also ranks disproportionately higher than for other roles (8%, vs 2% on average), which is also a bad process smell.
The recipe for productive engineers looks like simply to give them clear goals, and equip them with good DX and a lean dev process.
[I need] more clearly defined objectives, priorities, and vision for what we're building.
— Software engineer on a full-remote team
2) Productivity for Tech Leads
Tech Lead is a tricky role that is sometimes trapped between coding and management duties.
So it’s not surprising that the biggest threat for them is being unsure about what to focus on — TLs are the category that suffers the most from context switch and lack of clarity.
Coherently with the context switch issue, they are also the category that prizes working from home the most (35% vs 15% on average), as beneficial for their productivity.
Working from home when there's clear goals and no need for meetings
— Tech Lead on a hybrid team, about what makes them productive.
So, looking at individual answers and crunching the numbers, the recipe for effective tech leads looks like:
🍱 Setting clear boundaries — between their duties to avoid confusion about priorities.
🧘 Enabling appropriate focus time — and limiting meetings and interruptions.
3) Productivity for Staff+ Engineers
Staff+ Engineers report similar issues than Tech Leads, but with an even stronger focus on clarity, and a strong desire for autonomy.
Staff+ roles are hard to generalize, but they are usually about large scale, cross-team initiatives with strong impact.
As such, their work is subject to several dependencies from other teams, which shows up in their need for autonomy across all cohorts (13% vs 4% on average) being the highest.
Autonomy, trust, clarity on what to do and focus time
— Staff engineer in a full remote, 25-100 engineers team
4) Productivity for Engineering Managers
Managers are the largest cohort across all respondents. While this leads to less deviance from average values, some differences are substantial nonetheless.
With respect to the other categories — especially those with strong individual contribution — managers benefit less from isolation, in the form of fewer meetings and working from home.
While they still have a strong need for focus time, that shouldn’t come at the expense of the necessary meetings, which are seen as a necessary evil.
Conversely, some of the managers’ needs are radically different from those of the other respondents. Receiving more feedback, understanding their impact, and enforcing clear deadlines, all rank disproportionately higher (>3x) than in other cohorts.
While managers feel the same need for clarity as others, they seem less sure about the nature of their work, seeking more collaboration and feedback.
5) Productivity by work setup
We have found that work setup (full remote / hybrid / office-based) has a huge impact on what makes individuals and teams productive.
The first thing to notice is that percentages are just lower overall for full remote setups. This is possible because people could answer by entering multiple things (which is also why if you sum percentages it gives more than 100%).
This means that full remote folks just feel more productive, which was already on full display in productivity ratings from before:
People from these setups also complain about different things: full remote teams have better dev & release processes, spend less time on maintenance, get less distracted and less blocked by others, while also doing less meetings.
Conversely, they worry about engagement and doing meaningful work. However, these concerns 1) feel healthier to us — like a higher layer on Maslow’s pyramid, and 2) don’t show up at an alarming level anyway.
Not being motivated by the specific task I ought to be doing
— Tech lead on a full remote team
📈 Engineering metrics
We wrapped up the survey with a series of questions about the usage of engineering metrics for productivity. This has been a hot topic lately, which we covered both in the newsletter and in the podcast multiple times.
Here are some of the strongest insights from the survey:
1) Metrics usage is early
Only 35% of respondents’ teams track and use engineering metrics. This also varies a lot based on work setup: the majority of full remote teams use metrics, while only 17% of office based ones do.
2) Velocity & Cycle Time are #1
Velocity and Cycle Time are the most popular engineering metrics, closely followed by the full set of DORA metrics (which cycle time is a subset of), and PR turnaround.
There is a difference, though.
Velocity and Cycle Time seem to be used by different types of teams: full remote teams who use metrics largely prefer Cycle Time and DORA, while hybrid and office based ones use Velocity the most 👇
I believe this gap is a reflection of different eras, or generations, of dev practices.
Cycle Time and DORA are backed by modern research and it is not surprising that they correlate well with full remote teams, who are just more likely to be young and forward thinking than hybrids, which in turn are more likely to operate on a Jira+Scrum+Velocity model.
Again, nothing wrong with the latter, but this gap in opinions and practices does exist for the better or worse, and is reflected in the survey numbers.
3) In-house still wins over dedicated tools
Perhaps surprisingly, most teams who track metrics do so via either custom dashboards, or through the tools they already use for ticketing and version control.
Jira is especially popular for inspecting Velocity and Cycle time — it may not have the nuance of dedicated tools, but it has the advantage of being already there.
4) Metrics’ bad reputation is overrated
Finally, we find that metrics’ supposedly bad reputation across engineers and managers doesn’t match reality.
Most respondents (60%+) are positive about their impact, and numbers don’t vary substantially based on roles (e.g. engineers vs managers) or current usage (those who already use them vs those who don’t and would like to try).
Measuring the productivity of our team as a whole and fostering a sense of individual responsibility towards team productivity has a great effect on individual productivity
— Engineering Manager on a large hybrid team that tracks DORA metrics.
📌 Bottom Line
And that’s it! So, here are our top 3 takeaways from the survey:
1) Hybrid is hard
Teams who work in hybrid mode have the hardest time.
It is tough to create an environment where the playing field is level for everybody: there is less incentive to build a truly remote-friendly environment than in full-remote teams, while at the same time you don’t get the full benefits of co-location.
Committing decisively to one of the two extremes might bring better results in terms of productivity — but of course it can be more challenging in other departments.
2) Productivity is mostly enabled by clarity and focus time
The recipe for good productivity looks surprisingly simple, and it stays the same whatever your role and work setup:
Good clarity — I know what is the most important thing I should do.
Plenty of focus time — I have the space and time to go after it.
These look like simple items, but you only achieve them when you have taken care of everything else: context switch, meetings, collaboration, etc.
The degree to which engineers and managers suffer from this is also different:
Engineers suffer more from lack of focus time → they largely know what they should do, but would like less interruptions and more room to do it.
Managers suffer more from lack of clarity → they feel more productive in the sense that they do many things, but they are unsure if these are the right ones. They pedal fast, but in an unclear direction.
So, while for most engineers the direction is clear, but they would like to pedal faster, managers feel like they pedal fast already, but in a questionable direction.
3) Engineering metrics are early and feel like an opportunity.
Only 35% of the respondents use engineering metrics on their teams, but a strong majority (60%) look at them as an opportunity. This sentiment is shared equally by all the surveyed roles.
Also, more than 50% of teams who use them do so via custom tracking or by using the tools they already use (Jira, Github). Both setups are limited with respect to what you can achieve with dedicated tools, so there are likely strong margins for improvement.
Personally, I am a strong believer in using data to augment your understanding of how your team works. You can use metrics to spot trends, start conversations, and back initiatives. I wrote a full guide about them and it’s one of the most popular Refactoring pieces ever.
Again, this work couldn’t be possible without the support from my friends at Hatica, which provides engineering analytics to more than 600+ engineering teams. If you liked this report and its findings, you will like what they are building 👇
And that’s it for today! See you next week 👋
Sincerely
Luca
Great insights Luca and I'm pretty much aligned with the results. Thanks for sharing!
This is great, loved it! Thank You for sharing.