💸 AI spend is outrunning returns

This idea is brought to you by today’s sponsor, DX!

Last week I published my own commentary on the Q2 AI Impact report by DX, which among other things, showed how quarterly median AI spend grew 28x year-over-year!

My expectation is that this AI honeymoon will not last forever, and tech leaders may already face tough budget conversations in the back half of 2026. That’s especially because it doesn’t seem that such spend is leading to outrageous outcomes so far.

But we’ll see! You can find the full report below 👇

Check out the full AI impact report

🌱 Build relationships before you need them

Especially in tech, networking often gets framed in a very transactional way. You meet people because one day you may need a referral, an intro, a customer, or a job.

I think this is exactly the wrong way to look at it.

I have found that the best relationships are created before there is any immediate need attached to them. You build rapport around common interests, you understand what the other person cares about, you are helpful when you can, and you do not keep score.

Long-term, this is both 1) the better behavior as a human being, and 2) the better strategy.

Because you never really know who will be able to help you in the future, or how. Opportunities often come through weak signals, second-order connections, and people who simply remember that interacting with you was... pleasant. Before it was any useful.

So my advice is simple: do not wait until you need something to show up.

But how?!

It can be anything, really. Share useful material (”I read this article today and thought you might like it”). Make intros. Reply thoughtfully. Follow up when someone is working on something interesting. Be generous in small ways that compound over time.

I wrote more about this in this full piece on growing your network 👇

Refactoring

How to Grow your Network 🍻

For as long as I can remember, a good share of the people I admired have recommended me to grow my network…

Read more

4 years ago · 21 likes · 2 comments · Luca Rossi

🔧 Automate the copy-paste (not the judgment)

A lot of team process work is not actually process work. It’s copy-paste disguised as process.

Raise your hand if you ever updated a project management ticket because the real status lives elsewhere. Or you asked an engineer for a delivery date because the current dashboard is stale and can’t be trusted.

In my conversation with Antonia Scheidel, Director of Engineering at Duolingo, I liked her distinction between two kinds of information:

1. Information that already exists somewhere

2. Information that only exists in people’s heads

The first kind should usually be automated. E.g.:

• If an experiment was launched on the appropriate experimentation platform, humans should not have to copy that into Jira.

• If a stakeholder needs to know that a step is ready for review, the system should notify them.

The second kind, instead, still needs people.

In normal teams’ workflows, there are plenty of judgment calls that cannot be reliably inferred from a tool, and that’s where you want to direct human attention. Things like delivery estimates, blockers, timeline changes, tradeoffs.

So, the shape of good automation looks like:

1. Connect the systems of record

2. Remove the glue work

3. Ask people for the (remaining) information that genuinely requires their judgment.

Antonia built this successfully at Duolingo, and you can find the full interview here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 We 3.5x’d Our Pull Requests with AI: Now We Catch Fewer Bugs

5 min • by Pawel Brodzinski

A detailed, real-world write-up about the impact of AI on an engineering team’s code reviews. PRs went from 200 to 800 lines on average, and reviewers inevitably started missing more things. Worth checking out.

🥈 American AI is locked down and proprietary. It’s losing.

3 min • by Ben Werdmuller

Ben makes a sharp argument that open models may win for the same reasons open infra usually wins: distribution, portability, and permissionless innovation. I am not sure the geopolitical framing is the whole story, but I really like the business point.

🥉 Reverse-engineering is cheap now

1 min • by Simon Willison

Tiny post, big ideas. Agents make throwaway code extremely cheap, so entire categories of “not worth it” automation suddenly become worth trying. Simon makes the example of reverse-engineering home devices, which I have tried myself many times! If AI makes weird small projects viable, I am all for it.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

One of the things I like the most about the DX team is that they run an incredible amount of research. They run surveys all year round, and keep a pulse of where the industry is going around several topics — especially AI and developer experience.

They recently released the Q2 2026 AI impact report, the latest in a quarterly series tracking how AI is reshaping engineering teams. I regularly check out these reports, and one thing I noticed is how they gradually moved from discussing adoption to discussing impact.

The feeling is that AI is everywhere now, and the main challenge for engineering leaders is not to spearhead adoption anymore, but rather justify the exponentially-increasing AI budgets, by demonstrating what AI is actually delivering.

I got preview access to the report, and today we will comment on a few key highlights.

So here is the agenda:

• 📊 Speed is real but uneven — what 500+ teams reveal about where AI gains concentrate, and who’s being left behind.

• 🧱 Orgs are not multiplying AI output — developer experience is declining even as velocity climbs.

• 🔬 Code is easier to change but harder to trust — a surprising divergence between maintainability and confidence.

• 🚶‍♂️‍➡️ Spend is outrunning returns — AI budgets are up 28x, but innovation ratio is flat.

• 🧭 What this means for engineering leaders — how to shift from tool adoption to system optimization.

Let’s dive in!

I want to thank Justin and the DX team for partnering on this! I am a big fan of their work and you should check out the full report here.

Download the full report

📊 Speed is real but uneven

Across the 500+ respondents, over 50% of the code is now generated by AI. This metric has accelerated rapidly, increasing from 34% in Q1 2026.

Median weekly throughput rose 37% over four quarters (1.42 to 1.94 PRs/eng/week), and deployment frequency is up double digits across most segments. However, gains are concentrated among small organizations (less than 100 engineers) and tech-sector teams, pulling away from larger and traditional-industry teams, and the gap is widening.

This echoes what we have seen in other reports recently, from the Acceleration Whiplash, to the State of Software Delivery earlier this year.

The speed at which the tools keep improving makes gains compound for teams who invest into AI workflows, so the distribution of team productivity becomes a steeper and steeper power law, with the top percentiles pulling away from the pack.

🧱 Orgs are not multiplying AI output

DX measures developer experience with a proprietary index called DXI (we talked about it with Abi Noda), which, over the last four quarters, has dropped on average from 67 to 65.

Again this is not an even drop: AI is improving some aspects of the devex (docs quality, code maintainability, onboarding speed) while making others worse: larger PRs, slower reviews, less incremental delivery. In aggregate, the net effect is negative.

Median PR sizes have nearly doubled during the same period — and of course increases in PR size are an early proxy for technical debt, bugs, review friction, and all kinds of nasty stuff.

So, velocity metrics alone will tell you things are improving, but devex metrics will tell you that the system (and developers) are strained, and we are far from a sustainable, positive flywheel.

My personal feeling about this is that so far we have mostly tried to slap AI onto the existing steps of the dev process, while running them the same way we always have. This might bring incremental gains, and even work well for some of the steps, but it completely breaks others, like code reviews.

It’s obviously foolish to think that AI can make developers write code 3x faster (as it already does), while assuming we’ll need to keep reviewing every single LOC manually.

Read more

Brought to you by our friends at Atono!

You hand off a story to an agent. The code works, it ships, but it’s not what you meant.

As agents move faster, the decisions that shape what gets built need to travel with the work. Most teams haven’t systematized this yet, so the result is correct implementations… of the wrong thing!

Atono just published their From Intent to Production at Agent Speed guide, showing how product knowledge becomes your leverage point for keeping decisions intact, even at agent speed.

We’ve previously collaborated on pieces like The State of Product Development 2026 and The AI Productivity Paradox — this guide goes deeper into product delivery for agentic workflows 👇

Download the guide

1) 🏗️ Agents are great scaffolding

A mistake I see more and more with AI workflows (including mine!) is using AI for everything, even for parts that would be better done in other ways.

That’s easy to fall into, because when you are automating a new recurring task, an agent is often the fastest way to start. You can give it the goal in natural language, let it run the process end to end, and observe what happens. This is not only quick to do, it’s also useful because, at the beginning, you often do not know the real shape of the work yet.

So the agent helps you discover things like:

• which steps repeat every time

• which data is missing or messy

• where approvals are needed

• which parts require actual judgment

• and more

Once the workflow becomes clear, though, the boring parts should be extracted into something more deterministic: scripts, APIs, cron jobs, workflow definitions, or whatever tool fits the job. That makes the workflow faster, cheaper, and easier to observe and debug.

These days I try to keep AI where flexibility is valuable (e.g. judgment calls), and replace it where determinism is cheap. Most often, it’s a progression: I start with an agent because it is fast, observe where the task repeats, harden the stable steps, and leave the ambiguous parts to AI.

I wrote more about this in the full piece on orchestrating AI workflows 👇

Refactoring

How to Orchestrate AI Workflows

Hey, Luca here! Welcome to a weekly essay from Refactoring…

Read more

3 months ago · 60 likes · 1 comment · Luca Rossi

2) 💬 Giving feedback should be ordinary

Feedback is often treated like a special event. It’s the performance review, or the difficult conversation we should have had long ago, or the moment when something went wrong enough that somebody has to say something.

The problem is that, if feedback only appears in those moments, people learn to associate it with danger. They hear “can I give you some feedback?” and brace for impact.

In my conversation with Jean Hsu and Cate Huston, two professional coaches, Cate shared a better habit: every time someone does something good, tell them.

When people receive frequent, specific positive feedback, they learn that you are paying attention to their work as a whole, not only waiting for mistakes. Then, even when you eventually need to say something corrective, it lands differently.

You can also spread this beyond yourself. When someone praises a colleague privately, ask: “have you told them?” Over time, feedback stops being capital-F Feedback and becomes normal signal in the system.

You can find the full interview with Jean and Cate here 👇

3) 📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 The Tower Keeps Rising

3 min • by Armin Ronacher

I really like the Babel analogy applied to how we cooperate as humans and, ultimately, software. For the first time, “construction” can keep going even after humans have lost the shared language needed to reason about the system. Food for thought.

🥈 Generated and Suppressed Demand

3 min • by Will Larson

A useful mental model for why fixing a struggling team can feel like nothing improved. Sometimes the team gets better, and the reward for it is... more work! All the previously hidden demand finally shows up. Great management advice, as always from Will.

🥉 Control the Ideas, Not the Code

6 min • by Salvatore “Antirez” Sanfilippo

Antirez makes a clear point that many engineers are still resisting: if AI writes more code, reviewing every line is probably the wrong use of your time. You have more leverage when you focus on ideas and design.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by:

Unblocked is the context layer for modern engineering teams. Get a free three-week trial at getunblocked.com/refactoring

Today's guest is Anush Elangovan, VP of AI Software at AMD.

With Anush, we talked about how he thinks about open source as a key strategic driver in the AI era, and how his team is able to stay super fast while ensuring quality by combining their internal work with their external contributions.

And then we got into a rabbit hole about how to get big companies ready for AI, how to think about human capital, and how Anush personally spends his time. And spoiler alert, he is back to doing IC work for up to 40% of his time.

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute, handcrafted takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. 🔓 AMD’s open-source AI stack

2. 🤖 Navigating AI-generated code contributions

3. ⚡ Speed as the only moat

4. 🏗️ Getting legacy systems AI-ready

5. 🧠 Why leaders should get their hands dirty

⠀Let’s dive in 👇

Read more

That’s because I write a lot about my AI workflows with Tolaria, but many people (rightfully) tell me that things are more complex on a real team.

So I invited the team at Augment to share how they transformed their own engineering org, redesigning their main dev workflows. It’s free for everyone and you can RSVP below!

Learn more and RSVP!

Hi there! A lot of people these days ask me what AI models I use for coding, which I usually explain in my monthly recaps.

Some people also ask me “why” I use one model vs another, and that’s a way more complicated question to answer. AI models are hard to evaluate, because there are several factors to consider, and because, despite what public benchmarks say, each team situation is unique.

So today let’s try to think from first principles about how you should choose what LLM to use for coding, factoring everything in. Also let’s try to find some durable angles, so that you can create a mental model for yourself that stays valid for more than a couple of weeks.

Here is the agenda for today:

• 🔺 Accuracy / Speed / Cost — the three dimensions of model selection. But they are messier than you might think.

• 📊 Public benchmarks are not enough — you can’t trust leaderboards, and how your codebase discipline sets your floor.

• 🧪 How to evaluate AI models — a maturity path from “just don’t” to building your own eval.

• 🔀 Model routing — not all tasks are equally hard, so stop paying frontier prices for all of them.

• 🏠 Self-hosting: almost never — cloud-hosted cheap models beat your own hardware, every time.

Let’s dive in

🔺 Model Selection 101

With some degree of simplification, today you have three macro choices:

1. Top frontier models — e.g. Fable 5 or GPT 5.6

2. Large open weights models, hosted in the cloud — e.g. GLM 5.2 on OpenRouter

3. Smaller open weights models, self-hosted — e.g. see DeepSeek 4 Flash via DwarfStar by Antirez

These choices are actually more nuanced than this, because e.g. within frontier models, you may tweak the level of reasoning, or intentionally pick a smaller model that is cheaper and faster, because you feel it is a better fit for your task.

In fact, when judging models, there are three dimensions to consider:

• 🔬 Accuracy — % of times the AI gets the thing right

• ⚡ Speed — tokens per second

• 💸 Cost — cost per 1M tokens (multiplied by tokens required)

To make things even messier, these dimensions are not orthogonal. For example:

• A model with higher accuracy may translate into fewer round trips, which may translate into fewer tokens being used.

• With many models you can reduce the amount of reasoning to improve speed while reducing accuracy.

So, in most cases, performance is tricky to evaluate. Top frontier models guarantee max accuracy, but do you really need it?

Most people have anecdotal experience of small tasks for which fast + cheap models delivered the same level of accuracy. But how do you recognize them?

Also, it’s trickier than that, because even for bigger tasks you might find that a faster model, while less accurate, iterates faster over mistakes, leading to a lower cost overall. That’s what Jamie Turner, founder at Convex, was already saying last year:

We optimize for fast feedback loops over perfect output. It’s better to get 80% correct code in 2 seconds than 95% correct code in 2 minutes.

Jamie made the case for optimizing for a quick feedback loop, so that developers don’t lose their train of thoughts while waiting for slow responses, even at the expense of a few more errors.

This is even more compelling when you factor in that with the current crop of LLMs, accuracy and cost are not in a linear relationship: a model that is 80% as accurate as the top frontier one might cost 1/10th of it.

Two good examples are GLM 5.2 and Grok 4.5. They both score pretty close to the top OpenAI and Anthropic models, while costing a small fraction of them.

Looking at these numbers is a good way to start some kind of quantitative reasoning around models, but I don’t think it’s enough 👇

📊 Public benchmarks are not enough

I have grown skeptical of public benchmarks, for two reasons:

Read more

🪜 The 8 Levels of Context Maturity

This is brought to you by our friends at Unblocked!

The token spend shows that AI is in your engineering workflow, but the throughput doesn’t!

Raise your hand if you can relate. A lot of the times, this is a context problem. I personally invest a lot in making sure AI gets good context about past decisions, what exists, and what needs to do, and I get a lot of help from Unblocked about this.

On July 23rd, the Unblocked team is running a free webinar that maps the 8 levels of context maturity: where most teams are stuck, what the ceiling looks like at each stage, and what it takes to make the most out of your agents.

Very recommended 👇

Join live July 23rd (FREE)

1) 🗳️ Anticipate exec questions

Whether you are an engineer or a manager, one of the most underrated skills you can have is making your ideas easier to approve!

This is often tricky, especially when presenting to executives, who may be removed from our work and don’t understand technical details. In this case, you can put yourself in their shoes and *anticipate *many of their questions, like:

• How much will it cost?

• What do we get for the money?

• Why now?

• What happens if we do nothing?

• What other options did we consider?

• How will we measure success?

• What are the risks, and how do we reduce them?

• Who owns delivery?

• Which other teams or business areas are affected?

Some of these will feel annoying or nitpicky, or even not really applicable to your situation, but you may get asked them anyway! The more senior someone is, the wider their span of responsibility becomes, and the less time they can spend understanding any single problem. Expect broad, sometimes even off-topic questions.

So your job becomes to compress the important context for your interlocutor, translating complexity into things like trade-offs, outcomes, risks, and timing.

A good proposal should make the obvious executive questions boring, because they are already answered. That is how you turn a good technical idea into something the business can actually say yes to.

I wrote more about this in the full piece with Anna Shipman on the Engineer → Executive Translation Layer 👇

Refactoring

The Engineer → Executive Translation Layer 🔀

Hey, Luca here! Today we are hosting a fantastic new article by Anna Shipman, CTO at Kooth…

Read more

9 months ago · 31 likes · Luca Rossi and Anna Shipman

2) 🎧 Quiet leadership prevents noisy emergencies

Tech culture loves the person who saves the day.

The engineer who stays up late to fix production. The heroic intervention that makes everybody clap.

These days I rewatched my first conversation with Rands, and I love how he pushed back on this instinct. Rands simply says: if someone saves the day, the first question should be “why did the day need saving?”

A lot of good leadership is invisible because the bad thing simply never happens. It’s when:

• You act on a yellow flag before it becomes a red flag.

• You give feedback early, before resentment piles up.

• You listen long and hard enough to understand what is actually going on, instead of rushing to action.

Nobody claps when nothing explodes, but that’s how it works.

To make this possible, Rands stresses over the importance of listening as an active leadership tool. You need to be collecting the weak signals, the small patterns, which lead to equally small (but crucial) interventions every day.

You can find the full interview with Rands here 👇

3) 📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 Viability of Local Models for Coding

9 min • by Birgitta Böckeler

A useful reality check on local coding models. The short version is they are getting much better, but the constraints are still very real: RAM, speed, context, tool calling, and reliability all matter. Great piece by Birgitta who we also interviewed last year.

🥈 In Defense of AI Mandates

4 min • by Charity Majors

I love how Charity frames AI mandates as a funding device. This means. if leaders want people to learn a new way of working, they have to make room for the temporary slowdown, the annoyance, and the mess. Charity was also one of the very first guests of our podcast.

🥉 We should be more tired than the model

2 min • by Vicki Boykis

An interesting take that I largely do not agree with, but gave me something to think about. I love the idea of using LLMs in a way that I can *grow *(as opposed to my skills atrophying here and there), but I don’t see myself going back to writing code manually in order to do that. That’s too big of a tradeoff. But will look for ways to make this more palatable!

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by:

Unblocked is the context layer for modern engineering teams. Get a free three-week trial at getunblocked.com/refactoring

Today’s guest is Geoffrey Litt, Design Engineer at Notion.

With Geoffrey, we talked about malleable software, a type of platform that kind of always existed, but is now getting new meaning and opportunity thanks to AI.

We also discussed how Notion itself changed its place in the ecosystem, with the AI agents first, and now with the new developer platform, where you can ship code directly inside Notion. And finally, we talked about how Notion builds Notion, design as shipping code to production, internal prototyping, feature flags, and more.

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. 🧱 What is malleable software?

2. 🚲 The nightmare bicycle and progressive disclosure

3. 🤖 Notion’s AI agent platform and developer tools

4. 🎨 Design engineering and shipping code to production

5. 🔮 What’s next for Notion

⠀Let’s dive in 👇

Read more

Every week I write an article to 170K+ engineers about how make good software together, and interview a world-class tech leader. I also build and maintain Tolaria in the open, publishing my workflows and learnings here.

Subscribe now

I happen to know many startups who develop B2B products for large companies. I worked in one, invested in another one, and have friends who work in others.

An evergreen problem about these products is organizing work in a way that successfully takes into account:

1. Roadmap / long-term product strategy

2. Impromptu requests from key customers

This is tricky from all points of view: business, tech, and team organization. I have thoughts about this, so let’s start from first principles. Really the very basics.

We build tech to solve problems to customers, that pay us in return for it. At any given time we can have many of such customers. Based on this, the type the work we can do to solve such problems lays on a spectrum from bespoke to one-size-fits-all:

• 100% bespoke — is consultancy work. You just solve the problem, IP is of the client, zero reuse.

• 100% one-size-fits-all — is product work. You develop features that can be used by all (or many of) the customers of your product.

These types of work are very different but, fortunately, in most real-life situations it’s obvious where your work should land on the spectrum.

First of all, a lot of the ambiguity is cleared up upfront, by deciding the type of business you want to build. Knowing that you want to build an agency vs a B2B SaaS takes a lot of cards off the table:

• If you run an agency, the problems you solve change all the time, and there is most often not a clear path to building reusable pieces.

• Conversely, if you ran a B2B SaaS with e.g. thousands of customers, any of these customers might want something custom, but it’s obvious that it’s not worth to build it just for them.

About the latter, though, what if these customers are not thousands, but hundreds — or maybe just tens — and each of them is very valuable to you?

That’s where things get messy. Welcome to enterprise B2B SaaS.

So here is the agenda for today:

• ⚖️ Customer work vs product work — why enterprise B2B creates a unique tension between different types of work, and how you should think about them.

• ⁉️ Is it worth to do bespoke customer work? — the right questions to ask yourself before committing to custom work for a single customer.

• 🎽 How to design team structure — use dedicated roles like solution engineers of FDEs to shield your roadmap from customer requests.

• 📏 How to design tech strategy — empower solution engineers with great platform, so they move fast and build things close to product shape.

• 🔀 How to converge — treat customer work as validated product input, and create avenues to merge it back over time.

Let’s dive in!

⚖️ Customer work vs product work

If a customer of yours is paying you $100K for a yearly contract, instead of e.g. $2K, and you have, say, only 12 of those customers, all of a sudden their requests, however absurd, look... different.

This customer has leverage — and you have probably spent a lot to acquire them, so it can make complete business sense to do something specific to make them (and only them) happy.

We said this is a spectrum, but for the sake of simplicity let’s call this type of work “customer work“, as opposed to more general “product work“, which is done in the service of everyone.

The problem with customer work vs product work is that they have completely different shapes, when it comes to fundamental factors like:

• When and how such work spawns

• How urgent it is

• How big it is

• How it fits your overall strategy

One of the goals of your standard product workflow is to control for these factors, in order to normalize them. So items you plan for are: comparable in size, planned in timely fashion, all fit your strategy, and so on.

Customer work spawns, by definition, from external forces, which makes all of the above out of your control. It also doesn’t land with the same level of discipline that you might put into your product process, so it’s work that is often: urgent, big, and unrelated to your plans.

This leads to questions:

• When is it worth to do these things?

• How should I design my company accordingly? Speaking of team, tech, and process?

Let’s start with the first one.

Read more

🎟️ Join our webinar on how to become an agentic engineering org!

On July 22nd I will host an exclusive webinar together with the team at Augment Code!

Most teams adopt coding agents and see a 20% lift at most, and then plateau. The most common reason is that the bottleneck moves, and you should lift your whole dev process, not just coding.

In this session, the Augment Code team shares how they transformed their own engineering org by redesigning one workflow at a time, and demos two of them live on a real codebase: agentic code review and automated incident management, where agents now resolve the majority of incidents end-to-end.

You’ll leave with a playbook for finding your org’s current bottleneck and a workflow you can replicate with your team this week 👇

Learn more and RSVP!

1) 🧹 AI makes code hygiene enforceable

I have written many times about this recently, but I’ll keep hammering on this point because it is so important, and I believe many people haven’t fully grasped it yet.

Good code hygiene is, at once, 1) something everybody agrees with in theory, and 2) quietly and often sacrifices in practice, because it’s hard to maintain.

Things like good test coverage, small files, high cohesion + low coupling, low cyclomatic complexity, have all historically needed people who care enough to push for it, review for it, and absorb the cost of fixing it.

For the first time, AI allows to turn hygiene into a system that can be enforced at almost no additional cost. You can give agents rules about how to write tests, keep files small, or refactor before adding new behavior.

Many of such rules can also be enforced by gates: coverage checks, code health thresholds, linters, static analysis, and more. Guidance tells the agent what the floor looks like, and gates make such floor non-negotiable. And you can design these checks to run early, so the feedback arrives while the work is still cheap to fix.

The #1 concern of engineers about AI coding is still code quality, but you can actually use AI to increase the code quality of your team. I wrote more about this in the full piece on the compounding software factory 👇

Refactoring

The Compounding Software Factory 📈

Hey there, Luca here…

Read more

3 months ago · 58 likes · 2 comments · Luca Rossi

2) 🪞 Find a broader identity for yourself

A lot of the anxiety around AI is really about who you are (and who you are going to become) as a professional, rather than about tools.

Based on your sense of identity, AI can feel either like an opportunity, or a threat.

If you define yourself narrowly — e.g. “I am a backend engineer who is great at algorithms”, or “I love the thrill of solving puzzles with code” — then AI might feel like it is walking into your house and stealing furniture.

Earlier this year I talked about this with my friend Thiago Ghisi, who had a long engineering leadership career, and then used his sabbatical to step back and look for broader patterns in his life and work.

He is now studying clinical psychology— and sure, if you frame him only as a Director of Engineering, this might look like a strange detour. But if you frame him as someone who helps high-achieving people grow, you can suddenly connect more dots.

I think this is a useful exercise for engineers too. Instead of thinking in terms of skills, ask yourself:

• Why do I do what I do?

• What is a broader version of my role?

• What kinds of problems do I actually care about solving?

A broader identity makes you more adaptable, and opens up more opportunities. You can find the full piece below 👇

Refactoring

Finding yourself in the AI era ✨

Hey, Luca here! This is a ✨ bonus free essay ✨ from Refactoring. Based on our schedule it should have been a paid one! But I really care about this topic so I decided to make it free for everyone…

Read more

6 months ago · 77 likes · 6 comments · Luca Rossi

3) 📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 The Cost YAGNI Was Never About

4 min • by Kent Beck

I love how Kent reframes YAGNI for the AI era. The cost has never been writing the actual code, but rather spending optionality too early, and pulling investment forward before the feature can pay you back. Great article everyone should read.

🥈 The Wrong Abstraction

4 min • by Sandi Metz

This is a great segue to the Kent’s article above. Duplication is usually cheaper than the wrong abstraction, because the wrong abstraction eventually collects all kinds of bad stuff — parameters, conditionals, and sunk-cost arguments. Sometimes the fastest way forward is going back, but this is usually very hard to do.

🥉 AI and Liability

4 min • by Bruce Schneier

Including this great piece by Bruce even if it’s not properly about software engineering, because it’s so relevant today. Companies should be liable for what their AI agents say and do, just like they are for employees or published content. If your product presents an answer with your logo on it, you should own it, and not use “the model said so” as an escape hatch.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

By now, developing Tolaria is a big chunk of my working time (about 40%, to be precise). I like this balance and I am happy I can regularly write articles about AI coding coming from my own experience, as opposed to pure research.

I also don’t want to end up writing only about that, and so far I have kept a schedule of about once a month. This feels like is a good cadence: it’s feel overwhelming for readers, and gives me time to produce actual updates to the workflow, as opposed to posting always the same things.

So, today I am covering a few workflow updates, but most of all I am introducing a new set of partnerships I am very proud of.

So let’s dive in!

💧 The State of Tolaria

First of all you may wonder: how is Tolaria doing? Pretty well, if you ask me.

From a growth perspective, it has now almost 18K stars on Github, 100K+ downloads, and thousands of daily active users.

From a product perspective, this month we delivered a ton of updates, including some massive ones, like the support for native spreadsheets.

You can now create and edit spreadsheets inside Tolaria with Excel-compatible formulas in a completely open and portable format, which is basically CSV with a frontmatter on top.

We also improved a lot of the editor experience, adding collapsible headings, block navigation, better spacing and typography, all while making everything faster than ever. If you haven’t tried it in a while, you definitely should!

I really enjoy developing (and using!) Tolaria, so the question recently has rather become: how do I make this work sustainable, while keeping everything free and open?

Enter our tech alliance!

🦸‍♂️ Creating our own Tech Alliance

The tools I use to develop Tolaria are no secret — you can figure out most of them by looking at the repo, the AGENTS file, hooks, and my own articles.

So, over the last month I reached out to the ones I use the most to figure out if we could work together to make the development of Tolaria more sustainable on my end via open source sponsorships, in return for me spreading the word about how I use such tools, and providing good feedback on how to improve them.

The result is an alliance that as of today includes four tools: CodeScene, CircleCI, Codacy, and Unblocked.

These partnerships allow Tolaria to stay free and open forever, and even give us the opportunity to accelerate by hiring a great product engineer (if you think you are a great fit, please reach out and say hi!).

So today I will tell you more about my workflow, which includes these tools — but first an ethical statement: I will only ally (and stay allied) with tools I personally use and believe are the best way to do what they do. As you will see these are non overlapping: each has its own job, so they really make for my tech stack. Over time we will also expand on this and create exclusive deals for Refactoring subscribers!

Before we talk about the tools, it’s worth refreshing how I think about controls for code written by AI. I have written about it in this recent piece 👇

In a nutshell, I steer AI output in three main ways:

• ↪️ Guides — instructions in the AGENTS file and skills. Not 100% reliable: the AI may or may not follow them.

• 🔄 Gates — deterministic checks that don’t let bad code move forward if some conditions are not met.

• ↩️ Guards — fallback procedures that typically run once a day to fix what goes through the cracks.

Out of these, the most important part is the gates, because they have the (deterministic) power to stop bad code, so the more I can express this way, the better. Most of the tools below are integrated this way.

So let’s go 👇

🩺 Code Health — with CodeScene

CodeScene probably needs no introduction because I have written about it many times here. I am a big fan of Adam Tornhill‘s work, which to me is about two big things:

• A proprietary algorithm to calculate code health, that considers 20+ factors.

• The “hotspot” approach, which looks at git history and suggests refactoring targets prioritizing files that are changed the most often.

I integrate CodeScene as a hard gate in local hooks: new code can’t be committed if it’s not a 10/10. For modifying old code, the AI needs to follow the boy scout rule: leave the code better than it found it. So it measures the code health before touching the file, and after, and the score needs to be the same or higher.

After any commit that improves the overall codebase health, the AI also increases the thresholds, to create a positive flywheel.

After four months working like this, now the code has perfect health, and the current threshold is literally 10/10!

🔒 Security and Quality — with Codacy

Codacy works on code quality too, but from a different angle: rather than scoring files, it surfaces individual issues.

Issues come with severity levels and belong to various categories, like security, performance, compatibility, code being error prone, and more.

Codacy has an MCP that I use to steer my Codex locally, and to implement an actual blocking gate before it pushes any code. All new code has to be issue-free, and when touching old code, the AI also needs to fix existing problems, other than making the business changes.

I am also working with the Codacy team on their new product Verity (you can join the beta for free here), that aims to steer agents’ work by creating a knowledge base of past decisions.

This is interesting to me because I do that with ADRs, and would love to create more reliable controls against code that violates them. Verity is promising about that, and should also track token efficiency and a bunch of process metrics that I am interested in improving over time.

✅ Validation — with CircleCI

A few months ago I made a change in how I run our test suite, creating local hooks with Husky so that AI agents can immediately see if controls fail, instead of waiting for the remote CI.

This was very effective at improving the agents’ work, at the expense of completely hogging my machine every time. Some parts of the test suite are particularly heavy (e.g. playwright), and running them in local means the whole thing takes time — about 15 mins — and parallelizing things is out of question, because resources are not enough. It’s tricky to run even just 2 worktrees at the same time: things sometimes literally crash.

I talked with Rob Zuber from CircleCI about this, and they implemented a great solution: local hooks that are run remotely, in what they call Chunk Sidecars.

Sidecars are preconfigured environments that run alongside your local workflow, and validate changes as they happen.

This allows to 1) free up your machine resources, and 2) make things faster by introducing real concurrency.

This way I have been able to reduce the suite run time from 15 to 4 mins, while also being able to spawn more worktrees safely. A massive win!

📖 Fetching knowledge — with Unblocked

Unblocked is a context engine: it takes information from several sources (e.g. Github, Notion, Slack, Datadog, ...) so you can ask questions about it.

I have known Dennis, Claire, and the team at Unblocked for a long time, but didn’t think of using it for Tolaria until just recently, because I thought that Unblocked would shine when you have many information sources, and we legitimately don’t have many.

I was wrong.

I plugged it about one month ago, connecting it to Github and Sentry. Sentry is useful to triangulate issues, but the Github connection would basically be worth by itself, because it indexes the code, the ADRs, and all the history of changes.

I have been asking questions about how we do this and that and checking side by side with Codex (which instead just greps and fetches things on the go) and the difference in speed and accuracy is just obvious.

Even more importantly, Codex itself uses Unblocked via the MCP to understand past decisions and fetch relevant ADRs.

Big props to the Unblocked guys because this was an easy win and largely plug-and-play!

🔭 What’s next

Now that Tolaria has sustainable funding, I am going to invest even more on it. Again, I am exploring hiring a strong product engineer, so if you think you can be a good fit, feel free to reach out and say hi!

As for future work, the most important thing ahead is the mobile version.

I already have a working prototype for iPad, but there are still a lot of rough edges. Also, the iPad version is the easy part: for smartphones we need to completely reinvent the UI — I have been working on it 💪

And that’s it for today! I wish you a great week

Sincerely 👋
Luca

1) 🧭 Organize information based on action

Most knowledge bases slowly become piles of uninteresting information. After a while everything starts looking the same: meeting notes, ideas, projects, living side by side and you don’t know exactly how and why.

I believe when you store a new piece of information, there are two questions you need to ask yourself:

• What is this?

• What is this useful for?

The second question is often overlooked, and it’s a core idea behind Portent, which I published last month.

Portent includes default types — which answer the question “what is this” — but also a set of default relationships (belongs to, related to, etc) to help you figure out how to connect notes together, based on what they should help you with.

So, a meeting may belong to a project, a procedure may belong to a responsibility, a resource may be related to a topic, and so on.

Connecting notes together is hard work, but it’s paramount work! Otherwise you easily get stuck: you optimize too much for memory, and not enough for motion.

The goal is to know what to do next, rather than remember more stuff.

You can find the full article below:

Refactoring

Introducing Portent ☘️

Today it’s almost one month since I released Tolaria, and during this time I have spoken with many people and teams that are trying to use it to create and manage internal knowledge bases…

Read more

2 months ago · 63 likes · 7 comments · Luca Rossi

2) 📊 Treat code health as a data product

Code quality conversations are often extremely vague.

Everyone wants cleaner code, fewer bugs, easier changes, but unless these things become visible and measurable, they remain mostly opinions.

One thing that I liked in my conversation with Stuart Caborn is that Loveholidays treated code health like a data product.

They collected signals, put them on dashboards, trended them over time, compared teams, and checked whether the numbers matched how engineers felt in sentiment surveys.

The AI angle makes this even more important. If code health data is available through tools and MCP servers, humans and agents can ask questions directly:

• How is my team doing compared with yesterday?

• Are there any areas that got worse this week?

• Where should an agent be more careful?

At that point code health becomes a feedback system that informs your dev process, and it’s exactly the kind of feedback AI needs. The better the codebase, the easier for agents to understand it, change it, and spend fewer tokens doing so.

You can find the full interview with Stuart here 👇

3) 📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 Earn Your Scepticism

10 min • by James Stanier

James makes a distinction I like a lot: scepticism as a conclusion vs scepticism as an identity. The useful version comes from actually using the tools, staying specific about what fails, and keeping your mind open as the evidence changes. This applies just as much to AI enthusiasts as to sceptics.

🥈 Expertise in the Age of AI

5 min • by Brian Kihoon Lee

Brian uses calculators as the analogy for coding agents: the tool removes a lot of mechanical work, but makes the underlying intuition even more valuable. I do not agree with every prediction in here, but the core point is sharp: skipping the struggle may leave you unable to judge the output.

🥉 Find a New Role - Worksheet

4 min • by Lara Hogan

Lara turns career decisions into four simple lists: must-haves, nice-to-haves, don’t-cares, and what you are optimizing for. This is practical and useful, especially when titles, prestige, and compensation make a role look more appealing than it really is.

Quick heads up! Joel and Melinda, coaches in residence at Refactoring, are part of a group offering full coaching scholarships to 29 engineering leaders from underrepresented groups in tech. Applications are open until the end of the day on 30th June.

Learn more and apply here

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by:

Unblocked is the context layer for modern engineering teams. Get a free three-week trial at getunblocked.com/refactoring

Today’s guest is Michael Lopp, better known as Rands, who is writer of several popular books about engineering management, including “Managing Humans”, which is a personal favorite of mine.

Live at LDX3, we discussed his lifelong writing habit, the importance of giving feedback through stories and the leverage of leadership experience in the AI era.

Quick heads up! Joel and Melinda, coaches in residence at Refactoring, are part of a group offering full coaching scholarships to 29 engineering leaders from underrepresented groups in tech. Applications are open until the end of the day on 30th June.

Learn more and apply here

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. ✍️ Writing as a lifelong habit

2. 🏷️ The power of naming things

3. 💬 Stories as a feedback tool

4. 🤖 AI and the return of the coding manager

5. 📖 Managing Humans, Managing Robots

Let’s dive in 👇

Read more

To figure out what’s really happening in software engineering, I usually do three things:

1. I try things myself — recently by working on Tolaria.

2. I speak with people who know better — on the podcast and in the community.

3. I go through relevant research — to go big picture and get a grasp of the state of the industry.

Good research is hard to find, but a few weeks ago I was sent a copy of this report by the guys at Faros (thank you for that 🙏), which was very interesting for two reasons:

• Quantitatively — it’s one of the largest reports I have ever seen about AI coding: it surveyed 22K+ developers across 4K+ teams.

• Qualitatively — it’s very thorough. It doesn’t stop at measuring activity, but tries to understand the downstream impact of AI on all the stages of the SDLC.

It also paints an interesting trajectory when compared with other similar works from the past months, like the State of Software Delivery, which we covered just recently, and DORA’s State of AI Assisted Development from the end of last year.

So today I will report and comment on the main findings, trying to interpret them and connect the dots with the other things I have been observing lately.

Here is the agenda, with my main takeaways:

1. 🏁 Finishing work is hard — starting is now easy, and shipping is hard.

2. 🔍 Reviews are utterly broken — it’s now obvious that we need to do something about them.

3. 🔥 Poor quality is reaching prod (anyway) — production is suffering, big time.

4. 🪴 The way forward — some words of hope to close this on a high note!

Let’s dive in!

I want to thank the guys at Faros for giving me preview access to these results and partnering on this. If you want to download the full report, you can find it below 👇

Check out the full report

🏁 Finishing work is hard

Before we get to the first idea, there is an important preamble to be made.

We have said many times that AI is an amplifier, and that the *average* numbers of these reports don’t say a lot because they mix up the good teams that are getting a lot out of AI, with the average/bad ones that are not getting anything, or sometimes even negative results.

Now, Faros literally opens the report by saying that no — this is not the case anymore:

Across every downstream stage, the signal is the same: volume is up, quality is down, and the gap between the two is widening as adoption deepens. High-performing teams are experiencing the same downstream degradation as the median and average ones.

Words of hope! But what does it mean that volume is up and quality is down?

If we look at software dev as a pipeline, this means three things:

1. More work gets started, but

2. The amount of work that gets shipped is pretty much the same, and

3. The quality of such work is noticeably lower

Let’s go through all of this, starting with the most important point to me: starting work is now easier, but finishing it is harder than ever.

In particular, developers do more work indeed, by virtue of working on more things at the same time:

• +67% PRs touched / day

• +18% separate tasks

• +33% tasks completed, as measured by opening a final PR about them.

The problem is that such work doesn’t get to production just as fast. In fact, teams are noticeably slower at the last mile — shipping:

• +26% in-progress tasks stalled for 7+ days

• +14% work restarts per developer (tasks returning in progress after moving to another stage)

• -12% deployments per week

Qualitatively this is the same trend that both the State of Software Delivery and the latest DORA report found, but quantitatively worse. In fact, the numbers about coding activity and those about deployment frequency are respectively better and worse than previous reports, pointing at even more polarization.

One of the most visible results of this trend is lead time skyrocketing to a +480%, fueled by an avg +80% waiting time between each of the pipeline steps.

This is not surprising if we remember that:

So if WIP grows and throughput stays the same, lead time is bound to grow. But why does the throughput stay the same? Where’s the bottleneck?

Enter the elephant in the room: code reviews 👇

🔍 Reviews are completely broken

Read more

🎟️ Discover 8 levels of context maturity in AI engineering

This edition is brought to you by our friends at Unblocked!

AI shows up in 60% of engineering work. But only about 20% of it can be handed off without someone babysitting the output. That’s because agents are missing context.

This 8-stage context maturity model gives a real answer on why you haven’t seen meaningful productivity gains for all the tokens burned.

Join live June 24 (FREE) to learn:

• Why more MCPs provides agents access but not understanding

• What it takes to deploy agents you can trust without supervision

• How a context layer solves for quality, efficiency and cost

Learn more and register

1) 🔍 Is validation really the new bottleneck?!

So are the pundits (like myself) right? Is validation the new bottleneck? I think it depends, but for Tolaria I surely spend a lot of time on it.

Every task that gets completed by the AI is shipped in a new alpha version and moved to the “in review” section, where I review it against my production vault.

Here, a few things may happen:

• It’s good — I move the task to “to release”

• It’s ~90% good — I do the final tweaks from my MacBook’s Codex

• Needs work, but it’s still good progress — this is the trickiest to decide. If it’s incomplete but still useful, I have a bias towards progress and I release it anyway, creating a separate task for the improvements.

• Wrong or not usable — this is where the task is sent “to rework”, with a QA comment about what needs to be fixed.

I do a big batch of reviews first thing in the morning, for tasks that were done overnight, and then usually again after lunch, and before dinner, where I also try to make sure the AI has enough to do for the night.

So today validation feels like the biggest bottleneck indeed, also because I believe the right course of action most of the time is to keep specs light and let the AI figure out stuff by itself.

For most features, even when the first pass is not exactly how I wanted it, there is an argument to be made that if it’s good progress it should be released anyway, and have users chip in and give feedback. This has worked well many times, and evolved the product in the right direction faster than if I had waited for a perfect first version.

Of course nothing of this is exactly new thinking (MVPs, etc), but I think what’s changing here is the magnitude. I find that the right course of action is constantly to release a bit more control than what I am comfortable with, and this boundary keeps moving.

I wrote more about this in the full piece on how I run Tolaria as a open source project 👇

Refactoring

How I Run the Tolaria Project

Hey, Luca here! As you probably know by now, two weeks ago I launched Tolaria, a desktop app for Mac, Linux, and Windows, to manage markdown knowledge bases — for humans and AI…

Read more

3 months ago · 50 likes · 2 comments · Luca Rossi

2) 🌍 Async vs sync comms at Todoist

Remote work debates often get stuck in a binary: async good, meetings bad, or the reverse.

What I liked in my conversation with Gonçalo Silva, CTO of Doist, is that their view is much more practical. Doist has been remote-first for more than a decade, long before the COVID era, and they have learned that async-first does not mean async-only.

Async is fantastic when people need focus, when the topic is well-bounded, and when a thoughtful written update is enough. Doist replaced a lot of status rituals with async threads: people write updates when it works best for them, and others consume them on their own schedule.

But some conversations are simply bad async candidates. Gonçalo’s heuristic is to look at the number of iterations you expect:

“If you’re disagreeing with somebody and you feel like you’re not quickly advancing towards convergence, that is probably a great prompt to just hop into a meeting. We settled a conversation that had been going on for multiple days in a call in 14 minutes.”

If the topic starts from a blank canvas, involves disagreement, needs multiple rounds of back-and-forth, synchronous chats are the faster tool, and should not be seen as a failure of async culture.

You can find the full interview with Gonçalo here 👇

3) 📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 AI Demands More Engineering Discipline. Not Less

12 min • by Charity Majors

When Charity writes, we listen. The real product of a software team is shared understanding, and AI-generated code makes that easier to destroy at scale. For this reason, discipline now matters more, not less.

🥈 Revised Rules of Engineering Leadership

8 min • by Will Larson

Will’s piece is a great attempt at updating classic engineering leadership rules and ideas for the agent era. A lot of great takes about migrations, domain expertise, harnesses, and more. Including practical examples!

🥉 Humans and Agents in Software Engineering Loops

7 min • by Kief Morris

This connects nicely with the loop-engineering conversation: humans should not micromanage every generated line of code, but design and improve the loop that lets agents build, test, and learn. Great article hosted on Martin Fowler’s blog.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Every week I write an article about how to ship fast and make good software, and interview a world-class tech leader. I also build and maintain Tolaria in the open, publishing my workflows and learnings here.

Last week, Claude Code’s Boris Cherny got trending on X for saying this at the annual Anthropic conference:

I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops. And this is transition we’re going to see for the rest of the year

So, “my job is to write loops.”

Peter Steinberger (creator of OpenClaw) and several others picked up on it and agreed 👇

But before you roll your eyes and think here it goes another fad, new buzzwords, loop engineering, and so on, I’ll go out and say that I like this direction.

Loop engineering looks strictly better than prompt engineering because it forces you into systems thinking, as opposed to just focusing on what’s needed to deliver the current task. A loop is supposed to run indefinitely, so the focus shifts to reliability, sustainability, and how to make sure things stay good — vs diverge and go to shit.

But how do you ensure that?

I wrote about this several times lately, including last week, where I covered how I steer AI work with the 3Gs: guides, gates, and guards. These are not super new concepts: they build on what other smart people are doing, and they are my attempt to contribute to this global conversation.

But a problem I find in this conversation (which I am guilty of) is that it largely focuses on the tech side of things, while treating other disciplines as second-class citizens.

Namely, product.

When you hear people saying that software dev is now a factory, and the bottleneck is your product taste, review, validation, etc, what they are implying is that they have no idea about how to scale those things. In the AI race, product is behind engineering — big time!

The last time we discussed this on Refactoring has been when we published The State of Product Development, where we saw that only 9% of teams used AI to help product with specs (yay), vs 90%+ for coding. Since then I released Tolaria, started doing a lot of product dev myself, and tried a lot of different solutions and workflows to scale myself properly, both in tech and product.

So today I want to focus on product, taking pages from what works for coding, and exploring how we can make the same progress there.

To discuss this, I am also bringing in again Doug Peete from Atono, who has a privileged vantage point by working with thousands of product teams, and is working actively to fix this. We also talked a lot about AI & product in our recent podcast interview.

So here is the agenda:

• ⚔️ Product is harder than coding — an uncomfortable take we need to come to terms with.

• 📋 Towards Product ADRs — building on a successful tech practice.

• 📖 Glossary — a possible missing abstraction in how we develop product.

• 🔄 Product process — how to put everything together and work better with AI.

Let’s dive in!

Disclaimer: I am a fan of what Atono is building and I am grateful to have it as a partner on this piece. You should check it out!

Learn more about Atono

However, I will only write my unbiased opinion about the practices and tools covered, Atono included.

⚔️ Why product is harder than coding

Product is harder than coding, at least conceptually. I know this take will get many engineers angry, but think about it. For a product choice to be good, there is a lot to be taken into account:

• 🔭 Strategy — do customers want this? Do the right customers want this? How is this discoverable? Is this making things unnecessarily complex? How does it fit our business strategy?

• 🔬 Tactics — where do we put this? what’s the right UX? does it need a shortcut? should this live in the right-click menu? ...

A lot of these choices depend on aspects that we are not good at codifying into hard rules. It’s what we call judgment and taste.

Software engineering, on the other hand, is structurally easier, for two reasons:

• It’s downstream of product direction, so it has a narrower design space.

• Its deterministic nature allows it to be tested for correctness and non-functional qualities (performance, cohesion/coupling, complexity) in a more reliable way than product.

As a result, gates for coding agents are effective at enforcing good code, even when they have relatively little context about product. Static analysis, TDD, code health tooling, are all devices that help AI agents write good enough code—granted, not the code that the most talented engineer on earth would write—but still passable for many situations.

So how do we make product a bit more like coding? To answer this question, I will take a page from the least deterministic part of my coding harness: ADRs.

🏗️ ADRs

As I have written many times, I instruct the AI to create ADRs to store meaningful tech decisions. Each ADR has to include why that decision was made, what options were considered (and eventually discarded), and take care of superseding old ADRs about the same thing.

ADRs have been, so far, surprisingly successful. As I am writing this, Tolaria has 137 ADRs, and I see the AI using them all the time. It refers to them in specs, makes decisions that stay true to what exists, and reliably supersedes what’s not relevant anymore.

So, as of today, for most features I am comfortable not giving a lot of (tech) design input, because I know AI will pick it up from past decisions. I still review new ADRs, but they are 95% good — I do it more for my personal curiosity than because it’s a good use of my time.

The ADR workflow is, interestingly, completely non-deterministic. It’s a way to inject good judgment into the AI not by creating rules, but by repeatedly showing how we have done things in the past.

This is why I am hopeful about doing something similar to product 👇

📋 Towards Product ADRs

If ADRs are for tech work, what’s the equivalent for product? Mimicking ADRs, a Product Decision Record might record:

• 👁️ Intent — what the user needs to accomplish.

• 🎨 Design — how we decided to do it.

• ⚖️ Tradeoffs — alternatives that have been discarded and why.

If we roughly agree on this, the next question would be: what does actually deserve a PDR? To me, it’s any product decision we want to remember in case we meet a similar problem in the future. Anything genuinely new that we changed/added to the product, as opposed to reusing a proven pattern that we already understand well.

E.g. In Tolaria, I wouldn’t want to write a PDR to remember that we added “copy file link” as a right-click action to the current note — because there isn’t any novelty or interesting decision being made there.

Do PDRs map squarely to well-known product artifacts, like PRDs, Epics, or Stories? I don’t know about that either. Those other devices primarily exist to communicate what needs to be done. It’s great when they include more about the why, but it’s not clear to me whether e.g. a full PRD or user story should be stored as-is and work well as a PDR. Maybe!

The other reason why I tend to believe PDRs deserve a life on their own is that relevant decisions are largely independent of the size of the work. There might be a large Epic (e.g. integrating a new AI agent assistant into Tolaria) that doesn’t bring meaningful new product direction, and a small story that instead introduces new a interesting pattern.

Doug is experimenting with PDRs at Atono, calling them design decisions and attaching them directly to the stories they relate to — not as a separate doc for now. This doesn’t resolve the question of where a PDR lives when the decision is bigger than any single story, but looks good on the smaller end of the spectrum: when the decision is scoped to one piece of work, attaching it directly might hold up well in practice.

So, in general, even if there is a lot that we don’t know about their shape, we can agree that it’s valuable to record product decisions, in a similar way to what we do with architecture ones. And if we do, we should also agree that this is not enough 👇

📖 Product Glossary

Other than ADRs, I maintain other tech docs in the Tolaria repo, which are called ARCHITECTURE.md and ABSTRACTIONS.md. Without getting into too much detail, these are recap docs, that summarize the current state of our tech.

These are necessary, like views in a database, to avoid the AI to navigate eventually hundreds of ADRs (between those active and superseded) every time, just to reconstruct what exists. The recap docs give the AI the 10,000ft view, while individual ADRs are used to zoom into specific aspects.

What should this look like for product? Doug believes the right concept here is that of a Glossary.

The Glossary should list your main product abstractions: why they exist, what they are for, and how they are used. It maps your domain language, which in turn is shaped by your product surface.

An example from Tolaria may include concepts like:

• The Sidebar — which gives users ways to filter notes in their workspace

• The Note List — to make users navigate the current selection of notes

• The Breadcrumb Bar — to display the most important actions a user can do over the current note.

But also concepts that don’t map 1:1 to single interface elements, like the Git integration, or the Note organization workflow (capture / organize / archive).

It’s obvious these are not PDR candidates: they are long-lived abstractions that morph over time, and need a description that evolves with them. So, if anything, they are derived from a sum of product decisions.

Doug believes a shared glossary is the cornerstone that enables good AI-assisted stories and PRDs — because now the AI can ground them in your vocabulary, your past choices, and what exists. And he has data to back this up.

By talking with some of the teams we surveyed, which report abysmal low AI adoption for product specs (9%), it turns out a major cause is the amount of rework needed by generic AI specs. Making them good is just too much work, so most PMs keep writing specs manually. Doug and Atono have been testing glossary-backed AI stories internally and with customers, and they are seeing rework dropping from 60% to 20%.

(worth noting that 60% rework, as a baseline, is a ton. What if coding agents got things wrong 6 times out of 10? We would probably give up on them, which is what many PMs do with AI right now)

So the overall goal is to build foundational product knowledge: a map of what things mean in this specific product, which is not the same thing as a list of features.

So let’s say we have all of this in place, what does a new product process look like?

🔄 Product process

Any product work starts with capturing some intent — what the user needs to accomplish, how it maps to the product strategy, how we know this is successful, and so on.

This is 100% human (for now!) and is more about the why than the what.

Our input on the what can progressively shrink the more we help the AI figure out good (draft?) specs that respect the current product surface (glossary + PDRs).

Then, based on the feature, the maturity level of your process, and how greedy vs safe you want it to be, you may either:

• Have the AI do a first pass, so that your judgement is informed by an actual prototype, or

• Stop and tweak the draft spec before the AI goes forward.

Then you review and iterate on implementation as long as needed, and finally ship, possibly including new PDRs and Glossary updates.

Over the next weeks I will try to incorporate some of this into the Tolaria workflow, possibly retrofitting some PDRs and glossary concepts from the past. Will let you know how it works!

And that’s it for today! I wish you a great week

Sincerely 👋
Luca

📊 Engineering leaders don’t have a way to measure AI results

This newsletter is brought to you by our friends at DX!

DX has spent the last year studying what this measurement gap means in practice.

On June 18th at 09:30 PT, Distinguished Scientist Brian Houck and Deputy CTO Justin Reock are presenting their research on agent experience, AI readiness, and what the data says about productivity in an agentic world.

You can register for free below 👇

Register here for free!

1) 🧱 Keep deterministic plumbing outside AI

A lot of AI workflow demos make it look like the agent should own the whole process: understand the goal, decide the steps, call tools, and maybe even schedule the next run.

This is quick (and exciting!) when you are prototyping, but it is arguably a bad default for recurring work.

The boring parts of a workflow—routing, retries, scheduling, validation, etc.—need to be dependable, rather than “smart”. For things that I run regularly (e.g. all Tolaria procedures) I want to know which steps failed, what inputs they saw, whether they retried, and what state was left behind.

So the architecture I like more and more is hybrid:

• Orchestration owns the skeleton — when the workflow runs, what each step consumes and produces, how failures recover, and what humans can inspect.

• AI owns the messy parts — classification, summarization, judgment calls, and human-readable updates.

This does not make AI less important. If anything, it makes it more useful, because it stops wasting tokens and attention on plumbing that traditional software is a better fit for.

So, agents are a fantastic way to discover/prototype a workflow. But once the shape is known, the right move is to extract the stable pieces into code and keep AI where ambiguity actually exists.

I wrote more about this in the full piece on orchestrating AI workflows 👇

Refactoring

How to Orchestrate AI Workflows

Hey, Luca here! Welcome to a weekly essay from Refactoring…

Read more

3 months ago · 55 likes · 1 comment · Luca Rossi

2) 🌐 Open source in the age of AI

Last week Ladybird announced they are stopping accepting public PRs — the latest of a series of projects doing the same.

This made me think of my chat with Chris Lattner, where we discussed the impact of AI on open source.

Chris has spent a huge part of his career building massively successful open source projects — LLVM, Swift, MLIR, Mojo — and is concerned about the current state of things. As it’s now obvious, AI makes it much cheaper to produce contributions, but it does not make reviewing them equally cheaper.

“With AI tools, what’s happening is that a lot of maintainers are getting overrun. The contributor doesn’t have to do nearly as much work, but the reviewer has to do the same — at a bigger scale. I think it’s going to lead to new contributors not getting the attention they deserve.”

This is important because, in open source, patches have always been about mentoring and trust-building. Maintainers invest time in people, people learn the project, and some of them eventually become future stewards.

Chris also connects this to intellectual property. If agents can rewrite proprietary tools, translate GPL code into MIT-looking projects, then copyright and licensing become much harder to reason about. As it’s often the case, legal systems lag behind reality.

But the optimistic version is that this also pushes companies toward openness. If secrecy and proprietary headers become weaker moats, the better strategy is to build strong communities, move fast, and compete on innovation rather than control.

We should decide whether to see the glass half empty or half full!

You can find the full interview with Chris here 👇

3) 📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 Domain Expertise Has Always Been the Real Moat

3 min • by Aaron Brethorst

Short, sharp, and very true: AI makes writing software cheaper, but it does not make knowing what “correct” means any cheaper. The best engineers were always the ones building a model of the domain in their head, as opposed to simply translating tickets into code. That part is still a moat!

🥈 What Happens When the Coding Becomes the Least Interesting Part of the Work

8 min • by Obie Fernandez

I liked this because it frames AI coding less as “the machine writes code” and more as “the machine forces you to make senior thinking explicit.” If you care mostly about typing code, agents are annoying. If you care about judgment, tradeoffs, and intent, they free up the interesting part.

🥉 Changing How We Develop Ladybird

3 min • by Andreas Kling

Ladybird is stopping public pull requests and moving to maintainer-only code changes. The rationale is that, in the AI era, patch size is no longer a good proxy for effort or good faith. Open source maintainers are going to rethink trust, review, and responsibility a lot more than we admit. Thinking about this myself with Tolaria.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by:

Unblocked is the context layer for modern engineering teams. Get a free three-week trial at getunblocked.com/refactoring

Today's guest is Lara Hogan, an author, public speaker and coach for managers and leaders across the whole tech industry. And she's had a long career as VP of Engineering at FLY, Kickstarter, Director at Etsy, and more.

With Lara we talked about her journey in tech, which started with a philosophy degree. Then, taking from her work as a coach, we talked more about how successful teams are adopting AI and what you can do to make engineers feel safe about it. And finally, we explored the BICEPS framework and how you can better understand people's needs at work.

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. 🎓 From philosophy to engineering leadership

2. 🤖 How the best teams are adopting AI

3. 💡 The AHA meetings: learning together

4. 🧠 The BICEPS framework and feeling safe at work

5. 🧱 Practical steps for managers and engineers

⠀Let’s dive in 👇

Read more

TRM is AI intelligence for public safety — and it’s hiring! 📣

This newsletter is brought to you by our friends at TRM!

TRM models turn petabytes of data into actionable intelligence across the threat landscape: ransomware groups, human trafficking networks, darknet markets, terrorist financing, and more!

Threats don't wait. And neither does the TRM team.

Engineers at TRM work at a scale most never touch: real-time graph traversal, petabyte-scale pipelines, and models that power investigations with real-world impact. If you want your code to catch criminals, this is where you build it 👇

Check open roles

🧮 Map and score devex problems

Most teams have structured ways to work on customer problems — which we call “product” — but do not apply the same rigor to developer experience issues.

Earlier this year I advocated to do a listening tour with your engineers to uncover devex problems. An important part of the tour is not to stop at the uncovering stage: you should map, score, and prioritize issues so you work on the right things.

Mapping means taking stories — situations where bad things happened — group them together into friction points, and map them to steps of your dev process.

You can reuse the story map format for displaying this, but you don’t have to necessarily do it that way. You can use a spreadsheet or anything else.

The outcome might be something that looks like a three-level investigation that includes steps, problems, and stories 👇

After you collect problems, it’s time to score them. To do that, the most useful coordinates I have found are reach and impact:

• Reach — how many engineers are impacted by this

• Impact — how much disruption it causes

If this makes you think of RICE, we are not including:

• Confidence — because it’s often tricky to figure out and I haven’t found it that useful in real life, nor

• Effort — because we are only talking about problems here, not solutions.

Also, I am a fan of using simple scores, like low-medium-high, or 1-5.

After that, you should focus on the one true bottleneck™ — I wrote more about that in the full article 👇

Refactoring

The Listening Tour 📞

For the last month I have been running a program for CTOs and VPs of small to mid-size companies to help them make their engineering teams ship faster…

Read more

6 months ago · 36 likes · 3 comments · Luca Rossi

🪜 Career direction starts with what you want

Career planning starts getting useful when you stop asking only what the ladder says and start asking what you actually want.

Many engineers sleepwalk through their careers by following the next obvious step: promotion, bigger scope, manager track, staff track, better company, higher comp. Those may be good goals, but they are not a substitute for knowing what kind of work makes you thrive.

Last month I interviewed Jean and Cate, two professional career coaches, and they told me they are a fan of asking two simple questions:

• What was a peak moment in your career?

• What does it look like for you to thrive?

They asked them to me and I had to think for a minute or so. These questions are designed to pull attention away from abstract “prestige“ and toward real-world evidence. When did work feel energizing? What kind of contribution mattered? What environment made you stronger?

Also, the answer does not need to become a ten-year plan. In uncertain times, an horizon of a few months may be more useful: try something, observe how it feels, adjust, repeat.

Career direction is built every day from good reflection and experiments.

You can find the full interview with Jean and Cate below 👇

Also on Spotify and Youtube

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 Why Japanese Companies Do So Many Different Things

20 min • by David Oks

This is such an awesome article that goes deep into explaining how (and why) Japanese companies are wildly different from US (and Western in general) ones. So much food for thought from looking at things like lifetime employment, horizontal coordination, diversification, and more. A must read.

🥈 The Slide

5 min • by Rands

Putting this appropriately here after I interviewed Rands at LDX3 just last week, and asked him about stories and this article in particular. When someone keeps ignoring your feedback, don’t lecture harder — slide up next to them with a personal story about the time you struggled with the same thing.

🥉 Making a New Plan

2 min • by Cate Huston

An ever-underrated leadership skill is admitting you’re wrong and making a new plan when new information is available. Also, it’s easy to judge others for not doing that, until you realize we all do it from time to time. Short and sharp, as always in Cate’s style.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Every week I write an article about how to ship fast and make good software, and interview a world-class tech leader. I also build and maintain Tolaria in the open, publishing my workflows and learnings here.

Refactoring is read every week by 170,000+ engineers and managers from all over the world!

Hey! When I launched Tolaria I promised I would publish ~monthly updates to my AI coding workflow. The last one was in April, so I am late!

In May I also published how I run Tolaria as a project, but this was more about general orchestration, so I wanted to post an update that is more specific to coding.

I will start by saying that things are going... well.

Tolaria by now has a few thousands daily users, most whom are engineers. These are quite eager to report bugs, submit PRs, and suggest improvements on social media, Github discussions, email, and more — so keeping up with everything is quite a lot of work.

Still, I am happy with where we stand. As I write these, we got:

• 6 open issues — vs 417 closed.

• 14 open PRs — vs 306 closed.

• 1 day — average bug fix time.

• 99.1% crash-free rate — as measured by Sentry.

• ~28 commits / day — on average.

• 2 hours a day — I work on Tolaria, give or take.

I released Tolaria on April 22nd, so all of this happened in little more than a month, which is crazy when you think about it. From the day of the release, I added about ~1000 commits, and the codebase has grown a bit, but not massively: right now it’s 150K LOC, plus ~100K LOC of tests.

Most importantly, I don’t feel my velocity has decreased.

If anything, I am probably producing more output per unit of input than two months ago, so the effectiveness feels higher. Code quality also is higher, we are not drowning in regressions, and the rate at which bugs are reported feels stable.

So today I will cover the tweaks I made to my AI coding over the last ~2 months. To catch up with the full process, you can also go back to the previous articles here:

Here is the agenda for today:

1. 🚥 Guides, Gates, and Guards — my mental model for the various types of AI instructions. The core of the article.

2. 🤖 From Claude to Codex — I moved to Codex and (for now) I am not going back.

3. 💸 How much I am spending — monthly cost is now down >90%

4. 🔭 What’s next / missing — what I am trying to improve right now

Let’s dive in!

🚥 Guides, Gates and Guards

When it comes to the dev process, I think about the types of inputs that I can give to AI in three steps:

1. ↪️ Guides — rules and explainers of how to do things.

2. 🔄 Gates — deterministic checks that block bad things and force the AI to steer its output.

3. ↩️ Guards — fallback procedures to improve what falls through the cracks.

These steps are also somewhat sequential:

• Guides are the basic context that the AI uses to start the work

• Gates get into play to steer the work while it’s in progress

• Guards are the last line of defense against enshittification.

Let’s look at all of them:

1) Guides ↪️

We can as well call them rules, but then we’d lose the 3G’s memorability!

These days you write guides for AI agents in two main ways:

• AGENTS.md / CLAUDE.md — the core instructions that the agent always loads.

• Skills — specific instructions to do particular tasks. A skill gets used only when the task requires it.

It’s worth noting that the boundaries between the two surfaces are not very clear. It’s often dubious whether you should create a separate skill or just slam everything into the AGENTS.md, and even benchmarks are inconclusive about the effectiveness of the various strategies.

I believe that’s because, at their core, they are the same thing: instructions to the agent.

My dev process is simple enough that I can afford to keep everything into the AGENTS.md, and not use any specific skill. I have tried a few and never felt things got noticeably better, so I opted for keeping things simple.

You can inspect the actual file — it’s not perfect, mind you, but gives you a more precise idea.

In my AGENTS.md, the most important instructions are about:

• TDD — the agents has to write tests first and do the classic red → green → refactor.

• Boy scout rule — leave the code in a better shape than you found it. It should also measure the before/after quantitatively with CodeScene and Codacy.

• UI — always use shadcn/ui components when possible, and inspect our design language before implementing new UI (we got basic tokens for colors, typography, spacing, etc).

• Localization — extract and translate all the new strings in the supported languages. I use the Lara CLI for this.

Read more

📋 Unblocked is the context layer for modern teams

This newsletter is brought to you by our friends at Unblocked!

Agents can generate code, but getting it right for your system, team conventions, and past decisions is the hard part. MCPs give agents access to information but not understanding – so you end up wasting time and tokens in correction loops.

For this reason, the teams pulling ahead use a context layer to give agents exactly what they need.

Unblocked is the context layer that turns code, docs, tickets, and conversations into actionable context, so engineers move faster and agents stay on track 👇

Learn more about Unblocked

1) 📈 Measure AI effectiveness by leverage

There are a lot of *bad* ways to measure how well your team uses AI, including counting lines of code, tokens spent, and more!

To me, we should measure effectiveness, which is a function of leverage: how much useful output you get per unit of human input.

This is different from measuring AI usage. E.g. a team can maximize AI-generated code by writing incredibly detailed specs that are basically pseudo-code. The AI may write the final syntax, but the human still did most of the hard work.

That is not real leverage!

So a better question is: how much context does a human need to provide before the AI can do the job well? Some of the prompt will always be task-specific. If you ask for a new button, the AI needs to know where the button goes and what it should do. But a lot of the prompt should not have to be repeated every time: design rules, product conventions, architecture patterns, testing style, existing components, release constraints.

So you can think at a scale that goes like this:

1. 🏜️ Negative leverage — people spend more time fighting the AI than doing the work themselves.

2. 🌱 Low leverage — the AI writes code, but only after humans write exhaustive instructions.

3. 🪴 High leverage — the AI can reuse enough context from the system that prompts become short and results stay good.

4. 🌳 Maximum leverage — the AI one-shots meaningful work from tiny prompts because the factory already knows how good work should be done.

I wrote more about this in the recent article series about the new software factory 👇

Refactoring

How to Grow your Software Factory

Last month I published The Era of the Software Factory, where I commented on the latest State of Software Delivery together with Rob Zuber, a good friend and CTO of CircleCI…

Read more

4 months ago · 34 likes · 2 comments · Luca Rossi

2) ⬆️ Bottom line up front

One of the most popular articles we published on Refactoring last year is from Anna Shipman, CTO at Kooth and one of my favorite authors.

She described the translation layer that is needed to make engineering understood by executives, and the article is packed with great examples and mental models.

One of the pieces of advice that stuck with me is to always lead your communication (spoken or written) with an executive summary that includes the most important information: what action you are looking for.

Or, as Anna calls it: bottom line up front.

An exec summary tells the reader in a few short sentences what this document is going to cover. If this is a presentation rather than a document, include an executive summary slide. If this is a conversation, set it up right by explaining first what the purpose of the conversation is. If you are asking for a decision, make sure you’ve included what the decision is.

Here are some example exec summaries.

For a document:

This proposal recommends consolidating our infrastructure onto a single cloud provider to reduce operating costs by 20% over three years, improve reliability, and accelerate feature delivery. The alternative is to retain our split between AWS and Azure. The work will reduce capacity of the platform team by 50% for six months, delaying completion of the work on pipelines until Q4 2026.

We request an additional budget of £50k for the engineering team to have access to an AI coding assistance tool. We have demonstrated in our trial that use of Cursor increases throughput of work by 25% meaning over a year we would deliver extra features equivalent to the salary of four additional engineers, therefore this would pay for itself by end Q1 next year.

In a conversation you might say:

“I want to propose we start upgrading our data handling now rather than in Q3 as planned because our latest planning session raised a flag that we may not hit the deadline. We’d have to pause the work on the new dashboards to do that.”

“There’s a strong business case for investing more in developer experience in next year’s budget. I can show you the ROI calculation we’ve done, but I would like your take before we move forward with the budget preparation.”

You can find the full article by Anna here:

Refactoring

The Engineer → Executive Translation Layer 🔀

Hey, Luca here! Today we are hosting a fantastic new article by Anna Shipman, CTO at Kooth…

Read more

9 months ago · 29 likes · Luca Rossi and Anna Shipman

3) 📣 What makes for a great talk

By the time you read this newsletter I’ll probably be in London to attend LDX3. This year I will interview the one and only Rands on stage, for a live episode of the Refactoring podcast.

So, while preparing for this, I thought about my interview with Meri Williams last year. Meri is the long-time host of LeadDev conferences, and every year they have to run through hundreds of talk submissions to select only a handful of truly great ones.

I asked them what makes for an amazing talk, and they said it’s three things:

1. 📖 Personal story — or a case study. Anyway, a real experience that grounds the content.

2. 🧠 Framework — the author abstracted their experience into a theory that can be applied broadly.

3. ⚡ Practical application — the story + framework combination has clear takeaways that the audience can implement.

You can find the full interview with Meri here:

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca