Today’s episode is brought to you by: Notion

Learn more about Notion’s Developer Platform today at https://ntn.so/refactoring

Today's guest is Francisco Trindade, VP of Engineering, founder of two companies, and author of “Leading Effective Software Teams”, which I just read and loved.
So in this chat, you will learn how to think about your team as a system, how to optimize flow in the developer process, and why you should absolutely hire junior engineers.

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute, handcrafted takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

Agenda:

1. 📘 Why a book about teams as systems

2. 🧠 Systems thinking in day-to-day management

3. 🌊 Optimizing flow and measuring what matters

4. 🎯 Autonomy and shaping work for people

5. 🌱 Why junior engineers still matter

⠀Let’s dive in 👇

Read more

Every week I write an article to 170K+ engineers about how to make good software together. To get access to all our articles, resources, and our private community, subscribe to the full version.

More and more products these days show up inside agent hosts like ChatGPT, Claude, Grok Bot, as plugins that the model can use. In many cases, agents also act proactively and recommend plugins that fit their current job.

This means, as many predicted, that agents are becoming important distribution platforms, with many users discovering new products from inside them.

Such plugins, under the hood, are powered by MCP servers, which wrap and expose the tool's functionality.

On paper, these look easy to build. In practice, less so.

I bet everyone who has played with agents for a while has found themselves routinely battling with MCPs that are either broken, or incomplete, or don’t work as intended. I surely have, and I keep seeing embarrassing stories popping up on the internet about it.

Just last week, Figma got backlash for having hardcoded the list of allowed clients in their MCP server, effectively disabling popular tools like Pi and OpenCode. One of the creators of the MCP protocol himself chimed in to note that this is not how the protocol should work.

Why does this happen? I believe many teams just underestimate what it takes to create good MCP servers. They think about them as simple API wrappers, assign the task to an intern, and call it done.

Last year I wrote How to Design APIs for an AI World, because it was already clear that APIs needed to be adapted for when the caller was an agent rather than deterministic software. Fast forward to today, a lot has changed about how agents interact with software, so I am writing a sequel that 1) dives deeper, and 2) covers more of the product surface.

To do this well, I am also looping in Luigi Pederzani and the guys at Manufact, who I have known for a long time. They are the authors of the open-source framework mcp-use and have been working on MCPs from the very early days, so they are the most knowledgeable people I know about this space.

I will also be pulling real examples from products that are doing this right, like Notion, Block, Stripe, and Sentry, so you’ll leave this article with practical references that you can take inspiration from.

So here is the agenda:

• 🧩 Jobs instead of endpoints — deciding which tools should exist in the first place.

• 🧭 Designing for an agent caller — descriptions, improvisation, errors, and UI.

• 🔄 The production lifecycle — what is different from running a normal backend.

• ⚖️ Self-host vs managed — figuring out when each one makes sense.

• ✅ Checklist — what to do with your team next week.

Let’s dive in!

Disclaimer: I am thankful to Manufact for partnering on this and sharing how they see MCP design and production in the wild. I am a fan of what they build and you should check it out.

Learn more about Manufact

As always, I will only write my unbiased opinion on the practices and tools covered, Manufact included.

🧩 Jobs instead of endpoints

So, where do you start?

In my experience, the most common mistake is simply taking the endpoints you already have and turning each of them into a tool. REST (or GraphQL) APIs are usually built around resources and CRUD verbs, which work great for software that knows exactly what it wants, and can express it as atomic operations.

An agent, instead, is trying to complete a job on behalf of a user, like “file this bug with a reproduction and an assignee”, or “summarize the blockers of this sprint”.

If jobs depart too much from what single API endpoints do, the model spends tokens (and time) finding the right paths, chains five calls where one would do, and may still fail the workflow the user actually wanted.

Instead, it’s useful to work top-down, starting with the end in mind. List the handful of end-to-end workflows people actually ask agents to do with your product, and then design one tool, or a tight chain of tools, for each of them.

This also gives you a rule of thumb for granularity:

• Too fine-grained — and you pay for multi-hop agent loops (LLM seconds times N) on top of a confused tool choice.

• Too coarse — like one mega-tool with a kitchen-sink schema, and arguments get brittle, recovery gets hard, and auth gets messy.

A good default is tools that finish a job in one or two calls, batching related reads when you can and keeping multi-step flows for the domains that truly need them. When you do need them, make the intermediate state explicit in the responses, so the model can continue without guessing.

A useful example is Notion, because they shipped both versions.

In the Notion API, the content of a page is a tree of blocks (paragraphs, headings, to-dos and so on), and their first, open-source MCP server mapped each endpoint 1:1 to a tool, so agents had to work with blocks too. Blocks are a great abstraction for an API: they are granular and composable, and they give apps that render or sync Notion content precise control over every detail.

An agent, however, usually has a bigger job in mind, like writing up a meeting recap or adding a section to a plan, where the natural unit is the page. Notion’s own takeaway was that 1:1 mapping “created suboptimal experiences for AI agents.”

So, on their new hosted server, they eventually changed the abstraction. There is no tool to create, update or delete individual blocks anymore: pages go in and out as Notion-flavored Markdown, and one page-level call replaces a chain of block calls:

The work of turning Markdown into nested blocks, or finding the right block to change, moved inside the server.

Block tells a similar story in their MCP playbook: the first version of their internal Linear MCP had 30+ tools that mirrored the API, and a question like “what issues is bob@example.com working on” took four to six calls.

🧭 Designing for an agent caller

Once you know which tools should exist, the next question is how they should behave.

The thing to keep in mind here is that your caller is different in many ways from the software that typically hits your API. We covered this for APIs in general last year, and most of those ideas apply to MCP, with possibly even less forgiveness.

To me, here are the most important ideas to remember:

1) Descriptions are runtime

Developers read your API docs once and internalize the patterns, while agents re-read tool names and descriptions at every turn, inside a limited context window, and decide what to call based on that alone.

So, you should 1) name tools the way a user would phrase the request, and 2) use the description for the stuff that actually drives behavior: constraints, preconditions, and when not to use the tool. I would also put description edits through review and some testing, because changing a sentence there can change what the model does as much as changing code.

Sentry’s MCP server does this well, for example, in its search_errors tool:

The model learns what the tool is for, and what to use instead.

2) The caller improvises

Agents may invent argument shapes, call tools out of order, retry in odd ways and forget that setup has already happened, and the host will often retry on their behalf.

So the principles we discussed for AI-facing APIs are your defense here, which in practice means strict schemas where it matters, safe defaults, and idempotent writes on anything risky. For high-stakes actions, you can refuse with a clear message and offer a human-in-the-loop path when the domain needs it.

Stripe does this well for APIs: every POST accepts an Idempotency-Key, and a retry with the same key returns the first response instead of creating a second object. On MCP tools, you can do the same with an idempotency key argument, and declare how each tool behaves with the protocol’s tool annotations:

Clients can use these hints to decide when to ask for confirmation, as Block’s Goose does.

3) Errors should teach the next attempt

A key difference between agents and deterministic software is that the former can (and will try to) recover from errors by itself. So you should help with it.

This means a bare {“code”: 400} is fine for deterministic software, where a developer will eventually read the logs and fix the call, while an agent benefits from errors that are more descriptive and tell it what to do next, like the expected format, an example, or a different tool to use instead.

Block’s playbook shows a good one from Goose, for files over 400KB:

It says what went wrong, why, and what to try instead.

It also pays to log which errors agents hit most, because many of them will point to the tool's surface and hint at what you can improve. Half of your failures might be “invalid priority” because the schema says high and the server expects P1.

4) The job can come with its own UI

Lately, MCP servers can do more than expose tools.

With MCP Apps, a tool can point to an HTML resource that the host renders in a sandbox, so your product can show its own interface right inside the conversation.

This matters for jobs where the user wants to see and touch the result. In a design tool, for example, the user may open a project, select a screen, ask the AI to change it, and see the result in the same canvas. Doing that through text alone would mean describing objects back and forth, while here the app can tell the agent exactly what the user selected.

ChatGPT goes a step further with MCP Extensions, which turn an MCP server into a plugin with specific entry points: a launcher in the sidebar, a tab inside a conversation, and handlers that open files with a given extension in your own viewer. These are host-specific, and some of them (like file entry points) are desktop-only for now. Luigi wrote a good walkthrough of this.

🔄 The production lifecycle

The second intuition to update is about what happens after launch. Shipping an MCP server feels like shipping a small backend service, and in many ways it is, but keeping it healthy also involves work that a regular backend doesn’t have.

Differences mostly show up in three places:

1) Managing hosts

Your MCP will be called by ChatGPT, Claude, and a long tail of other clients, each with its own behavior and update cycle, and the same server can produce very different outcomes across them.

Manufact has an interesting testing write-up that walks through an issue-tracker example where Codex manages to work around server bugs and pass, while Claude fails the same suite because of sneaky defects in the tools. If you only test with your favorite client, it is easy to ship bugs like these without ever seeing them, so make golden prompts on a few real hosts part of normal product QA.

2) Updating the protocol

In about 18 months, MCP has changed a lot: from HTTP+SSE to Streamable HTTP, then tighter OAuth rules, and then the 2026-07-28 revision, which the maintainers describe as the largest since launch.

Just in recent changes, sessions and the initialize handshake are gone, every request is self-contained, and auth has been changed substantially.

Being MCP a client-server protocol, you need to incorporate changes ASAP, because even when your own server code doesn’t change, the one in the clients will!

3) Distributing to directories

Distribution through agents comes with its own process, too.

ChatGPT app review, the Claude Connectors directory, Cursor Marketplace, Muse and Grok custom connectors all want slightly different assets and reviewer access. Each platform has its own packaging, review, and update flow, so you should track the requirements for each destination, keep the endpoint stable, and test the experience after every change.

Put together, this is closer to keeping a mobile app healthy across OS versions and app stores than to maintaining an API behind an SDK you control. None of this is rocket science, but it’s a significant (and growing) surface to keep track of.

⚖️ Self-host or managed?

So, all that said, should you run all of this yourself? It depends on how much of the lifecycle above actually applies to you:

• Self-hosting — is fine when the MCP is internal (e.g. stdio), used by one team or IDE, the blast radius is low, you are okay owning SDK upgrades when the protocol changes, and you are not listed in marketplaces.

• Managed — a hosted platform pays off when you ship a remote MCP as part of your product across many directories (ChatGPT, Claude, Cursor, ...), you want cross-client test suites against real hosts and checks before each submission, and all of that work sits on your critical path without being what makes your product special.

As a quick plug, the second scenario is what the team at Manufact is building for. Their MCP cloud lets you connect a repo and deploy, test against clients, monitor usage, all with mcp-use as open-source framework underneath.

If you are already hosting your server elsewhere, they also run cross-client tests and publishing-readiness checks without moving the deployment. I love their approach and you can learn more about it here.

✅ Checklist for next week

If you have been thinking about developing (or improving?) your own MCP server, and want something concrete for the team to begin with, here is a quick checklist:

1. List the most important agent workflows — that should exist (say < 7), and delete the endpoint-dump tools that do not serve them.

2. Rewrite tool names and descriptions — as if they were prompts, and put description changes through review.

3. Add actionable errors — and idempotent writes to the risky tools.

4. Pick three hosts — and run a golden-prompt test of the full experience, UI and permissions included.

5. Assign an owner — for MCP health, with a re-check after every major MCP, Claude or ChatGPT release.

6. Decide self-host vs managed — for deploy, test, monitor and publish, and write down the reason.

💬 Join our community!

Hey, thank you for making it this far! If you liked this piece, consider subscribing to the full version of Refactoring. Paid members get exclusive access to our private community of 1500+ tech leaders, where every month we pick one theme and dig into it together.

In October the theme is management in the age of AI, and on Monday, October 19 we have our next live Mastermind on rethinking the manager's role.

Get access to the Community

And that’s it for today! I wish you a great week.

Sincerely
Luca

Hey, Luca here! Welcome to a new edition of the 💡 Monday Ideas 💡 — ideas and readings to start the week on the right foot.

I also publish an original essay every Wednesday. Here is the one from last week in case you missed it:

Hyperscale Remediation: Autonomously Secure Your Software Estate

This week’s newsletter is brought to you by our friends at Blitzy!

Security teams don’t need another list of alerts. They need to know which vulnerabilities are reachable, exposed, and safe to fix. Blitzy understands your entire software estate and, through Proactive Insights, surfaces known and previously undiscovered vulnerabilities in the enterprise codebase and remediates them autonomously. It traces each finding through execution paths, identifies dependencies, and validates remediation. Blitzy then turns prioritized risks into batches of tested pull requests for engineers to review.

The result: less alert noise, faster remediation, and confidence that each fix is safe. Find risk in context and remediate it securely at enterprise scale.

Try on your codebase today

📚 Knowledge capture compounds engineering quality

In April, Rob Zuber and I wrapped up our software factory series with an article on compounding. We asked: how do teams actually get better over time, instead of just shipping more?

I believe knowledge capture is one of the few things in engineering that truly compounds.

Teams usually capture too little of it. They especially lose decisions, forgetting why systems work the way they do.

There is basically no excuse today for not getting this right, because capturing and processing information is much cheaper than before. Meeting notes, ADRs, product decisions, can all be captured, summarized, updated, and reused by agents.

So docs become an asset that improves the development loop, instead of a tax. And the more context the team captures, the more coherent decisions the AI can make on the next project.

This is especially important for building the right thing, as opposed to simply the next thing. A codebase becomes easier to steer when future work can see past decisions.

Read the full article here 👇

Refactoring

The Compounding Software Factory 📈

Hey there, Luca here…

Read more

6 months ago · 61 likes · 2 comments · Luca Rossi

🤝 Good engineering is an exercise in human relationships

Two years ago I had Kent Beck on the podcast, and we talked about Extreme Programming and TDD. I rewatched that interview recently to figure out what has changed (and what has not) since then, and a part that stuck with me is when he said good engineering is an exercise in human relationships.

This is easy to miss because you may just treat XP as a bag of techniques: short cycles, tests, pairing, reviews, and so on. But all of these only work if people communicate and give each other feedback.

This ties in with Kent’s idea that Agile goes largely against traditional power structures. Waterfall represents a fatal attraction for many teams because it replaces grassroots communication with a plan that gets transmitted down through hierarchy.

If the human relationships are missing, you can install the practices and still not have the actual engineering.

You can find the full conversation with Kent here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 The Kids Are Alright

6 min • by Francisco Trindade

Francisco takes on the question of whether we should still hire junior engineers, and his answer is a strong yes. If the job of a senior becomes leading a project by orchestrating agents, the job of a junior will be to lead simpler projects in the same way. And if juniors have nothing meaningful to do on your team, that probably says more about your process than about them. I loved this one.

🥈 Software Factories Are a Local Maximum

3 min • by Aviv Ben-Yosef

Aviv makes a great point: agents can make us lose the healthy friction that made us think about why we were building something. I have written a lot about software factories myself, and I 100% agree with him.

🥉 How to Write an Effective Software Design Doc

15 min • by Michael Lynch

Michael wrote design docs at Google and Microsoft, and this is his full playbook on when a doc is worth it, what belongs in it, and what doesn’t. I feel this is even more relevant today, since a good design doc is pretty much the context you want to give your agents (or have them help you write anyway).

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

About four years ago I published Do you really need a Staging environment? — the first article of mine to really go viral. Front page of Hacker News, more than 2,000 likes on Linkedin, and hundreds of comments across social media.

Many of such comments were just... angry! In fact, even if the title was framed as a question, I argued that, in most cases, staging environments did more harm than good. And many people didn’t like that.

Fast forward to today a lot of things have changed. Teams ship more code, faster, and more often. Does this make staging (and/or equivalent intermediate envs) more or less valuable? I see you saying “it depends” — so it depends on what?

So today I want to revisit the original thesis, figure out what still holds, vs what needs to change.

Here is what we will cover:

• ⬅️ What I argued in 2022 — a short callback to refresh our memory.

• ✅ What still holds — parity, batching, and when shared staging still pays off.

• 🤖 What AI changes — more throughput, worse env lies, safety net in prod.

• 🛡️ Guardrails for dev — guides, gates, and guards upstream of merge.

• 🚢 Ephemeral preview envs — review on isolated, prod-shaped envs.

• 🔬 Observability and controls — recover fast when something slips.

• 🎯 Final checklist — what “good” looks like now, without the sermon.

Let’s dive in!

Disclaimer: I am thankful to Upsun for partnering on this piece and for pushing me to revisit this topic properly in 2026. I am a fan of what they build around Git-native environments, and you should check them out.

Learn more about Upsun

However, as always I will only write my honest opinion on the practices and tools covered, Upsun included.

⬅️ What I argued in 2022 (briefly)

The original piece tried to get clear about when a shared staging / pre-prod environment helps, vs when it mostly adds cost and delay.

The two problems I saw (and still see) the most often are:

1. Staging is often unreliable — because keeping it at parity with prod (i.e. data and infra) is hard and expensive. So most teams cut corners: a fraction of the database, different instance sizes, and just some of the services.

2. It makes releases many times slower — an extra release level means waiting for the env, batching several features “because we are releasing anyway,” deploys that become morning-only (or only “at the start of the week”), and the classic “whose change broke staging?” moments.

The way out already existed in 2022: small diffs, strong automated tests, feature flags, canary / progressive delivery, and real investment in observability in production. This way problems in prod are smaller, and you are fast at catching them anyway, even without staging.

On top of this, bonus points if you setup two of my favorite things:

• Preview links for QA — envs that spawn on the fly for features that need dedicated product review.

• Remote dev environments — that live in the cloud and can inherently be closer to prod than a developer laptop.

This is the gist of what I wrote back then — if you want the longer version (including when staging still earns its keep), here is the original:

✅ What still holds

Read more

I also publish an original essay every Wednesday. Here is the one from last week in case you missed it:

🖥️ Coding agents need more than a fast loop

This week’s newsletter is brought to you by our friends at Blitzy!

Coding agents have sped up individual tasks, but they haven’t solved the coordination, validation, and handoffs that determine how quickly software ships, especially the enterprise one!

Tomorrow join Sid, CTO and co-founder of Blitzy, for a live session on what changes when a system can hold context across an entire codebase—not just a single file or function.

He’ll walk through a real project in Blitzy, covering reverse engineering, tech spec generation, the Agent Action Plan, code generation, and PR Review, including where human review gates the process. There will also be a Q&A at the end 👇

Save your seat!

🚀 Create high-agency engineers

Out of the various interviews / case studies we did last year, one of those that aged better is the one with Greg from Graphite.

I think it’s because we talked very little about tech, and more about values and culture. In particular, Graphite is obsessed with building a team where engineers have high agency.

What does a high-agency engineer look like? For Greg, it’s someone who owns problems end-to-end, which allows for:

• More productivity — less back-and-forth, less waiting, less “that’s not my job”.

• More fulfillment — people who have real impact stay engaged, grow faster, and stick around longer.

High agency, though, it’s not (only) a personal trait — it needs to be enabled by your team environment. For Greg, at a team level, agency comes from three things:

1. Purpose — a clear vision of where you are going and why it matters.

2. Permission — freedom to go after big things, make decisions, and occasionally get things wrong.

3. Simplicity — cognitive load low enough that ownership feels like a superpower, not a burden.

To make this happen, for example, Graphite is extremely light on product management: a couple of PMs for 30+ engineers. Engineers need to fill the gaps: talk to stakeholders, prototype, dogfood, and push work forward without waiting to be unblocked.

To be honest, they also have an unfair advantage: they build a dev tool, so their engineers are their users. Which means product intuition comes for free.

But for everyone else, the principle holds: get engineers closer to customers. The difference is you have to invest more intentionally about it:

• Bring engineers to customers — sales calls, user interviews, support escalations. First-hand experience and empathy beats summaries.

• Make context accessible — record calls, share clips, pipe tickets into Slack.

• Dogfood relentlessly — even if the product is not “for” engineers, find ways to make them use it.

I wrote a full case study about how Graphite ships 👇

Refactoring

How Graphite Ships 🚢

It’s a weird time for engineering teams…

Read more

10 months ago · 19 likes · Luca Rossi and Greg Foster

🧭 Growth trajectory >> AI fluency

Greg’s story made me think about my interview with Austen McDonald, former hiring committee chair at Meta, and author of Mastering Behavioral Interviews, where we discussed how AI is reshaping how engineers get assessed.

Austen recommends evaluating AI skills in both technical and behavioral rounds, but the questions should be different. In behavioral interviews, stories are king. So you may ask something like “Tell me about a time you used AI and it didn’t work out.”, instead of abstract prompts like “How do you think about AI?”

But the strongest predictor is not current AI usage at all. It is growth signal: is this person staying on top of industry trends, running side projects, adapting when the ground moves? This has always been important, but now more than ever.

“The most important thing about this person is not how they use AI right now, but how they are going to be adapting to using AI as the landscape changes.”

Austen particularly focuses on ownership. As AI handles more of the stack, “full stack” expands toward more areas of the business: customers, go-to-market. Does the candidate reach for more end-to-end ownership? Are they willing to do so?

You can find the full conversation with Austen here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 If We Do Not Stop to Help Each Other, What Do We Become?

3 min • by Jeff Atwood

Jeff shares a great message from a developer who once got through a dangerous deployment because strangers on Stack Overflow chose to help him. An LLM could have provided the same (or a better?!) answer, but not the feeling that somebody cared. We need that reminder!

🥈 Dishwasher Mode for AI Tasks

1 min • by Clemens Adolphs

A dishwasher is slower than washing plates by hand, but we don’t care, because it still frees up our time. Clemens uses the same frame for AI: give it small, annoying tasks off the critical path, then care more about reliable completion than raw speed. I liked this a lot.

🥉 The Sensible Defaults Playbook

2 min • by Thoughtworks

Thoughtworks shares its collection of good defaults. Their approach is to have defaults remove recurring friction, but without becoming dogma. Start with practices that usually work, adapt them to the context, and review them as the environment changes.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by: Notion

Learn more about Notion’s Developer Platform today at https://ntn.so/refactoring

Today's guest is Adam Tornhill, founder of CodeScene and author of Your Code as a Crime Scene.

With Adam, we talked about how his team refactored the whole Street Fighter 3 codebase, which is more than 300K lines of code, in just a few days and spending a few thousand dollars in tokens, and measured how that led to better and cheaper AI development of new features.

And then we discussed how the CodeScene team does agentic development, what the process looks like, what the bottlenecks are, and what are the crucial parts that Adam believes we still need to do as humans.

Show notes:

• The Street Fighter case study

• Controlling the uncertainty machine: do you still need to read the code?

• Making legacy code AI-ready

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute, handcrafted takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. 🕹️ Refactoring Street Fighter with AI agents

2. 📖 The refactoring playbook

3. 🧪 Testing, docs, and the new fundamentals

4. 🤖 CodeScene’s agentic development process

5. 🔮 Bottlenecks and the future of the developer role

Let’s dive in 👇

1) 🕹️ Refactoring Street Fighter with AI agents

Adam’s team at CodeScene, in collaboration with researcher Marcus Borg, set out to explore large-scale refactoring using AI agents. They needed a code base that was non-trivial, legacy, and universally understandable, so they picked a decompiled version of Street Fighter 3, a 25-year-old game with 300,000 lines of C code.

When they first analyzed the code base with CodeScene’s own metrics, it scored a 5.4 on a 10-point scale, roughly the industry average for enterprise code and very far from what Adam calls “AI-optimal code” (9.5 or higher). The team had two goals: uplift the code health as much as possible, and preserve the existing behavior of the game.

“I’ve never ever seen anything close to this before. A rewrite at that scale would have been a 12-to-18-month project. And now that can be automated for a fraction of the cost at a fraction of the time.”

Street Fighter turned out to be a particularly good target because the game already had a clever test approach: computing the hash of each rendered frame and comparing them. This gave the agents a reliable way to verify that refactoring didn’t break anything.

The preparation phase took about two weeks (as a side project), with the team exploring different models and patterns. The actual uplift then took just a few days and cost roughly $4,000 in API tokens. They primarily used Claude Code with Opus, because while GPT models were competent at refactoring, they weren’t as good at documenting and extending the playbook.

2) 📖 The refactoring playbook

Read more

Every week I write an article to 170K+ engineers about how to make good software together. To get access to all our articles, resources, and our private community, subscribe to the full version.

Lately, I have become more and more interested in how teams are using AI inside real production workflows.

Sure, your Claude skills are cool, and so is your AGENTS.md, but I want to see how this tech graduates into stuff we run on our servers, reliably, all the time, and creating value.

This is also not the first time I have written about this:

• In May, I published How to Orchestrate AI Workflows, which is a primer for moving from AI loops to more structured orchestration.

• Earlier this month I published Getting Started with AI Governance, to talk about security, control, and how “if you want the car to go fast, you need brakes!”

So today I want to go deeper into what it means to create good AI workflows at scale, based on conversations I had with the people who are doing this for real. I took a lot from my recent chat with Jan, from friends I met during recent travels, and especially from Ludovic and the guys at Kestra, that I am looping in again after the huge success of the first piece we wrote together!

Here is the agenda:

• 👁️ Observability — what to watch on agentic runs, plus a practical checklist.

• 🛡️ Boundaries — how keep sensitive data where it lives, and separating workers from orchestration.

• 🧑‍✈️ Human in the loop — when to put a gate, and how to resume without replaying an hour of work.

• 🔌 How to get started — prefer governed flows as tools for the agents you already trust.

• 📚 Resources — a few places to go deeper on the practices above.

Let’s dive in!

Disclaimer: I am thankful to Kestra for partnering on this and providing ideas and insights about the orchestration industry. I am a fan of what they build and you should check it out.

Learn more about Kestra

However, as always, I will only write my honest opinion on the practices and tools covered, Kestra included.

👁️ Observability

A surprisingly high number of the teams I spoke with built some AI observability tools in-house.

To me this feels like when, around 2022, I ran the first surveys about what tools teams used to measure engineering metrics, and about ~50% of respondents had patched something together themselves, e.g. pulling numbers from Github and putting them on Grafana.

Likewise, today many teams are logging basic info themselves. For every run, this is typically token spend and the final answer of the agent. This is good! You get your hands dirty, learn what you really need, and wrap your head around the idea of measuring this stuff.

However, just like with engineering metrics, these setups fail as soon as you need to drill down into specific steps of the agent runs. Which, as long as your runs are short, might not be a big deal, but as soon as you have a long (and expensive!) recurring job failing halfway through, you may want more control over what happened and why.

In my experience, good observability for such workflows means you can reliably get the following information, ideally from the same system:

1. Intermediate step outputs — not only the final blob. Each meaningful step should leave something inspectable: a fetched record, a classification, a draft that was about to be sent.

2. Tokens and cost per run — these are the basics everyone measures, and for good reason. This should be ideally broken down per step / model / tool.

3. Tool and MCP calls — which servers, which tools, with what inputs and outputs.

4. Who or what triggered the run — a human (manual trigger), a schedule, a webhook, or another agent session. Origin and session tags matter when you audit later.

5. State at failure — which step failed, what was already committed, what is safe to retry.

If these still feel a bit abstract, here are a few questions you can ask yourself (or your team!) about your setup:

Read more

I also publish an original essay every Wednesday. Here is the one from last week in case you missed it:

📚 Master production-grade orchestration, for free!

This week’s newsletter is brought to you by our friends at Kestra!

A few months ago we published an article about how to orchestrate AI workflows, together with the team at Kestra, an awesome open source project used by the teams at Apple, JPMorgan, and more.

Today I am happy to help them spread the word about the Kestra Academy, where you can find self-paced courses on infrastructure automation, AI workflows, and standardizing orchestration across your engineering org.

In the academy, you can learn how to govern complex workflows and start running production pipelines the same week.

You should check it out! 👇

Learn how to run orchestration

🏁 Faster coding vs faster delivery

Most AI coding tools optimize the inner loop: you describe a feature, watch the agent write the code, and get something working a few minutes later.

This is where AI has delivered the most visible improvements, but it is just a small part of how software gets shipped. There is an outer loop too: deciding what is worth doing, shaping and reviewing the solution, releasing it, and learning from production.

If you don’t improve both, the constraints move elsewhere:

• Reviews — agents create more PRs than people can reasonably inspect.

• Shaping product — the team can build faster than it can decide what good work looks like.

• Customer signal — shipping the wrong thing faster is still waste.

At a small scale, this is the same acceleration whiplash I have experienced with Tolaria. Starting work is cheap, but my ability to review it, decide what is good enough, and create the next piece of work does not scale at the same rate.

So the goal should be end-to-end throughput, which requires a workflow that encompasses both the inner and the outer loop.

I wrote a full case study about how the guys at TRM are doing this 👇

🧪 Agents will game your tests

When I spoke with Anush Elangovan, VP of AI Software at AMD, he described a funnel where creativity is allowed to run wild at the top, but progressively more human judgment is needed as code moves toward production.

He also shared an entertaining story about building Spur, an AI-native job scheduler. The agents writing the code were so “smart” that they found a shortcut: instead of actually running unit tests on hardware, they simulated the hardware and reported success.

“The agents were so smart that they would simulate the hardware and pass the unit test and say ‘yes, success.’ We had to say no, no, no — you never simulate the hardware. You have to actually run the test on hardware before you can promote a PR.”

That forced the team to build a superset of the guardrails they previously had for humans. Test harnesses now need to account for adversarial agent behavior. The result, Anush said, is a quality system roughly 10x more comprehensive than what was needed when humans were the only contributors.

I have found this to be a useful posture: assume agents will optimize for passing the tests, not necessarily respecting their spirit, and design accordingly.

You can find the full conversation with Anush here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 The Manager’s Path in the Age of AI

13 min • by Camille Fournier

Camille asks what still belongs to managers when AI raises individual output and eats the small day-to-day interactions that build trust and rapport across an org. This is the written version of the talk I heard at LDX3. I loved 100% of it and it gave me a lot to think about.

🥈 Code Yellow, Code Red

11 min • by James Stanier

James walks through declaring a Code Yellow after months of instability eroded some customer trust. Running it well means clear communication, defined goals, and company-wide support to fix problems, without blame.

🥉 Trying the Software Factory Pattern

3 min • by Will Larson

Will is trying a “software factory” loop: agents audit the project’s goals and metrics, keep track of state on Linear, work on the non-blocked tasks, and re-check when the description goes stale. This was short and practical, and a good read.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Run a better engineering team

See you next week!
Luca

The Refactoring Community hosts a ton of great conversations in threads, but, most of all, every month we run two live sessions, hosted by our awesome Community Lead Melinda Seckington, and always joined by yours truly. These sessions are:

• Mastermind — leaders bring a real problem and we work on it.

• AI Club — people show what they learned about AI that month, including the parts that did not work.

This monthly recap makes sure the good stuff doesn’t stay confined to Zoom. So here is the agenda for today:

• 🧠 How to think about productivity now — what we discussed in the September Mastermind.

• 🪄 September AI Club — terminal personal OSes, picking the right models, and agents that live in the cloud.

• 📅 What’s next — AI Club on 2 Oct, Mastermind on 19 Oct.

• 💬 How this works — and how to join.

Let’s dive in!

🧠 Redefining developer productivity

In this month’s Mastermind, we talked about what “good” looks like when most of the volume metrics (PRs, commits, lines of code) feel worse as proxies than they used to. Btw, have they ever been good? Probably not, but they are arguably worse now because of AI.

One of the biggest points of discussion was the tension between:

• Investing in automation — making the “harness” and overall pipeline better, versus

• Just shipping things — with familiar human-in-the-loop tools, especially under a short deadline.

Why is there a tension? Because with engineers increasingly acting like engineering managers of agents, there is often higher leverage in using their time to improve the system, rather than closing the ticket in front of them. But it’s still tricky to balance the two

Personally, I have found that, before digging into a task, it’s worth asking yourself whether this is a system problem, worth fixing for posterity once and for all, or just an immediate, one-off ask.

My rule of thumb is: as soon as you find yourself doing something twice, make sure you don’t do it three times.

We also talked about what to measure instead. Some people pointed at the team actually delivering on their cycle (or Sprint) goal, while others discussed the amount of features shipped, or cycle time. I think a very good leading signal is simply to directly ask engineers how they feel about work, and where they felt they were wasting time that week. Good conversations always deliver an incredible amount of signal.

There is also a ton of work that is hard to measure, aka invisible work. Things like mentoring, “insurance” maintenance, and the informal chats that keep teams moving do not show up well under hard metrics, and they get squeezed when people optimize for numbers. It’s also unclear how to score that work cleanly. No one had an answer here!

As always, you will find the full writeup in the Mastermind space (if you are in the community).

🪄 September AI Club

In this month’s AI Club, four people shared their AI wins for five minutes each, then we did the quickfire round again. Melinda wrote it up properly here. I liked a few threads in particular:

• AI as an accountability partner — Kishan has built something like a personal OS that mostly runs from the terminal: daily check-ins that read journals and plans, a weekly retro on what energized / drained him, and a folder structure so ideas (notes, photos, voice memos) get filed without conscious sorting. He also shifts models by task (heavier for planning, lighter for execution) and periodically audits the core memory file itself, because instructions written in one place do not always get pulled into the right context.

• The same prompt, very different outputs — Julie ran the same reflection prompt across a chat interface and a coding CLI on the newest model family. The chat version came back quite readable, while CLI version needed a lot of correction. Several people in the room had noticed the same split: this generation of models can be stronger at one-shot coding and weaker at other things, like… clear writing! So, do not assume “newer” means better across the board.

• Moving the operation to the cloud — I shared my own migration of agents off the Mac Mini onto cloud machines (after a power outage and too many agents fighting over one box). Coding stays local on a kanban, while everything upstream moved. I wrote the longer version in My agents are moving to the cloud.

• Working solo on something that cannot fail — Andrea is deep in a high-stakes embedded / security project, mostly alone after the rest of the team went on leave. He walked us through how he set up two tiers of docs: customer-facing ones, reviewed manually line by line, and internal brainstorming ones, mostly for AI consumption and not reviewed directly. Autonomy helps a lot, but every now and then it does funny things, like fabricating a citation that doesn’t exist, so he keeps the high-stakes parts of the ops on a tight leash.

All in all, a very practical session. Less about shiny demos, more about how people actually run their week with these tools.

📅 What’s next

Both the Mastermind and the AI Club, like all community activities, are for paid members of Refactoring. Here are the next editions 👇 RSVP on the event page to help us size the room:

• AI Club • Fri 2 Oct, 14:00 CEST — five-minute shares, questions, and the one-sentence fireside round at the end.

• Mastermind • Mon 19 Oct, 18:00 CEST — third Monday of the month, as usual!

Bring your questions! These sessions work because of the members, not because someone gives a talk.

💬 How this works

The Refactoring Community is a private space for paid subscribers of Refactoring: it hosts async threads, Q&A, and monthly live rooms like these.

It is home to more than 1500 engineers and managers, and it’s my favorite place on the internet!

To join it, subscribe to the full version of Refactoring!

Join the full Refactoring 🙌

That’s it for today! See you on the next events!

I wish you a great weekend 👋
Luca

Every week I write an article to 170K+ engineers about how to make good software together. To get access to all our articles, resources, and our private community, subscribe to the full version below:

Run better engineering teams

I am back from two weeks of travel, during which I took some time off but also attended two great conferences: LDX3 in New York, and ProductLab in Berlin.

Conferences, for me, are a great way to say hi to friends and to check the pulse on the trends and sentiment in tech. This time made no exception: I learned a lot by attending talks and workshops, and simply by talking with people.

So this week’s edition covers my personal highlights from such talks and conversations.

Here is the agenda for today:

• 🚚 Doing migrations right — Will Larson on Imprint’s delivery journey, framed as a set of migrations.

• 🗣️ Guiding people through change — Meg Adams’ diagnostic tools for having hard conversations.

• 🧭 The Manager’s Path in the Age of AI — Camille Fournier on what still belongs to managers in 2026.

• 📊 DX quarterly numbers — Justin Reock’s latest AI impact data, plus how I think about these metrics.

• 🎙️ How n8n is embracing AI — highlights from my conversation with Jan Oberhauser, founder of n8n, in Berlin.

Let’s dive in!

1) 🚚 Doing migrations right

LDX3 opened with a great keynote by Will Larson, who described the delivery journey of his team at Imprint: from manual gates everywhere, to CI/CD, to agentic workflows.

Will’s talk was extremely practical (we need more stories like this), and framed these steps as migrations. We know a lot about migrating code, and most of the same logic can be applied to processes and workflows.

Here are the parts of Will’s approach that stuck with me:

• Understanding people’s problems — there’s a reason why people do things the current way, even if it looks flawed. To build something better, you need to understand current problems deeply, which is hard!

• Getting buy-in — Will stressed how, even if there was clear top-down input about the direction, the implementation was a collaborative effort where people were involved at all stages. At some point, Will wanted to migrate from Jira to Linear, and started by involving the people who were the biggest Jira fans. He knew that if he could address their concerns, the rest of the team would be easier to migrate to.

• Draw hard lines — inclusion and collaboration matters, but once a direction is taken, resources need to go to supporting that direction. In picking AI coding agents, Will settled on making Claude Code the recommended coding tool: people who wanted to use something else could do it, but would not receive support from the platform team.

• Sweat the details — with any migration there is always a long tail of stuff to be fixed and edges to be smoothed until things truly work for everyone. Keep working on it even when it seems that “most of the work is done”.

2) 🗣️ Guiding people through change

During LDX3, I attended a workshop by my friend Meg Adams, who I also interviewed last year, about having good conversations with people and guiding them through change.

Read more

I also publish an original essay every Wednesday. Here is the one from last week in case you missed it:

🔨 Autonomous Software Dev for the Enterprise

This week’s newsletter is brought to you by our friends at Blitzy!

Blitzy is built for large, complex software projects that other coding agents cannot handle: large-scale refactors, scaled vulnerability remediation, and undocumented legacy systems.

Blitzy’s Sandbox lets engineers evaluate Bitzy on their own software estate, at their own pace. Eligible organizations can connect real applications, reverse-engineer up to 1 million lines of code, generate up to 25,000 lines of E2E tested code, and surface prioritized security vulnerabilities across their software estate.

Try Blitzy on your codebase

🌱 Grow juniors with guardrails and judgment

One of the most important qualities of junior engineers is that they can turn into senior engineers. But that does not happen automatically.

There is a dream scenario where juniors grow, become senior, and help rotate headcount when seniors leave. But there is also a nightmarish one where juniors are thrown in the trenches, abuse AI in ways they do not understand, nobody checks on them, and the product eventually goes to shit.

When the growth cycle breaks, it is usually because juniors do not have a tight enough feedback loop: 1) one that keeps the work high quality, and 2) one that helps them learn from doing it.

To me, the loop looks two-fold:

• Invest in an amazing harness — create guardrails, rules, and gates, and encode everything that can ensure quality automatically. This helps everyone: seniors, AI, and juniors who can learn from the same rules.

• Pair juniors with seniors — let young folks experiment and move fast, but keep seniors close enough to provide judgment and supply systems thinking.

There is no way around the fact that AI makes the junior path harder in some ways, because more judgment is needed earlier. But that is also why freezing junior hiring is such a fragile bet: you optimize for today’s output, at the expense of starving the pipeline that should produce tomorrow’s seniors.

I wrote more about this in the full piece 👇

📱 Most AI products are still “m.google.com”

When I spoke with Charlie Guo, developer experience engineer at OpenAI, we walked through three eras of AI-assisted development.

First came models that could autocomplete snippets, which became tools like GitHub Copilot. Then came AI-native IDEs like Cursor, where the model sat inside the editor. Now we are entering a third phase: standalone apps that orchestrate agents, and move beyond the text editor entirely.

Charlie believes most products are still stuck in the middle of that transition:

The m.google.com of AI was (or is): take this product and cram a chat box on the sidebar, and then chat with it on the side while you’re doing stuff in the app.

That’s similar to when the iPhone arrived: companies took desktop websites and mostly squeezed them into a vertical layout. It took nearly a decade to invent native mobile products: Uber, DoorDash, things that could not really exist on a desktop.

We are in that same messy middle with AI. A chat sidebar is not necessarily bad, but it’s just an easy retrofit. Native agent experiences are still being invented.

You can find the full conversation with Charlie here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 What psychological costs does AI adoption create for engineers?

4 min • by Lizzie Matusov

Lizzie covers a case study from a large Danish software company, one year into an org-wide AI rollout. Regular engineers feel “accountability anxiety” and “craft identity disruption” as tools take over work, while architects (more augmented than replaced) barely show the same costs. Interesting research, worth looking into.

🥈 De-Brainrot Vacations

4 min • by Daniel V.

This piece by Daniel really resonated with me. He noticed that after years of engineering, plus AI and phone dopamine, his thinking got slower and lazier. So he used a countryside vacation to reset his brain by reading hard books and doing math and physics as a hobby. This is peak mental marathoners!

🥉 The Internet Is Kind of a Predatory Cesspit Now

8 min • by Stephen Diehl

Stephen argues that, over the years, “predation” moved from the periphery of the internet to its very core, and that a lot of ordinary people have been (often unwillingly) conscripted into it. I have less of a bleak view about this, but I see where Stephen is coming from and I don’t necessarily disagree. Worth reading!

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by: Notion

Learn more about Notion’s Developer Platform today at https://ntn.so/refactoring

Today's guest is Rahul Raina, co-founder of TRM Labs, which is a YC-backed company that works on solutions and products for security, fighting criminal networks, AI attackers, and more.

With Rahul, we had a great chat that covered a lot of ground, starting with the tough reality of cyber defense today, and why attackers by default always have the upper hand when AI and technology improves. And then we moved to a deep dive into how the TRM team works and how they rebuild their whole product development workflow for AI, including reshuffling all the technical roles, compressing the developer process steps, and creating dedicated artifacts for human reviews.

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute, handcrafted takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. 🛡️ The asymmetry of cyber defense

2. 🔄 Inner loop vs. outer loop

3. 🏗️ Redesigning the shipping framework

4. 🧪 Building a culture of experimentation

5. 🔍 Human review in an AI-driven workflow

⠀Let’s dive in 👇

Read more

Lately I have been talking with both software engineers and security people about this moment in AI rollout, and it’s interesting how they feel it from opposite ends.

At first glance, there seems to be a real tradeoff between making AI useful, and keeping it safe:

• On one side, the best stories about AI impact always involve connecting it to pretty much everything you got: tickets, docs, CRM, Slack, telemetry, and more.

• On the other side, this creates a big surface for threat that even the latest models are extremely vulnerable to.

As of today, most agent setups I see are still completely ungoverned. What can this agent touch (MCP servers, browser, data, ...)? Who connected it? What customer data can actually leave the building?

Many teams are conflicted about putting real governance in place because that’s perceived as a brake, in a moment in time where you are expected to accelerate instead.

I have a contrarian take on this, that I want to discuss today.

In 2006, Peter Summerlin, a professor in Landscape Architecture in Mississippi, ran a simple study about kids’ behavior in playgrounds:

A simple study was conducted to discover the effects of a fence around a playground and the consequent impact it would have on preschool children. Teachers were to take their children to a local playground in which there was no fence during their normal recess hour. The kids were to play as normal. The same group was to be taken to a comparable playground in which there was a defined border designated by a fence.

In the first scenario, the children remained huddled around their teacher, fearful of leaving out of her sight. The later scenario exhibited drastically different results, with the children feeling free to explore within the given boundaries.

So, in playgrounds with fences, kids actually used more space.

I believe good governance can be an accelerant. Like kids, once people know where fences are, people can be empowered to do more within them. But that’s only if governance is good, which is a big if!

Today I will try to explore what this looks like, in the context of AI. Here is what we will cover:

• 🛡️ Bounded freedom — why governance speeds AI rollout, instead of killing it.

• 🧱 What you should actually govern — people, agents, data, tools, and actions.

• 🔒 AI operates on a subset of human trust — the analogy between human and AI governance only goes so far.

Let’s dive in!

To explore this, today I am also bringing in Becky Brooks from MCP Manager. They work at the forefront of this topic, and have helped me a lot in making my mind up.

I am grateful to have them as a partner on this piece, and you should check out their MCP gateway, which companies use to manage how data and tools flow to and from AI.

Learn more about MCP Manager

In any case, I will only write my unbiased opinion about the practices and tools covered, MCP Manager included.

🛡️ Bounded freedom

A lot of ambitious teams that I know delay governance and compliance, with the idea of going AI-native first, and cleaning up later. In my experience, this order is backwards.

Without clarity, people react in two possible ways:

• They freeze — they are unsure about being punished for this or that, and they just go the safe route, doing with AI much less than they would be able to.

• They go underground — I was on a train once, and a guy in the seat in front of me was using his company laptop with one hand, and ChatGPT on his personal phone with the other hand, copy-pasting stuff between the two. You don’t want that to happen, obviously.

There is a study by Google Cloud from late 2025 that confirms this. Teams with clear governance in place get much farther, and faster, on agentic adoption than those without:

• Teams with full governance → 46% adoption

• Teams with partial governance → 25% adoption

• Teams with “still in development” governance → 12% adoption

Of course, correlation is not causation, but it still makes it hard to look at governance as only a tax.

And when you think about it, that’s already what happens in specific domains, like coding. We talk about guardrails for AI coding all the time, and I even published my own approach with guides, gates, and guards, which is, admittedly, quite strict.

Having strict guardrails is what allows me to be more liberal with the tasks I give to my agents, and what they can ship.

So if good governance is guardrails for data, tools, and people, what does this look like? Let’s try to figure out.

🧱 What you should actually govern

Working from first principles, what is it that you should actually govern? To me, it looks like four things:

1. People — who may use which AI client or gateway (by role, team, seniority). E.g. engineers get the coding setup, interns do not get production CRM tools, etc.

2. Agents / clients — which AI surfaces are approved at all. Company Claude yes, personal ChatGPT no, and so on.

3. Data & tools — what the agent can see (and call). Drive folders, Slack channels, Jira projects, databases, CRM, MCP servers.

4. Actions — what it may do once it is connected. Read, write, delete, open a PR, message a customer.

It’s interesting to note that this looks exactly like the same line of reasoning we should apply to people, even without AI. And I would argue that we should start with governance for people first, and address AI later, because the opposite is impossible.

Read more

Every week I write an article to 170K+ engineers about how to make good software together. To get access to all our articles, resources, and our private community, subscribe to the full version below:

Run better engineering teams

Hey there! Welcome to a new edition of my monthly updates on my AI coding workflow.

Once a month I publish an article about how I develop Tolaria, including an overall recap plus the latest updates, and some useful resources. I only ship these when there is something real to say, so you will not get the same post every month just with a different date on top!

I consider part of my work (combined between Tolaria & Refactoring) to always try the latest and greatest, so you will only get thoughts that come from real usage and experiments.

So here is the agenda for today:

• 🔄 Workflow recap — for those who missed the previous episodes.

• 💧 Tolaria updates — product updates from this month, and what’s coming next.

• ☁️ Agents move to the cloud — a general trend plus my personal experience of gradually moving away from my Mac Mini (and my laptop).

• 🤖 Moving to Grok Bot — from a single local orchestrator agent to a panel of five, hosted in the cloud.

• ➡️ What’s next — what I am excited to improve for next time.

Let’s dive in!

🔄 Workflow recap

For those who missed the previous episodes, here are some pointers:

• Product management — how I do traffic control between issues, PRs, and product requests, and orchestrate the overall product backlog.

• AI Coding — my own approach to scaling coding with agents, via guides, gates, and guards. Mostly remained unchanged from last month.

• Tools I use — deeper dive into how I use the tools from the alliance, and others too.

• Summer AI Coding Updates — last month’s edition (forking our editor, custom performance gates, and notes about the latest GPT models).

Also, this series (and the whole development of Tolaria) is made possible by the great tools I use everyday, that also sponsor the project, which are:

1) CodeScene 🔬

It guarantees great code health across the codebase. My coding agent uses their CodeHealth MCP as a guardrail, enforcing 10/10 health. I have been using it from day one of Tolaria — very recommended!

2) Codacy / Verity 🔒

It’s quality & security as gates: an all-in-one PR reviewer. Codacy also launched Verity (beta), which helps Claude Code fix quality, security and intent issues on every run, and builds a knowledge base of decisions to make every session smarter than the last. That’s basically what I have been doing manually with ADRs, so I am excited to see where this is going!

3) Unblocked 📑

I use Unblocked as context engine, both for my coding agent to use, and for me to ask questions to. I am also testing their latest Unblocked Code coding agent as a straightforward way to get small tweaks done.

4) CircleCI 🔧

I use CircleCI to run local validation remotely, via their chunk sidecars, and I am also migrating the whole remote CI to it. I love their focus on speed and reliability, which is important to me in order for validation not to become too big of a bottleneck for the coding agents.

Now on to the new stuff, but first... product updates. Because my workflow is only as good as what I ship with it 👇

💧 Tolaria updates

This month for Tolaria is lighter on new features and heavier on two bets that I am still working on. These are the mobile apps and a hosted version of Tolaria. These are also two of the things people are asking the most for (mobile is #1 far and away), so I’ll give some updates.

1) Mobile

By now I have a working iPad prototype already, with the UI rebuilt in React Native. It’s a universal app, so it also works on Android tablets (at least in the simulator, because I don’t have one to try it on).

I started with the tablet first for two reasons: 1) the UI is easier to map almost 1:1, and 2) selfishly, I do a lot of work from my iPad so I need an iPad version of Tolaria.

What’s still missing/unfinished are mainly the Git and AI integrations, with Git being particularly tricky (will elaborate on it over the next few weeks!)

Once the tablet version is ready, though, the smartphone version is quite straightforward. The UI will be simplified a bit, but technically speaking it’s not a big deal.

2) Hosted Tolaria (exploration)

I am also thinking about releasing a web version of Tolaria, that is self-hostable but also available with a managed option, so the vault is reachable without the need for a local native app.

This would be good for many reasons:

1. Makes sharing files easier, opening up to web sharing a-la Google Docs

2. Obviously makes vaults accessible from anywhere (also reducing the need for mobile native apps)

3. Could turn into a side business model for Tolaria, with the managed version being a paid option.

About the third point: Tolaria is run and developed as a sidekick to our Refactoring business, and we also have good sponsors (thank you ❤️) so I am under zero pressure to profit further from it, but a hosted version 1) could be genuinely useful for some people (they asked for it) and 2) of course it would cost money to run, so it should be paid.

This is still in exploration stage, so I am happy to hear your thoughts and ideas about it!

☁️ Agents move to the cloud

By now, I think we have all seen the half-open laptop meme. As you can see in my previous posts, for a long time, I have run all my product development on my machines, and in particular:

• Orchestration via OpenClaw first, and Hermes later.

• Coding via Codex instances on my Mac Mini, and occasionally on my Macbook.

I believe that as 1) we understand better what agents need, and 2) agents themselves become more capable and can use computers more broadly, we are necessarily going to move away from local environments and increasingly ship these in cloud VMs, at least for agents doing actual work.

In a way, this is the same thing that happened with regular applications. We are not hosting websites and apps in our basement anymore (at least not the majority of us?!) — we are using the cloud for that. Cattle vs pets, and all that jazz.

I believe the same will happen with AI agents, because the benefits are just overwhelming. I don’t think local agents will disappear, because it will always be good to run a good agent on the computer you actually use, but when it comes to deploying many of these boys, there’s no way we’ll be doing these on our machines. So most of the agentic work will turn to the cloud.

For me, this migration had already started slowly, with experiments here and there, but got a decisive acceleration when I tried Grok Bot this month.

Grok Bot has been a revelation because of how well-made, convenient, and useful it is, and I migrated all my agent work (except coding) to it.

If you are unfamiliar with it, Grok Bot is an agent environment built by the team at Cursor, where you can spin new agents that are automatically hosted in the cloud, on Linux VMs. On their machines, the agents can do anything they want: install stuff, use tools, use their browser, and so on. Mind you, this is nothing inherent new or that you couldn’t setup yourself, but it’s just 100x more convenient than setting things up manually with SSH, Tailscale, Termius, containers, and so on.

So, once you try it, it’s pretty obvious all agent environments should work this way — and I bet they will. Because having agents hosted in the cloud makes them replicable, disposable, makes it easy to run many of them concurrently, gives you good mobile access out of the box, and a loong tail of quality of life improvements.

So here is how Grok Bot changed my agents setup:

🍻 From Brian to... Brian & Friends

If you followed the previous episodes, you know that I have been running just one main agent, as a second version of myself, for all the orchestration work that is not coding. It’s called Brian, and it’s my “Chief of Staff”.

Over time, Brian clearly started doing too many things, taking care of: product work for Tolaria, all kinds of work for Refactoring, and various scheduling and traffic control for my own life.

Grok Bot allowed me to easily spin off Brian’s work into multiple agents, so that now I have five of them:

1) Brian

Stays as Chief of Staff.

He is the default agent I reach out to for anything misc, and is on top of everything that happens that I need to be aware of.

Every morning he sends me a brief about what I need to take care of during the day, writes me a recap at the end of the day, and occasionally sends stuff my way during the day when something urgent needs my attention.

Now, however, he is also being helped by his friends 👇

2) Karn

Runs product management for Tolaria.

It has access to all the product signal that comes from PostHog, Sentry, Github, Canny, emails I receive about Tolaria, and people talking about it on social media. Like Brian before it, it does traffic control, triages and dedupes items, specs them, and puts them on the coding kanban board.

He also sends me a brief every day with the key decisions I need to make about Tolaria, including tasks I need to review 👇

3) Edith

Is my Editor-in-Chief.

While I insist on writing all actual articles by hand, Edith intervenes at the beginning and at the end of the process, by doing:

• Research — for any new article, it pulls related ideas from past articles I wrote, interesting new articles I should read, podcast interviews, and more. It is phenomenally helpful in compressing and improving the research work.

• Feedback — after I write a full draft, it gives feedback on everything from basic grammar to inconsistencies in the structure, contradictions, and so on. It is good at challenging what I write so I can reflect more on it.

It is also “in charge” of the newsletter publishing calendar, so it makes sure the various drafts are ready at the right time, podcast interviews are lined up, articles done in collaboration with others are shared in advance, and so on.

4) Sponsy

Helps with Sponsorships.

At Refactoring, sponsorships are led by Matteo, my closest collaborator, and they involve a ton of operations. Sponsorships deals have a whole lifecycle, like standard sales deals, plus they spawn tasks that are specific to the newsletter work, like:

• Collecting the copy to include in the newsletter

• Sending previews

• Reporting numbers after ~1 week

• and more!

Sponsy helps a lot with mapping the status of the various deals, what needs to happen every day.

5) Grow

Grow is the newest addition to the bot stack (or should I say team?!). It tracks a bunch of metrics for the newsletter health (open rate, growth, click rate, ...) and helps with operations related to the paid ads campaigns we run.

In particular, it does a lot of work that would otherwise be manual: for example, importing new subscribers from campaigns into Substack, and assigning the correct tags to them so we can create email sequences.

It also sends daily reports about how campaigns are going, like:

Closed week: 163 Substack Meta regs (vs 203 prior, −40 / −20%). Received-mail unique 7d open 40.0% (60/150) vs 44.8% prior (−4.8pp). Morning was 36.1%; afternoon refresh after more people received mail (raw 60/163; 13 not received yet). Paid converts: 2 (vs 3 prior).

🔭 What’s next

This setup is about one month old and is by no means perfect, but it just shows how much promise there is in creating fleets of small-scoped agents to scale work.

To make this viable, though, it needs to be easy to create and steer these guys, which I think is what Grok Bot is uniquely succeeding at.

When agents are successful at 1) doing their thing, plus 2) reporting to you, you can legitimately do most of your work in a chat environment, or with voice notes, without using the tools yourself. Which is basically what you see in the GPT 6 Astra launch video.

I think this is a good direction to point ourselves at, as a proxy for good delegation. So, good questions you may ask yourself are:

• “How can I use my computer less?“

• “How can I do more work from my phone, just sending voice instructions?”

In a way, it’s like when you see pictures of the desk of the President of United States, and there is no computer. How does the guy (ok, the previous guys) get things done? He just talks to people. People come and tell him what needs to happen, what decisions he needs to make, who he needs to speak with, and he does. He doesn’t need tools.

This might be an extreme, but I think you see my point.

📌 Bottom line

And that’s it for today! Here are the main takeaways:

• ☁️ Most agent work goes to the cloud — local agents won’t disappear on the computer you actually use, but deploying the bulk of them on your Mac Mini (or laptop) stops making sense. Same arc as applications in the cloud.

• 🤖 Grok Bot made the jump easy — cloud Linux VMs, install what you need, with browser included. I moved all non-coding agent work there because it is just much more convenient than the SSH / Tailscale / Termius stack.

• 🍻 Team of small agents > One mega agent — work feels better now that I have a small fleet of focused agents instead of Brian doing all of the work.

• 🔭 Aim at less computer, more chat — when agents do the work and report back, more of your day can live in chat or voice notes. So a proxy for good delegation becomes: how can I use my computer less?

I wish you a great week!

Sincerely 👋
Luca

I also publish an original essay every Wednesday. Here is the one from last week in case you missed it:

🔒 Autonomously secure your software estate!

This week’s newsletter is brought to you by our friends at Blitzy!

Security teams don’t need another list of alerts: they need to know which vulnerabilities are reachable, exposed, and safe to fix!

Blitzy understands your entire software estate and, through Proactive Insights, surfaces known and previously undiscovered vulnerabilities in your codebase, and remediates them autonomously.

It traces each finding through execution paths, identifies dependencies, and validates remediation. Then it turns prioritized risks into batches of tested PRs for engineers to review.

The result is less alert noise, faster remediation, and confidence that each fix is safe. Find risk in context and remediate it securely at enterprise scale!

Try on your codebase today

⚔️ Match quality controls to the risk

Next week I will be in New York to attend the LDX3 conference, which made me think about the best ideas I got from last year’s edition in London.

One that particularly resonates with me, especially now that I work on Tolaria, is about quality controls.

The most common approach to software quality is to treat every change the same. Everything goes through the same QA process, regardless of what it touches or what happens if it fails. This is simple to setup, but it tends to push teams toward one of two bad extremes:

• Too much control — every release becomes slow and painful, even when the change is small and reversible.

• Too little control — the team moves fast until failures pile up and you end up spending most of your time cleaning up.

At LDX3, Christine Pinto argued that the confidence you need before shipping should be proportional to two things: customer impact and technical risk.

A tiny change with a narrow blast radius does not need the same process as one about payments or critical infra. The farther a change moves up and to the right on those two dimensions, the stronger your evidence should be before you release it.

I also liked Christine’s framing of quality as a maturity journey:

1. 🦸 Crusader — one person champions quality and convinces others that it matters.

2. 🪴 Coach — that person teaches the team and turns individual judgment into shared principles.

3. 🌳 System — those principles become processes that no longer depend on a hero.

The goal is to build the right level of confidence for the risk at hand, then make that judgment part of how the team works all the time. I wrote more about this in the full piece 👇

Refactoring

My Favorite Ideas from LDX3 🎟️

Last month I spent two days in London for the LDX3 conference…

Read more

a year ago · 27 likes · 3 comments · Luca Rossi

🎙️ Storytelling is built through reps

When I spoke with Stephanie Wong, Head of Technical Marketing at Google Cloud, she told me that people often ask how she makes explaining complex topics look so natural.

Her answer is not super glamorous: what people see is the result of 10,000+ hours of practice.

Her story is especially useful when it comes to storytelling, because for some reason that’s often treated like a personality trait. Some people supposedly have it, while engineers who struggle to explain their work assume they are simply not storytellers.

Instead, Stephanie’s method looks much more like product development:

1. Understand the audience — know what they need and give them something concrete and hands-on.

2. Iterate — move through ideation, prototyping, testing, and releasing, like you would do for a product, instead of waiting for the perfect explanation.

3. Stay consistent and authentic — always bring your own perspective, then learn from what lands and what does not.

Technical audiences are particularly wary of fluff. Credibility comes from knowing the subject, but relatability also matters, and that comes from making your personal take visible. You need both.

So, don’t wait until you feel naturally good at communication: explain one thing, notice where people get confused, rewrite it, and tell the story again. Over time, the rough edges disappear and the result starts to feel effortless.

Natural is a feeling the audience should have, but on your side, it is reps!

You can find the full conversation with Stephanie here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 Breaking Claude Code Opus 5 Auto Mode

9 min • by Johann Rehberger

A reminder that even the most powerful models are still extremely vulnerable. A website asks Claude Code to summarize some content, nudges it from WebFetch to curl, and then turns the model’s own Python decoder into an exploit. Food for thought if your agents are touching untrusted content.

🥈 Selling Out

15 min • by Sean Goedecke

Sean’s title is provocative, but the thesis is very important: professionalism is a role, and playing it well doesn’t have to consume your life. Everyone chooses how much integrity to trade for wealth and career success, and that’s mostly ok, things are not black or white. A sharp essay about work and personal identity.

🥉 Small Models Have Arrived

3 min • by Calvin French-Owen

We spend so much time looking at the smartest models that it’s easy to miss how far the cheap, fast ones have progressed. Calvin argues that this is (even more than frontier models) what makes many new products viable, because not every task needs a genius: most business work just needs something responsive, cheap, and that keeps the ball moving.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Today’s episode is brought to you by: Notion

Learn more about Notion’s Developer Platform today at https://ntn.so/refactoring

Today's guest is Dennis Pilarinos, who is the founder and CEO of Unblocked and a long-time developer tools expert, having founded BuddyBuild and having served as a director at Apple, Amazon and Microsoft.

With Dennis, we talked about the latest trends in software development. We discussed why more tokens are not translating into more productivity, how to guarantee code quality with AI and what's the role of context in making your agents succeed. We also explored how the Unblocked team is doing it, navigating their setup and a few use cases about how their shipping features in production.

🥇 Interview Summary

If you are a 🔒 paid subscriber 🔒 you will find my own summary of the interview below.

It’s the 5-minute, handcrafted takeaways of what we talked about, with timestamps to the relevant video moments, for those who don’t have time to sit through the 1-hour chat.

Here is the agenda for today:

1. 🔥 More tokens, less progress

2. 🔍 Code review is the new bottleneck

3. 🧠 Context is the missing piece

4. ⚙️ How Unblocked runs internally

5. 🚀 What’s next: remote agents and Unblocked Code

Let’s dive in 👇

Read more

I loved it, celebrated it as the best piece I read that week, and since then I have kept thinking about various ideas that built up on it, and that I want to share today1.

First of all, let’s start with the title: what does thrive mean?

Each of us has (or should find) their own version of success, but a broad one that I particularly like is: becoming better versions of ourselves. I like it because a lot of the good things in life are downstream from growing as a person. You become better, and better things eventually come to you. That’s true in work, relationships, health, and pretty much anywhere.

Of course, that’s not the only way good things can come to you. Many come by chance, or luck if you may, but if you want to find an angle that is actually under your control, you can hardly go wrong by trying to upgrade yourself over and over.

And here is the problem with AI: I don’t feel that as a tool, by default, it helps with that. Pretty much the opposite. If you blindly use AI to the best of its capabilities, chances are you gradually get worse at several things.

So, when I think about my usage of AI, that’s the angle I try to use. How can I make it so that 1) AI helps me at producing more and better work, while 2) making me better as a professional (and a human being)?

We’ll be exploring this today:

• ⚖️ What AI takes from us — for everything tech gives, something else you now have to nurture.

• 🏃‍♂️ Mental marathoners — when intelligence is cheap, stretching yourself becomes your edge.

• 🧱 What makes us marathoners — instead of riding as passengers.

• 🧭 How do you know you are on the right track — better work, learning, taste, energy.

Let’s dive in!

⚖️ What AI takes from us

The fact that AI makes us worse at some things may seem counterintuitive, especially coming from a work tool, but it’s not really surprising if we look at the broader trajectory of technology.

In fact, progress is not strictly linear and incremental. For everything that tech has given us, there are also things we now need to intentionally safeguard and nurture, more than we used to in the past, in order not to lose them:

• An obvious one is physical health — regular, commercial gyms were not a thing until the 70s, when our lives gradually became more sedentary and all of a sudden fitness became something you had to be intentional about.

• Another one is attention span and our ability to focus — today, we kinda all agree they are under attack, but we hardly ever thought about them before the social media era.

So what is AI going to take from us?

This is a tough question, because AI is likely the broadest tech improvement we have seen in our lifetimes. By simplifying it a bit, it delivers two main things: intelligence and knowledge.

Knowledge is, obviously, about knowing things, while intelligence in this context is the ability to reason on a problem and apply logic to come up with a solution. Most work (and life) tasks require some combination of knowledge and intelligence to be done.

Now, we may nitpick about the definition of AGI, but, as of today already, we have to admit AI does a better job than people for probably >90% of such tasks. That’s the reality, folks. And that’s because, even in those rare cases where an expert still has an edge (better knowledge), or you can reason through a problem better than a frontier model (better intelligence), AI still most often wins because of how cheaper and faster it does the thing. Even when the work done by AI is just, say, 80% as good as a human expert, it may still win because it’s 80% faster and cheaper, and that’s a desirable tradeoff.

Should we be excited about this, because of the opportunities it brings, or scared, because we feel increasingly disposable? It depends.

Personally, I believe this is fine as long as we use AI in a way that puts us, as individuals, on a trajectory of personal growth, instead of regression.

Is this even possible? I believe so. Chess and Go are great examples of this: AI surpassed humans long ago, but it also allowed humans to get immensely better by watching AI play, and now both games are more popular than ever.

But what does it take to get there? Let’s talk about volition and... marathons 👇

🏃‍♂️ Mental marathoners

In his article, David says that when intelligence is plentiful, volition becomes valuable. In this case, it’s volition to stretch your own abilities and become a better version of yourself. David calls the people who are willing to go through this: mental marathoners.

I like the marathon picture for this, because marathons do not fit a practical purpose, outside that of stretching what you can achieve as a human. When it comes to health, you can (and it’s probably easier to) get in great shape without extreme 40km runs, and of course, when it comes to pure utility, a car covers that distance just fine.

So, just like marathoners run because they feel the need to run and challenge themselves, mental marathoners are supposedly those who feel a need for cognition, and are going to use AI to get better and sharper, as opposed to being passive passengers of this tech.

Now, where David and I disagree is in framing such a need as an innate trait — you either have this hunger or you don’t. I don’t think this is the case, and in general I don’t find this to be a productive way of looking at things. The productive way, instead, is by asking: what can we do to become hungrier for cognition? Or why don’t we feel like it right now?

In other words, assuming we are not either born hungry or not, what can we do to become hungrier? This is an interesting question because it is relevant at all levels:

Read more

🏦 Invest in high-potential startups

This week’s newsletter is brought to you by Alumni Ventures!

Our friends at Alumni Ventures are giving Refactoring readers early access to high-potential startup opportunities, including some of today’s most exciting AI, deep tech, quantum, and cybersecurity companies, co-invested alongside top VC firms like Andreessen Horowitz and Y Combinator.

As a Refactoring reader, you get:

• A curated deal flow of high-potential AI startups

• AV already investing alongside elite lead venture firms in these deals

• No cost to see deals

• No obligation to invest

Check out the current deals

Back to this week’s ideas!

🎲 Consider the onboarding roulette

At Graphite, every day a random employee gets their account deleted.

So the next time they open the product, they go through the full onboarding flow again: link repos, tune settings, the whole thing. Greg Foster, Graphite’s CTO, calls it the onboarding roulette: basically everyone feels onboarding roughly once a month.

They do this because onboarding is often a massive product blind spot. You test it once, maybe two years ago, and then never again, while new users keep hitting it every day.

Instead, when the team has to onboard regularly, rough edges show up fast, and get easily polished.

Greg said it got love and hate when he first introduced it, but now it’s mostly love, and the whole onboarding now only takes about three minutes.

Of course, your mileage may vary: you probably should not delete accounts if you are building a hospital system! But the principle holds: whatever your users do frequently, your team should do frequently too.

Maybe that is:

• Signing up with a fresh account once a quarter, or

• Going through your own support flow when something breaks, or

• Using your mobile app on a slow connection

The point is to design a process so the team feels the same friction users feel, regularly.

I wrote more about this in the full piece 👇

Refactoring

How Graphite Ships 🚢

It’s a weird time for engineering teams…

Read more

10 months ago · 18 likes · Luca Rossi and Greg Foster

💡 Make everyone share an AI surprise

When I spoke with Lara Hogan a couple of months ago, she introduced me to what she calls AI “Aha” meetings.

Every week, or every other week, the whole team shares something they learned about AI. A win, a failure, a surprise, or just something weird. And everybody has to share, managers included: nobody gets to skip.

In spirit, this is similar to a postmortem, just lighter and more often. People share knowledge, figure out patterns, and learn what works and doesn’t work.

Frequency matters because what someone figures out this week about an AI tool can become stale next week. If the team does not have a tight loop, people keep working from last month’s outdated instincts.

I loved this idea and we have even implemented our own version in the Refactoring Community, called the AI Club.

You can find the full conversation with Lara here 👇

📚 Weekly Readings

Finally, here are the best articles I have read this week:

🥇 Fences, not Sandboxes

13 min • by Steve Yegge

Steve spent ten weeks running ~50 agents on Fable-class models to ship his game, and... they grew a legal system! This is a very interesting write-up, as always with everything Steve writes.

🥈 Stealing Blame

3 min • by Aviv Ben-Yosef

Do you steal blame? When a project blows up, the easy move is to explain why it wasn’t your fault. Great leaders, instead, look for their share of blame. That buys you the truth: people talk more when they know someone has their back. Great short piece by Aviv.

🥉 PostgreSQL for Everything.

7 min • by Raphael Bauer

Gentle reminder that we don’t need fancy tech for most stuff. Postgres can do an awful lot these days, and most people are not aware of it.

And that’s it for today! If you are finding this newsletter valuable, subscribe to the full version!

1700+ engineers and managers have joined already, and they receive our flagship weekly long-form articles about how to ship faster and work better together! Learn more about the benefits of the paid plan here.

Make your team ship faster!

See you next week!
Luca

Lately, I have been thinking about how little of the community actually shows up in the newsletter.

The community hosts a ton of great conversations in threads, but, most of all, every month we run two live sessions, hosted by our awesome coach-in-residence Melinda Seckington, and always joined by yours truly. These sessions are:

• Mastermind — leaders bring a real problem and we work on it.

• AI Club — people show what they learned about AI that month, including the parts that did not work.

But a lot of the good stuff stays in Zoom. So, once a month, I want to start sending out a short recap. This should work on multiple levels: 1) it makes you learn useful things, 2) it gives you a feel of whether you should join the community, and 3) it gives you a heads-up of the upcoming events.

So here is the agenda for today:

• 🏝️ How to take time off right — what we discussed in the August Mastermind.

• 🪄 August AI Club — verbosity, legacy migrations, and a weirder kind of tired.

• 📅 What’s next — next events on Sep 4th and Sep 21st.

• 💬 How this works — and how to join.

Let’s dive in!

🏝️ Taking time off without the team stalling

In this month’s Mastermind, we talked about going on leave without making your inbox or your team fall apart.

One of the biggest points of discussion was about whether it’s okay or not to check in while away. Personally, I have found there is no hard-and-fast rule for that. Something you might consider is that checking Slack and replying on Slack are different things. You can glance at it, sure. But the moment you reply, you are telling everyone you are reachable. You are setting an example (for the better or worse) and presenting yourself in a way that is hard to walk back.

Whatever you choose for yourself, it’s critical that your ownership of things moves to someone else. Otherwise, if some recurring thing is still waiting on you, it’s not really time off. Voluntary, optional check-in is different from someone still relying on you to do things.

At the end of the discussion, I feel like we landed on some kind of model to make this work:

The deputy is particularly important because it should be the one person who can make the unexpected calls, and ideally the only one who is allowed to escalate to you for emergencies.

Doing all of this right is incredibly valuable, not just for your peace of mind while away, but because you may come back and figure out that some of the handovers can actually... stay that way. Many people reported it, and I confirm that from my own experience. When people are given the opportunity to fill bigger shoes, you might be surprised by how well they fare.

If you want the full writeup, you can find the community thread here (if you are in the community! If you are not, scroll to the end for instructions).

🪄 August AI Club

In this month’s AI Club, a few people shared their AI experiences, including both wins and failures. I liked three stories in particular:

• Agents that will not shut up — two small skills came up for Claude: one that leads with the next action and bans the preamble, and one that rewrites into simplified technical English (the register airplane manuals use). We all agreed that agents can be incredibly verbose, so these skills can definitely help.

• Treat the codebase as the source of truth — someone was migrating a 25-year-old Salesforce app and drowning in meeting notes that constantly would go stale mid-build. They eventually instructed agents to treat the codebase as the single source of truth, and docs, tickets, and transcripts as “best-effort context”. We also agreed that, out of all docs, the why behind a decision is the part worth writing down on purpose, because that’s what agents can’t likely reconstruct by looking at code. A couple of folks also have agents that update docs as they ship, plus a nightly pass that checks whether yesterday’s work left the docs behind.

• Faster leads to a different kind of tired — it feels that automating the work is not making people less tired. Quite the opposite. Constantly reviewing what the model hands you back can make you more tired faster, as opposed to doing creative yourself. It’s also unclear how to do better on this. Some people try to speed themselves up to keep up with the models. Some protect the human part of the thinking. Some intentionally do less.

All in all, a very interesting conversation, where we focused more on the human parts of the process, than the technology of it.

Full notes are here.

📅 What’s next

Both the Mastermind and the AI Club, like all community activities, are for paid members of Refactoring. Here are the next editions 👇 you can RSVP on the event page to help us size the room and plan accordingly:

• 🪄 AI Club • Fri 4 Sep, 14:00 CEST — five-minute shares, questions, and a one-sentence fireside round at the end.

• 🧠 Mastermind • Mon 21 Sep, 14:00 CEST — this month we’ll talk about developer productivity and what “good” looks like when measuring output is getting less useful.

Bring your questions! These sessions work because of the members, not because someone gives a talk.

💬 How this works

The Refactoring Community is a private space for paid subscribers of Refactoring: it hosts async threads, Q&A, and monthly live rooms like these.

It is home to more than 1500 engineers and managers, and it’s my favorite place on the internet!

To join it, subscribe to the full version of Refactoring!

Join the full Refactoring 🙌

That’s it for today! See you on the 4th, or the 21st, or both!

I wish you a great weekend 👋
Luca

Every week I write an article to 170K+ engineers about how make good software together, and interview a world-class tech leader. I also build and maintain Tolaria in the open, publishing my workflows and learnings here.

Recently I have started researching and publishing more case studies about how real companies are doing product development with AI. That’s because I feel that many people are like: “enough with the thought leadership!” — and that includes me!

This time I am doing a deep dive into the AI workflows of the team at Unblocked, which I have known for a long time. I sat down with Dennis and Brandon, who walked me through some of their most interesting processes.

The team at Unblocked is especially relevant because they got skin in the game. They ship a product that turns the team’s code, docs, tickets, and conversations, into actionable context for agents. So they dogfood all of this every day. That’s a strong incentive to work as close as possible to the frontier, rather than doing things “as we have always done” (cit.)

I also love that a lot of what they do is public: they maintain an open-source cookbook with several recipes (skills), which you can fork and copy.

So here is the agenda for today:

• 🎚️ Dynamic autonomy — scoring runs for risk and confidence, and behaving accordingly.

• 🧪 Case study #1: vibe check — one skill, two triggers, three exits.

• 🚨 Case study #2: AI-assisted incidents — humans drive, cleanup can go further

• 📘 Creating a cookbook around it — interactive vs headless, and how the score moves

• 📊 Delivery numbers — what the metrics say so far.

Let’s dive in!

Disclaimer: I am a fan of what Unblocked is building and I am grateful to have it as a partner on this piece. You should check it out!

Learn more about Unblocked

However, I will only write my unbiased opinion about the practices and tools covered, Unblocked included.

🎚️ Dynamic autonomy

One of the most popular ideas about AI workflows is the so-called autonomy slider — that is, for any given process you should design how much leeway to give to the AI, vs how much humans should intervene (and when, and where).

Autonomy is, of course, a spectrum. If perfect autonomy looks like a loop, we talk about humans-in-the-loop to express that some parts are designed for humans to interrupt the AI and do things.

One of the most interesting ideas I got from Dennis and Brandon is about making this autonomy slider dynamic: a lot of their workflows are designed to have runs scored for risk and confidence. Such a score guides how much autonomy the agent gets at that time, and where a human has to step in.

This is a flexible concept that can be applied to many different workflows: shipping code, processing bug reports, intercepting incidents, and more.

In fact, most of these include extremely diverse situations, so it’s tricky to hard-code human gates into the workflows themselves. You constantly risk having either:

• Safe work unnecessarily waiting in queue, or

• Risky work getting through when it should have been stopped and inspected first.

To make an example, let’s say a bug report spawns from a customer success chat, or from Datadog telemetry. The AI evaluates two questions:

• How confident is the AI about this job being real, and being able to do it?

• How risky is this change?

This leads to a score that drives the next action:

• 🔴 High risk, low confidence — just write a ticket and hand it to a human.

• 🟡 In the middle, roughly 50 to 70% — Slack a person who can answer more questions about the work itself, with the context already attached.

• 🟢 Low risk, high confidence — write the code directly and open a PR.

The same idea is used on PRs. A change that is evaluated as low-risk can go through without a human review, while a high-risk one puts a human reviewer in the loop.

So, let’s get more into the details of this.

🧪 Case study #1: vibe check

The first process Brandon walked me through was something they call vibe check.

Read more