Hi Andrea! Can you be more specific about manual tests? Do you mean manually testing new features before they get released, or old ones to catch regressions, or both?
Hi Andrea, sorry for the suuper slow reply! Things got packed at the end of the year.
End-to-end tests (manual or not) to me belong *both* to review time (in a dev or preview environment) and to deploy time (straight in prod).
You test things first before you merge the branch (that's where automation should kick in, but you can also do a manual check if you don't have automated tests in place) and after the feature is in prod.
You can also do a canary release to *disable* the feature in prod for regular users until you have done proper testing in prod.
Also, manual testing of critical paths is perfectly fine to me in a lot of situations. Automating this stuff is hard — making it worthwhile depends on a lot of factors. I also wrote more about this here: https://refactoring.fm/p/how-to-do-qa-in-2024
It is mostly productivity! Fixing tech debt is mostly about changing things under the hood, so it's not improvements (on the product side) but rather stuff that makes your engineering team go faster!
Awesome newsletter issue Luca and thanks for the mention!
Thank you for your insights, Nicola!
Metrics can be a great leading indicator, showing you whether things are improving or not
Absolutely! Then it's on you to learn more through good conversations.
Hi Luca, nice reading, thanks so much.
A question about the cycle: where would you place manual tests by a business/PM's team?
In your pic, and in general in the main picture that I can understand by reading many of your articles, there is no mention to the manual tests...
Is it something that in your factories, where maybe you are so much... optimized..., you don't need to perform??
Thanks
Hi Andrea! Can you be more specific about manual tests? Do you mean manually testing new features before they get released, or old ones to catch regressions, or both?
Hi Luca, both!
Hi Andrea, sorry for the suuper slow reply! Things got packed at the end of the year.
End-to-end tests (manual or not) to me belong *both* to review time (in a dev or preview environment) and to deploy time (straight in prod).
You test things first before you merge the branch (that's where automation should kick in, but you can also do a manual check if you don't have automated tests in place) and after the feature is in prod.
You can also do a canary release to *disable* the feature in prod for regular users until you have done proper testing in prod.
Also, manual testing of critical paths is perfectly fine to me in a lot of situations. Automating this stuff is hard — making it worthwhile depends on a lot of factors. I also wrote more about this here: https://refactoring.fm/p/how-to-do-qa-in-2024
Where does Tech Debt fall in the four categories according to you? in Improvements or in Productivity?
It is mostly productivity! Fixing tech debt is mostly about changing things under the hood, so it's not improvements (on the product side) but rather stuff that makes your engineering team go faster!
Thanks for clarifying. Improvements = Product Enhancements. Tech Debt & (internal) Productivity go hand in hand.
Awesome post Luca. More than anything, super concrete and actionable.
Thank you Sebastian! 🙏
Awesome work! I'm thrilled to see that we're making progress in the right direction with the increased use of data-driven platforms.
Absolutely!
Stellar edition Luca 💡 highly valuable and a great place to start for anyone looking to grow their engineering metrics practice
Thank you Conor! It means a lot coming from you 🙏
Awesome
Very solid post, thank you for sharing!! We also post about engineering and metrics, you can check one of our articles here https://www.metridev.com/metrics/engineering-productivity-what-is-it/
nice article