How to Integrate LLMs into your Tech Stack 🔌
How to Integrate LLMs into your Tech Stack 🔌
Exploring prompt engineering, fine-tuning, RAG systems, and more.
A few weeks ago I received this question via email, by Daniel:
What are the best approaches to integrate LLMs into workflows? What are the alternatives to using ChatGPT / GPT API out of the box? I would love an article where you go through the various options.
Over the last year I have received many similar questions about AI. I have always kind of restrained myself from writing such an article because:
Things were moving too fast — there was a high chance that anything I could write would become legacy in just a few months.
Success stories were few — honestly, despite all the hype, it has long been unclear to me what the best use cases of AI are.
Fast forward to today, I feel the dust is settling and we have a better grasp of what AI is good / not good at. Tech progress, too, has slowed down a bit (maybe too much?) and some patterns have emerged.
So, this piece covers the best approaches for embedding Large Language Models into your workflows and your tech stack:
🎯 Use cases — what should you use LLMs for?
✏️ Prompt engineering — the most popular approaches and when to use them.
🔧 Fine-tuning — for when prompt engineering is not enough.
🔍 Retrieval systems — how to integrate LLMs with external information.
🖥️ Self-hosting — should you host an open-source model or use APIs?
This subject is quite technical, so for writing about it I asked for help to two of the most knowledgeable people that I know of:
🙋♂️ Andrea Rossi — my brother! PhD in Machine Learning and ML Engineer at Google. Andrea has extensive first-hand experience on the technical side of AI, by implementing models himself and working on all the parts of the pipeline.
🙋♂️ Disheng Qiu — VP of Engineering at Translated. When it comes to AI, Translated does it all: it has an internal research team, creates its own models from scratch, and embeds AI into all their products. They also collaborate with some of the biggest tech companies out there, like Nvidia, Airbnb, and Amazon.
Let’s dive in!