LLMs, RAG, and AI agents - the simplest explanation you'll ever find

Before we start this video, I feel it is important to tell you that I haven't slept all night. I don't know why you need to know that. I just felt it was relevant to share before we actually begin. We're going to talk about LLMs, RAG, and AI agents. And I'm going to give you the simplest and shortest explanation of all three of these things. Let's begin. So first of all, LLMs. LLMs are basically large language models. These are simply AI models that are trained on human language and they can basically just produce language right you can ask them anything and they can give you an answer in natural human language so when you look at stuff like chat gpd or cloud or whatever these are simply llms you can give them a prompt and they will give you some sort of output in natural human language and they don't have any capability beyond that and the thing with llms is that they are trained on a huge amount of data they're trained on basically the entire internet, right? Which is a ton of training data. But the problem is that this is not something that's specific to your use case. If you want to train an LLM on your specific data and you want to make sure that the LLM only responds from your data set, then you need to use something called RAG. And RAG is basically just the power of LLMs augmented with real-time data, right So think something like maybe an LLM trained on your product documentation Let say this is a document that not going to be something that the llm is trained on by default right and so in order to train the llm on this particular data set right your own particular unique data set you're going to want to use something called drag this basically stands for retrieval augmented generation let me explain what all of those things mean so the first step is retrieval right and during retrieval you're going to retrieve relevant data from your data set that is relevant to the user's query right once you do that you're going to pass that into the augmentation stage and you're going to pass this pass the relevant data that you just got into the llms prompt and once you've done that you're going to move on to the generation stage where you're going to give the llm all of that new context all the new real-time data that you just got and ask it to generate a response to the user's query based on all of that real-time information. And on a very high level, that's basically how RAG works. So if you want a question answering machine like ChatGPT or something that is trained on your particular data set, you need to use RAG. Now, lastly, AI agents. This is the most fascinating part of the entire video and arguably the most fascinating piece of technology in all three of these An AI agent is basically when you have an LLM that going to be connected to a big bunch of tools And these tools are just functions right So think just regular functions inside your code right that what a tool is in this context and what these basically do is they enhance the llm's functionality by just giving it more tools in its arsenal that it can use to solve your problems for you right so let's say for instance we could have a files tool that could allow the llm to read and write files and folders on your computer right we could pass in functions like read file and then it would pass in a file path over here or something or we could have something like write file the file path and the file content we could have something like that and again these are just basic functions that you write in normal programs but you basically give the llm access to these functions so that it can call it anytime it wants you could have a search tool that's going to search google for any query that the llm wants to get real-time information on you could have some custom api i don't know for any integration or something that you need to do and you want the llm to have access to it you can have any of these things over here right and now the way this is going to work is you have an llm that's basically the brain of the ai agent and And then you have a bunch of tools connected to it that it can call as needed, right? So we're going to have a little user over here. And that user is going to pass in some sort of a query to the LLM right Like for instance let say the query is something like write an SEO optimized article on AI agents right Let say we have a query like that So the LLM is going to go over to the search tool and it's going to find out all of the latest information on AI agents that it can. It's going to read all the articles that you return over here in this function. It's going to use the files tool to make sure that there isn't any article like this that already exists or something. It might use this third party API, depending on whatever it is, to get some more information or whatever, or do some sort of task, right? And then it's going to use the files tool again to write the file to your system. And then it's going to just notify you that, hey, this is done. You can check it out, right? And once all of this work is done, it's going to respond back to the user over here. So essentially that's how AI agents work. You give them a task to do and they use the tools that are within their grasp to complete that task for you and then they're just going to notify you that hey this is done. Obviously you can expand this functionality to include memory, past conversation history and other kinds of features right but that gets pretty advanced. At a fundamental level this is what AI agents are. They're an LLM that has access to a bunch of tools and the LLM is going to call these tools in a loop until the task that you give it is completely done. So that's a quick explanation of LLMs, RAG, and AI agents and how each of these things work.