How To Pick The Most Cost Effective Model For Your Task

It's no secret that LLM tokens are extremely expensive. Companies are spending like hundreds of millions of dollars, billions of dollars on these tokens and not really seeing a good return out of them. And all they're really doing is just burning their money away, right? Meta, the company behind Facebook and Instagram, huge addiction of mine, is spending hundreds of millions of dollars every year. I have actually read an actual like news article which claims that they've spent $900 million in just clod tokens, just paying $900 million to Anthropic, no questions asked. Uber, I made an entire video on Uber, which reportedly just ran through its entire like $3.4 billion of budget for the entire year of 2026 in just four months, spending the entire budget on AI tokens. And unless Gemini hallucinated all of this data and basically ruined my entire video, the fact is most companies are spending a really large amount of money on AI tokens and they're using it very inefficiently, right? Now, there are different ways to improve your usage of AI, like, you know, writing better prompts, providing good context and all that. We're not going to talk about all those things. They are important, but they are not the topic of discussion for this video, because in this video, I want to talk about how to choose a cost-effective model for your use cases, right? The problem with a lot of companies is they're using really expensive, really frontier models, and they're using it for really basic things. Like I actually covered this in my Uber video as well. You can check that out on my channel. Because these companies are mandating AI usage so much, every employee has to like use like a certain amount of LLM tokens. And if they don't use it, they basically get penalized for it. So you know what these employees are doing to, you know, drive up their LLM token use. Yeah, they're using these frontier models that are basically AGI to summarize emails and draft PRs and shit. Stuff like that, which can easily be done by a very small, very cheap model, like even something like Gemma 4 by Google, honestly or like a Quen 27B something like that right Anyway point is a lot of you folks are using models that are really intelligent but you not using them for intelligent tasks And so what I want to do in this video is give you a very, very simple approach to picking cost-effective models. You're going to look at this approach and you're going to think, what the fuck? This is so fucking obvious. Why is this kid teaching me something that I could have done on my own? Yeah, if you could have done it on your own, you would not be watching this video. So yeah. Now, without further ado let's get into it so this is the entire process and you need to follow this for every single task that you're automating with some sort of an llm here's what we're going to do we're going to pick a pencil so i can draw and explain things as we go along the very first thing you need to do is you need to obviously have a task to automate right you should have a task in mind and you should have some sort of a prompt that automates that task for you right the prompt that you're going to feed into the llm or the ai agent or whatever what you're going to do is take that prompt take it to the most frontier intelligent model that you have in your stack no matter what models you're using whether that's from openai or entropic or some combination of them whatever right feed that prompt and that task into the most intelligent model you have in your stack and score the outputs you get from it right literally just take a score from like let's say zero to a hundred and just score the model based on that right run that same prompt like i don't know 5 10 15 times see the outputs that you get and by the way as you keep doing this more and more you'll also be able to find ways to improve the prompt that you're using make it more specific give the ai better high for high quality examples explain any edge cases that it may encounter just give it better instructions and a better more refined prompt so that the outputs improve as a result of that as well but yeah you want to run this like 10 or 15 times you want to score every single output you get and then what you want to do is you want to go over to a cheaper model so remember we started off with a frontier model right now we move to a cheaper model a smaller model than that and we do the exact same thing we run the prompt the same prompt the same task on a much cheaper model we run this like 10 or 15 times and we score this out of 100 as well and then what we're going going to do is we going to analyze the output quality we going to see where the output quality starts to degrade right so as you keep repeating this process because you will need to repeat this like a couple times at least what's going to end up happening is since you're using cheaper and cheaper smaller and smaller smaller models at every single iteration at some point the output quality is going to start to like drop right when you're using a frontier model in the very first step the output quality might be like 98 97 99 100 out of 100 right that's really good but this is a very intelligent frontier model right when you move to a cheaper model than that right the output quality might you know stay at 97 98 maybe it'll drop to like 94 95 93 you basically just keep moving down and down to cheaper and smaller models until the output quality degrades to a point that is unacceptable to you right let's say you're looking for 90 reliability or something or like output scores that are 90 or above right as soon as a cheaper model starts to fall below that in most of the you know iterations that you run it on and it's giving you scores like i don't know 85 83 82 that is much lower than what we started with right and so the output quality is degrading a lot and so at that point you basically want to switch to a more intelligent model just slightly more intelligent than what we started with. And that will be the model that we will use for this particular task, right? And if the output quality is good enough, then we just keep using the cheaper model. Or we can go one step further and try using an even cheaper model than that and see if that one works for our tasks and it produces good outputs as well. And what ends up happening in the very end is we end up selecting the best model for our use cases, one that can actually produce really high quality outputs as well, because we test that with our scores on every single iteration, and has the cheapest cost as well, because we keep using cheaper and cheaper models until we find the one that continues to produce high quality output, while still being as cheap as humanly possible, right? Humanly possible, that makes no sense. So yeah, that is the entire process Pick a task you want to automate write the prompt score the outputs with a frontier model run it like five or 10 times get the average score then try a cheaper model run the exact same process, run it five or 10 times, take the average score, then keep going down and down to cheaper and smaller models until you find a point where the output quality falls below your requirement for this particular task. Once you get to that point, either select that cheap model or switch to a slightly more intelligent model and use that model. Otherwise, just keep trying cheaper models until you find one that produces high quality output as per your requirements and is still extremely cheap to use. And if you just do this, you'll find that most business tasks can be automated by really small, cheap models as well. And you can save the frontier expensive models for your highest reasoning required tasks, right? So yeah, hope that helps. Let's end this before it gets way too long. By the way, I guess I should probably show you some sort of a example of this whole thing as well. Okay, so this is literally all you need to do. And I guess I should have explained this in the beginning of the video. But yeah, this is all you do. Like you literally just create a spreadsheet like this, that is supposed to be score, not whatever I wrote. You put the task over here that you're trying to do, like, I don't know, make a $1 million business with no mistakes. You put the prompt that you're using. I'm just going to put some example stuff over here. And let's say I'm using Cloud Sonnet. And we will literally run this like five or 10 times. And, you know, let's say we get scores like 97, 95, 98, 99, 97 again, right? We just take their average. And that is the score that we get for this particular task with this particular prompt with this particular model. Then we just move over to a cheaper model and a cheaper model and a cheaper model until we find the one that produces really good output scores, but is also really cheap for us to use. So yeah, that is the entire process. Hope that helps. Follow this. It's a lot of manual work for sure, but it'll drive costs down by a lot. And you can continue to use all these models without necessarily bankrupting your company. Or if you're using this personally, bankrupting your own wallet, right? Yeah, hope that helps. See ya.