Optimizing AI Generated Codebases - How I Do It

Okay guys, we need to talk about AI generated code bases and how to optimize their performance because let's be honest, a lot of you are just wipe coders who are just generating absolute dog shit and creating slop that does not run fast enough for actual end users to, you know, use. And so I'm a software engineer. I've been building production apps for clients for the last, I don't know, five, six years, you know, apps that get actual users. And so I know how to make an app run fast. And I've recently applied the same concepts to an AI generated code base that I built for myself. And I want to walk you through some of the lessons, I guess, that I learned from doing that. And my goal with this video is to help you write faster apps, right? To basically just improve the performance of your app. Because let's be honest, you're using AI to generate all the code anyway. You may as well use it to make your app run faster, right? Now, before we even talk about how to make an app run faster, I want to talk about why we should do it in the first place. The problem is, as we move up each abstraction layer, we lose performance and speed gains, right? If you think about something like, where did computing start, right? With punch cards and, you know, binary zeros and ones and shit. And then we added the C programming language, which at the time was considered a very high level programming language, but it introduced an abstraction layer, right? Sure, it made us, you know, it unlocked new possibilities because now more people could program and build stuff with computers, but it was an abstraction layer, right? We lost a bunch of performance because we couldn't fiddle around with binary zeros and digits, and we couldn't get the absolute best performance out of our CPUs. That's completely fine because our computers are so much faster now, right? And then we kept adding more and more abstractions on top, right? Languages like Python and JavaScript, they are really simple, really easy for beginners to learn and to build cool apps with them. But they're a lot slower than a language like C or Rust, right? Or hell, even Go, which is honestly just as simple and just as easy to use as Python. But yeah, each time you move up the abstraction layer, you're losing performance and you're losing speed, but you're gaining so much more because you're able to, you know, learn those tools more easily and use them to build cool shit. And AI is just the exact same thing. AI is just a really big abstraction layer, right? You're using AI to write code. And it's a lot easier because instead of having to learn the syntax of a programming language, all you have to do is just tell the AI to go generate some code based on some, you know, normal everyday English that you use to describe your task. So obviously, the possibilities that it unlocks for you are amazing. but again, we lose performance and speed each time we move up above each abstraction layer. Now, I'm not saying that you need to stop using AI code because frankly, we're way past that. If anyone is not using AI generated code, you're completely left behind. If anyone is still typing code by hand, you are just completely going to be left behind. You're already behind. And so the key is not to, you know, go down the abstraction layer and use some old generation tools. The key is to learn how to optimize AI generated code and to solve performance bottlenecks. And AI is not going to do this for you by default, unless you ask it the right way, unless you give it the right prompts. I'm going to show you how to do that. But the very first thing I want you to do is to go and learn about common performance issues in apps and how to solve them. And you can literally just Google this and you'll get some really great answers or you can just go over to ChatGPT, tell it to teach you about common performance issues that most applications suffer from and it's going to give you a big bunch of lessons and topics that you can go read about more and go learn about them The reason I want you to learn about all these things is because this is going to help you write better prompts right You can just you know tell some coding agent hey go make this app faster and expect it to produce a really good result, right? You need to learn how to identify performance bottlenecks and how to solve them. That means you need to understand what kind of common issues an application might go through that might cause it to slow down. You then need to figure out how to identify those same issues in your own apps. And then you need to figure out why they are occurring in your app and how you can fix them, what kind of specific solution you're going to implement in order to fix those issues. And so you need to go learn about these things. You can learn about things like, just for starters, things like load balancing, which is basically where instead of having, let's say, one machine or one server to handle your entire application, you can distribute it across like 10 different machines or 10 different servers. You could have a website that's hosted on like 10 different machines and each of them serve a little chunk of your traffic and just distribute the workload, right? That's load balancing. That's one of the performance techniques that you can learn, right? Another one is parallelization. Now, this is really cool. And what it basically means is that instead of using, let's say, one CPU thread or one CPU core to execute a task, you use multiple of them or even all of them, right? And this is really useful because most languages by default are not going to be parallelized. Most of the programs you write, by default, they're only going to use one CPU core. And you know that you probably have like 4, 8, 16, I don't know how many cores you have in your CPU, right? If you use all of them, you can execute a task way faster than if you were to use just one of them, right? so this is another really cool performance technique for you to learn right along with that go learn about efficient algorithms right an algorithm as you probably know from your computer science class is that it's basically just a bunch of steps right you want to solve a certain problem you follow a bunch of steps to solve it those steps could have unnecessary steps those steps could do some unnecessary really slow things and that's basically a bad algorithm right and so you want to learn about some really common algorithms, you know, people talk about data structures and algorithms. That's basically what this is. You want to learn data structures and algorithms and where a certain data structure or an algorithm fits best, what kind of problems it solves, why people use it, and in what situations people use it. A lot of people shit on data structures and algorithms, but this is exactly the kind of stuff that is going to help you write faster and more performant apps. As a very concrete example, I'm going to show you, yeah. So for those of you who think that data structures and algorithms are not useful, consider this. You probably know or maybe you don't know about the big O notation. So I'm just going to explain it very briefly over here. Let's say you've got a list of items, right? Let's say you've got like 100 items in a list and you want to find one particular item in that list. What are you going to do you'll use whatever find or index function you have in your programming language but how long is that lookup going to take right you've got a hundred items in your in your list you want to find one particular item how long is that going to take it's going to take o of n what that means is it's going to take you a hundred a hundred cpu cycles to find that you know particular item why because the program has to loop through the entire list check each item one by one until it finds the item you're looking for now that item could be at the very start of the list or it could be at the very end of the list but it's going to take you like a hundred c a hundred in cpu time to find that item because you have you have a hundred items in that list right as opposed to if you were to use something like a hash map hash maps are just like objects in javascript or dictionaries in python like key value pairs it the exact same thing if you have a hash map of let say a hundred items and you want to look up one item how long is that going to take Just one CPU time Literally just one Lookups in hash maps are instant Google that, okay? Google the, you know, bigger notation or the time complexity for hash maps. You're going to find that the lookups in a hash map are literally instant. You could have 10,000 items in this hash map and the lookup for a single item, for a single key, is literally going to be instant. It's going to take one CPU time every single time. If you were to have a list with 10,000 items, yeah, 10,000 lookups. So that's just a very brief, very concrete example of why learning data structures and algorithms can be very useful, right? Because if you need to look up something in a very large data set, would you rather that data set be a list or an array? Or it should be a hash map, right? Now, the code base that I made more performant is basically my Phantom Editor AI content and video editing workflow. I've talked about it like a thousand times on this channel, so I'm not going to go too deep into it. But it's basically a big collection of Python scripts that automate different tasks for me in content creation and in video editing and all that stuff. right and the way i optimized it is first of all i just noticed that it was running really slow video encoding and other tasks were taking way too long and so what i did at first was literally just ask my coding agent hey what can we do to optimize this code base what can we do to make these steps run more you know efficiently run faster all that stuff right i just referenced the right files to it and i asked it to analyze those files and identify key performance improvements that we we can make, right? And that is always the first step when you're trying to use AI to do these kinds of things for you, right? Give it the right files that it's supposed to read, all the code files, all the scripts, and then just let it do its thing, right? Let it research and analyze and come up with some ideas that you can then start to fiddle around with, right? Now, the AI took some time, it took some time, it read a bunch of files, and then it came up with a bunch of ideas, right? I went through each and every single one of those ideas, and a lot of them were really good, and we implemented them and it just boosted the performance of this whole setup way more right i'm not going to go through this entire document i'm just going to tell you a bunch of really specific things that we did just to give you a bunch of ideas on what you could do as well right so the first thing we did was there were a bunch of steps in the entire processing of a video that were being repeated right and each of those steps was encoding a video file every single time even though we didn't actually use that right so for example webcam mask generation right the you know little bottom rectangle you see where you say you're seeing my face right that's the webcam mask right in order to place it over there i was using an ffmpeg script and that script would would basically encode a video file with that webcam mask right there, right? That's one video encoding. Right after that's done, we would perform silence trimming to eliminate all of the silent bits from the video, right? That's another video encoding. And we would just do this like three or four times, even though it is completely unnecessary and we are not using those intermediate files at all, right? And so the first major optimization that we did was we basically just eliminated all of those video encoding tasks. We just did the intermediate tasks like placing the webcam mask at the right position, applying all the silence trimming and all that. And we did all of that without actually encoding the video. And we moved the entire video encoding task to the very end. So we only encode the video once while doing all of these different tasks together, right? And that made the entire code base like five times faster. Along with that, we use a little neural network or something some AI model I don know called DeepFilter to perform noise reduction across the entire video file right the problem with this is that it literally only uses one cpu core to execute right we talked about parallelization over here right this is where it actually applies so deep filter only runs on one cpu core and so all the other cpu cores on my machine were just completely unused which means this is taking like one what is it this is taking four times as much time as it should have been taking. How did I fix that? I literally broke all of the audio into 60 second segments, right, using FFmpeg, and then I would have Deep Filter run on those 60 second segments on each of my CPU cores. And once I implemented this, I could now use all of my CPU cores to perform noise reduction like four times faster because I've got four CPU cores over here in this machine. I can now do this thing four times faster as opposed to what I was doing before, which was only using one CPU core. Another really cool optimization we did was reordering the pipeline steps, right? And so what we did was previously we were doing audio processing at the very first step, right? And now what we do is we perform silence trimming first, which cuts down the video duration by like, I don't know, 20, 30, 40 percent, because a lot of times I'm just silent during the video. We cut down that entire video and then we perform audio processing and noise reduction, which reduces the total audio processing duration by a lot, right? And so it just makes the audio processing run way faster. Also, I use Grok and the OpenAI Whisper models to transcribe my videos. And Grok has some limits on the request size and whatnot and the file size and stuff. And for really long videos, this can turn into like multiple requests. The way I, you know, optimize this is instead of running each request sequentially, like one by one, I just run them in like multiple workers, multiple threads simultaneously. And this again, falls into the parallelization and concurrency, you know, optimization technique, where instead of doing things one by one, you're basically just doing them all together. And this just saves you a lot of time as well because sending these requests doesn't cost me anything. It's an IO bound task, right? It's an input output bound tasks. My CPU doesn't need to process anything. It just sends the request and then all the processing happens on Grok's servers, right? So I lose nothing by sending like 10 requests altogether to their servers, right? But I get all of the outputs of the responses way faster. And finally, I standardize the parameters, Things like using as many threads as FFmpeg needs to do the video encoding task as fast as it possibly can. Things like using a consistent preset. Things like literally stream copying the audio of the file once it's done processing, as opposed to re-encoding the audio every single time. So I'm sure many of those steps don't make a lot of sense because they're unique to my project. The point is, you need to learn about these different performance techniques that you can use, and then you need to go apply them to your project. So just follow the steps that I laid out over here. Go and learn about common performance problems that you can then start to identify and then solve in your own personal apps. Pick an app that you've built that is operating really slowly. Ask an AI agent to analyze different files or different features or functions in that code base that are running very, very slowly. Ask it to come up with performance improvements that you can make to that feature and then go research about those solutions, right? Go learn more about those solutions. Heck, even ask your AI agent to explain those solutions to you. And as you do this more and more, you're going to understand how to actually make your apps run faster, right? This is how you write better and faster apps in the AI coding age where all of your code is just slop produced by Fable 5.5 or whatever model we have nowadays. This is how it's done, all right? So hope that was helpful. See you in the next one.