Auto-generated transcript AI has gotten really good at writing code but still not very good at building good software and that's the major reason why you see most of the good and half decent AI apps out there are built by people who already have some previous experience building software maybe they're software engineers they're web developers whatever they are right but they have some previous experience building software and that's because there's a lot of different skill sets within software development besides just writing more and more code that are crucial when you're trying to build a good app and in this video i'm going to go through all of them for you so if you're some vibe coder or you're a junior developer at some company or heck you just want to level up your software development skills in the age of ai coding this video is going to be extremely helpful for you because i'm going to tell you exactly what skills are valuable now that ai has basically started writing all the code for you so with all that being said let's head over to my screen where I can teach you exactly what you need to know so here's what we're gonna do I'm going to explain the following 10 topics to you right we're gonna go discuss we're gonna go through databases data structures algorithms system design and everything else you see on this list right I'm not gonna teach you everything there is to know about this topic because that will take like 500 more videos of this kind what i am going to do is give you an overview of this topic what it is how we use it and what kind of problems it solves and why it's still important today and i'll give you some sort of a roadmap or some learning resources so that you can go out there and learn these things for yourself right now as you can see i built this entire little whiteboard presentation for you so let's start with databases so databases one of the most important topics you need to learn as a developer today despite ai writing all of your code for you right you need to know how databases work you need to specifically know about relational versus non-relational databases and when you should be using which one right in most cases you would be using a relational database like postgres or mysql or something like that but there are many cases where something like a document database like mongodb or something else works really well and and you need to know when to use which. You also need to know about vector databases. Now, vector databases are really different from traditional databases in that they don't necessarily store data the way you think about it. They store vectors, as in really large numbers, basically. And without going into too much detail about them over here, they're basically used a lot in AI, in LLMs, in RAG, in all sorts of different AI workflows and models, right? So this is extremely important for you to learn. Some examples of vector databases are ChromaDB, Pinecone. There's also a Postgres extension called PGVector, I believe. And you can use that as well. But yeah, learn vector databases. Along with that, learn about query optimization. Because AI can write a bunch of SQL queries for you, but you have no way to tell whether they're good queries or not. You also don't have any way to optimize them if you need to, right? One query could take 10 whole seconds to finish executing, while another more efficient query could take something like, I don't know, 50 milliseconds, right? That's a huge difference. And so learn about query optimization. I'm not telling you to learn SQL syntax or whatever. I'm saying learn how to optimize queries and write better queries. Learn about things like indexing, about joins, left outer joins, right outer joins and how they work and what they do to the database columns and everything. learn how to analyze the performance of every single SQL query you run. I know that Postgres has its own internal tools that can help you to analyze the performance of any SQL query. I'm pretty sure other database systems have their own equivalent of that as well. So go learn how to use them and how to write good queries with them. Now, lastly, learn about scaling because you can build toy projects all you want, but at some point you need to get them out to real users, right and when i say real users i don't mean your family or friends i mean hundreds or thousands or tens of thousands of real users right and when you get to a scale like that you need to know how things like read replicas work how things like sharding works right different optimization strategies like these that help you to scale your app to hundreds of thousands or even millions of users right so when you're building real apps that ship to thousands of users out there in production you need to know about these things as well. Now, with databases covered, let's talk about data structures and algorithms. So the reason you need to know all these different data structures and algorithms is mainly so you can write more efficient and better code, right? Like an algorithm is basically just a series of steps, right? It's not something complicated. But the main value there is that you need to learn to write better algorithms, you need to learn to use better data structure so that you can increase the performance of your application. For example, if you've got some API, you've got a really horribly optimized endpoint over here that takes 10 whole seconds to finish executing. No human being is going to sit through that, right? You're going to lose users obviously Whereas if you had a really highly optimized endpoint with good queries with fast performance that finishes running in literally just 200 milliseconds which is like 0 seconds if my math is correct you going to retain a lot more users Not only that, but your cloud infrastructure costs are going to be lower because you're not using as much compute. You're going to finish your requests faster, all sorts of good things, right? And the difference between a really poorly written endpoint and a very fast performing endpoint can really just be the data structures and the algorithms you use to do all the operations in that endpoint. And as a very simple but really practical example, if you've done map lookups in programming, right, like hash maps, dictionaries, basically a dictionary in Python, right, that's called a map. If you've done those before, you'll know they're extremely fast, even if you have thousands or tens of thousands of items inside your dictionary or hash map, right? why are they so fast because they have an O of one lookup time as in no matter how large that map gets the time to find an item inside it is still constant whereas if you try to look up some items in an array that has a time complexity of O of n which basically means the larger that array gets the more time is going to take to find items in the array so super simple example of using good data structures. If you've got a really large, you know, list or whatever, and you need to look up something in it, that's going to take a really long time. If you instead had a map, like a hash map of that entire list, looking up something in that map would take like a millisecond probably. Now, where do you go to learn these things? Well, there's a roadmap to learn data structures and algorithms over this link, which you can open up and it'll just give you a big bunch of topics to learn and you can just get started there now let's talk about systems design system design is basically how you design and architect your entire application right it's not necessarily the code you write it's more how you structure everything how you design everything to be to satisfy these four main criteria right so scalability what do i mean by scalability are you designing an application that can easily be scaled to thousands or tens of thousands of users because if you're not you need to fix that maintainability is your application going to be easy to update and refactor later on or is it some giant ai generated mess of code that nobody is ever going to be able to touch right because these things matter availability is your system designed to be available or is it just going to crash 20 times a day like github does nowadays like when you see things like 99.9 percent uptime on websites this is what they're talking about they're talking about availability and availability isn't strictly limited to your web servers being online it can also mean different individual features inside your application do you let's say a certain feature and your app stops functioning properly right is that going to affect the rest of your application is that going to take down the rest of your application or can it's can it gracefully handle its failures go down individually if it has to but let the rest of your application keep running right because those things matter as well and lastly let's talk about reliability is your system reliable can users make changes to it and expect them to stay there right if you look at github for example nowadays it's malfunctioning really horribly to the point where you can't even trust a pr that is merged to actually retain those changes like you could you could merge a pr on on GitHub nowadays, and you wouldn't even be sure if it actually merged those changes or not. Like that's the kind of bugs that GitHub is now going through. So obviously you don't want that. So can your system or your app be completely reliable for users? Can I make some changes in it and expect to see those changes live permanently without some sort of malfunction? Because like, look, a lot of people go really deep into UI and UX, right? Designing the right user experience for users, let me tell you something that's going to save you a heck of a lot of trouble. The best user experience is when your app does what it's fucking supposed to do, all right? So have some reliability. Make sure that your app does what it says it can do. Now, let's talk about API design and API security. I'm not going to go too deep into these. Basically, when it comes to API design, it really just means how you're designing your endpoints, how you're structuring your URLs and different URL formats and everything, right? And API security is kind of related to that. You need to make sure your API is completely secure. Users can't, you know, hack it in some way. Like super simple example, you need to have rate limits so users can, for example, try out like 10,000 different email and password combinations on your login endpoints, right? Like some brute force hacking program. You need to have all sorts of different security practices over here, right? I'm not going to go too deep into these things right now, but I am going to link you to two amazing resources from roadmap.sh to learn about API design and API security. And as long as you're doing the things that they mentioned over here, you're going to be ahead of 99.9% of developers when it comes to API design and API security So now let talk about CI and cd all right ci cd basically stands for continuous integration and continuous deployment and over here i basically talking about stuff like github actions or some other ci cd pipeline that you're using right the reason you need to use these things is so you can automatically run linters and checks and different you know verification tools for your code base and also so you can run your tests your entire test suite and make sure that your app actually works and you didn't break anything by making some changes right that's the continuous integration part and once you're done with that you can deploy to whatever environment you want whether that's a beta environment that's a staging environment or it's a production environment right you can automatically deploy to all these different environments and if you have any other tasks that you need to do before deployment or after deployment right you can configure some pre-deployment and post-deployment tasks to run automatically and that works really well as well and again the reason you need these things is because like you can do all these things manually right but you would have to do them again and again manually every single time you make a change to your code base and deploy that right so instead of that why not just have a simple pipeline that does it for you right so that's why you need to know these things go ahead and learn them as well now let's talk about caching now when it comes to caching just go ahead and learn redis right like redis is basically the industry standard when it comes to caching so just go ahead and learn that and you'll be pretty much golden now the reason why you need caching is because there's going to be some expensive end points in your api where you're running some really computationally expensive sql query or something or you're doing some other expensive operation that's going to take a lot of time, a lot of computational energy, and you don't want to run that every single time a user requests it because the result may not have changed, right? Let's say I'm running some search function on some government website documents where those documents rarely ever change, right? Let's say they take months to update, right? And let's say for the purposes of this example that it just takes a really long time to run that operation. right that search operation it takes five seconds let's just say if i run this operation every single time i have to search something there that's going to take forever right five seconds is a lot of time especially when you add that up to thousands or hundreds of thousands of users right that may be viewing that website at the same time as me right so that's a lot of unnecessary load on your servers especially because these results are not really changing anytime soon right so in these cases what you do is you cache the result of the db query inside of redis inside of a cache and then for all subsequent requests after that the server will just open up the memory cache and it's going to fetch the result from over there right and this is extremely fast because it doesn't need to go and run some computations or queries anymore it literally just fetches the result from the system's ram and just sends it over to the user as a response extremely fast so learn redis learn how it works learn the problems it solves and anytime you're designing an api with a bunch of expensive endpoints that don't really change often make sure you use redis or some other caching solution over there and you'll be very amazed with the results now let's talk about messaging so the reason i'm talking about messaging is because you will often especially if you're building a really complicated really large app you're going to have multiple different services that need to communicate with each other right and this is going to be more apparent if you're using a microservices architecture but even without that you're usually going to have multiple different services that need to communicate with each other right and there's basically two ways to make them communicate with each other one of them is a horrible way and the other is a really good way. So let's talk about both. If you're a junior developer, if you don't know what message brokers are, you would probably do something like this, right? You would have a bunch of services like a backend API or a frontend application or something else, or you have multiple different backend servers that handle different parts of the API or whatever, right? You have a bunch of services and you need to make them communicate with each other and keep each other in sync, right? What you might try is some sort of a direct communication kind of a method, right? Where each server basically sends a request to the other one, and they communicate whatever they want to say, right? You send requests to server two, server two sends a response back, you send a request to server one, server one sends a response back. This is a horrible way for communicating between different services. And it's horrible because it's bottlenecked because here's what's actually happening when server one needs to communicate with server 3 it needs to send a request to server 3 wait for server 3 to respond and then server 3 needs to send a response back to server 1 wait for server 1 to acknowledge it and then the request ends that's just unnecessary work that these two services are doing right and the same thing would happen if i tried communicating between server 1 and server 2 right they need to wait for each other to respond and acknowledge the messages and stuff And this is a horrible way to communicate between different services Now the better approach to make your services communicate with each other is to use a message broker Now, a message broker is basically a program or some server that is going to listen to events from your various services, and it's going to store them in a queue, like over here, right? You can see we have a bunch of messages over here in this queue. The message broker is going to store them in here, and anytime a service wants to send a message, like let's say service one wants to communicate with service two, right? What it's going to do is it's going to send a message over to the message broker, which is then going to be stored inside the message broker queue, and then it can just go back to work. Like service one doesn't need to do anything else after this. It sends a message to the message broker. It doesn't need to do anything else. It can go right back to work. It doesn't need to wait for some sort of a response from the other server. then service 2 is going to receive that message from the message broker and then it can process it and do whatever it needs to do based on that right whereas server 1 can just go back to work without needing to wait for some sort of acknowledgement or whatever right so that's a way better way of communicating between different services right now there's a couple cool message brokers out there some really popular ones are rabbit mq redis can also be used as a message queue i know i said that it's used as a cache but it's also used as a message queue and there's a bunch of other ones as well basically go learn about these things and you're gonna figure it out and just don't try to make your services communicate directly with each other i know a lot of you junior developers do this stop doing that it's really inefficient you might not notice the horrible side effects in your toy apps because you don't have any users but in a real production application you're going to need a message broker that can handle these things for you so start learning those things now now finally let's talk about docker and kubernetes so here's what docker does docker is an application that's going to take your entire app and everything it needs right like the entire application source code and everything the dependencies like the node modules and the python packages that you need to run this application the os packages like some system level libraries or packages you need to run this application right like i don't know ffm peg or something like curl or wget or some other you know utility or some library the database that you need the web server like nginx or apache or caddy or whatever you're using it's going to take basically everything that your application needs to run properly and it's going to package it all up inside of one big container right now this container is completely isolated from the rest of the system environment and it's extremely easy to reproduce as in you can create like hundreds of instances of this docker container very easily and you can run this anywhere on any machine that has docker because you've literally packaged your entire application in a docker container right All the things your application needs to run are in this container. So as long as you can just send this container over to someone, they can run your application very easily. Now, Docker is extremely useful on its own already, but it really shines when you combine it with something like Kubernetes, which is basically a cluster management slash container management tool that helps you manage multiple different Docker containers more efficiently. And this is going to become really important for you to learn as you build larger and larger applications for many different you know like thousands and thousands of users right so over here i have an example kubernetes cluster right which has 12 docker containers inside it how does kubernetes help us over here well it's going to manage the containers for us if some container goes down or has some trouble or it's malfunctioning kubernetes is going to fix it for us or it's going to restart it for us it's going to do health checks on every single container and make sure everything is running properly, right? If a container stops running or if a container has some sort of issue, it's going to replace that container, restart it, something, right? It's going to fix it for us. It also helps with progressive deployment. So basically, if you want to deploy a new feature, Kubernetes is going to help you deploy it without any downtime. You don't need to take your entire application down in order to deploy that update Because you can literally let some of these containers keep running while you deploy it. So let's say you've got a new update. You can let these three containers deploy the new update while the other nine are still running the current application, right? And then once the new update is live over here, you can put these containers back online and you can then update the other nine containers over here. so basically your users will experience zero downtime like the site will never go down whilst you're updating everything and deploying your new features so that's docker and kubernetes and with that we've covered the 10 topics that i need you to learn if you want to be a good developer especially in today's day and age the only thing ai has really automated is syntax you don't need to write code yourself anymore but you do need to understand the fundamentals and that's this video is going to help you do all right go ahead and learn all 10 of these topics and you're going to be a much better developer than you are right now and most importantly do all of this