Auto-generated transcript Hello there guys, welcome back to the go redis clone series and welcome to the final part of this series Alright in the previous part we implemented the info command over here Implemented is the wrong word. We actually just implemented like half of the command we're going to finish this in this part of the series and Then we're going to basically just end this project because I think we've built enough of this already We've built so many different features in this project and I think I'm ready to move on. So before I actually Continue this we have to implement the persistence category and the general category before I do that Let me just go up here and declare the info command handler And then down here, let's create a command handler for it Now, to actually get the info output, right, this is going to be a string, we want to have another method over here and we can just call this, this is going to be another pointer receiver and we can just call this something like print and this can also take the app state and it's going to return a string. There we go. So first of all, let's just call info dot build over here and pass in the state. And this will just build the info object, add all of the information that we're having over here. And then let's craft the message that we're going to return. So let's say message string, and it's going to be just an empty string, obviously. Then let's create a sub function over here and call it print category. All right, this is going to actually print a category in the info struct over here. can see we have five categories over here right the server client memory persistence general and all of these are going to be printed in the same way right so there's no need to duplicate the logic let's just create a print category function over here and we can just pass in the actual category and it will just print it for us so this is going to take a couple of arguments which will be the header which is a string and then the actual data right which will be a map of string of string and we'll return a string as well uh and also make sure to add the colon over here since we're declaring a new variable i don't know why i forget to do that sometimes um but uh yeah so first of all let's just create a string over here and add the actual header let's say hash this is how we declare a heading in um this output this is how redis actually displays headings in the info command output and let's say Let's print a string over here, which will be the header and let's also add a new line after this Now we can basically just loop over the entire map write the data in This category and then we can just say the s string we can append to it. You can say FMT dot s print F print two strings over here separated by a colon right the first will be the key and then the value and then also add a new line after this finally let's return the string with one more new line over here and that's all we need to do now we can just say something like something like message and then append to the message and say just call the print category function pass in the header which will be server passing the data which will be info dot server then just copy paste this three times second the category will be client the data will be in info dot client and then finally memory and info dot memory and then at long last let's return the message to the caller and that's all we need to do now let's actually create this info struct we need a place to store the info InfoStruct right so let's do that in app state as well because we store everything in app state apparently let's create an info struct over here this will be a pointer and Let's um go down here in new app state and call new info over here And yeah, that's all we need to do so back in the handlers over here now. We can say something like state dot info dot print and Pass in the state this will return the message that we want to send Then we can just return a value of type bulk. The output of the info command is a bulk string. And the bulk value will just be message. And we can actually test this out as well. If I run the server and then if I run a client over here, I can run the info command. And it's going to return all of this information over here, right? Awesome. So now let's configure the other two categories, persistence and general stats. Let's go down here to the build function and say info.persistence. This will be a map again. First, the rdb bgsave in progress field. This just tells you if a bgsave is currently in progress. And I think we already tracked this in state.bgsave running. Then we also have a field called rdb last saved time. So basically the last time the rdb file was saved, right? And we don't actually track this right now. So let's actually do that. Let's open up the app state file over here. And let's just create a new struct up here. Call it rdb stats. There we go. This will be a struct. And let me just name this something better. rdb stats. There we go. And this is going to have two fields the RDB last saved timestamp Which will be an inch 64 and the RDB saves So the number of times that the RDB snapshot has been created which will be an integer let attach this to the um app state by saying rdb stats equals rdb stats let do the same thing for append only aof files as well let's say aof stats this will be another struct and yeah before i actually write all of these things into the info struct let's just create all of the actual structs let's just track all of the data that we need to track and then we'll just add it all to the info struct over there. For the AOF stuff we don't really need to track a lot we just need to track the number of times that we have done an AOF rewrite. All right let's attach that to the app state as well let's say AOF state or AOF stats is AOF stats there we go. Now finally this is this is all we need to do to track the um what is it the uh persistence category but for the general category we want to have another struct called a general stats and this will have a couple of um fields first of all the total connections received which will be an integer and the total commands processed which will also be an integer and then the expired keys which will be an integer and also the evicted keys which will also be an integer let's add this down here as well let's say general stats is general stats add it all to the app state and down here in the new app state function where we create the actual app state let's just um create an empty object for all of these different stats right like this okay awesome all done now let's actually start tracking all of these things right so for the let's start with the um rdb stats we want to go over to the rdb save function which is um in the rdb file over here there we go and every time we save the rdb file we want to update the last save timestamp and the number of saves so let's go down here to the very end over here and say state.rdbstats.lastsave timestamp is time.now.unix and the state.rdbstats.rdbsaves just increment this one time and now we're tracking all of the rdb stats now for the aof rewrites field we can go over to handlers.go again in the bg rewrite aof file or the command handler we can say down here state dot aof stats dot aof rewrites and just increment this by one right so each time we rewrite the aof file we can just increment the rewrites by one now for the um for the total connections received field over here in general stats we can go over to main.go in the handle con function where we increment the client count and down here we can just say something like state.general stats.connected or the total connections received and just increment this right and there's no need to decrement this because this is the total connections field right not the current connections and for the commands process you can probably guess this we can just go over to handlers.go up here in the handle function which receives every single command we can say down here state.generalstats.totalcommandsprocessed and increment it. Now for the expired and evicted keys both of these operations happen in the db.go file right. So we can track it over here. First of all in the try expire function over here this is where we actually expire an item. We can pass in the app state over here. App state why can't I there we go And if the key should expire and if we delete it then we can just say state dot general stats Dot expired keys and increment it over here And down here where we call this try expire function. We make sure to pass in the state We're not actually receiving the state in this get function. So let's add it over here as well By saying state and app state now back in handlers.go in the get function over here where we call db.get we want to pass in the key name as well as the app state there we go and now we can basically track the number of expired keys finally we just need to track the evicted keys and we evict our keys in this evict keys function up here so for this one we can just say over here in the evict until mem create function we can basically return how many keys were deleted this can just be an integer so this will return an integer every time we um delete a key we can and first of all let me just create an integer up here we'll call it n and every time we delete a key we'll just increment n and then at the end we'll just return this n value and then over here every time we call this evict until mem freed function this This is going to return the number of keys that were evicted. So we can assign these as such. And we can just say state.generalstats.evictedkeys plus equals. So add to this evicted keys field. And then just add the evicted keys that we received from this evict until mem read function over here. And now we're tracking all of these things properly. So let's add these to the info struct at long last. Let's say rdb last save time is fmt.sprint state.rdbstats.lastsavetimestamp and rdbsaves equals fmt.sprint state.rdbstats.rdbsaves. There we go. And aofenabled will be fmt.sprint state We tracked this already in the config We also have one more field over here called AOF rewrite in Progress and I don think we track this yet State dot a yeah, we don't track this yet. So let's add this as well to the app state So kind of like we track the RDB BG save running up here in the app state. We can say something like AOF Rewrite running or something like that and this can just be another boolean and in the info struck over here we can just return the rewrite running boolean over here now to actually track this we want to go to the handlers dot go function again the handlers of the file again and in the BG rewrite AOF file over here this is actually where we rewrite the AOF file right so every time we call this rewrite function we can say up here a of rewrite running equals true and as soon as the rewrite is finished we can say that a wife rewrite running is false and you could even do something like a left rewrite running equals true up at the top of the function and then just create a defer function that would basically say a web dot state dot a web free ride running equals false and then call that up here right so as soon as this go routine ends this function will run and a web free ride running will become false awesome of course both approaches are valid you can track it down here as well in the rewrite function or you can do it on a per function basis right in the entire function yeah finally let's Let's track the AOF rewrites over here in the info struct as well. Let's say fmt.sprint state.aof stats.aofrewrites and that's all we need to do. Awesome. So that's all for the persistence category. Now for the general category. Let's say map string string total connections received. fmt.sprint state.generalstats.totalconnectionsreceived total commands processed fmt.sprint state.generalstats total commands processed See, this is super simple stuff, right? The hard part is basically just tracking all of these stats across your entire program. but um returning this to the user that's super easy right so we can say expired keys and evicted keys over here is fmt.sprint state.general stats dot evicted keys just copy paste this this will finally be expired keys and over here change it as well to expired keys there we go and then down here in the print function let's add both of these categories persistence which will take info dot persistence and general which will take info dot general and i think that's all we need to do let's rerun the server go run dots there we go start a new client run the info command on it and yeah Awesome, so this is working we get the used memory and everything else I can try to I guess set a new key To let's say XYZ. I guess if I try to get the info command again you'll see that the I Guess the commands processed is three now Connections received one because we're only using one connection right now connected clients is also one that all makes sense sense. We get the process ID, the executable, the config file, uptime in seconds and all that. We get the used memory. We get the used memory peak as well. I can actually delete a key like delete key one and then grab the info again. And you'll see that the used memory peak is 201, but the used memory right now is 106. Awesome. And I wonder what keys are still being stored over here. Okay, so the new key is being stored. What is its value? Okay, a large number. Let's try expiring that key, right? So let's say expire new key in three seconds. Let's grab the TTL of this key and it's already expired. Awesome. So let's grab the info again. And okay, so it didn't actually increment this. Why not? Okay, so the evicted keys works, but the expired keys doesn't. I wonder why. I guess we're not tracking it correctly right now. Let's go back to DB. Go in the try expire function. Oh, never mind this is because in the Handlers go in the TTL handler instead of calling the try expire function We're trying to expire it on our own right so we shouldn't be doing that We should just be running DB dot try expire over here passing the key name, which I guess is K the item which is over here it's key let's rename this key to item just so it makes sense and then in the try expire call over here let's pass in the item and then let's pass in the app state this will return a boolean which will tell us whether it was expired or not then we don't need all of this logic over here we can just say if expired return all of this we actually do need the expiry in seconds let me get that back where was it yeah this stuff okay so let's actually track the expiry seconds there we go but to actually try expiring the the key we should call DB dot try expire because that is what will actually track the stats as well we're done with this let's run the server again because I just want to make sure that this is actually tracking all of the keys properly let me see what keys we have right now. OK, we have a bunch of keys. Let say expired name in three seconds Let grab it TTL And there we go It expired Let grab the info and you will see that one of the keys was expired awesome So we're tracking all of the stats perfectly now and that is the final thing that I wanted to implement in this server. Now before I actually end this project I want to fix one last bug in the project which is in aof.go over here. You'll recall that the AOF synchronization process over here doesn't actually respect the maximum memory limit of the server, right? So you can see over here that we're using 395 bytes and even all the way up to 493 bytes, right? When we sync the AOF records, even though the maximum memory of this server in the configuration is 256 bytes, right? So the AOF sync is not respecting the maximum memory limit. And the reason it's not respecting it is because we're passing in an empty config over here. Now to fix this, we can pass in the actual maximum memory and the eviction policy and stuff directly in this sync function. The reason I'm not passing the actual config object is because then it would also have stuff like AOF enabled and AOF file and fsync and stuff. And then every time you try to sync it, it will just add to the AOF records again and again, right? So every single value you restore from the AOF file, it will just add it to the AOF file again because the config would have all of these fields enabled right which is why we set the config to a blank struct in the first place right anyway this sync function should take a bunch of new arguments now and they are max memory which is an inch 64 eviction policy which is of type eviction and mem samples which is an inch 64. there we go and we can assign this to the config Object over here. We can say max memory is max memory Eviction is eviction policy mem samples is mem samples There we go, and I guess this is an int so we can convert this back to an int in the argument as well Awesome now, let's actually pass these in the main dot go file up here here where are we calling this over here okay so this is going to take the maximum memory from conk.maxmem the eviction policy as well and then the mem samples all right so we're only grabbing the configuration directives that we need to respect the memory limit we're not grabbing all of the other configuration variables over here so now that we have this let's stop the server and run it again and yeah awesome it works so you can see in the logs over here previously we were not respecting the memory limit but now when we sync the AOF records you can see that the memory increases as we add more keys but as soon as we cross the 256 bytes memory limit it starts evicting keys even from the AOF backup over here right it starts evicting keys and always make sure to stay below the 256 bytes memory limit right so this works as well the AOF file the AOF functions respect the memory limit of the server as well and yeah now we're done with this and this is turning into a very long video I've been recording for more than 30 minutes this is going to be a very long video but yeah that's the final video for this project and with this we are done it's been a very interesting remarkable journey um i've been building this project for the last 20 videos so yeah so many cool things that i built in this project so many awesome new features so many new things that i learned about redis about databases about about how all of these things work under the hood right behind the scenes you just don't even know about these things right like you use redis You use Postgres you use my sequel you use all of these other tools, right? All of these databases to store your data and you don't even know how they work under the hood, right? all of the different like features that they have all of the different design decisions that they have to make to Make sure that they're delivering a performant server, right? all the different trade-offs that they have to consider to make sure that this database system that they're building can run and scale up to thousands and tens of thousands of you know operations per second and all those wonderful things right there's just so much that goes into this into projects like these right and what I built over here is very simple right compared to actual Redis compared to all of the other database systems out there what I've built over here is very very simple and bare bones right you would think that this is a giant project right but yeah it's nothing compared to all of the other database systems out there that are actually running in production, actually powering enterprise level software, right? But yeah, this has been a very fun project. Hopefully, if you were following along the entire journey, you learned a bunch of things yourself as well. And you're now a much better programmer just as I am. And yeah, thank you for following along this entire series. It's been a very fun journey. Let's see what comes next right I'll probably build another cool project and go or I might actually start learning C or rust or something like that I've been very interested in learning C for a while and just becoming much more used to low-level programming and how systems and hardware works under the hood right so I might actually build a new project in C or just continue coding and go for a while but we'll see again thank you for watching this video thank you for following along with the whole series so far hopefully you learned something hopefully you got some value out of it and yeah I'll see you in the next video which will be about who knows what because I have no idea what comes next for the channel after this project but yeah I guess we'll see thank you for watching and see you in whatever video I film next bye bye