Auto-generated transcript Good morning boys and girls. I don't know when you're gonna be watching this but at the time of recording this video It is like 10 a.m. So good morning and Yeah, we're gonna continue building this go redis clone. Let's do this The very first thing I want to do is fix a horrible horrible bug that I forgot to fix in DB.GO if I go down here to the get function, let me just Yeah, here we go. So So you can see over here that we're locking the database for reading, right? Because we're trying to get a value and then we're unlocking it down here, right? This is fine. But the problem is, what if the item doesn't exist, right? It just returns this and it doesn't unlock the new text for reading, right? Same thing over here. If the item is expired and it just returns this and it doesn't unlock the database for reading, right? So this is a huge bug and we need to fix it. All we need to do is down here just say db.mu.readunlock and then say the same thing down here as well. When you're using mutexes in Go, the best way to lock and unlock them is to just lock it at the top and then just say something like defer.mu.readunlock, something like that, right? So to unlock a mutex, the best thing to do is to just use defer at the top so that you can guarantee that this is going to run at the end of the function no matter what happens, right? And the reason I'm not using this approach is because basically just for performance reasons, I want to unlock the item as soon as the like, I want to only lock the item while we're actually using the shared variables and for no longer, right? Now in this case, you could actually use defer over here because in this case, we're only unlocking the database at the end of the function. So for this specific function, we can just remove all of these unlock commands and just use a defer at the top. But if, for instance, I had some extra processing to do over here, right, some really long processing that didn't actually modify the database in any way, then that would just be a waste of system resources, right? because you would keep the database locked even though you're not really using it in any way down here, right? In which case, you would have to unlock the database before you do this other processing, right? So, that's just some stuff to keep in mind when you're trying to use mutexes in Go. But yeah, in this case, we can just use defer to unlock the database. Now, moving on, I want to organize all of the code that we've written in the past 20 videos, I don't know, into a bunch of different files. So let's do this. First of all, I want to create an item.go file. Package main. Oh my god. And then just extract the entire item struct and its methods into this new file over here. There we go. And in this approximate mem usage function, we can change this key to an item just so the naming makes sense. Save everything and we're done. Let's do the same thing for this transaction and tx command structs as well. Let's create a transaction.go file over here. There we go. There we go. Awesome. Save all of that as well. Let's do the same thing for the app state which is in main.go down here. So let's create an app state.go file package main and then copy paste the entire app state struct over there. There we go. Save everything. It is nice to do these kinds of things every now and then, right? You don't just want to just write as much code as possible. You want to make sure that the code you're writing is maintainable, scalable, easy to read, easy to understand and all of those wonderful things, right? So always just try to organize your code as best as you can. We're also going to do the same thing for this client struct. So let's create that file. There we go. And let's take all of this, paste it over here, save everything and we're done. Now, up here in the main function, we can remove this wait group over here as well and over here because we don't need it. Because this listener.accept function is going to hang the main thread for us, right? The reason we were using the wait group is to basically keep the program running as long as possible until all of the connections have been served, right? We don need to use the wait group because this l function is just going to hang until we receive a connection You can see it waits for and returns the next connection So it will literally just keep the program running indefinitely forever until we receive a connection So this is all we need right We don need the wait group anymore Now in the previous video, we implemented a bunch of eviction policies that are over here. We implemented all keys random, all keys LRU, all keys LFU. Next, we're supposed to implement the where is it down here, the volatile eviction policies. But I'm not going to do that. Actually, I've decided I'm going to implement a bunch of other commands and other features. I'm not going to implement all of these eviction policies. I am going to actually explain how to implement them. So if you want to do this, you can think of it like a like a exercise that I'm assigning you or something Like a practice project that you can work on the way to implement all of these volatile eviction policies is basically you want to Where is the DB struck you want to have another store over here that only stores the expirable keys so something like exp store and and this will also be a map string and item, right? So this is kind of like a second database store, but it will only store expirable keys. And the reason we want to store expirable keys separately is because it's just easier to filter them out, right? If you try to filter out the expirable keys from the store, it's just going to be very bad for performance, right? Because imagine you have thousands or tens of thousands of keys, right? and you have to filter out only the expirable keys from the store right that would require you to literally loop over the uh if i write this down somewhere that would require you to literally loop over the entire database store and filter out the what is it the um expirable keys so it would say something like, I don't know, is expirable. And this method doesn't exist. Obviously, I'm just showing you an example. If is expirable, then, you know, run the eviction policy or something, right? This is horrible, right? Because you are literally looping over the entire database store, right? Just to filter out the expirable keys. So the performant way to do this, to implement all of these volatile eviction policies and not necessarily nuke the server is to literally have a second store in the database truck that only stores expirable keys right so you would have to keep this in sync with the original store up here and you would basically just um down here in the delete function in the set and the get functions you would basically instead of storing expirable keys in the store you would also store them in the expirable store so kind of like a duplicate right and then in the eviction policies up here in the evict keys function what you would do is you would just add a bunch of cases over here for the volatile um eviction policies like volatile lfu all you need to do then is to just grab the list of samples from the expirable store not from the original database stored, the expirable stored, right? Grab the list of samples up here and then all you need to do is sort them by least frequently used or least recently used, depending on which eviction policy you're using. If you're using volatile random, then there's no need to sort. You can just start evicting keys, right? If you're using volatile LRU or LFU, you can already sort them by using these little snippets over here, right? And then you can just evict until you're done, right? As for volatile TTL, this is also easy. All you need to do is sort by which keys are going to expire first, right? And then you delete the keys that are going to expire sooner rather than later. So this is all super simple to implement. I've literally explained everything you need to do. So think of it like a practice project and go implement this on your own if you're interested what i'm going to do is move on from these eviction policies and build the monitor command so let's do that let's add a um where is it the handlers.go right let's add the actual handler over here and the monitor command basically what it does is it allows you to monitor or receive every single command that every other client is going to run right so if you have bunch of clients connected to this server and one of the clients is monitoring the server every single command that every single client tries to run that monitor client is going to receive it so that's super useful let's implement a handler over here as well see client there we go V for value State for app state There we go. Awesome. Now, before we actually implement this, we want to go to app state again over here and add the actual monitors list. We want to store every single client that is monitoring us. So this will be a slice, a list of client pointers, right? We want to store every single client that is trying to monitor the server. And then every single time we run a command, we want to send that command to all of these monitors as well, right? Now we can go and implement the monitor command as well. This is super simple. All we need to do is just append to state.monitors and append the client that we have over here. And then we just return a value of type string. And this will just be a simple OK success message. That's all we need to do. All right. So this is going to add the client to the monitors list. Now, we also need to be able to remove a client from the monitors list when it stops monitoring the server. The way to do that is to go back to main.go over here. And in the handle connection function, every time you receive a new client, we also know when the client is when the connection is closed down here right so we want to make sure that we remove any of the monitoring clients from the monitors list over here in this handle con function so let's do that as well let's create a defer function in this handle con function let's say defer func and create a new monitors list by saying state dot monitors colon and then zero. What this is going to do is it's going to create a new slice with the same capacity as state.monitors. So we don't need to reallocate any memory for this new slice, but it's not going to fetch any elements from state.monitors. So basically this will create an empty slice with the same capacity as state.monitors. And again, this is just for performance. So we don't need to reallocate any extra memory for this new slice. Now let's loop over the entire state.monitors list. Let's say if the monitor does not equal the current client, which means this client is not monitoring the server, then we can append it to the new monitors list like this. There we go. And then we just say state, what the hell, state.monitors is equal to new and then down here let's call this function as well there we go so basically let me just walk you through everything over here all we're doing is creating an empty slice with the same capacity as state.monitors so we don't need to reallocate memory for this slice every time we append to it then we're just looping over the state.monitors list and we're saying if the monitor does not equal the current client so basically if this client is not monitoring the server then just append it and just assign it to state.monitors, right? That's all we need to do. So now if a monitoring client, right, if a client that is trying to monitor the server stops monitoring the server, it will be removed from the state.monitors list. So we have the functionality to add to the list of monitors and we also have the functionality to remove from the list of monitors. The last thing we need to do is go up to the handle function over here in handlers.go and every time we receive a command we want to send it to all of the clients that are trying to monitor the server. So let's do that as well and I'm actually going to do this in a go routine just so it doesn't interfere with the current client's execution right. We don't want the current client to hang just because we're trying to send a command to all other clients, right? So let's do this in a go routine. Again, just for performance, let's loop over the entire monitors list in state.monitors. Let's say if the monitor is not the current client, because we don't want to send a command to the current client, right? We want to send it to all of the monitoring clients. Even if this client is monitoring the server, if it's also the one sending the command, we don't want to send it back to it, right? That's just weird, right? So make sure to check for that. And then let's actually create a new function to send this reply. Let's open up client.go and add a new function over here. that gonna be I guess a pointer receiver to client call it C and then just call the function something like write monitor log and it should also take a value pointer which will be the actual command that we trying to um send to the server let's call it over here in the go routine in the handle function let's say mon monitor dot write monitor log and pass in the value which is the command that we received from the client, right? Okay, so first of all, let me just add a log over here. It's always a good practice to log these kinds of things. Let's say relaying command to monitor and then print the monitor address over here. Let's say c.localaddress.string. There we go. Now, let me show you how Redis actually sends the monitor messages. So this is going to be a string. And the first thing we want to add over here is the Unix timestamp of the current time when the command was run. So let's say time.now, call this .unix. There we go. And then we also want to print the connection address, right? So the client's address. And to do that, we can just say c.con.localaddress.string. So basically the IP address of the client that is trying to run this command, right? And then we want to print the Actual command that is being sent over here So, you know that all of the commands that the Redis client sends are just arrays like arrays of bulk strings So let's loop over that by saying something like we range we dot array and then append to the message by saying fmt.sprintf and then print the bulk string in this value but also cover it with double quotes and make sure to use backslashes to escape the entire thing. So now we have our message. Now we need a writer to actually send it to the connection. So let's call new writer, pass in the connection of this client then write the message to it and this will take a value which will be a reference to a new value struck this will be of type string and the string will just be the message variable we can actually just move this to a new variable as well just to make things a bit more clearer we can say something like reply is this value over here and And then just pass in a reference to that in the write function. Then obviously make sure to flush the output out to the network connection to make sure that it is sent. And that is all we need to do. Awesome. Now let's actually test this out. We're going to need a couple of clients over here. So first of all, let's just stop the Redis server. There we go. And then let's run our server over here. So it's running. let me open up one client and call the monitor command i got the okay success message and now i'm going to start receiving commands over here let's open up a second client over here and you can see we already got the logs over here right so this is the unix timestamp when the command was run this is the ip address of the client that is sending the command i am obviously running all of this on my local machine so the ip address is just going to be the same right because i don't have multiple machines to run this on but uh i can run any single command i want over here and it's going to get logged over here right i can open up a fourth client as well or i guess the the third client over here there we go and i can i don't know set name to to Hassan again, which is the same value it was set to before. And you can see that it gets sent over here to the monitoring clients. I can press control C to stop the monitoring. And now I can obviously use this first client as a normal client as well. And yeah, so everything works. I can just remove all of these other clients. And yeah, we built the monitor command. awesome all right that's all we wanted to do in this video let's um let's end it over here it is getting pretty long 30 minute video already yeah let's stop it over here in the next video we're going to probably implement the info command which is basically a command that gives you info about the redis server how much memory it's using how much cpu it's using where it's hosted and all of those other things right so we're going to try implementing that in the next video and yeah thank you for watching this one like comment subscribe and stay tuned for the next video see ya