Auto-generated transcript Hey guys, welcome back. We're gonna continue to build this Go Redis clone thingy. But first of all, what I want to do is I want to recap how RESP works, Redis Serialization Protocol. We talked about this a bit in the last part as well, but I don't think I explained it as well as I could. So, over here I have a bunch of examples of RESP, different messages in RESP. And I'm just gonna go through each of them and just show you what each of them means, alright? And then hopefully this will give you a much better understanding of resp and how it what it looks like how it works and all That and how we can then parse it and just use it right so Let me just make some space over here So Let's go through this. Okay message over here because this is the simplest example I have right now, so First of all this plus symbol this means a string so different data types in rest have a different like symbol to represent them, right? I think I make, yeah, here we go. So we have the string type over here represented by the plus sign. And then after that, we have the actual string message, right? So right now I just have an okay message over here. And then after that, to signal the end of this message, we have these new line characters, backslash R and backslash N. What backslash R does is it just takes the cursor back to the beginning of the line and backslash n will move it to the new line right so you don't need to go too deep into those characters you just need to know that these are new line characters all right they're just going to signal the end of the message or the end of this part of the message right so that's how strings are represented in resp all right next we can look at errors now errors are going to be represented by this dash sign all right and we can add that over here as well i can add a new error data type over here represented by this dash symbol all right so to send an error or to receive an error in resp you have to use this dash symbol and then this er keyword over here and then after that you can have an error message all right then once you've added the error message you can then add these new line characters to signal the end of the message right that's how Redis will know that the that this is the end of the message and it can stop reading the data after this right now moving on from that this dollar sign represents the bulk string and this is what we talked about in the last part of the video as well this bulk string right over here alright so when it comes to bulk strings we also need to know the length of the string why because let me explain in in regular strings right you don't need to know the actual length of the string because you know that this string will continue until we reach these and these new line characters all right so you can just continue reading the entire data until you reach these new line characters so you don't need to know the length of the string but when it comes to bulk strings bulk strings can contain multiple words and stuff right so you do need to know the length of the string so you can so you can read the exact number of bytes and know more than that right it's a performance thing over here which is why you know in to use both strings because use this data this dollar sign symbol to represent a bulk string and then we add the length of the bulk string so right now over here I have six which is the length of this string right so you have the length of the bulk string and then you add these new line characters and then you add the actual string value and then you again add these new line characters to signal the end of this bulk string alright so So that's all the data types that we're going to use. Finally the array data type which we looked at in the last video as well. I'll quickly go through this again. The Redis client sends all of its messages, all of its commands as an array of bulk strings. So it's important that we cover this one as well and we learn how this works properly. So arrays are represented by this star symbol and then after that symbol we have the length or the number of elements in the array, right? The length of the array. And you can see I have this data type over here as well. We built this in the last part of the video. So you can tell that this is an array of two elements based on this, right? Once you add the length, then you can add another new line character, and then you can add the actual bulk strings over here, right? So you can see over here, if I just format this a bit better, this is the exact same format that we discussed these bulk strings over here right so nothing has really changed right all we need to know is that this is an array which we can tell from the star symbol and this is an array of two elements which we can tell from this two over here right after that we just add the actual elements of the array in rest format and that's about it right there's no there's nothing else to discuss over here this these are just simple bulk strings right so that's how rest works that's what it looks like that's how we can represent and work with different data types in rest and hopefully now you have a much better understanding of how this works right so i can actually close this now and write some code and continue building this project right so in this part of the video we're going to be building command handlers like get set retrieve exists delete all of these other things right so the actual database data handling part of this project that's what we're going to build in this part of the video first let's just quickly recap what we've got so far we have a value struct we have some value types we read the array that we get from the client and this is just to read the bulk string this method and yeah that's about it right so let's let's actually run this server now and i'll actually also add a little So log statement over here. We'll say listening on port 6379. Just so we know that the server is actually running, right? And over here as well I can add connection accepted, right? Just some basic logs over here. Let's run this and oops, yeah I forgot to shut down the original Redis server so let's do that as well. There we go and now let's run this. There we go. So it's running. Now I can run the client and let's just send a bunch of commands right. So this is what we have so far. Now we need to parse these and just use them to actually build all of the data logic right so let close all of this and start writing the command handlers first let's create let's go down here and let's create a type for these handlers so i'm going to call it handler and what this is going to be is a function that takes a pointer to a value and also returns a pointer to a value i'll explain how this works but basically what you're gonna have is you're gonna send in the actual value that is the commands that the redis client sends right which we read from this read array method right over here we'll pass these into the handler and then the handler will read all of the commands and arguments and everything and then you know do all of the data stuff over there and then it will return a new value that will be a reply that we can send back to the client right like an okay success message or something like that right now let's also create a handlers map now this will be a string and it will hold handler objects right let's initialize this as well and now let's create a handle function over here this is going to be this is just going to take the array that the client sends and and it will just select the appropriate handle handler and let it do its thing right so this is going to take the actual connection and also the value all right and there we go now the actual command is going to be in the first element right of the of the array so let's do v dot array zero and we know that this array holds a bunch of bulk string elements right so we just get that value over here so that's the actual command right now let's grab the handler from the handlers list by doing this there we go I forgot to actually declare this variable okay now let's handle this okay error as well we need to make sure that this handler actually exists obviously and that's what I'm doing over here if this doesn't exist then let's just log dot fatal or not even log dot fatal we don't need to actually shut down the server just because the client sends a wrong message right let's just print this right invalid command and then just print the actual command and then return because don't want to continue executing if the handler is wrong right and then we just call the handler with the value right that we get over there now before we go any further we also want to be able to reply to the client right so the way to do that is with using con dot right right that's what we're doing up here to send a message back to the client but this is kind of like a bit too low level right I want another abstraction over here so let's create a writer struck I'm gonna call this writer it's gonna be a struck and we'll have a writer field which will be an IO dot writer not reader writer and then let's create a new writer function this is going to return a pointer to the writer and it's going to take an IO dot writer this will be the connection that we pass in all right because the problem over here is sure we can reply to the client like this right but we have to construct the reply messages in rest format ourselves right with this plus symbol with the new line characters and all that I don't want to do that every single time I want to reply to the client right so this we're going to create this writer struct over here and what this will do is it will just serialize all of our value structs over here into a REST format right so what I'm going to be able to do is I'm going to be able to create a value shrug give it a type give it some bulk string data or something and it will just and then I can just pass that into the writer and the writer will convert it to REST format on its own right so that just saves me a lot of work so let's return a pointer over here and let's add the bulk field. Now I'm going to use buff.io to create a new writer from this io.writer. Now with that done let's create a write method. This will be a pointer receiver to the writer struct and we'll call this write and it will take a value. There we go. The actual reply will be a string obviously and then we can just use switch a switch statement over here on V dot type and then we can just handle each of the data types here now before I do that I want to add a couple more error types so we already have an error type I want to also add a null value type over here and this is not going to have any like symbol over here right but But we do want a null data type because what we can do is, so let's say you try to get a key that doesn't exist in the database, right? We want to be able to return a null value in that case. So let's do this. First let's handle strings, right? Now strings are just going to be, let's do this, there we go. Strings will need the data symbol, which is the plus sign, right? So let's add that. And then they just need the actual string message, right? After that they just need backslash R and backslash N, the new line characters, right? So let's add the type here and then the actual string, alright? Now let's handle the bulk. In this case the reply should be the symbol first of all and then the length of the bulk string, right? So this will be a digit, a number. Let's add both of these. the type and then the length of the bulk string there we go and then after that we add the new line characters and then after that again we have to add the actual bulk string right there we go and then finally new line characters again and that's how you how you represent bulk strings next let's do i was going to do array but let's not bother with that because we're not really gonna send arrays back in any response in this project so that's done let's not even bother with that let represent the errors as well errors are going to be just like strings so we want to have the value type and then the actual error message I don have an error field over here we need to add that and then finally the new line characters right let's add that value that error field over here this will just be a string there we go and then finally let's handle the null value so to send a null message what we can do is we can just send a bulk string with a length of negative one right and then just the new line characters there we go so now we have a reply message over here now we need to actually send this to the client so let's grab the writer over here which is which will be the connection that that we pass in. Then we need to convert this reply into a byte slice instead of a string. And then we just pass that in. And boom, that's all we need to do. It's giving me an error, not even an error, but a warning over here because we don't need to use this method. So let's just turn this into a string literal. There we go. And now we have a writer over here. now we can go back up to this handle function so each of these handlers is going to return a value pointer right so we can grab that call it reply we can initialize a new writer by saying new writer and then pass in the connection over here and then we can say w dot write and we can pass in the reply over here there we go now up here we want to actually handle this we want to actually call this function i realize sometimes i just say the weirdest things if that happens i'm sorry i'm trying to explain this as best as i can but anyway uh now we handle this we want to pass in the connection and we want to pass in the value that we get over here so a pointer to the value this will first we'll read the array that we get from the connection right from the client and then we'll just pass that into this handle function and that's about it then this handle function will take care of everything else so now we can actually create a bunch of commands right so let's start with get we're going to this is going to be a handler right so this will take a pointer to the value and will return a new pointer to a new value first of all let's get the arguments for this command right now these are going to be in the array now we know that the first element in the array is the art is the command right so everything after that is going to be the arguments right and in the case of get we only need one argument so if the arguments are more one that's an error right so we return that we return an error message by just creating a value struck with the type of error and the error message over here can be first the error keyword right remember this is important this is how Redis knows that this is an error this error keyword and the dash symbol right So make sure you include that now we can just say invalid number of arguments or get command, right? Now assuming that we get the appropriate number of arguments Which is just one now we can get the name over here the name of the key that We're trying to fetch from the database. This will be the first Argument right because there's only one argument in this list and now we need to actually create a database, right? so for now I'm just going to create a map over here we'll just say this is a string map right as such and we can just use this to grab the um value from over here right and of course this argument is going to be a value type so we should grab the bulk string over here and then let's grab the value as well from using from the database as such and make sure to use this okay coma okay idiom so we can make sure that there is a value over here and if there isn't then we return a null type just to indicate that there is no value over here and if there is a value then we can return the correct value over here this will be a bulk string and will assign the value into the bulk string over here as such awesome so we have the get function now let's also create a set function just so we can test both of them together right again this will take a value and it will return a new value which will be the reply now set is going to have at least two arguments right or exactly two arguments the key that you want to set and the value that you want to set it to right so let's make sure that we are only receiving two arguments over here and if we're not then we can return an error and the message can be invalid number of arguments for set command. Now what we can do is we can assign this into the database right. So let's grab the key which will be the first argument and the value which will be the second argument and then we can assign that into the database. So this key will be assigned to this value. There we go and And now we've set the value in the database correctly. Now we can return a success message like this. We'll just give it a type of string and the STR can be okay, right? So the same success message that we use over here, all right? So now we have the set and the get commands, right? Let's add these to the handlers list up here. we'll say get equals get and set equals set there we go now you've added these we've added these handlers over here and the handle function will call the appropriate handler right so now we can actually test this out and see how it works let run the server it running let run the redis CLI there we go and you can ignore this little invalid command error over here you can see that in the handlers list we only handling get and set right now not this command and these dog stuff right so every time the Redis client starts up it sends this command message to the to the server and what this does is it basically returns a bunch of information about the server right we haven't created a handler for that yet which is why we're getting this invalid command error so just ignore this you don't need to worry about it right now let's actually try and get name now nothing at all and actually actually before before we proceed I forgot to where is it I forgot to remove this where's the main function there we go we don't need this cons odd right anymore right because the handle function is going to return the replies for us we don't need it we don't need to do it like that so remove that con dot right from the main function and then let's try this again run the client now it's not running that's weird oh oh never mind never mind let's also handle the command over here because I forgot to I forgot how this works we actually do need to handle this command message over here first because if I don't send a reply to this then the client is just not going to start up right you saw that the client just doesn't start up anymore so let's create a handler for this as well and we don't need to do anything fancy over here this is just going to return an okay message right we don't need to actually return anything other than that so let's create a handler and just return the a new value of type string and the message will just be okay there we go now the command will actually run now the server can actually run properly and the client can actually start up properly so let's run this again let's clear this and run the CLI again and it's still not starting up I wonder why okay so we still have one tiny issue to fix we are sending the replies over here by writing them to the connection but the problem is that sometimes they don't get sent to the client because we don't flush them out right so let's do that as well we need to get the writer interface and convert it to a buff.io.writer as such and then just call the flush command right the flush function what this is going to do is it's going to write any buffer data to the underlying writer right so how how writing data works is it sometimes stores the the data you write into a buffer and then it just delays writing it to the actual file or the actual You know connection and the way to actually force it to write it to the connection or the file or whatever is to just call This flush method right? This is going to send it out of the buffer and into the whatever connection or file or whatever resource you have open in This writer right so let's make sure to do that as well now save everything run the client and Run the client and server there we go now let's try to get name we get nil why because we don't have any key stored over here right now let's try setting name to Hudson and it gives me a message okay right so that's the success message message now let's see if the name got assigned and it did awesome right so yeah we built a key value store over here obviously this is very basic and everything but this works right I can set the age or something to I don't know 21 I can get the age and there we go we get the age right awesome so this is a complete data value data store right a complete key value store just like redis yes the implementation is pretty basic but it gets the job done right now there's two more things we need to add and we're not going to do this in this part of the video because it's getting really long honestly i'm going to end the video over here we've built the get and set commands and even the this command command over here right we made a ton of progress over here we built a little database object over here now there's two more things we want to implement in this project which we'll do in the next video and that is concurrency we want to make sure that this server is concurrent we want to make sure that getting data, setting data, handling connections, all of this stuff is concurrent and we'll use go routines and mutex to implement that concurrency. We need this because in a real world application you're not going to have just one client and one server, right? You can have one server and multiple clients from different locations or whatever, right? Accessing the same database, accessing the same server, right? So we want to make this whole thing concurrent. We're going to do that in the next video and I also want to build some sort of data persistence right we want to be able to store this data on our disk somehow right because redis is originally just a database stored in memory right which is what we're doing over here this is just going to be stored in memory and as soon as I shut down the server like this all the data gets destroyed right it just gets destroyed so we want build a data persistence method as well there's two major ways that Redis uses the that Redis implements data persistence and we'll talk about both of them in the next video so that's what we're going to build in the next video data persistence making sure that the data we store in this database also gets stored in disk so we don't lose it and concurrency making this whole server concurrent so that multiple clients can connect to it and interact with it without like stepping over each other right that's what we're going to do in the next video and in this video we just built a bunch of command handlers and just built a basic key value store over here so yeah that's all that's all for this video I hope you learned something I hope you are enjoying the series so far we're making good progress here building this Redis clone I hope you're learning new things just as I am stay tuned for the next part and I hope you enjoyed this part if you did like subscribe all of those wonderful things just drive up the algorithm for me and yeah that's all thank you so much for watching and supporting this channel and everything thank you so much and I will see you in the next video bye bye