Syncing projects with Google Drive - Part 2 (Config Files and Drive API)

Alright guys, welcome back to this Go Drive Singer project playlist. This is part 2. In part 1, we basically built all of the OAuth connection and authorization stuff, where we connected to the Google Drive API, grabbed the access token, saved it, so on and so forth, and just asked it to list the first 10 files in my Google Drive. And it worked perfectly fine, as you can see. It just connected to the API and saved the credential file, the token.json, over here, and it gave me the first 10 files in my Google Drive. Awesome! Now, in this video, we're going to first of all do a bit of cleanup over here because I don't like my main function having so much, so much of the business logic. I try to separate all of those things out into different files and functions and just organize everything better instead of having all of the business logic of my application in the main function. I just don't think that's a good practice. So we're going to do a bit of cleanup first, and then hopefully, if the video doesn't get too long, we're going to also create a config struct and start to get configuration from the, from a file. Let's just say from a configuration file where we will put stuff like, hey, which folders do I want to synchronize? Which folders or files do I want to ignore? And so on and so forth, right? We'll put all those things into a config.json file, and then our application will read that and then upload the files and folders that we want to upload based on those rules that we specify in the configuration. All right, so in order to do this, right, we need to first be able to handle that config file before we actually implement all of the file uploads and folders and stuff, right? So let's get started. First of all, what I want to do is go all the way down here and create a new struct, and we'll call this Drive. And this will have two properties. First of all, it will have the config, and it will have the API. The API will just be a drive.Service pointer, and config will be a new struct called Config. There we go. I'll configure this in just a bit. But for the Drive, for the Drive struct, let's create a new function, call it NewDrive. This will take a config pointer, and it will return a Drive pointer as such. There we go. Return Drive and pass in the config over there. And there we go. Now let's move this config to its own file over here. Call it config.go. Package main. All this is in package main. Now, before we actually write the config file and everything, let me finish the Drive struct first. So let's add a pointer receiver over here. This will be called Connect, and it will take the force refresh boolean and also a context over here, which will be a context.Context object. Now, to actually implement this, we can go up here to the main function and basically just copy all of the stuff that we have over here. Let's take all of this, put it in here, and say that the d.api equals service, right? d.api is just this drive.Service field in this Drive struct over here, and we're just going to assign that to this service, right, that we get from drive.NewService. This will help clean up our main function as well, and yeah, that's all we need to do for now. Now we can just go up here to the main function again, and we can just say Drive drv equals NewDrive. Pass in a config. I'm just going to say config equals a new Config object and just pass that in over here because we haven't implemented the config stuff right now, right? So create the new Drive, run the Connect function, force refresh, let's say true for now. Context, let's pass in, let's create a new context up here, and it will just be context.Background for now. And boom, there we go. By the way, if you guys don't know what context is used for in Go, you should probably go learn it because it is very useful, especially in concurrent programs. You can basically use context to cancel your, cancel different tasks and just manage different tasks when they being run in different goroutines and stuff. So instead of context.Background, you can also have context.WithCancel, for instance, and what this will allow you to do is you can basically cancel a task if you want to, right? You could say something like, and this is a completely side lesson, by the way, but you could could say something like context.WithCancel, and this would return a context and also a cancel function. You could then, what you could do is you could pass in this context into some other function, I guess, pass in the context, and this other function would run inside a goroutine, let's say, which means it's running concurrently and you have no control over it anymore. But in this main function, you could run the cancel function, and then this some other function would have to just stop its execution right there. You can also add cleanup routines and stuff like that, and yeah, this is a very fascinating package to learn and to use. I'm not going to go too deep into those things. I'm just going to use context.Background for now, but you should know this is very useful to have. Anyway, yeah, learn the context package in Go. Anyway, this srv variable, we're going to change this to drive.api, and that's all we need to do to make this work. Let's quickly make sure that our stuff is running over here, and it is perfect. As you can see, it gave me the files that we need, the files, the first 10 files in my Google Drive over here. Now, it didn't ask me to refresh my token by going to that browser URL and everything, right? Why? Because we've already saved the token over here in token.json, and until it expires, we can just keep using it over here. Once it does expire, then we'll get an error, and we'll need to refresh the token after that. But for now, this works. So with that done, let's continue implementing the config struct over here. This is going to take a bunch of fields over here. First of all, it's going to take a base folder, which will be a string. This will be where we store all of the projects that we upload and synchronize to Google Drive, because look, we use Google Drive to store all kinds of things, right? We don't want all of those things mixing up with our programming projects, so we need one specific folder to store all of our programming projects in. In my case, I'm just using this programming folder over here, right, which will hold all of my programming projects, and that's what this base folder is going to do. So that's the first thing we need in our config. Then a list of folders, which will be a list of pointers to the folder object. And let's implement that as well real quick. Let's say folder, which will be a struct. This will have a name, which will be the token, the folder name, and then a path, which is the absolute path to the folder on your local machine that we can then open up and look at the files inside that folder and up and then upload all of them to Google Drive. Okay, then it's going to have an ignore, which will be a string list. This is basically all the files that we want to ignore. And then it's going to also have the use git ignore over here, which will be a boolean. And then let's also add a struct tag over here. Let's say JSON use underscore git ignore, just to tell the JSON package what the field name will be in the config file. And this use git ignore will basically allow us to read the git ignore file in every single project, and it will just grab all of the ignored files and folders from that git ignore file and just skip all of them in the upload. So this will be very convenient to use because we don't have to specify all of the files and folders that we don't want to upload to Google Drive. We can just add a git ignore file just like we do with all of our other projects, right? And in this case, I don't want to upload the client.json. I don't want to upload token.json. And that's all we need to do, right? Now we can just use this git ignore file in this program as well, in this application we're building. And we don't need to specify individually which files and folders we want to ignore in every single project, right? So this is a very convenient little feature that we're going to build in this application. Now let's create a function called read config. This will take a file path, which will be a string, and it will return a pointer to the config object. Let's say file and error equals OS dot open. Pass in the file path. Handle the error as well. Let's say log dot fatal f cannot read config file at and then print the file path and then let's also print the error. Pass in both of those things over here and then obviously a new line at the end. And of course, defer f.close. And then let's create a new config object by saying c equals pointer to config, and then let's use the built-in JSON object. Clear new decoder. Pass in the file and say decode. Pass in C over here, the config, and then we can return C over here. And by the way, this is the beauty of using JSON as the configuration language over here. You will recall, if you remember our, like, if you watch the Redis clone that I built in Go in just like the past couple of videos, you will know that in Redis you didn't actually use JSON for configuration files, right? You had to use their own specific language, and so parsing the config was a lot more complicated in Redis than it is in this custom project that we're building, right? If you use JSON for your config files, you can literally just add this one line over here, and the built-in JSON package will take care of everything for you. So with that done, let's also create a config.json file over here and let's add some config. So first of all, the base folder, in my case, this will be programming. In your case, this can be whatever you want: projects or code or whatever, right? Ignore, which will be a list of strings. For my purposes, first of all, I'm just going to use node modules, which is a Python virtual environment directory name, and then dot git, so we don't back up all the source control stuff, right? Use underscore git ignore. It's going to be true because this is going to be a super useful feature of this application. And finally, folders. This will be a list of objects. Each object will have a path for now, and for now, I'm just going to back up the current directory, which is home hassan programming drive syncer video. Let's paste that in over here, and that is our config file all done and ready. By the way, this ignore, this is a global ignore. This is the stuff that you want to ignore in every single project, right? So we don't have to repeat that in every single project, in every single git ignore file and stuff, right? This stuff will just be skipped in every single project that you synchronize in this application. And then for the individual, the individual projects where we want to ignore some specific files, we can use the git ignore file. So with that done, let's go back over here to the config config.go file and try to read this. So let's go to main. Let's, instead of using an empty config struct over here, let's say read config and pass in the file path, which in our case is just config.json, right? And we can remove the ampersand here because this is already going to return a reference. Let's actually print this as well. There we go. Now let's run this and see what happens. Okay, so it's nil right now. I wonder why. Oh, oh, oh, I forgot to pass in a reference over here. Just make sure you add this ampersand over here in the decode function call, just to turn this into a reference, and then it should work. So as you can see, this is the config struct, the config object. It got the base folder programming. It's got the ignore stuff and so on and so forth, right? Okay, awesome. By the way, this memory address you see over here, this is just the folder list. Since this is the list of pointers, there's a list of pointers, and pointers are just memory addresses in Go. That's what it printed over here. But this is a list of folders, right? This is the first folder that we added over here, so just keep that in mind. But yeah, this works. We can read our config. Awesome. But there's one final tiny issue that we need to solve. You will notice that we're not actually able to, like, let me just show you. Let's print the list of folders over here. We don't need to print the config over here anymore. Let's just print a list of folders, and you'll see what I mean. Okay, so that didn't work because this is still a memory address. But let me try, instead of that, let me just try looping over the whole thing. What I basically wanna show you is that you can get the folder path over here, but you won't get the folder name, which is something that we really need. So we're going to have to get that ourselves. But let me just show you what this looks like first, passing the F, which will be a pointer, and print the path and the name, and probably separate both of them with, let's say, an is equal to sign or something, and see what that gives us. Yeah, see, so the name field is empty, and we get an empty string over here, is equal to, and this is the path. So it can read the path just fine, but there is no name. We need a folder name over here, and we can grab that by just looking at the path over here and just taking this final folder name over here, right? Like, look at the entire path and just take this final folder name, and that will be the folder name, right? So let's do that. Remove all of this, and I'm going to create a new function down here. Let's say `addFolderNames`. This will take the config again. Let's loop over all of the folders again, `range c.Folders`. Let's say if the folder name is equal to an empty string, you'll see why this check is important in just a second. We'll say `f.Name` equals, you can use the `filepath.Base` function, and this will return the last element of the path, which is exactly what we need in this case. Let's pass in the folder path over here, and that's all we need to do. Now let's call this function over here, `addFolderNames`, pass in the config over here, and then try printing all of the folders inside this config. Let's do the same thing we did before: print the name, separate them with an is equal to sign, and print the path. Let's run this again, and this time you can see. And by the way, let me probably, I should probably just comment all of this out because this is not doing any good for me right now. There we go. Okay, so I just commented out all the Google Drive upload stuff, and there we go. Okay, so now we're getting the file name, the folder name, and then the folder path as well. Okay, so this works now. Why do I check whether it's an empty string? Well, because it's only going to be an empty string if the user didn't specify the folder name in the config. So if, for instance, we don't want to use this name, we can also do something like name, and we can say custom name over here, and this should read that name. And yeah, there we go. See, so we can use a custom name for this folder as well if we want to in the config. But if we, let's say, we forget to do that, then the default is just going to be this folder name over here, right? And that's going to work perfectly fine as well. So I think I'm gonna stop the video over there. Again, this is getting way too long. 24 minutes. The last video was like 46 minutes of recording, right? I still haven't edited that, but it'll probably end up being like a 30, 40-minute video, which is insane. So yeah, let me try, just, just let me try and keep the videos to a very short length, like a 15, 20-minute video per video. And yeah, for now we're done with this. We built this config struct. We also built this drive struct over here to organize stuff better and just remove some of the business logic from the main function. We're going to do a bit more cleanup in the next video, and we're also going to implement the files and folders upload stuff in the next video, or at least we're going to start implementing that because there's a lot of stuff to do in that section as well. And yeah, there's a ton of stuff to do. So for now, we built this config struct, and it is working perfectly fine, as you can see. In the next video, we're going to try and build the upload functionality for files and folders. So stay tuned for that. New videos coming very, very soon. And yeah, that's all. Thank you for watching. Like, comment, subscribe, share. Thank you for following along this new series, and I hope you learned something, and I hope you have a good time here. We're going to— we're going to continue building all sorts of cool, interesting, complicated projects as much as we can. And I'm going to be sharing all of the work I do on this YouTube channel. So if you haven't subscribed, then make sure you do that. And yeah, thank you for following along. I will see you in the next video. Bye-bye.