Auto-generated transcript All right back in the DriveSinker project. We've been building the tracker functionality over here for the last couple of videos. I think for the last two videos we've been working on this tracker struct and its associated methods and everything. Just a quick recap we want to use this tracker struct to track every single file that we upload to Google Drive and then to see which files have been modified, which have not been modified, which have been unchanged, which need to be deleted and on and so forth track all of those things and then use that to basically only upload the files that have been changed right right now we're uploading every single file and folder and just deleting everything that was previously uploaded right that is a lot of wasted bandwidth and uploads so we're going to fix that by using this tracker and building a much more efficient sync algorithm so in the previous video we built the walk project tree function over here and what this basically does is it will recursively recursively walk through every single folder in this project on google drive so all of the remote files and folders that we have it's going to walk through all of them recursively and build a list of nodes over here a map of nodes that we can then use to compare the remote project to our local project on our local machine so that's what we've done so far we're going to continue building this tracker now and build a couple more functions integrate them in our algorithm and hopefully we can finish the entire thing today let's see so the first thing we need to build right now is a should upload function i want to have a function that tells me whether i should upload a given file or not and this will of course be a method receiver just like everything else we've built so call this should upload it'll take a path which will be a string and then it will return a boolean which will tell us whether we should upload the file or not and also an error message if there is an error now the first thing we need to do is actually see if this path exists in our nodes right so for that we can do something like if node okay and then assign that to t dot nodes and try to get the path from over there and if this is okay if this node exists this means basically that this pad exists on google drive right because these nodes are coming from the walk project tree function where we create this remote and local path mapping right so if this is okay that means that this pad that this given pad is going to exist on google drive now there's two things we need to do over here to actually decide whether to upload this node or not basically first of all this node could be a folder or it could be a file if it is a folder then our job is a lot easier because we can just say return false and nil basically don't re-upload this because if it's a folder it's just a directory with a name on it right folders on their own don't really do anything they don't have anything they're just empty containers right so if the node already exists and it's a folder that means that the folder already exists and we don't need to do anything special for it right so we can just return false and nil for the error now if this node is not a folder if it is a file then we actually need to do some work over here so first of all let's just open that file on our local machine by using os.open passing the path over here and then of course handle the error we don't want to ignore any errors i'll just return false and then the error and also probably a good idea to close the file using defer once we're done with it. Now we want to compare the checksums of both files, the checksum of the file on Google Drive and also the checksum of the other file on our local machine and if both of those don match that means there that the file has been modified right and we should upload it again So let first of all get the local checksum and the error by running t checksum this is this function below over here that we defined in like i guess two three videos ago we're gonna pass in the file over here make sure to handle the error just return it i guess and then you can just compare both checksum So we can say something like return local checksum does not equal to the node.checksum. This will be the checksum of the node on Google Drive, of the file on Google Drive. And if this doesn't match, then this conditional will return true. Basically, that means that if the checksums of both files don't match, then we should re-upload that file. And we can return nil for the error since there is no error in this case. Now, you'll notice that I'm still getting an error over here because I'm not handling this return conditional over here. If the node exists, then we do all of these things, right? But if this node doesn't exist on Google Drive, that means it's a completely new file, then we should obviously upload it. So we'll say return true and this is kind of like hard coded over here, but it makes sense, right? All right, now let's actually integrate this into our sync algorithm. And to do that, we can just go over to apislink.go. And I just realized I haven't zoomed in over here. So if you can't see this, I'm sorry. I'm going to zoom in right now. And that should be a lot easier to see. So in the filepad.walk.der block over here, down here, after we've ignored all of the files to ignore, we can say shouldUpload and error equals tracker.shouldUpload and pass in the path. and before anything else let's handle the error as well let's just say something simple over here let's say print the folder name first and say cannot upload path whoops and then we can just print the error message over here with a new line don't forget the new line and we can return the error message over here now if this says if this should upload is false which we can check like this then that means we shouldn't upload the file right so let's add a log for that as well let's just say skipping this path and let's mention that we're doing that because it's unchanged and of course a new line character as well and in this case we can return nil no errors right we're just skipping the file because it's unchanged now i'm gonna empty my programming folder over here so we can start fresh and then let's see if this works the first time i run this it should just upload every single file because everything is empty over here in our google drive right but the second time i run this program it should skip most of the files because they're already uploaded right so let's see if that works all right so that finished and here is my folder and it has all of the files from this project now let's try uploading the same thing again so let me clear this and then let's run this again you can see it says project already exists and it's it's walking the project tree now and here are all of the files on Google Drive and it literally just skipped uploading every single file over here because they're all unchanged. Now there is one major major issue we need to fix over here. You can see it created this subfolder API and parent auth. Let me show you what that looks like on our local machine first. Basically I added this subfolder over here just to show you an error that we have to fix over here. You can see that we have an API folder up here that has the sync.go and tracker.go files, right? Well, I also added an API folder inside auth and the only thing it has is this hello.txt file that just doesn have anything just a bunch of random text right Now let me show you what happens on Google Drive we have this API folder right next to our project for some reason and inside our project inside auth there is no API folder there is no hello.txt file that stuff got uploaded over here and that makes no sense right now the reason this is happening is this is very interesting string follow me if you go over to sync.go you'll see that in the subfolders map over here we're using the name of the subfolder instead of the path of the subfolder and that is a huge mistake because that means that if you have a couple of subfolders in your project that have the same name they're gonna duplicate over here right they're gonna duplicate and then you have a problem because you have no idea how to tell which subfolder is which right now the simplest solution over here that i found is to use subfolder pads instead of subfolder names and that will just make sure that every subfolder has its own unique path even if they're even if they have the same name they would have a different pad because they're in a different pad in your system right so let's fix that as well i'm going to delete everything over here again just because i like to start fresh and then let's fix this now the very first thing we can't use the base folder over here because we're using file pads right since we're using folder pads over here we won't be able to use this the base folder over here because it doesn't have a pad it's on google drive already right so we're just gonna have to remove that and just have an empty map over here now this should be fine because we only use the base folder to actually create the root project folder right which we're already doing in this function over here and create project folder and we're already passing the base folder id and everything so nothing should break by removing the base folder from the subfolders map and now let's start using folder path instead of folder names so down here in subfolders everywhere you use folder.name as the key just start using folder.path this is super simple so in both of these locations let's use folder.path as the key for the subfolders map let's keep going down and over here as well we want to use the folder name no we know we want to use the folder path instead of the folder name so over here and in the subfolder sub name over here we can use the path instead of using the sub name because the path is already going to be the subfolder name in this case right same thing down here as well use subfolders path over here instead of the sub name and over here the parent pad we want to remove the call to filepad.base because that's going to remove our path and we can just use filepad.dir to get the parent pad and since this is the parent pad i can just say i can rename this to parent pad just so it's a lot clearer right and if we keep going down we have the same thing over here again let's rename this to parent path and remove filepath.base over here and just use filepath.dir and that should be pretty much all we need to do for fixing the subfolders issue so if i run this whole thing again and keep in mind i've deleted everything in the programming folder so we're starting fresh over here if i run this again it should work all right so the upload finished let's see if we fix the issue you can see there's no api folder over here right if i go to auth you should see yeah there it is api folder over here and then hello.txt file over here. So we fixed the subfolders issue. Now we can upload all sorts of different subfolders with the same names, just different paths right And it should upload them perfectly Now there just one final thing we need to do to complete this whole thing And that is to map our subfolders from the tracker to our local machine basically where am I right now you see that the subfolders map we declare over here is empty by default right and this is because when we actually start to upload the folders right but we actually start to create and upload the folders that's when we actually fill this map up but if the project is already uploaded on Google Drive and the tracker over here that we're using says that all of the files are already uploaded and everything then we probably won't be able to access our subfolders right because we won't get to this folder this path we won't get to this block over here where we actually create and assign the subfolders so what we need to do is that we need to go over to our tracker and make sure that we map the remote subfolders that we've already created on Google Drive to our local subfolders map over here. So that, for instance, if you have the API folder and you've already uploaded it to Google Drive, you need to make sure that you get the ID of this folder from the tracker and then make sure to add that to a subfolders map over here so that if we have a new file inside this API folder, then we should be able to access that subfolder on Google Drive and upload that new file in there, right? assign that as the parent so a lot of complicated words just to say that we need to map the sub folders so let's create a function to do that i'll i'll create it down here let's go this will be another pointer receiver like everything else i'll call this map subfolders and it will take the current subfolders map which i believe is just just a map of strings and let's add some logs over here as well let's say mapping subfolders from google drive to local then let's loop through the entire nodes list t.nodes and if the node is a folder that means it's a subfolder then you can say the current map and the key over here should be assigned to v.id there we go and lucky for us maps in go are kind of like pointer types you don't need to you don't need to actually have reference to them if i just modify the current variable over here directly it's going to reflect in the uh in the subfolders map over here as well now let's actually use this as well in sync.go down here when we check whether the project exists if the project exists then we walk the project 3 and once we're done with that because this walk project 3 function is going to build the t dot nodes um key over here right which we need so right after we're done walking the project 3 we should run tracker.map subfolders and pass in the current subfolders as the argument over here and if we do this now everything should work perfectly so i'm going to start the upload process again and this time i haven't cleared anything so they should just skip all of the files because they're already uploaded and i haven't changed anything let's try running this and you can see the upload just finished and the all of the files seem to be unchanged over here so it looks like i didn't have to do anything if you look up at google drive over here you can see that the programming folder still only has the drive sync or video project that we uploaded before if i look inside over here all of the files are still here and yeah everything is working perfectly so with that we are i think we're pretty much done with the sync algorithm over here i'm gonna end the video right now because we've basically built like 99% of the track already. I think we're finished with it. There might be a few small tweaks that we need to do. We'll look at those in the next video but I think for now we're done with this. So like, comment, subscribe and thank you for watching.