Require final approval for sensitive tasks - Automation Tip #6

Okay, automation tip number six, always require final approval from a human being for sensitive tasks. Now, this is the last automation tip I've recorded so far, at least until I come up with more to record. If you haven't seen the rest of these, I've basically made a playlist of automation tips that I learned after building tons of automations for myself and for my clients. And I'll put a playlist link or something in the description and you can go check out the previous ones as well the tip for today is to always require final approval by a human being and let me just show you some examples of some i guess horror stories you could call them that just show you why this tip is so important right when you don't have a manual approval step in the end of sensitive actions this is the kind of stuff that happens right some autonomous ai agent is going to delete your entire company's database delete all the backups and make it so that you never recover all of that sensitive data just because it hallucinated some bad output over here and the thing is things like these don't happen because the llm is being malicious or the llm received some bad prompt or something the prompt is very you know normal if i can just find it over here yeah it was working on a routine task and then it encountered some credential mismatch and decided that the best way to fix it is to just delete a railway volume like this is the thing with these machines right this is the thing with these llms they working on normal routine tasks you would never expect them to do something like delete an entire database but somehow they just decide that this is the best solution and then they just go implement it and the entire database is gone now and if this company just had a simple manual review step like hey this action is gonna go the ai is gonna take this action do you want to approve that or reject that if they just had a simple step like that in the you know automation this would have never happened and if that wasn't enough very recently just in july open ai hacked hugging face hugging face is one of the most like leading websites and platforms when it comes to AI development, AI models, and all that other stuff, right? So you know they take their security very seriously. And some OpenAI GPD model was able to just hack their website, break through all of their security protections. And the reason why it did it, you want to know the reason? Because the reason is fascinating. All OpenAI was doing was literally just trying to benchmark the new model on some cybersecurity evaluation, right? And apparently the evaluation was so fucking difficult that GPT thought that the best way to solve it is to just literally go and hack a real website and grab the answers Like the evaluation basically asked it a bunch of questions right i don know the specific questions but it asked it some questions and it just failed to compute them it wasn't intelligent enough to compute them so it thought that why not go and hack a website steal its data and inside that data will be the answers and so that's exactly what it did it wasn't a malicious prompt it was literally just trying to get a benchmark score on some cybersecurity eval and this is what happened. And in the spirit of true capitalism and not being left behind by your competitors, Anthropic did the same thing. They hacked three other websites during their cybersecurity evaluations. So I could show you a hundred more examples just like this, but the point I'm trying to make is you always need a human being to review the LLM steps, especially when it comes to sensitive tasks, right? Like I know these models are getting better and better and I mean even I myself am using them for more and more things and just letting them run autonomously but only for non-sensitive tasks. When I need the agent to do something really sensitive, I always review the action it's going to take before I approve it or reject it and I'll admit so far I haven't had the agent suggest some really drastic action like deleting a database or hacking some website. I haven't had that experience yet, all right, but But I still not going to remove the human review step I still going to always check the agent actions for sensitive tasks right Because you never know when this is going to break And when it does break, I don't want to go through the kind of experience that all these other guys are going through, right? So how do you implement this? Because this is very simple to implement, right? Let the agent do all of the non-sensitive work autonomously, right? Because most of the time, it's just going to be fine. And so it's worth it to let the agent do all of the non-sensitive tasks you've got. But when it comes to something really sensitive, right, something like, I don't know, changing some values in a database or something, always review the agent's actions and approve them or reject them, right? Heck, even if you let the AI agent do all the sensitive work, just make sure you have some step in there that allows you to review the agent's actions before making the change permanent right something like changing rows in a database or sending some email or message that you can't easily undo right or some other permanent action like that that is just really hard to undo and just make some sort of permanent change in your systems right always always always have a manual review step in those cases because that's the only way you can make sure the agent doesn't do something catastrophic like delete your entire production database Super simple automation tip, huge ROI to avoid fuck-ups, and very, very easy to implement. Try it out.