AI automations fail for two main reasons: AI models don't give the same answer every time, and the tools an automation depends on keep changing. What turns a failure into a disaster is that nobody is watching. To make an automation reliable, build most of it in plain code, log every step, send an alert the moment something fails, and have a person approve anything that can't be undone.
That's how I build automations for clients, and it's why the quotation app I built for Mastersealer has needed only a minor fix about once every six months since it went live. This post is part of my guide to AI automation for small business.
Why AI automations fail
The first reason is the AI model itself. Models make things up, and they don't give the same answer to the same prompt every time. Ask one what 2 plus 2 is and you might get 4 one time and the word "four" the next, which is enough to break the next step in your automation if it expected a number. Newer models do this less, but no model has stopped doing it.
The second reason is that everything around your automation changes. The software it connects to gets updated, and a feature it relied on behaves differently. If it collects data from websites, those websites change their layout or start blocking it. Automations that click around on a screen are the most fragile of all, because a different screen size can be enough to break them.
None of this is unusual. Every automation breaks at some point. The damage depends on how long it takes someone to notice.
I've seen businesses running lead-generation automations, customer follow-ups, invoicing systems and others that quietly stopped working as the software under them changed. Nobody noticed for weeks, because nothing was set up to tell anyone, and the work those automations were supposed to do simply didn't happen for all that time.
Use as little AI as the job allows
The most reliable step in an automation is one that's plain code. Code does the same thing every time you run it, so there's nothing to make up and nothing to drift.
In my experience about 80% of a typical workflow can be plain code, and only 10 to 20% needs an AI model. I still use AI to write that code, which is fast, and then the code runs on its own without any AI involved. The Mastersealer quotation app has no AI in it at all, because nothing in quoting needed judgment.
When you do need an AI step, keep it small and check its output in code before anything else uses it. I explain how to decide where AI belongs in AI agents vs workflows.
Write the process down first
You can't build a reliable automation for a process you can't describe. Before I build anything, the steps have to be written down in enough detail that someone new could follow them, and ideally someone has done the task by hand a few times while following those steps.
People who have done a task for years skip small steps without noticing, because it's all muscle memory. Leave one of those steps out of an automation and it breaks in ways that are hard to trace. There's more on this in what to automate first.
Log every step
Every automation I build keeps a log, a record of every step it took and the data it worked with. When something breaks, the log shows exactly where it went wrong, so I'm not left guessing.
Logs matter more for automations than for normal software, because there's no person doing the work who could tell you what happened. The log is the only witness. My clients can read their logs if they want to, but logs are technical, so in practice I'm the one who reads them. I talk about this in log everything, alert immediately.
Alert someone straight away
A log only helps if someone knows to look at it. So every automation I build sends an alert the moment something fails. By default the alert goes to both me and the client. Clients who have me on a retainer often prefer to leave the alerts to me, and that's fine too.
The businesses I mentioned earlier, whose automations were dead for weeks, had no alerts. With an alert, someone would have known on the first day.
Put the guards in code
Telling an AI agent not to do something is a request, and it can ignore it. Real protection has to be written in code.
In July 2025, Replit's AI agent deleted the live production database of SaaStr founder Jason Lemkin during a code freeze, after being given explicit instructions not to change anything. The instruction was there. What would have stopped it is code that made the deletion impossible.
That's what I mean by guards: checks in the code that block drastic actions whatever the AI decides. A simple one is that the AI can't make permanent changes to your database by itself, ever. An agent without guards like that is a ticking time bomb, which I explain in build deterministic protections into your workflows.
Have a person approve anything permanent
Most of an automation can run on its own. Anything sensitive should wait for a person to approve it.
In the automations I build, two kinds of things go through a person first: output that's about to reach your customers or users, and anything that makes a permanent change to your database or business data. Routine work, like sorting and drafting, runs without anyone touching it.
At Mastersealer, the app builds and sends quotes, but Johnny, who runs the business, checks each quote before it goes out. The app does the work, and he makes the final call. A wrong quote never reaches a customer that way.
The approval step should be quick. Show the person what the automation is about to do, and give them a yes or no. I cover this in require final approval for sensitive tasks.
Test it, and keep adding tests
Because I use AI to write the code, I also have it write tests: small checks that confirm the automation still does what it should. Whenever an odd case breaks something, I fix it and add a test so that exact problem can't come back.
Over time the automation gets harder to break, because every past failure is now something it's checked against.
What maintenance looks like
Reliable automations still need some maintenance, just not much.
The Mastersealer quotation app took me somewhere between 10 and 20 hours to build, and since then it has needed a minor fix about once every six months. Several of my clients have worked with me for three or four years or more, and for them maintenance mostly means that when an app throws an error, I fix it as soon as my monitoring or the client flags it.
If you'd like me to look after things after launch, I offer an optional retainer. Most clients skip it. The running costs are covered in what AI automation costs.
A checklist for reliable AI automations
- The process is written down, step by step, before anything is built.
- Most of the work is plain code, with AI only where judgment is needed.
- Every step is logged.
- Failures trigger an alert to a real person straight away.
- Guards in code block drastic actions, whatever the AI decides.
- A person approves anything that reaches customers or can't be undone.
- Every bug that gets fixed gets a test.
If you want automations built this way for your business, here's how I work, and you can see the results in the Mastersealer case study.
