Getting an AI workflow working is satisfying, which is probably why it’s so easy to forget that “working” today doesn’t mean you can stop checking it tomorrow.
Heather Baker has seen that problem from the inside. She built and sold an agency, later ran the company that acquired it to £14m revenue, and now writes The Humans in the Loop and runs The AI Edit.
In this guest post, she looks at what happens after AI gets switched on, when the model changes, the business moves on or the system simply stops without anybody noticing. 👇🏻
If you hired someone and never looked at their work, you’d call that negligence.
It’s surprisingly easy to do the same thing with AI. Something gets set up, tested and switched on. It works, so everyone moves on.
Three months later, the model may be different and the process around it may have moved on. The information it relies on may no longer be current, or the workflow might have stopped without producing the kind of error that gets anybody’s attention.
You only find out when somebody eventually notices. We’re putting more AI systems into businesses without always deciding who owns them once they’re live.
New here? Millennial Masters shares practical lessons from founders on building better companies and dealing with the parts of the job nobody prepares you for. Subscribe for the next one.
The model can change underneath you
When you build an AI workflow, you test what comes out, decide it’s good enough and move on. But not every model name points to something fixed forever. OpenAI’s chat-latest model, for example, points to the latest Instant model used in ChatGPT. OpenAI says the underlying model snapshot is regularly updated.
We saw another version of this when OpenAI retired GPT-4o from ChatGPT in February 2026. GPTs using retired models were automatically moved to newer equivalents. OpenAI was clear that its API wasn’t changing at that point, so this wasn’t a case of every Make or Zapier automation being silently switched.
Something you tested and approved can still end up running on a different model or version later. The output may be perfectly fine. Unless somebody checks it again, you don’t know.
The technology doesn’t have to change for the same problem to appear. Sometimes your business changes instead.
Your process changed, your AI didn’t
The production process for my Insider’s AI Briefing ran through a series of automations inside Notion. They took me half a day to build and were worth every minute, until I changed the thing they were automating.
I moved the session from Zoom to Substack Live. The way I set it up changed, but the automations didn’t. They carried on asking me for Zoom links when what I needed was a Substack Live link. They were doing exactly what I’d told them to do. The instruction had simply expired, and nothing in the system could tell me that.
I know they need updating. Fixing them takes time I haven’t found yet, so I’m running on substandard automations and I’m the person whose job it is to notice. There’s a harder version of this problem when the workflow stops doing anything at all.
The workflow stopped and nobody knew
I run a Zap that watches a folder in Outlook and turns anything I drop in there into a task. Then I migrated my email, which reset the Zaps attached to it, and that one stopped.
I didn’t know. There was no error message because, from Zapier’s point of view, nothing had failed. It simply wasn’t being asked to do anything anymore.
Three weeks later, someone chased me about a guest post I owed him. I went looking and found the task sitting in the Outlook folder where I’d left it. There were nine others behind it.
This is one of the hardest failures to catch because there’s nothing obvious to investigate. A wrong answer will usually reveal itself eventually. Missing work can sit there until the person waiting for it asks where it is.
A system can also keep working while the thing it knows slowly goes out of date.
Nobody was keeping it current
A client had a custom GPT trained on the company’s tone of voice. It was good, and the team used it every day for client-facing work. Then the person who built it left.
The GPT still worked and sounded roughly right. The problem was that the company kept changing while the GPT stayed where it was.
Nobody had decided who was responsible for keeping it current because they’d treated it as something that could be set up once and left alone. It never stopped answering, so there was no obvious moment when somebody had to question it.
What sounded exactly right in January was merely adequate by August, but it still sounded just as confident. Keeping a system current still doesn’t guarantee the output is right.
Plausible isn’t the same as right
AI produces work that often looks finished before anybody has properly checked it. The language is fluent, the formatting looks clean and there may be nothing obvious telling you to stop. That makes it easy to trust too quickly.
Even Deloitte got caught by this. It partially refunded the Australian government after a report produced using an Azure OpenAI model contained fabricated citations and a made-up court quote. The problems weren’t caught before publication. A University of Sydney researcher spotted them afterwards.
If work like that can get through Deloitte’s checks, looking finished clearly isn’t enough. And when it goes wrong, the AI doesn’t get embarrassed. You do.
So who owns it?
For most small businesses, this doesn’t need to start with another hire. It needs a name against each system.
Somebody should know what it was built to do, check what it produces often enough to notice when that changes and have the authority to switch it off.
For every AI system running in your business, ask:
Who owns this?
When did they last look at what it produced?
What would have to happen for them to notice it had stopped being good?
If nobody’s name is against it, it isn’t a system you’re running. It’s a system that’s still switched on.
👤 Heather Baker is the creator of The Humans in the Loop and founder of The AI Edit, helping leaders understand and use AI in their businesses. She previously built and sold an agency before becoming CEO of the company that acquired it and growing the business to £14m revenue.
