What's missing from most "AI workflow" conversations is ownership, not oversight. Someone specific needs to be accountable for reviewing outputs on a schedule, the same way someone owns a P&L line. Curious how she'd suggest assigning that ownership when the whole point of automating the workflow was to remove a headcount line in the first place?
I suspect we’ll see a few businesses discover they automated the task and accidentally automated away the person who knew when the task was going wrong.
If the whole point is to remove a headcount, then you need to find something that can adequately replace that headcount. Just because AI exists doesn't mean it can do that.
Naming an owner answers who's accountable, not how they'd notice something drifted. Heather's Zap never threw an error, it just quietly stopped being asked to do anything, and her client's GPT didn't sound any less confident as it went stale. A calendar check from a named owner wouldn't catch either one. A canary would catch both, a known input with a known correct output run on the same cadence as the real work, that fails loudly the moment its own answer drifts. That doesn't need a headcount line, just an alert somebody's watching.
Yup agree with you, and the way we handle it for clients is escalation, not just an owner: the canary's what triggers it and if the person watching doesn't react, it goes to ops (us in this case), and we step in, either contacting them directly, or, if it's part of our contract, validating the output ourselves. That's what actually closes the loop, the alert has a name on it same as the automation does, otherwise it would be just two unwatched processes instead of one :)
Totally agree with you, a human keeping watch matters regardless, the canary's not a replacement for that, just gives the person something concrete to check instead of waiting for your Zap to go quiet or the GPT to go stale while still sounding sure of itself.
AI workflows need ongoing monitoring, clear accountability, and regular review because a system that works today can quietly become unreliable as models and processes change.
That's the trap exactly — the person who knew when the task was going wrong was doing invisible work that never showed up in a job description. Thanks for the link, I'll dig into it.
What's missing from most "AI workflow" conversations is ownership, not oversight. Someone specific needs to be accountable for reviewing outputs on a schedule, the same way someone owns a P&L line. Curious how she'd suggest assigning that ownership when the whole point of automating the workflow was to remove a headcount line in the first place?
I suspect we’ll see a few businesses discover they automated the task and accidentally automated away the person who knew when the task was going wrong.
I've had another guest sharing a similar example recently, but from a different angle: https://millennialmasters.net/p/ai-human-judgement
If the whole point is to remove a headcount, then you need to find something that can adequately replace that headcount. Just because AI exists doesn't mean it can do that.
Naming an owner answers who's accountable, not how they'd notice something drifted. Heather's Zap never threw an error, it just quietly stopped being asked to do anything, and her client's GPT didn't sound any less confident as it went stale. A calendar check from a named owner wouldn't catch either one. A canary would catch both, a known input with a known correct output run on the same cadence as the real work, that fails loudly the moment its own answer drifts. That doesn't need a headcount line, just an alert somebody's watching.
A canary with nobody responsible for the cage sounds suspiciously like another AI workflow waiting to become somebody’s problem. 🫣
Yup agree with you, and the way we handle it for clients is escalation, not just an owner: the canary's what triggers it and if the person watching doesn't react, it goes to ops (us in this case), and we step in, either contacting them directly, or, if it's part of our contract, validating the output ourselves. That's what actually closes the loop, the alert has a name on it same as the automation does, otherwise it would be just two unwatched processes instead of one :)
Or it just needs someone thinking and being on guard to notice.
Totally agree with you, a human keeping watch matters regardless, the canary's not a replacement for that, just gives the person something concrete to check instead of waiting for your Zap to go quiet or the GPT to go stale while still sounding sure of itself.
AI workflows need ongoing monitoring, clear accountability, and regular review because a system that works today can quietly become unreliable as models and processes change.
That's the trap exactly — the person who knew when the task was going wrong was doing invisible work that never showed up in a job description. Thanks for the link, I'll dig into it.
A workflow with nobody's name on it is really just something switched on, not running.
And “switched on” is probably enough to make everyone assume somebody else is looking after it. 🤷🏼
Rogue agents everywhere!
Plausible indeed isn’t the same as right.
And that's the problem with AI Evals - they can only pull one lever: Increased output "plausibility."
This is what worries me too. Convincing output that happens to be wrong can travel surprisingly far before someone notices.