When AI Gets It Wrong: What the Failures Taught Me More Than the Wins

There's a version of the AI adoption story that's entirely wins. Better outputs, faster thinking, sharper decisions. I've told that story on stage. It's not false. But it's incomplete.

The more honest version includes the failures. And the failures, it turns out, taught me more.

The confident wrong answer

Early in my AI-as-infrastructure experiment, I used an AI to research some market data for a client keynote. Specific figures, specific claims, well within the AI's apparent knowledge domain. It gave me confident, well-formatted, compelling statistics.

I used them in a draft. Then, in the verification pass I do before anything goes into a final deck, I couldn't find the source. Because there wasn't one. The AI had generated plausible-sounding numbers that didn't exist.

This is a well-documented behaviour. The term hallucination is actually useful, not because the AI is lying, but because it's generating something that feels real without the underlying reality to support it.

The lesson wasn't "don't use AI for research." The lesson was about the nature of trust with a new tool. You don't trust it the way you trust a sourced document. You trust it the way you trust a brilliant but occasionally overconfident colleague who sometimes fills gaps in their knowledge with confident-sounding extrapolation.

The limits that clarified the value

Beyond factual errors, I ran into limits that were more interesting than frustrating. Moments where AI couldn't do what I was asking, and the asking itself revealed something.

I tried to use AI to help me navigate a complex interpersonal situation with someone I work closely with. I gave it context, asked for perspective, pushed for nuance. What it gave me was technically reasonable and emotionally hollow. It could describe the dynamics. It couldn't feel the weight of them.

That's not a criticism. That's a clarification of what AI is and isn't. And having that clarification made me better at using it for the things it's genuinely good for, and better at recognising when I need something else entirely.

Why I think failure is the curriculum

In my workshops, I often say that organisations learn more from failed innovation attempts than from successful ones (we in the industry call that the INVALIDATION test), if they've built the psychological safety to examine the failure honestly. The same applies here.

The failures I had with AI over this period gave me a calibrated understanding of where to trust it and where to check it. Where to lean on it and where to stay the primary thinker. Where it adds signal and where it adds noise.

That calibration isn't something you can learn from reading about AI. You have to develop it through use, including the uncomfortable use where it fails you.

Today, I have built in fact-checking, backtesting, and safety harnesses into my global instructions, so my AI knows to ensure my work is solid. I don't take it for granted; I'll ask invalidation questions all the time. But it has taught me to be mindful.

Practical AI: Deliberately break it

This week, try to find the edges of your AI tool. Ask it something where you already know the answer, something specific, nuanced, and within your domain of expertise.

See if it gets it right. Push it further. Ask for a source. Ask it to go deeper. See where the quality degrades.

The goal isn't to catch AI out. It's to build your own calibration map, a working sense of where this tool is reliable and where it requires your verification and judgment.

That map is one of the most valuable things you can develop as an AI user. Understanding how to uncover this, and adapt will be the single biggest gap betwen a novis and an advanced AI user in the coming year.