You Know the AI Is Wrong. You'll Do What It Says Anyway.

The GPS said to turn right. You'd driven this route a dozen times. You knew, with genuine certainty, that the right turn ended in a construction dead-end that had been there for three months. You turned right anyway. Researchers call this automation bias — and most people do a version of it dozens of times a day.

Or maybe your version looks different. An AI tool suggests a sentence that's factually off. You've worked in this field for seven years and you can feel it's wrong. You paste it in anyway, maybe with a slight edit, maybe without. Or the algorithm recommends a product that makes no sense for your situation, and something in you doesn't quite trust it, but you click add to cart because, well, it must know something you don't.

This isn't a personality flaw. It's not laziness or intellectual cowardice. It has a specific research paper behind it, and a mechanism that operates even — especially — when you're trying to think carefully.

What Kathleen Mosier and Linda Skitka actually found

In 1996, two researchers published a paper that should have fundamentally changed how every organization deployed technology. It didn't. But it should change how you deploy it in your own life.

Kathleen Mosier and Linda Skitka, in their foundational paper "Human Decision Makers and Automated Decision Aids: Made for Each Other?", ran a series of experiments in which participants performed complex, cognitively demanding tasks — the kind where checking multiple sources of information carefully was genuinely necessary to get the right answer. Half the participants worked with a decision aid: an automated system that flagged issues and made recommendations. Half worked without one.

The results were startling in their consistency.

Participants with access to the decision aid made more errors, not fewer. And the errors fell into two distinct, troubling patterns. First, omission errors: when the automated system failed to flag a real problem, participants were far less likely to catch it themselves than the participants working without any aid at all. The presence of a system designed to help them actually impaired their independent vigilance. They'd stopped looking as carefully because something else was looking for them.

Second, and worse: commission errors. When the automated system gave a demonstrably, clearly wrong recommendation, participants regularly followed it anyway — overriding their own correct judgment to comply with an incorrect automated suggestion. In some trials, participants had information in front of them that directly contradicted the system's recommendation. They followed the system.

Read that again. They had the correct answer. They had the evidence. They threw it out because a machine said otherwise.

Mosier and Skitka called this automation bias: the systematic tendency to over-rely on an automated or algorithmic source, manifesting both as reduced vigilance when the system is silent and as deference to the system when it contradicts your own better judgment.

The mechanism: why effort plus authority is the perfect storm

Here's the part that most summaries of this research leave out, and it's the part that makes it genuinely actionable.

Automation bias doesn't appear equally across all tasks. It intensifies specifically under two conditions that almost always occur together: the task is effortful, and the automated source appears authoritative.

Think about what that means. You'd expect that the harder a task is, the more carefully you'd use all available information — including your own judgment — to get it right. The research says the opposite. When a task is cognitively demanding and you have access to a seemingly reliable automated source, your brain treats the source as a cue to reduce its own independent processing. Not because you're lazy. Because that's actually an efficient strategy, most of the time. If the machine is usually right and the task is hard, outsourcing judgment to the machine is a reasonable shortcut.

The problem is that this efficiency-seeking persists even when the machine is not right. And even when you have private evidence that it's not right.

The "authoritative source" piece matters enormously. Mosier and Skitka found the bias was amplified when participants perceived the decision aid as coming from a credible, expert, or reliable-seeming system. In 2026, think about what that means: AI tools don't just give you recommendations. They give you confident, fluent, citation-dense, well-formatted recommendations. They feel authoritative in a way that a casual colleague's suggestion doesn't. Every design choice — the clean interface, the instant response, the absence of visible uncertainty — reinforces the perception that the source knows what it's talking about.

Which means the technology that's supposed to augment your thinking is, under exactly the conditions where you'd most benefit from your own careful reasoning, doing the opposite.

Decision Fatigue: Why Your Morning Wrecks Deep Work

The distinction that matters: this isn't groupthink

It's worth being precise about what automation bias is not, because people conflate it with other cognitive patterns they've heard of.

Irving Janis's groupthink concerns what happens inside a human group when the pressure for consensus suppresses members from voicing dissent. That's a social dynamic: people silence their own views to preserve group harmony. Lee Ross's naive realism concerns how confident people feel in the objectivity of their own perception — the conviction that you're seeing the world straight while others are biased.

Automation bias is neither of those. It's not a social phenomenon requiring other humans. It's not about overestimating your own accuracy. It's specifically, narrowly about what happens when a non-human, algorithmic, or automated source enters the picture. The research found that a single individual, alone, with no social pressure whatsoever, and with private evidence in hand that the machine was wrong, would still defer to the machine at a striking rate.

This distinction matters practically. The traditional advice for groupthink — speak up, create psychological safety, designate a devil's advocate — doesn't address automation bias at all. You're not trying to overcome peer pressure. You're trying to overcome your own quiet, efficient, entirely involuntary tendency to offload cognitive work to a system that happens to be wrong on this particular occasion.

Why this is more relevant now than it was in 1996

Mosier and Skitka's original experiments used flight simulators and industrial monitoring systems. In 1996, most readers wouldn't interact with a decision aid more than occasionally in specialized professional contexts.

In 2026, you interact with automated decision aids dozens of times before lunch.

Your email client ranks which messages deserve attention. Your navigation app routes you based on criteria you never specified. Your document editor flags grammar. Your search engine decides which results are most relevant. Your AI assistant drafts, suggests, summarizes, and recommends — fluently, confidently, and at high speed.

None of these systems are infallible. Every single one has failure modes that are opaque to you. And every single one, according to the research, is quietly reducing the vigilance you'd otherwise bring to the same questions.

The counterintuitive implication here — the one that's hard to accept — is that having access to an AI tool for any given decision doesn't neutralize your cognitive biases. It adds one. If you think using an AI makes your reasoning more objective, Mosier and Skitka's data suggests you should think again. You've introduced a system that your brain is already treating as more reliable than your own careful evaluation, regardless of whether it's earned that trust in this specific case.

Seth Godin put it in a single sentence

Seth Godin published a short, quiet piece called "Eager to give up agency" that captured something Mosier and Skitka spent pages demonstrating with data: people are glad to defer. Not reluctant. Not coerced. Glad.

This is the uncomfortable observation hiding underneath the research. Automation bias isn't just a vulnerability — it's often experienced as relief. Handing judgment to an authoritative source reduces uncertainty. It reduces the discomfort of being wrong in your own name. It reduces effort. And the systems we interact with daily are designed, explicitly, to provide exactly that relief — to feel like a capable, trustworthy partner who's already thought things through so you don't have to.

The question isn't whether that's appealing. It obviously is. The question is what the actual research says happens to your decision quality when you accept the offer without checking it first.

Why Your Day Runs on Autopilot — The Real Science

How to catch your own automation bias before it catches you

This is not an argument against using AI tools, algorithms, or any other form of automated assistance. It's an argument for a single, specific habit that the research directly supports.

Form your view before you consult the tool.

That's the core of it. Before you read the AI's draft, write your own first sentence. Before you check the algorithm's recommendation, decide what you'd choose without it. Before you let the navigation app route you, recall what you know about the roads. Then consult. Then compare. The sequence is what matters.

Mosier and Skitka's data shows the bias operates at the point of initial exposure to the automated source. Once you've read the recommendation, the influence is already active — your independent evaluation is already compromised. The only position that's genuinely protected is the one you formed before you saw the output.

Here's how to make that practical:

1. Name the decision before you open the tool. Before launching the AI assistant or checking the algorithm's output, write down, in one sentence, what you already think or know. You don't have to be right. You just have to commit your own view to language before external input arrives.

2. Keep a decision log for anything consequential. When a significant decision involves an automated recommendation, record your independent judgment first, then the tool's output, then what you actually decided. Reviewing these logs over time reveals your own automation bias pattern — which domains you defer in, which you don't.

3. Ask "what would I conclude if the tool said the opposite?" This is the fastest reset. If the automated source had recommended the other option with equal confidence, would your evaluation change? If yes, you don't have independent evidence — you have a preference for the tool's output dressed up as your own reasoning.

4. Notice when "the machine knows something I don't" becomes a reflex rather than a reasoning step. That phrase is sometimes true and worth saying. It's also the exact cognitive move that automation bias leverages. The research doesn't say machines don't know things. It says people stop checking whether the machine is right on this particular question, in this particular moment. Those are different.

5. Match your review depth to the stakes. Not every automated recommendation needs independent verification — your email client's spam filter doesn't require a pre-formed view. But for decisions with real consequences — career, financial, relational, health — the evidence says independent evaluation isn't optional. It's the whole point.

What Are Mental Models and How to Use Them

The Design Your Evolution angle

There's a version of this research that's paralyzing: you can't trust your own judgment, you can't trust the machine, everything is compromised. That reading is wrong, and it misses what the data actually says.

Mosier and Skitka didn't find that independent human judgment is unreliable. They found it's underused — bypassed, not replaced — when an automated source enters the room. Your judgment, exercised first, before the algorithm speaks, is exactly as capable as it was before decision aids existed. Possibly more, because you know something the machine doesn't: your specific situation, your constraints, your history with this exact kind of problem.

Designing your evolution through this research means protecting something you already have. Not developing a new capability. Not installing a new system. Just refusing to let your own best thinking arrive after the recommendation, when the influence is already active, rather than before.

One question worth sitting with: the last time an algorithm or AI tool recommended something you weren't sure about, what did you do? Did you form your view first — or did you let the tool form it for you?

That's not a rhetorical question. It's the data point Mosier and Skitka were actually after.

---

Sources:

• Mosier, K. L., & Skitka, L. J. (1996). Human Decision Makers and Automated Decision Aids: Made for Each Other? In Automation and Human Performance: Theory and Applications. Lawrence Erlbaum Associates. Semantic Scholar
• Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/001872097778543886
• Cummings, M. L. (2004). Automation bias in intelligent time critical decision support systems. AIAA 3rd Intelligent Systems Conference. https://doi.org/10.2514/6.2004-6313
• Godin, S. (2026, July 29). Eager to give up agency. Seth's Blog. https://seths.blog