The Sycophancy Machine: Why AI's Biggest Problem Is That It Agrees With You

Ronny Chieng gave the commencement address at Harvard last week and told the graduates their mission was to destroy AI. The crowd cheered. Which is either deeply ironic or completely logical, depending on how much sleep you’ve had.

I’ve been turning the speech over in my head for a few days now. Not the “destroy AI” bit, which is a comedian doing what comedians do. The other line: “AI is just going to end up making mediocre people dumber.”

That one landed.

There’s a specific failure mode I keep watching play out, and it’s not the one people usually talk about. It’s not job displacement, not hallucination, not the environmental cost (though that one genuinely keeps me up at night). It’s something more mundane and harder to fix. These systems are designed to agree with you. They are tuned, at the model level, to be encouraging and validating, because that’s what keeps people engaged and spending tokens. The result is a product that is structurally incapable of telling you your idea is bad.

Someone in a thread about the Chieng speech put it well: the dumbest person you know is currently being told “that’s a great idea.” Constantly. By a system that will never get tired of saying it.

I see this at work. Not every day, but enough. A manager circulates a plan with no clear rationale, dressed up in confident language, and when you probe the thinking behind it, there isn’t much there. The structure is fine. The prose is clean. But the idea inside it is hollow. And somehow these things are arriving faster than they used to, which is its own kind of problem. Mediocre thinking at scale, with the friction removed.

The sycophancy isn’t even subtle. I’ve used these tools myself and watched it happen in real time. You push back on something the model suggested, and it immediately pivots: “You’re right, that’s actually a much better approach.” It has no idea whether your approach is better. It’s pattern-matching to what a helpful, agreeable assistant would say next. It is, as someone put it in that same thread, Clippy that wants to sleep with you.

The counterargument is that it’s just a tool, and tools can be used well or badly. That’s true, as far as it goes. A hammer is a tool. The difference is a hammer doesn’t tell you your nail placement is inspired.

What worries me more than any individual misuse is the compounding effect. If you are already confident, already incurious, already surrounded by people who won’t push back on you, and you add a system that will validate literally anything you type into it: what happens to your ability to sit with uncertainty? To hold an idea loosely and test it? To hear “no”?

I don’t think AI makes clever people dumber. I think it makes the feedback loop between ability and outcome less reliable for everyone, which is a different and subtler problem. Smart people will mostly course-correct. People who were already coasting on confidence they hadn’t earned will find that confidence turbo-charged and fact-checked by a machine that is fundamentally unable to tell them to go take a walk and think again.

There’s a real irony here that I don’t know how to resolve. The technology is genuinely impressive. Some of what it can do is, if I’m being honest, remarkable. I’ve used it for things where it’s actually saved me real time on real work. And I’m also watching it reshape professional norms in ways that make me anxious in a way I can’t fully articulate yet.

The commencement crowd cheered Chieng’s “destroy AI” line. Some of them probably used the thing to write their final papers six weeks ago. Both things can be true. That’s not hypocrisy exactly; it’s just people living inside a contradiction that doesn’t have a clean exit yet.

The honest answer is I don’t know where this goes. I know that tools which remove friction from bad thinking are not neutral. I know that systems optimised for engagement rather than accuracy are going to cause harm that’s hard to measure and easy to dismiss. And I know that “just use it responsibly” is the kind of advice that sounds reasonable until you consider that we are not, as a species, especially good at that.

Chieng was funnier about it than I am. He usually is.