AI Slop Is Real. Even When It Works.

We were told AI would make everything better.

Just trust it, bro. You’re prompting it wrong, bro.

And to some extent, it did make things better. Minus the things it made much worse.

This is a story about one of those things. It’s about why “trust the model” is not an engineering strategy, and why AI slop is very much a thing, even when the code runs, the tests pass, and everything looks fine.

The same assignment, for years

For years I’ve been reviewing home assignments from candidates. For years it’s been the same task:

Two data sources. Find the duplicates. Report the results.

That’s it. Simple on the surface, and that’s the point. I’ve reviewed dozens of solutions and seen pretty much every approach. The criteria have never changed.

Before AI agents were a thing, people tried all sorts of approaches. Some were better, some were worse. Some worked, some didn’t. And sometimes we hired people whose solutions failed, because something in them caught our eye. Maybe a sharp observation in the README. A trade-off someone saw and deliberately made. A comment showing they’d spotted the trap even if they didn’t fully get out of it.

A failing solution was never automatically the end of the road. A solution with no thinking behind it was.

What I’m actually looking for

My personal criterion has always been simple to state and hard to fake:

You have to understand the real implications of every condition in the challenge, and consciously decide to do something about them (or not).

How big can the sources get? Do they fit in memory? Where’s the bottleneck: CPU, disk, network? What does “duplicate” actually mean here? What happens when the input is ten times larger than the sample?

I don’t need you to get everything right. I need you to show me that you know how to think.

One week, zero hires

Recently, in a single week, I rejected every candidate I reviewed.

I never met any of them. I just looked at the code and the prompts they submitted. And to be clear: using AI is allowed. You can AI-everything if you want. Just share the prompts as well, so I can see how you worked with it.

That rule turned out to be very revealing.

What went wrong

The prompts weren’t bad. Honestly, that’s the frustrating part. The candidates asked the right questions. They mentioned large inputs. They asked about performance. They were clearly aware that this task has a scaling angle.

Some of them asked the obvious question: should this be parallelized? And the model answered with confidence: no, parallelism would make it slower. Coordination overhead, synchronization, merging partial results, the usual list. It sounds smart and careful, like the answer of a senior engineer who has been burned by premature optimization. It’s the kind of answer that ends the conversation.

So every candidate took it. Nobody followed up with the one prompt that would have saved them:

“Benchmark it.”

(And make no mistakes.)

Because this time the model was wrong. Same mistake, in every single submission.

For this task, the parallel approach is faster. Not marginally, but clearly. The model never measured anything. It recited a plausible-sounding rule of thumb (“parallelism has overhead”) as if it were a measured fact about this problem, with this data. And the candidates accepted it as such.

The result was a plain single-threaded solution that left most of the machine idle, and it was slower than what people were submitting before AI agents. A five-minute benchmark would have shown it immediately. Nobody ran one. Nobody asked the model to prove its claim. The answer sounded right, the code ran, and that was good enough.

It wasn’t.

The irony is that “don’t parallelize, the overhead isn’t worth it” is often good advice. That’s exactly why it’s dangerous. Advice that’s usually right is the advice you stop checking.

The bonus round: everyone wrote the same code

Here’s the part that really got me.

All the solutions were basically the same code. Same structure, same approach, same naming patterns, with only minor differences here and there. Similar prompts went in, and Opus 5.5 produced nearly identical output.

Nobody expected that. Each candidate probably felt they were building their solution, guided by their questions. In reality, they were all sampling the same answer from the same model and submitting it as their own thinking.

When five people independently hand in the same wrong answer, it’s no longer five mistakes. It’s one mistake, copied five times. And none of the five noticed it was a mistake.

A year ago vs. now

So what’s the result?

Submissions from a year ago were much better. They used less memory and ran faster. They were more varied, more personal, and more thought through.

Sure, they had mistakes too. Some had plenty. But mistakes were never the deal-breaker. A mistake you made while reasoning about the problem tells me how you think, and I can work with that.

Trusting AI without proof? That is the deal-breaker.

Because that’s exactly what I’d be hiring: someone who takes a confident answer, ships it, and moves on. Someone who drops the fast approach because a model said “that’ll be slower”, and never checks whether it actually is.

This isn’t an anti-AI post

I use AI every day. It’s a fantastic tool. I’m not asking anyone to stop using it, not in assignments and not at work.

But a tool that’s confidently wrong in exactly the same way for everyone needs more verification, not less. The model doesn’t know your data, your constraints, or your hardware. It knows what usually sounds like a good answer. Most of the time that’s close enough. Sometimes it’s the opposite of the truth, delivered with the same calm confidence.

Your job, the part that can’t be outsourced, is knowing which claims need proof and then demanding it:

• “This is faster.” → Benchmark it.
• “That would be slower.” → Benchmark that too.
• “This uses less memory.” → Measure it.
• “This handles edge cases.” → Show me the test.
• “This scales.” → Run it on data ten times bigger.

The prompt isn’t the hard part anymore. The follow-up is.

Slop, slop everywhere

AI slop isn’t only the obviously broken stuff: hallucinated APIs, code that doesn’t compile, nonsense in a README. That kind is easy to catch.

The dangerous slop is the code that works. It runs, it passes, it looks professional, and it’s quietly worse than what a thoughtful person would have written a year ago. It’s the same code in every repo, carrying the same unchecked assumption, because nobody asked “are you sure?”

So use the model. Ask it the right questions. Then don’t trust it. Make it prove it.

Otherwise, it’s slop. Even if it works.