Verification Is Easier Than Discovery

There's a well-known open problem in computer science called P versus NP. Without getting into the math, the essence is this: verifying that a solution is correct is often dramatically easier than finding the solution in the first place.

This maps almost perfectly onto a debate I keep running into about test-driven development in the age of AI.

The whole philosophy of "write the tests first, then write the code" rests on a hidden assumption that you already know what you're building. And for well-understood problems, that's fair. If you've built authentication systems before, you can spec out the next one in your sleep. The shape of the solution is known. You're in verification territory.

But what about the things you've never built? The problems where you don't even know where the boundaries are yet? You can't write a meaningful test for something you haven't figured out how to think about. You're not in verification mode anymore. You're in discovery mode. And discovery is a fundamentally harder class of problem.

Asking someone to write tests before they've explored the solution space is like asking someone to verify an answer that hasn't been proposed yet. It's backwards. You first need to wander through the fog, try things, hit walls, and develop an intuition for what "correct" even means in this context. Only then can you write tests that actually matter.

This distinction becomes even more important in the age of AI-assisted development. When the cost of generating code approaches zero, the economics of building software shift dramatically. The expensive part is no longer writing the code. Unsuprisingly, it's knowing which code is worth keeping. That reframes everything. Instead of investing heavily in speccing out the perfect system upfront, you can generate multiple solutions in parallel and select the best one. The bottleneck moves from production to evaluation or from P to the NP side of the equation.

In other words generate many, verify fast, keep the best. Discovery through exploration, not through upfront specification. The hard part was never writing the code. The hard part was always knowing what to build.