The Bottleneck Is Somewhere Else

The biggest problem with frontier labs right now is that tokens cost so much it is hard to tell what you are getting back. The ROI is unclear, and most people spending the money have not done the math.

On the surface, speed looks like the answer. If a job that used to take a long time now finishes in an afternoon, the spend looks obviously justified. Faster is cheaper, cheaper is better, end of story.

But speed only matters when the thing being sped up was the bottleneck. Say a piece of software used to take ten months to write and now takes ten days. You collapsed the writing time from months to days. Yet if it still takes ten months to get that software in front of customers, none of that time you saved actually moved the date it ships. The code was never the slow part. The real constraint sits somewhere else along the line.

This is an old lesson. Eliyahu Goldratt built The Goal around it: a production line moves only as fast as its slowest station. Speed up any other station and you just pile up inventory in front of the bottleneck. Software development can absolutely be made faster now. Whether that makes the whole pipeline faster is a separate question, and the honest answer is that it depends on the situation.

More features falls into the same trap. Cranking them out with an LLM feels productive, but every feature you add makes the product harder to grasp - for the people building it and the people using it. The more there is, the harder it gets to say what the product actually does and who it serves. Generating more of them is rarely the win it looks like. The bottleneck is somewhere else.

Peter Steinberger recently posted that he spent $1.3 million on tokens in a single month - reviewing pull requests, scanning commits for security issues, that kind of thing. You could read that as a sign of how much work got done. What goes unmentioned is whether the $1.3 million was worth it. The work happened, no doubt about that. But did it matter? Would skipping most of it have changed the outcome? Would the same money handed to ten strong engineers to read through the codebase of OpenClaw have produced better feedback on how to restructure the thing?

Nobody knows. That is the point. It was a flex, and it is a clean example of how muddy the ROI of tokens really is.

The bill is easy to see. The return is anyone's guess. The bottleneck is perhaps somewhere else.