back to reflections

Be Your Own Customer

When you build a product you know too much about it, and that knowledge blinds you to how it actually feels to use. Being your own customer is the cheapest way to get the cold, outside view back.
Petko D. Petkovon a break from CISO duties, building cbk.ai

When you build a product, you know too much about it. You know which button does what and which edge cases to step around out of habit. All that knowledge quietly blinds you. Your customer shows up with none of it and uses the product cold.

Being your own customer is the cheapest way to get that cold view back. The moment you sit down and use the thing to get real work done, the seams show. Bugs nobody tested for show up, and rough edges that looked fine in a demo start costing you time every day. You also notice the gap between how you imagined people would use it and how it actually gets used.

At ChatBotKit we lean on this hard. We build our own agents on the platform, and they now run close to half of our operations. When something is slow or awkward, we feel it before any customer does.

We do not wait for customers to build on top of us either. We ship our own products and put them in front of real users. Formshare, Afterdark, Rook, and a handful of others all run on the same platform we sell. There is also labs.chatbotkit.com, where we park the stranger ideas and early experiments. Everything there is built for us, on the same SDKs we hand to customers.

A few of the things this has surfaced:

  • Runaway agents need real guardrails. We burned a couple of thousand dollars on a single mistake before we built tighter ways to get notified and shut an agent down mid-run.
  • Cheaper models need more harness. Our agents used to run on frontier models. Today they all run on open-source ones, often ten to twenty times cheaper, once we accepted that the missing intelligence has to be made up for with a stronger harness around the model.
  • Some SDK patterns only showed their flaws once we depended on them. Building real products on the same APIs exposed places where the developer experience was rougher than we wanted, so we reworked those patterns and added more support.

And the list keeps going.

So by the time anyone else writes a line of code against those SDKs, we have already found the parts that break. The customer gets a product that survived its harshest user first, which is us.