From demo to production: what is missing when the AI has to do real work

The demo goes well. Someone shows a language model, the technology behind tools like ChatGPT, doing something genuinely useful with company data. The room is impressed, and a decision gets made to roll it out. Six months later it is still not in production. This happens often enough to be a pattern rather than bad luck.

The demo was never the hard part

A demo runs once, on a case someone picked, with a person watching. Production runs a thousand times, on cases nobody anticipated, with nobody watching. Almost everything that makes the second one hard is invisible in the first.

The gap has a name. The harness is everything around the model: the tools it can reach, the rules it follows, the limits on what it may do, what happens when it fails, and how anyone finds out. The model is the engine. The harness is the rest of the car.

What the harness has to answer

Which tools, at what level. Every capability an agent has is a capability it has on a bad day too. Read-only where reading is enough. Write access scoped to the specific records the task touches, not the system.

What happens on failure. Systems time out and return nonsense. The interesting question is whether the model retries, escalates, or stops. Left unspecified, it will improvise, and improvising is exactly what you do not want on the failure path.

Who approves what. Some actions should never happen without a person: sending to a customer, moving money, deleting anything. Others would be smothered by approval. Deciding which is which is a business decision, not a technical one, and it is usually the conversation nobody has had.

How you find out it went wrong. Not whether it will, it will. Whether the third bad output gets noticed, or the three hundredth.

What it costs to run. A demo's cost is a rounding error. The same workflow across a department, every day, is a line in a budget. Worth knowing before, not after.

Why this gets skipped

Because none of it is visible in the thing everyone is excited about. The harness produces no demo. It is the reason the demo can be trusted to run unattended, which is a much harder thing to show in a meeting.

It is also unfamiliar work. It looks like plumbing, and the instinct is to treat it as something to add later. Added later, it usually means rebuilding the thing that was demoed.

A reasonable order

Build the harness around one narrow task first, and get that task genuinely running. Boring, small, real. The rules you discover doing it are the rules that will apply to everything after, and you will discover several that nobody predicted.

Then widen. A second task on an existing harness is a fraction of the work of the first, which is the whole argument for building it deliberately rather than by accident.

Next step

If something has been demoed in your organisation and has not shipped, the blockage is usually one of the five questions above. Naming which one is normally a short conversation.