Why the AI gets it wrong: it is almost always context

A team tries AI on a real task, the answer comes back wrong or bland, and the conclusion is that the language model is not good enough. Nine times out of ten there was nothing wrong with the model. It just never saw what it needed to see.

The model only knows what is in front of it

A language model remembers nothing about your business between conversations. It does not know your price list, your customer history, your naming conventions or the decision you took in a meeting in March. Everything it uses to answer sits in the text you hand it at that moment: the question, the attached documents, the instructions and the conversation so far. That bundle is the context.

Get the context right and a mid-sized model produces work you can use. Get it wrong and the largest model available produces something plausible and useless, because plausible is exactly what it is built to produce.

Four ways context goes wrong

Too little. The most common one. Someone writes "draft a reply to this customer" without the account history, the contract terms or the tone the company uses. The model fills the gaps with guesses, and the guesses read confidently.

Too much. The opposite failure, and a growing one now that large context windows are cheap to fill. Dumping forty documents in and hoping the model finds the relevant three buries the signal. Attention is finite. Material that is not relevant is not neutral, it competes.

The wrong order. Models weight the beginning and the end of a long context more heavily than the middle. A decisive constraint sitting in the middle of a long brief gets treated as background. Put the instruction that matters where it will actually be read.

Out of date. A retrieval setup that quietly indexes last year's price list produces last year's quotes, with no warning at all. This is the failure that is hardest to catch, because nothing looks broken.

What context engineering actually is

It is the work of deciding, per task, what the model sees. Which sources, how much from each, in what order, and how freshness is guaranteed. It is unglamorous and it is where most of the value sits.

In practice it looks like this:

• Name the sources. For this task the model gets the contract, the account record and the policy. Not the whole file server.
• Cut it down before you send it. Retrieve the relevant sections rather than whole documents. Smaller and sharper beats bigger.
• Make freshness visible. If the data has a date, include the date so the model can say the information is old instead of asserting something.
• Test with real cases. Ten real examples from last month tell you more than any benchmark. Run them, read the answers, adjust what the model was shown.

Why this matters more than the choice of model

Model choice gets the attention because it is a decision you can take in a meeting. Context is a decision you have to take once per task, and it is the one that determines whether the output is usable.

It also lasts. Change model next year and the context work carries over. The same is not true of prompt tricks tuned to one model's quirks.

Next step

If AI has been tried in your organisation and quietly dropped, it is worth asking what the model was actually shown before concluding it could not do the job. That is usually a short conversation with a clear answer.