At noon on Tuesday, your finance team changes a rule: from now on, refunds over £75 need a manager’s approval. The refund request that arrives at five past twelve tells you a lot about how your company uses AI.
If every request goes to the model, the answer follows the new rule only if someone has already added it to the prompt, and you pay full price for a decision your team makes all day long. If your system reuses answers it gave before, the request costs almost nothing, and the refund goes through without the approval the new rule requires. If you retrain the model on the new policy, the answer will be right after the next training run, whenever that is.
You want three things from that answer. Right means the answer your team would give, with your rules and your exceptions. Current means it follows the rule as it stands today, from the moment it changes. Affordable means a decision your team settled long ago doesn’t cost a full model call every time it comes back.
The familiar approaches each reach two of the three at most, and the third is the price. Duplex reaches all three on the same work, because it learns from every result while the work happens.
Call the model every time, and it is right 59.8% of the time, with every decision paid for.
Calling the model every time pays for the same thinking again.
Calling the model on every request is the natural place to start, and for good reason. A model brings language, knowledge and reasoning to almost any question, it copes with things it has never seen, and it improves each time its maker ships a new version.
It also starts from nothing, every time. A refund exactly like yesterday’s is worked out from scratch and paid for again, so the bill grows with use and never shrinks as the work becomes familiar. And the model knows your team’s rules only when someone sends them along with the request: leave one out, and it decides by what it knew before.
At its best, then, calling the model every time is right and current, as long as every correction travels with every request, and it is never cheap. Keeping the corrections travelling is a job of its own: someone has to notice each change, add it to every prompt that needs it and take out the rule it replaced.
On a stream of 3,000 decisions in which the rules changed five times, the model on its own reached 59.8% accuracy, and only 2.3% on the decisions those changes affected. With nobody sending the new rules along, it went on applying the old ones, and paid for almost every decision as it did.
A decision reaches the model, which works it out from scratch, and the call is paid for.
A cache is cheap until the rules change.
A cache is the obvious answer to the bill. When a request looks like one answered before, it returns the answer it stored and skips the model call. On familiar work it is fast, cheap and, for as long as nothing changes, right.
What a cache knows is what was asked and what was answered, but not why. When the rule changes, nothing tells it which of its answers the change touched, and the requests look the same as before. So it goes on serving the old answers, confidently, until someone finds and clears each stale one by hand.
At its best, a cache is cheap and right on what it has seen before: two of the three, with current as the price. In the same test, the price came due. A cache lifted overall accuracy to 77.3%, and fell to 42.6% on the decisions the rule change affected.
Duplex learns the decision rather than the answer: the rule behind it, why it was right and where it applies. When the rule changes, it follows the new one from the next request, and everything else it learned stays as it was. On the decisions the change affected, Duplex was right 92.5% of the time.
Before the rule changes, a cache answers familiar requests correctly, and cheaply.
Duplex reaches all three.
Duplex is a second intelligence beside your AI model, and it changes who does what. The decisions your team has settled, Duplex answers itself, in milliseconds and with no model call. What is new goes to your AI, briefed with what Duplex knows about the case, and once your team confirms the result, Duplex learns from that too.
Each corner follows from learning while the work happens. The answers are right because Duplex learned them from results your team confirmed. They are current because a change applies from the next request, where retraining would wait for the next run. And they are affordable because a decision Duplex has learned needs no model call, so the more it learns, the less each request costs.
Over the same 3,000 decisions, Duplex beside the same model reached 94.6% accuracy, the highest in the test, with 90.3% fewer model calls than the model alone and 84.8% lower cost. A cache made fewer calls still, and paid for them in stale answers once the rule changed.
The trade between right, current and affordable was never a law of nature. It came from approaches that think again every time, never think again, or think again only when a training run comes round. An intelligence that learns from every result doesn’t have to choose: it knows what your team has settled and follows what has changed, and your model is paid only for what is new.