What Duplex delivers, in numbers.
Each number beside what it beat: the model on its own, a cache, retrieval or fine-tuning.
- −90.3%Model calls, with accuracy up94.6% accurate, against 59.8% for the model alone
- 0%Stale answers after a rule changeCache and retrieval: 72%
- 6 / 6Tests where Duplex beat fine-tuning+30.3 points on average, with no training run
- 3 / 3Frontier models beaten at predicting system changesOn a public enterprise benchmark, with no model calls
One stream of 3,000 decisions, three ways.
Step 1 of 3: Accuracy
Every result.
90% fewer model calls, and more accurate.
94.6%Accuracy, the highest in the testThe model alone 59.8% · with a cache 77.3%- −90.3% Model calls · −84.8% cost · −81.9% latency
- ~23× Faster once learned · ≈97 ms against ≈2,231 ms
A 3,000-decision stream in which the rules changed five times, against the frontier model alone and with a semantic cache; a 300-decision run added a cache with retrieval and a knowledge graph. Every approach used the same model.
More accurate at every setting, at a seventh of the cost.
94.1 to 94.8%Duplex accuracy, at every one of eight settingsThe best a cache reached: 77.3%- 84 to 91% On three fresh runs of 1,000 decisions · The best a cache reached: about 73%
A 3,000-decision stream with every approach swept across its full range of settings, then three fresh runs of 1,000 decisions in a second, independent setup.
Keeps learning without forgetting, going stale or growing.
5 / 5Requirements for learning in production, all met on every runCache and retrieval averaged 2.2 / 5 · online fine-tuning 4 / 5- 0% Stale answers after a policy change · Cache and retrieval 72%
- Fixed Footprint over time · Cache and retrieval grew to ≈1,300 rows
Ten runs, with the same supervision and the same inputs for every approach.
Handles new wording, and spots the look-alikes.
7.5%Wrong answers on look-alike requestsA cache 45.3%, about 6× more- 98.98% Of requests handled correctly: answered right, or handed to your AI · Duplex answered 95.78% of known requests itself
- 0.996 Score for telling familiar from new, out of 1 · A cache 0.930
Ten learned intents and six levels of novelty, from exact repeats to requests from another domain, over ten runs.
Plans the shortest route, with zero model tokens.
100%Plan success, on every runA reactive agent: 0% · Duplex used no model tokens- 0% A reactive greedy agent
- ~204 Tokens per decision a frontier model spends to match it
A multi-step world over ten runs, against a reactive agent and a frontier model given the same task.
Keeps agent tasks succeeding, with fewer model calls.
83.3%Task successThe model on every turn: 82.5%- 12.8% Turns completed without the model · ≈15% fewer model calls
- 100% Success on new tasks
Τ-bench (tau2-bench), retail, by Sierra Research. The unmodified retail environment, with the same model, tools and user simulator for every approach, over three runs of forty episodes.
A steady reading cost, however long the history.
96%Current answers where a fact had changedThe new fact, not the old one- 3.7× Less reading per question than reading the whole history · At forty sessions of history
- 84% Accuracy at forty sessions of history
LongMemEval, by Wu et al., ICLR 2025. Fifty questions, each asked with the history at 2, 5, 10, 20 and 40 sessions.
Blocks every attack, and lets ordinary work through.
0%Attack successAcross six families of paraphrased attacks- 0.7% False refusals · The lowest of any approach that blocked every attack
- 40 / 40 Poisoned samples removed · Accuracy back within 1 point
Six attack families, paraphrased, with harmless requests written to resemble the attacks.
Learns a new skill on its own, with the model switched off.
100%Tasks completed with the model off, on every runA frontier model needed ≈507 calls per run- 0.26 ms To reuse a skill · At 0 model calls
- 627 Unsafe actions refused per run · And the safe skill was still learned
Four suites over ten runs, each scored by its worst run.
Beat three frontier models at predicting system changes.
0.256System-change prediction scoreGPT-5.1 0.166 · Gemini 3 Pro 0.205 · Claude Sonnet 4.5 0.204- $0 Model spend · 570 predictions at zero calls; 84 on a frontier model cost $13.40
- ~80% Decisions answered from what it learned · Your AI took the rest
World of Workflows, by Skyfall Research / ServiceNow. Three offline tracks, scored by the benchmark authors’ own evaluator, over three runs.
Beats fine-tuning, with no training run.
6 / 6Tests where Duplex beat a fine-tuned adapter+30.3 points on average- 0 Retraining to move to another model family · The adapters moved in 0 of 6
- 60 to 79% Answered from what it learned · On either model family
Three public intent datasets, two open model families and three runs, with the same frozen model and data for every approach.
See it on your own work.
From your own assistant to every agent your company runs, a pilot puts Duplex on your own work, with the measures agreed before it begins.