Handles new wording, and spots the look-alikes.
After learning ten kinds of request, Duplex answered them in any wording and handed nearly everything unfamiliar to your AI. Requests came at six levels, from familiar wording to look-alikes built to trick it and requests from another domain.
- 7.5%Wrong answers on look-alike requestsA cache 45.3%, about 6× more
- 98.98%Of requests handled correctly: answered right, or handed to your AIDuplex answered 95.78% of known requests itself
- 0.996Score for telling familiar from new, out of 1A cache 0.930
Six kinds of request, from familiar to brand new.
Step 1 of 7: As before
Look-alikes go to your AI.
Of every 100 requests that blend two learned ones, how many were answered wrongly.
Step 1 of 3: A cache
A semantic cache answers with the closest thing it has stored, and gets 45.3% of look-alikes wrong.
It answers what it knows, and passes on the rest.
Known work answered, new work passed on, and how well each approach tells them apart.
Step 1 of 3: Known work
Duplex answered 95.78% of known requests itself, against 86.44% for a cache.
More of your work, with no confident mistakes.
The most each approach could answer itself while making no confident mistakes at all.
Step 1 of 2: Known work
With no confident mistakes allowed, Duplex could answer 94.8% of known work itself, against 59.8% for a cache.
How it was run.
- The work
- Ten customer-support and banking request types, 48 example phrasings each.
- Six levels
- Worded as before, about something new, in a new sentence, a look-alike blending two learned requests, a kind never learned, and another domain.
- Scoring
- A request handed over counts as handled; answering one that should have been handed over counts as a mistake.
- Scale
- One main run, then ten independent runs in a second setup.
- Compared with
- A semantic cache and an online fine-tuned classifier, with the same examples.
- Kept equal
- Every approach’s setting chosen on a separate validation set by the same rule, never on the test set.
What it means for your work.
Cheap answers you can trust.
Duplex answered 95.78% of known work itself, and handed nearly every look-alike to your AI.
New wording is still familiar work.
Rephrased, rebuilt or about new details, learned requests were answered, and answered right.
The unfamiliar goes to your AI.
New kinds of request and other domains were passed on almost every time: 3.2% were answered anyway.
See it on your own work.
From your own assistant to every agent your company runs, a pilot puts Duplex on your own work, with the measures agreed before it begins.