Benchmark · Agent tasks

τ²-bench

A model working as a customer-service agent under a written policy, with tools and a simulated customer, in one domain at a time (airline, retail, telecom, banking), in text or by voice. Each domain is scored on its own.

Score
% of tasks passed on the first try (pass^1), higher is better
Run by
Sierra
Status
Current, newest result 29 October 2026
Updated
as submissions land
Data
the published file, MIT, read 20 November 2026
Method
taubench.com/

Results: text: banking knowledge

4 models we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1Kimi K3 B37.1%Independenteffort max, retrieval alltools, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-08-04source
2Gemini 3.1 Pro (preview) AA26.0%Independenteffort high, retrieval alltools, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-05-05source
3Inkling B25.0%Independenteffort max, retrieval alltools, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-08-04source
4Qwen3.5-397B-A17B B9.8%Independenteffort enabled, retrieval text-emb-3-large, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-03-02source

Results: voice: banking knowledge

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1GPT-Live BBSubmitted by the makereffort Backend: GPT-6 Astra (high), retrieval alltools, submission type custom, tau2 version 1.0.1, user simulator gpt-5.5 (xhigh)2026-09-16source

Results: voice-v2-entity

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1GPT-Live BB60.3%Independentagent provider openai_live, submission type standard2026-10-20source

Results: voice-v2-banking

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1GPT-Live BB33.2%Independentagent provider openai_live, submission type standard2026-10-20source

Results: voice-v2

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1GPT-Live BB79.9%Independentagent provider openai_live, submission type standard2026-10-20source

Results: text: telecom (legacy)

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1DeepSeek-V3.2 CSubmitted by the makertau2 version v0.1.3, user simulator DeepSeek-V3.22025-12-01source

Results: text: telecom

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1Qwen3.5-397B-A17B B97.8%Independenteffort enabled, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-03-02source

Results: text: retail (legacy)

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1DeepSeek-V3.2 CSubmitted by the makertau2 version v0.1.3, user simulator DeepSeek-V3.22025-12-01source

Results: text: retail

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1Qwen3.5-397B-A17B B84.4%Independenteffort enabled, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-03-02source

Results: text: airline (legacy)

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1DeepSeek-V3.2 CSubmitted by the makertau2 version v0.1.3, user simulator DeepSeek-V3.22025-12-01source

Results: text: airline

1 model we list, each at its best setting, best first.

#ModelScoreRangeWho ran itSettingDate
1Qwen3.5-397B-A17B B81.5%Independenteffort enabled, submission type standard, tau2 version 1.0.1, user simulator gpt-5.22026-03-02source

A rank is among the models we list with a result, not a place on the whole leaderboard, and a model offered by several hosts is shown once, on its own listing. These results don't move our grades. Data: τ²-bench, Sierra Research, MIT, MIT.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.