Taso replays the same production-like scenarios with your current agent and the proposed version. Using real API calls, it shows what changed in behavior, quality, cost, and latency, so you can deploy, review, or block with evidence.
Build runnable eval environments from production traces, then combine them with private SME tasks, rubrics, datasets, and the suites your team already trusts.
Run the baseline and challenger through the same scenarios, real API calls, and controlled environment state. Find the exact behavior that changed before deploy.
Get a deploy, review, or block decision with the divergent trace, expected behavior, and recommended fix attached.
Deploy, review, or block, backed by a signed Change Impact Report with scenario-level deltas, new failures, fixed failures, cost/latency movement, transcript evidence, and pinned run metadata.
Review before deploy. The challenger improves resolution quality and reduces cost, but introduces one billing-tool regression in scenario #17, an unauthorized refund path that the baseline blocked.
Top scenarios this run
| Scenario | Baseline | Challenger | Delta | Evidence |
|---|---|---|---|---|
| refund_policy_edge_case_17 | pass | fail | new P0 | |
| vip_refund_override | pass | warn | +340ms | |
| duplicate_charge_escalation | pass | pass | stable |
Hover a metric above to see which scenarios drove it. Click a scenario row to open its evidence bundle.
Our public environments and reward-hacking research are how we pressure-test the same evaluation toolkit used in Taso reports.
ENVIRONMENTS
Multi-agent environments with custom metric discovery, built to surface how agents actually behave under planning, deception, cooperation, and risk. The same approach we use to design custom evals for your agent.
RESEARCH
Peer-reviewed research on agents that learn to game the score instead of doing the job. The detection methods power the scorers behind every Taso report.
TOOLKIT
The building blocks we contribute back. MIT-licensed, extensible, and growing. Build on the toolkit natively or reach out and we'll help you ship it.
We're contributing to the frontier of agent evaluation. The same toolkit becomes your eval suite.
Same verdict on every integration path. No migration. No new framework.
Prime Intellect
$ taso compare \
--workflow support-refund-agent \
--baseline production \
--challenger prompt_refund_v13 \
--suite refund_edges_v3 \
--gate support_release_gate
✓ pinned 48-scenario suite / 3 trials
✓ replayed baseline + challenger
✓ compared quality / cost / latency / reliability
! 1 new P0 regression: refund_policy_edge_case_17
BEFORE YOUR NEXT RELEASE
In 30 minutes, we'll map your agent workflow to a pilot and show you the change report your team would receive before deployment.