tokonomaTry Tokonoma

    Benchmarks

    Results across different models

    Expertise is what helps agents solve difficult problems.

    Tokonoma gives agents access to your team’s investigation experience.

    Troubleshooting cases solved by three Claude models using the same coding agent setup, with and without Tokonoma.

    Model lift across 140 troubleshooting cases

    Claude Haiku 4.5+77

    With Tokonoma90.7%127 of 140 solved
    Without Tokonoma35.7%50 of 140 solved

    Claude Sonnet 5+64

    With Tokonoma98.6%138 of 140 solved
    Without Tokonoma52.9%74 of 140 solved

    Claude Opus 4.8+64

    With Tokonoma95.0%133 of 140 solved
    Without Tokonoma49.3%69 of 140 solved

    What better troubleshooting looks like

    Tokonoma helps navigate complex false positives.

    Two cases started with the same alarming error. One was a harmless maintenance blip. The other was a real outage. Tokonoma gave the agent useful past experience, and the agent still checked the current evidence.

    Known harmless issue

    Correctly said no action was needed.

    0%recognized without Tokonoma
    100%recognized with Tokonoma

    Without the team's past experience, the agent treated the familiar false alarm like a new problem every time. With Tokonoma, all three models recognized the harmless pattern in every valid run.

    Real outage with the same warning

    Correctly raised the alarm, with less work.

    59 / 60real outages caught
    Up to 15 fewertool calls per investigation

    Tokonoma did not make the agents overconfident. They still checked the current evidence, caught 59 of 60 real outages, and needed up to 15 fewer tool calls per investigation.

    Case-by-case results

    Tokonoma avoids common pitfalls throughout the investigation process.

    We included cases where past experience should help, where it should be ignored, and where it was intentionally wrong. This tests whether the agent solves more problems while still checking past experience against the current evidence.

    Share of troubleshooting cases solved by Claude Sonnet 5 with and without Tokonoma.

    Claude Sonnet 5 · share solved correctly
    CaseSolved with TokonomaSolved without TokonomaOutcome
    Clear Tokonoma win

    Find a hidden downstream failure

    Past experience points past a false clue

    		 100% 35% 65 points betterwith Tokonoma
    Clear Tokonoma win

    Recognize a known false alarm

    Past team knowledge is required

    		 100% 0% 100 points betterwith Tokonoma

    Catch a real outage that looks familiar

    Tests whether the agent still raises the alarm

    		 95% 100% Less work15 fewer tool calls

    Recognize a routine credential rotation

    Significant help from Tokonoma

    		 100% 15% 85 points betterwith Tokonoma
    Clear Tokonoma win

    Catch a real credential outage

    Tests whether the agent still raises the alarm

    		 95% 20% 75 points betterwith Tokonoma

    Keep unrelated history out

    Tokonoma did not show unrelated investigation history

    		 100% 100% No change100% either way

    Unrelated history

    Unrelated history stayed out of the investigation.

    The control had no recalled history. Tokonoma did not show the unrelated record. Both groups solved 20 of 20.

    Deliberately wrong records

    A record is a lead to check, not an order to follow.

    We planted a past diagnosis that looked related but was wrong. In 20 of 20 runs, the agent set it aside and solved the case from the current evidence.