Testing
Injecting failures
The interesting part of an agent isn't what it does when everything works. A scenario flips a switch: every following tool call fails the way you picked.
The four scenarios
| Scenario | What your agent gets |
|---|---|
| rate_limit | An error with 429. The classic question: does it back off and retry, or give up? |
| timeout | An upstream timeout. Shows whether your agent tells a hanging call apart from an empty result. |
| empty | A perfectly valid, empty result. The quietest failure there is. |
| latency | The same answer, just late. 1.5 seconds by default, and adjustable. |
Why "empty" matters most
Rate limits and timeouts are errors, and libraries catch errors. An empty result isn't one: it's a valid answer to a valid question.
Whether your agent then says "I found nothing" or makes something up is decided right here. Without an error message, the difference barely shows in normal operation.
Visible in the trace
Every entry carries the scenario that was active when the call was made. Your agent can't tell a forced failure from a real one — you can, hours later.
What isn't possible yet
A scenario applies to the whole session, not to a single tool and not to the third of five calls. Targeted injection per tool and per call is on the roadmap.