← Notes

01 / Run limits

When should an agent stop?

A failed run can still produce useful work.

· 2 min read

What the teams report

Stripe limits Minions to two rounds of continuous integration (CI) checks. It then returns the branch to a person. More attempts cost time and compute. [1]

Dropbox uses a different limit for Deflaker, its tool to repair unstable tests. It carries notes and test logs between attempts. It stops after a successful fix or five attempts. [2]

“A turn counter is not a progress detector.”

DoorDash, on a repeated request that did not advance the turn counter. [3]

DoorDash added deadlines for each agent. A soft deadline requests verified findings. A hard deadline stops the agent. [3]

01AttemptMake a change
02CheckInspect the result
03Stop or retryApply the run limit

At the limit → Save the work and explain the failure.

Our illustration of a possible control flow. Each source uses different checks and limits.

Our observation

The limit needs a useful exit

An attempt limit and a time limit address different failures. Neither limit explains what the next person needs.

A useful exit can include the current work, failed checks, and a reason to stop. This is a design proposal, not a shared implementation.

The cases do not establish one correct retry count.

A question for your buildWhat will a person receive if the next attempt fails?

Sources

  1. Stripe: Minions, part 2CI checks and the return to a human operator.
  2. Dropbox: Introducing NovaDeflaker and its capped fix attempts.
  3. DoorDash: How we built an AI code reviewerRepeated requests and per-agent deadlines.