The agent loop

This page explains what happens between “you press enter” and “you get an answer” , especially on runs that take a long time.

Source: packages/core/src/agent/execution/agent-loop.ts

An agent run is a bounded loop: ask the model, execute whatever tools it asked for, feed the results back, repeat until it stops asking for tools. The loop itself is twenty lines. Everything that makes it survive a long autonomous run is in the guards around it.

One full iteration

flowchart TD
    START(["Iteration i of 100"]) --> COMPACT

    COMPACT["<b>Compact if needed</b><br/>tokens &gt; 80% of window?<br/>→ summarize, then resume"]
    COMPACT --> PRESSURE["<b>Build budget pressure</b><br/>just compacted → 'continue the task'<br/>else i/100 ≥ 50% → 'plan to wrap up'<br/>i/100 ≥ 70% → 'consolidate'<br/>i/100 ≥ 90% → 'finish now'<br/><i>ephemeral, not stored, repeats every iteration</i>"]
    PRESSURE --> ASK["<b>Ask the model</b><br/>messages + tools + reasoning effort"]

    ASK --> INT{"Interrupted?<br/>(double-Esc)"}
    INT -->|yes| STOP(["Break: keep partial output"])
    INT -->|no| RECORD["<b>Record usage</b><br/>tokens, cost,<br/>calibrate token counter"]

    RECORD --> APPEND["<b>Append assistant message</b><br/>content + reasoning + tool_calls"]
    APPEND --> TRIM["<b>Trim</b> to the token budget<br/>(turn-aware: never splits<br/>a tool call from its result)"]

    TRIM --> BRANCH{"Did the model<br/>request tools?"}
    BRANCH -->|"no"| FINAL["<b>Final response</b><br/>present, notify, exit loop"]
    BRANCH -->|"yes"| TOOLPHASE

    TOOLPHASE["<b>Tool phase</b><br/>see below"]
    TOOLPHASE --> NEXT(["Iteration i+1"])

    classDef guard fill:#f9a03f,stroke:#b3541e,color:#1a1a1a
    classDef work fill:#4f9d9d,stroke:#2f6d6d,color:#ffffff
    class COMPACT,PRESSURE,TRIM guard
    class ASK,TOOLPHASE work

The loop runs at most DEFAULT_MAX_ITERATIONS (100) times. It exits early on a final response (no tool calls) or a user interrupt.

The tool phase

flowchart TD
    IN(["Model requested N tool calls"]) --> TRACK["<b>Track</b> each call as<br/><code>name:arguments</code><br/>keep a rolling window of 10"]

    TRACK --> MELT{"<b>Meltdown?</b><br/>unique keys / 10 &lt; 40%"}
    MELT -->|yes| INJECT["Inject recovery message:<br/>'stop, summarize, try a<br/>different strategy'<br/>reset the window"]
    MELT -->|no| DEDUPE
    INJECT --> DEDUPE

    DEDUPE["<b>Collapse duplicates</b><br/>identical read-only calls<br/>in this batch run once"]
    DEDUPE --> EXEC

    EXEC["<b>Execute</b><br/>≤10 concurrent, each forked<br/>3-min default timeout<br/>approval-gated per tool"]
    EXEC --> VALIDATE{"Every call<br/>got a result?"}
    VALIDATE -->|no| FAIL(["Hard fail. This is a bug,<br/>not a recoverable state"])
    VALIDATE -->|yes| FORMAT["<b>Format results for context</b><br/>compress before storing"]

    FORMAT --> QUEUE{"User typed<br/>while we worked?"}
    QUEUE -->|yes| PUSH["Append their message,<br/>steer mid-run"]
    QUEUE -->|no| OUT
    PUSH --> OUT(["Next iteration"])

    classDef guard fill:#f9a03f,stroke:#b3541e,color:#1a1a1a
    classDef bad fill:#c1443c,stroke:#7d2b26,color:#ffffff
    class MELT,INJECT,FORMAT guard
    class FAIL bad

Missing results are a hard failure

If any requested tool call comes back without a result, the run fails loudly. It would be easy to paste in a placeholder and continue, but a message history where a tool_calls entry has no matching tool message is invalid to most providers, and the resulting error appears three iterations later somewhere unrelated. Failing at the source keeps the bug findable.

Mid-run steering

Tool execution is slow, and users type while it happens. Anything queued during the tool phase is appended as a user message before the next iteration, so you can redirect a long run without killing it.

Guard 1: budget pressure

An agent with 100 iterations and no sense of time will happily spend all 100 on research and produce nothing. So Jazz tells it where it stands:

timeline
    title Iteration budget (100 iterations)
    section 1–49 · Free rein
        No pressure : explore, research, spawn sub-agents
    section 50–69 · 50% notice
        "30 iterations left. The budget is a hard stop." : agent plans for the finish
    section 70–89 · 70% warning
        "18 iterations left. Begin consolidating results. Stop re-reading covered files."
        : agent shifts to synthesis
    section 90–100 · 90% critical
        "6 iterations left. Write your final output NOW."
        : agent lands the plane, nudge repeats every iteration

The tiers are cumulative, not one-shot: a model at 93% sees the critical message on its next iteration too, instead of the silence that followed the single 90% nudge. Each message names the remaining count in absolute terms — “90% used” is not actionable while “6 iterations left” is — and every tier is re-sent on every remaining iteration, so a run that is about to die of an empty loop keeps hearing about the wall.

The important detail: the pressure message is ephemeral. It’s appended to the array sent to the model for that one call and never pushed into the stored conversation:

const budgetMsg = buildBudgetPressureMessage(iterationIndex + 1, maxIterations);
const messagesForLLM = budgetMsg ? [...state.currentMessages, budgetMsg] : state.currentMessages;

If it were stored, iteration 98 would carry eight escalating “FINISH NOW” messages, each one costing tokens and confusing the transcript that later gets summarized. This way the nudge steers the run without polluting its history.

Sibling guards: cost, tokens, and wall-clock time

Three more budgets check in the same place, right after each iteration’s tool phase finishes, and are configured via maxCostUSD, maxTokens, and maxDurationMs; see run budgets:

• maxCostUSD re-prices the run’s accumulated tokens (plus any sub-agent spend rolled up via childCostUSD) against the model’s models.dev pricing. Skipped entirely when pricing is unknown. It never guess-aborts a run it cannot verify the spend of.
• maxTokens just sums totalPromptTokens + totalCompletionTokens. No pricing lookup, so it still enforces on a local/unpriced model where maxCostUSD cannot.
• maxDurationMs is a deadline. Each iteration races timeRunsOut, which polls workTimeLeftMs: the budget less ANSWER_RESERVE_RATIO (15%), counted from runMetrics.startedAt. So it interrupts a model call, a tool batch or a sub-agent in flight. Dangling tool calls are closed with a note and the loop moves on to the final answer. It also gets an ephemeral pressure message inside runIteration (buildTimeBudgetPressureMessage) at 50%, 80%, and 90% of the work time, and spawn_subagent hands the child the work time left through remainingRunBudget.

maxCostUSD and maxTokens share the iteration budget’s timing: checked between iterations, not a preemptive interrupt. A single expensive iteration (a costly tool call, or a sub-agent delegation that runs for a while) can push the total past the cap before the next check trips. A run stopped by any cap reports which one fired on the response: costCapped / tokenCapped / durationCapped.

When a deadline or an interrupt stops a tool batch part-way, the loop closes it from the batch’s ledger: completed calls keep their results, the others say whether they were interrupted while running or never started, and the response lists them as stoppedToolCalls (stopping a batch).

--timeout is enforced outside the loop: the CLI races the whole run against a deadline (packages/core/src/utils/run-deadline.ts) with Effect.raceFirst and ends it as a failure. Interrupting the run interrupts its tool fibers (the shell kills the command’s process group) and the LLM stream’s finalizer aborts the provider request. The CLI also hands the deadline to the loop as AgentRunnerOptions.deadline, and workTimeLeftMs takes the smaller of the two limits. The deadline’s work time is its remaining time less 15% of its original budget, re-read on every poll. An approval wait that extends the deadline (deadline.extend) therefore pushes the work stop out with it. The kill only fires when the run is still going after its answer reserve, for example a dead provider or start-up that hangs.

The final answer after a stop

A run stopped by a cap, the iteration limit, or a stall would otherwise return whatever its last step said. That step was usually a tool call, so the answer would be empty. Before finalizeRun, the loop sets state.finalAnswerReason and runs one more runIteration. That step asks getCompletion for text only (toolsOffered is false), appends finalAnswerInstruction to the ephemeral nudges, drops any tool calls the provider returns anyway, and does not wait on running sub-agents or take queued guidance. It is bounded by timeLeftMs(deps, 0), the time limits with no reserve held back, and skipped when none is left. A failure or timeout of that step is logged, and the run returns what it had. The cap flags (iterationLimited, costCapped, tokenCapped, durationCapped, stalled) stay set, so callers still see the answer as partial. Esc skips the step.

Guard 2: meltdown detection

The classic agent failure isn’t a crash, it’s a groove: the same search, the same file read, forever, until the budget is gone.

const keys = window.map((tc) => `${tc.name}:${tc.arguments}`);
const uniqueness = new Set(keys).size / windowSize;
return uniqueness < 0.4;

The key is composite. tool name plus arguments. This distinction is the whole design:

Keying on tool name alone would flag the second and third rows as meltdowns, which are exactly the behaviors you want. When a meltdown does trip, Jazz injects a message telling the agent to stop, summarize what it has, and either output or try a fundamentally different approach: then clears the window so it gets a fair chance. A second meltdown after that nudge (MAX_MELTDOWN_NUDGES) stops the run: the observer warns (onStalled) and the response carries stalled: true.

Unlike budget pressure, this message is stored. It’s a real event in the run’s history and the agent should keep remembering that its last approach didn’t work.

Guard 3: duplicate calls within one batch

Meltdown detection watches repetition across turns, and can only react once a window of 10 has filled. It says nothing about a single assistant turn that asks for the same file three times at once: a real shape, seen in practice as three identical read_file calls and two identical package.json ones inside one parallel batch, all dispatched in the same millisecond.

So before the batch runs, byte-identical calls are collapsed: keyed on name:arguments, exactly like meltdown detection, the first occurrence executes and the rest reuse its result. The saving is doubled: the tool runs once instead of N times, and its output enters context once instead of N times.

Only read-only tools are collapsed. Two identical mutating calls could legitimately be two separate effects, and guessing wrong there destroys work rather than saving it. A read-only call repeated with identical arguments cannot mean two different things.

Nothing leaves the transcript. Providers require one role: "tool" message per tool_call_id, so collapsed calls are mapped onto the surviving call’s result and every id still gets its message: the model sees the batch it asked for.

Streaming vs batch

The loop is shared. Only how it talks to the model and renders output differs, expressed as a CompletionStrategy:

Adding a third mode means implementing one interface, not forking the loop. That’s the reason for the indirection.

Both modes return the provider’s canonical completed response. In streaming mode that response is read after the event stream closes, so metadata finalized after the last visible delta—usage, reasoning signatures, and model-generated artifacts—is not lost.

Finalization

After the loop, win or lose:

flowchart LR
    EXIT(["Loop exits"]) --> WHY{"How?"}
    WHY -->|"final answer"| OK["Normal"]
    WHY -->|"hit 100 iterations"| LIMIT["Warn: iteration limit"]
    WHY -->|"empty response"| EMPTY["Warn: empty response"]
    WHY -->|"interrupted"| INT["Keep partial output"]

    OK --> METRICS
    LIMIT --> METRICS
    EMPTY --> METRICS
    INT --> METRICS

    METRICS["<b>Write metrics</b><br/>forked fiber, awaited on release,<br/>never blocks your answer,<br/>never lost on exit"]
    METRICS --> COST["<b>Cost</b><br/>own tokens × model price<br/>+ sub-agent cost"]
    COST --> RESP(["AgentResponse<br/>content · usage · costUSD · messages"])

    classDef core fill:#4f9d9d,stroke:#2f6d6d,color:#ffffff
    class METRICS,COST core

Two details worth noting:

• Metrics are written on a forked fiber that the loop’s release step awaits. You get your answer immediately, and the telemetry still lands even if the process is shutting down.
• Cost includes sub-agent spend. A run whose own tokens are unpriced (a local model) but which spawned a priced sub-agent still reports a figure: otherwise the number would silently understate what you spent.

Reading a run in the logs

Related

• Context management: what “compact” and “trim” actually do
• Tools & approval: the execution and gating detail
• Delegation: what the loop does when the agent delegates
• Architecture: where the loop sits in the runtime