# Design decisions

This page explains _why_ the harness is built this way, and what each choice gives up.

## Conversation source and viewport ownership

Conversation presentation uses one semantic source document and separate view
state. A response keeps its ID during streaming and settlement; a reader's
position refers to displayed content instead of a distance from the bottom.
Layouts retain compact geometry and bounded painted-row caches. Terminal output
must be verified independently from the renderer's intended buffer.

The alternatives were to continue compensating bottom offsets for every kind of
row change, or to give the native scrollbox the full conversation. Offset fixes
cannot identify the text a person was reading. Native viewport culling alone
does not bound retained historical children or layout work. Jazz keeps explicit
content anchoring and bounded realization, at the cost of maintaining a semantic
row index. Very large individual blocks still need temporary wrapping.

The source research was pinned on October 1, 2026:

- [OpenCode normalized messages and parts](https://github.com/anomalyco/opencode/blob/0112a92c416f5ad833d96e7a8308441f0a875d94/packages/tui/src/context/sync.tsx)
  informed stable identity and batched publication. Its inspected latest-100
  fullscreen collection was not evidence for unlimited retained history.
- [Hermes virtual-history reconciliation](https://github.com/NousResearch/hermes-agent/blob/234badf4012af380d23c91eae55d045a69c69ffb/ui-tui/src/hooks/useVirtualHistory.ts)
  distinguishes geometry changing above the reader from arriving content.
- [Pi viewport ownership](https://github.com/earendil-works/pi/blob/a4715ec9bffbfcb8a32a1a4100dcfd06c12c93e4/packages/tui/src/components/scroll-view.ts)
  and its [independent terminal oracle](https://github.com/earendil-works/pi/blob/a4715ec9bffbfcb8a32a1a4100dcfd06c12c93e4/packages/tui/test/virtual-terminal.ts)
  informed one navigation owner and testing emitted bytes.
- [Codex terminal invalidation](https://github.com/openai/codex/blob/444da310e108da16aaeb18fd790b0ac464f08aca/codex-rs/tui/src/custom_terminal.rs)
  demonstrates why forced repaint must repair blank physical cells too.

These are implementation references, not claims that an upstream application
has no rendering defects. See [Conversation presentation](./presentation-documents.md)
for the implemented contracts and [Terminal rendering tests](./terminal-rendering-tests.md)
for evidence at the output boundary.

---

## Map

```mermaid
mindmap
  root(("Jazz<br/>harness"))
    ("Keeping runs alive")
      ("Ephemeral budget pressure")
      ("Meltdown on tool diversity")
      ("Compaction, not truncation")
      ("Turn-aware trimming")
    ("Staying honest about cost")
      ("Two-tier token counting")
      ("Tool results reformatted")
      ("Tool results offloaded")
      ("Conversation prefix cached")
      ("Sub-agent cost roll-up")
    ("Running anywhere")
      ("stdout = payload")
      ("Risk tiers, one dial")
      ("Two-phase approval")
    ("Not locking you in")
      ("AI SDK as provider port")
      ("Catalog + on-disk snapshot")
      ("Effect-TS core")
```

---

## Keeping long runs alive

### Iteration budget with ephemeral pressure injection

**Decision.** 100 iterations per run. At 70% Jazz injects "begin consolidating"; at 90%,
"write your final output now". The message is appended to the request array for that one
call and never stored.

**Alternatives rejected.** A hard cap with no signalling: the agent gets cut off
mid-thought and you get nothing. Storing the warnings: by iteration 78 the history carries
eight escalating FINISH NOW messages that cost tokens, contradict each other, and poison the
next summarization.

**Cost accepted.** The agent can't see the pressure history, so it can't reason about "I've
been told this twice". In practice escalating tiers cover that.

📄 [`agent-loop.ts:40`](../../packages/core/src/agent/execution/agent-loop.ts#L40) · [Agent loop](./run-lifecycle.md#guard-1-budget-pressure)

### Meltdown detection keyed on name + arguments

**Decision.** Over the last 10 tool calls, if unique `name:arguments` keys fall below 40%,
inject a recovery message and reset the window.

**Alternatives rejected.** Counting repeats of the tool _name_: that flags
`web_search(q1) → web_fetch(u1) → web_search(q2)` as a meltdown, which is what research
looks like, and flags reading ten files in a row, which is what understanding a codebase
looks like. Both are the behaviors you're trying to encourage.

**Cost accepted.** An agent that loops with _slightly_ varied arguments, such as appending a
counter to the same query, slips through. Catching that needs semantic similarity, which
costs a model call per check.

📄 [`agent-loop.ts:101`](../../packages/core/src/agent/execution/agent-loop.ts#L101) · [Agent loop](./run-lifecycle.md#guard-2-meltdown-detection)

### Compaction at 80%, not truncation

**Decision.** When tokens pass 80% of the model's window, summarize the middle of the
conversation into one message and rebuild as `[system, summary, ...recent]`.

**Alternatives rejected.** A sliding window: it drops the oldest messages, which is where
the task definition and the plan live. Forty minutes in you keep a tool result about page 14
of a PDF and lose the reason you were reading it.

**Cost accepted.** An extra LLM call, added mid-run latency, and genuine information loss.
Mitigated by a configurable `llm.summarizerModel` (point it at something cheap), by making it
visible rather than silent, and by letting the agent trigger it deliberately via
`summarize_context`.

📄 [`summarizer.ts:222`](../../packages/core/src/agent/context/summarizer.ts#L222) · [Context management](./context-lifecycle.md#3--compaction-summarize-dont-truncate)

### Turn-aware trimming

**Decision.** Trimming protects the system message plus the last N _complete turns_, and
keeps an assistant message's `tool_calls` together with its `tool` results as a unit.

**Alternatives rejected.** Dropping the oldest K messages. It eventually splits a tool call
from its result, which is invalid to most providers, and the 400 shows up several
iterations later, far from the cause.

**Cost accepted.** Trimming is coarser: sometimes a whole turn is dropped where a couple of
messages would have sufficed.

📄 [`context-window-manager.ts:94`](../../packages/core/src/agent/context/context-window-manager.ts#L94) · [Context management](./context-lifecycle.md#2--trimming-turn-aware-never-mid-tool-call)

---

## Staying honest about cost

### Two-tier token counting with per-model calibration

**Decision.** Exact counts via `gpt-tokenizer` for OpenAI families; for everyone else a
family-seeded chars-per-token ratio that calibrates against the provider's own reported
`usage.promptTokens` after each call, smoothed and clamped to `[2, 6]`.

**Alternatives rejected.** One universal ratio (≈4 chars/token): under-counts Claude by
~15%, so you overrun the window you thought you were under. Anthropic's tokenizer package:
stale, Claude-2 era. Their `count_tokens` API: a network round trip on the hot path.

**Cost accepted.** The first call against an unfamiliar model uses a seed estimate and can
be off. It self-corrects after one exchange.

📄 [`token-counter.ts`](../../packages/core/src/agent/context/token-counter.ts) · [Context management](./context-lifecycle.md#1--counting-tokens)

### Tool results reformatted before entering context

**Decision.** Every tool result passes through `formatToolResultForContext` before being
appended, and its size is recorded per tool name.

**Alternatives rejected.** Storing raw payloads. Tool output (not conversation) is the
dominant context cost in long runs, and raw JSON is the least token-efficient way to say
anything.

**Cost accepted.** Formatting is lossy; a tool whose output genuinely needs full fidelity
must say so in its own formatter.

📄 [`tool-result-formatter.ts`](../../packages/core/src/utils/tool-result-formatter.ts)

### Offload old tool results, don't wait for 65%

**Decision.** Every iteration, persist large tool bodies to the conversation's work
directory, then replace every cycle except the live one with a pointer to
`retrieve_tool_result`. A failed write (read-only CI, container, Telegram host)
does not fail the run: the placeholder tells the model to re-run the original
tool. No window-fill gate.

**Alternatives rejected.** Waiting until 65% of the window, once per crossing:
a 200k model carried ~130k of already-read grep/file output on every round trip.
Clearing every turn without persist: the model cannot get the bytes back.
Requiring a writable disk. Jazz runs in CI, Docker, and chat bridges that can
read but not write.

**Cost accepted.** One cache miss per aged-out result (the prefix rewrites once,
then sticks). Retrieve is an extra round trip when the model still needs an old
body. Hosts that cannot write pay the same stub they had before, minus the wait.

📄 [`tool-result-offload.ts`](../../packages/core/src/agent/context/tool-result-offload.ts) · [`tool-result-clearing.ts`](../../packages/core/src/agent/context/tool-result-clearing.ts)

### Cache the conversation prefix, not just the system prompt

**Decision.** Anthropic-style providers get a cache breakpoint on the last
content part of the request as well as the system message. OpenAI requests
always carry `promptCacheKey: "conversation"`, including when reasoning is off.

**Alternatives rejected.** Caching only the system prompt: the dominant tokens
in a long run are history, and they were re-billed at full input price every
turn. Rewriting the prefix every iteration to shrink it: that busts the cache
and makes every remaining token expensive.

**Cost accepted.** A compaction or offload that rewrites an earlier message is
one cache miss. That is cheaper than never hitting the cache at all.

📄 [`ai-sdk-service.ts`](../../packages/adapters/src/llm/ai-sdk-service.ts)

### Sub-agent cost rolls up into the parent

**Decision.** A parent run reports its own cost plus all child cost, and emits a figure
whenever _either_ side is known.

**Alternatives rejected.** Reporting only the parent's own tokens: a local-model parent
that spawned three cloud sub-agents would report `$0.00` while your bill said otherwise.

**Cost accepted.** You can't read per-child spend off the top-level number; that lives in
the telemetry records.

📄 [`agent-loop.ts:221`](../../packages/core/src/agent/execution/agent-loop.ts#L221)

### Sub-agents run beside their parent; in chat they outlive its turn

**Decision.** `spawn_subagent` returns an `agentId` at once and the child is a fiber in a
supervisor's scope; `wait_subagents` collects it. There is one mode, not a waiting one and a
non-waiting one. What differs is who owns the supervisor:

- **A run** owns it by default (`jazz run`, workflows, bots, goal cycles, and every sub-agent's own
  children). Closing the run, however it ends, cancels every child still going, and an answer the
  parent gives while children run waits for them and goes back to the model once.
- **A chat** owns one per conversation and attaches each turn to it. The parent answers without
  waiting, the children keep working, and a child finishing while the chat waits for the user
  starts a turn whose message says which children finished. What a child shares with the turn that
  started it (the browser, typed secrets) is released once that turn's children have all finished,
  and spend a child reports after its turn was recorded goes to the ledger as a further entry under
  the turn's run id. A message the user sends ends a `wait_subagents` early, so a parent waiting on
  its children never holds the user off.

**Alternatives rejected.** Keeping a mode where the spawn call blocks until the child answers: it
saves one model call when a single child is started and awaited at once, but it is a second
concept for agents and people, and a blocked parent cannot steer. Running children as daemon jobs
that wake the parent later, the way `run_background_jobs` does: that needs the parent's budget,
approvals, egress state and cancellation carried across processes. Children live in the chat's
process instead, so they share its approvals, policy and presentation as they always did.

**Cost accepted.** One extra model call for a single delegation (spawn, wait, answer), and one more
for the turn a finished child starts, since its result is read through `wait_subagents` so the
egress taint sees it. A child cannot survive a park, a detach, or the end of the conversation.
Steering reaches a child only between its steps; only cancel is immediate.

📄 [`supervisor.ts`](../../packages/core/src/agent/subagents/supervisor.ts)

---

## Running anywhere

### stdout is the payload, stderr is everything else

**Decision.** `jazz run` writes only the answer (or exactly one JSON object) to stdout.
Status notices, tool chatter, headers, footers, and `--events` NDJSON all go to stderr.
`JAZZ_NO_TUI=1` is forced so Ink never touches stdout.

**Alternatives rejected.** A `--quiet` flag over the normal output path: you're still
filtering, and one new log line breaks every downstream parser. A dedicated `--format json`
that only _mostly_ suppresses chatter, which is the same problem later.

**Cost accepted.** Two streams to wire up in a bridge instead of one. That's the entire
cost, and it's what makes every non-terminal surface possible.

📄 [`execute.ts:30`](../../packages/cli/src/commands/run/execute.ts#L30) · [Headless](../surfaces/headless.md)

### Risk tiers instead of a tool allowlist

**Decision.** Every tool declares a risk level (`read-only` / `low-risk` / `high-risk`), and
one policy dial decides what runs unattended.

**Alternatives rejected.** A per-tool allowlist as the primary mechanism. It doesn't
generalize across surfaces: you'd maintain a different list for CI, cron, and each bridge,
and a new tool defaults to invisible rather than gated.

**Cost accepted.** Tiers are coarse: `high-risk` covers both `git push` and `rm -rf`.
Sharper control comes from the two escape hatches: a per-tool session allowlist and a
per-command allowlist for `execute_command`, and from trimming the agent's toolset, which
is the strongest control available.

📄 [`types/tools.ts:19`](../../packages/core/src/types/tools.ts#L19) · [Tools & approval](./tool-lifecycle.md)

### Two-phase execution (propose → approve → execute)

**Decision.** A gated tool doesn't act. It returns an approval request describing what it
_would_ do, including a preview diff for edits. Approval (human or policy) then invokes the
real execution tool.

**Alternatives rejected.** A boolean `dangerous` flag checked before calling. You can't show
a meaningful preview without doing the work, and "would this be destructive" gets evaluated
before the arguments are resolved.

**Cost accepted.** Every gated tool is two registry entries instead of one, and the registry
carries the propose→execute mapping.

**What it buys:** interactive and unattended runs use _the same code path_. There is no
separate headless mode that can drift from the interactive one: the only difference is who
answers.

📄 [`tool-executor.ts:192`](../../packages/core/src/agent/execution/tool-executor.ts#L192)

### Command approval matches on a parsed key, never a raw prefix

**Decision.** "Always approve `git status`" stores a key extracted from the command (binary

- first subcommand) and matches exactly or on a word boundary.

**Alternatives rejected.** Prefix-matching the raw command string. Then approving
`git status` also approves `git status && rm -rf /`.

**Cost accepted.** Approving `git` broadly takes several confirmations instead of one.

📄 [`tool-executor.ts:601`](../../packages/core/src/agent/execution/tool-executor.ts#L601)

---

## Not locking you in

### Vercel AI SDK as the provider port

**Decision.** One adapter (`ai-sdk-service.ts`) behind the `LLMService` interface, giving 18
providers including self-hosted Ollama, llama.cpp, vLLM, and SGLang.

**Alternatives rejected.** Hand-written clients per provider: every new provider becomes a
project, and streaming plus tool-calling plus reasoning quirks get reimplemented each time.

**Cost accepted.** Jazz is bounded by what the SDK normalizes, and inherits its bugs.
Provider-specific behavior that leaks through (reasoning-effort semantics especially) is
normalized in `adapters/src/llm/reasoning/`. AI SDK's internal retries are turned off
(`AI_SDK_MAX_RETRIES = 0`) so Jazz owns retry policy via Effect rather than having two
retry loops fighting.

📄 [`ai-sdk-service.ts`](../../packages/adapters/src/llm/ai-sdk-service.ts) · [Providers & models](./add-a-provider.md)

### Model catalog from models.dev, with an on-disk snapshot

**Decision.** Context windows and pricing come from models.dev, cached to
`~/.jazz/cache/models-dev.json`. Offline mode reads the snapshot; `JAZZ_MODELS_DEV_URL`
points at an internal mirror.

**Alternatives rejected.** Vendoring a pricing table: stale within weeks, and wrong pricing
is worse than none. Requiring the network: breaks airgapped installs, which are a supported
deployment.

**Cost accepted.** A brand-new model may be missing from the catalog; Jazz falls back to
provider-reported metadata and a 128k default. Ollama, llama.cpp, vLLM, and SGLang need no catalog to list models. Ollama and llama.cpp report additional capabilities; vLLM and SGLang model lists can report context length but cannot verify whether tool parsing is enabled on the server.

📄 [`models-dev.ts`](../../packages/core/src/utils/models-dev.ts) · [Airgapped](../getting-started/local-models.md)

### Effect-TS for the entire runtime

**Decision.** Typed errors, tracked effects, `Layer`-based dependency injection throughout.

**Alternatives rejected.** Plain `async/await` with thrown exceptions. In an agent runtime
the failure paths _are_ the product: a tool that times out, a provider that 429s, a
malformed tool argument. With exceptions those become `undefined` three frames away.

**Cost accepted.** A real learning curve. Effect is unfamiliar to most contributors and the
type errors are intimidating. That's precisely why [Code map](./architecture.md) leads with the
DI and Layer patterns, and why `core/` is kept dependency-free so it can be tested with
plain mocks.

### Lazy MCP connection

**Decision.** MCP servers are not connected at startup. Tools are registered per-agent based
on the agent's tool list, and a server connects the first time one of its tools is actually
invoked.

**Alternatives rejected.** Connecting everything at boot. MCP servers are child processes;
half a dozen of them turn `jazz` into a several-second startup and a CLI that hangs when one
server misbehaves.

**Cost accepted.** The first call to an MCP tool pays the connection cost, and a broken
server surfaces mid-run instead of at startup.

📄 [`register-tools.ts`](../../packages/core/src/agent/tools/register-tools.ts)

### Progressive skill loading

**Decision.** Three tools: `find_skills` (names + descriptions), `load_skill` (full
instructions), `load_skill_section` (a referenced file).

**Alternatives rejected.** Preloading every skill's instructions into the system prompt. A
hundred skills would consume the window before the user says anything. Re-injecting a loaded
skill into the system prompt on later turns: that would bust the cached system prefix, so
the playbook stays in the conversation as the `load_skill` tool result.

**Cost accepted.** Two extra round trips before the agent starts working with a skill. Later
turns must follow a playbook that lives in transcript history, not in the system prompt.

📄 [`skill.ts`](../../packages/core/src/agent/tools/skill.ts) · [Skills loading](../concepts/skills.md)

### Deferred tool schemas

**Decision.** Tool categories declare a `loadTier`: `eager` (schema sent every turn: files,
shell, todo, etc.) or `deferred` (name + one-line summary only. MCP servers, background jobs,
reminders, wake triggers, workspace, peers). A `search_tools` tool fetches a deferred tool's
full schema on demand; once fetched it stays callable for the rest of the run.

**Alternatives rejected.** Sending every registered tool's full schema every turn, including
every tool a connected MCP server advertises. Server tool counts are unbounded and
user-configured, so this cost grows without bound as someone adds servers, most of whose tools
never come up in a given conversation.

**Cost accepted.** An extra `search_tools` round trip before a deferred tool's first use per
run. Names/summaries must stay visible in the prompt regardless: hiding them entirely would
push the model toward replicating a listed tool with `execute_command` instead of discovering
it, which `execute_command`'s own description now warns against explicitly.

📄 [`search-tools.ts`](../../packages/core/src/agent/tools/search-tools.ts) · [Tools reference](../tools/index.md)

---

## Related

- [Agent loop](./run-lifecycle.md) · [Context management](./context-lifecycle.md) · [Tools & approval](./tool-lifecycle.md)
- [Code map](./architecture.md): where the code for all of this lives
- [Discussions](https://github.com/lvndry/jazz/discussions): decisions not yet made