How It Works

The harness is three pieces: a CLI that speaks only JSON, a supervisor that owns the endpoint and the code generation, and a disposable worker that hosts your app. Everything else in this page follows from that split.

CLI commands
    │  loopback URL + bearer token (auto-discovered from .pai/harness.json)
    ▼
supervisor — owns the port, token, file watcher, generation counter
    │  fresh private process per generation
    │  worker stdout/stderr → JSON-line diagnostics, read back with `logs`
    ▼
worker — imports your `pai` registry, serves the stock PAI HTTP receiver

The worker serves the same createPaiHonoReceiver your production adapters use, unmodified, so behavior observed through it is the behavior your app ships. What the harness adds — direct tool execution, identity resolution, hidden-thread listing — lives on separate /harness/* routes rather than in front of your agents' own, which is why send and a production request take the same path.

There is deliberately no foreground mode. A command that never returns would hang the turn of the agent that ran it, so start detaches, publishes readiness through the discovery file, and returns; the supervisor it spawns is an internal entry point that writes nothing to stdout.

Generations: never test stale code

Every reload starts a fresh worker process, so transitive imports are actually re-evaluated — editing a tool module three imports deep takes effect. The supervisor keeps the public URL, token, and instance id stable across the swap, so no command has to reconnect.

Each generation is a number, and it is load-bearing:

• Every request is pinned to the generation the CLI connected under. A request that straddles a reload fails with generation_mismatch instead of silently executing against different code.
• Every mutating result is stamped with daemon.generation, so an observed behavior can always be tied to an exact version of your source.
• While a reload is in flight, mutations are rejected with daemon_reloading and reads may drain. The supervisor drains active requests before starting the candidate and again after it boots, then swaps synchronously.
• If the candidate fails to import, the generation does not increment: status becomes reload_failed, and agent commands keep returning reload_failed (with the esbuild location of the error) until the source is fixed. The harness never silently falls back to the previous code.

Reload errors distinguish activeGeneration (what status describes now) from attemptedGeneration (the candidate that failed), so a repair loop never has to infer which generation a number refers to.

One caveat on "fails to import": workers load TypeScript through an esbuild transform, which strips types without checking them. A syntax error fails the reload loudly; a type error loads happily and may surface as a runtime fault instead. Run your project's typecheck alongside the harness when you are verifying a fix — the harness proves runtime behavior, not type soundness.

restart is the barrier

The watcher reloads on source changes with a short debounce, which is convenient but asynchronous, and an asynchronous reload is ambiguous to a script that must know which code it just tested. restart is the deterministic synchronization point: it queues its own fresh-process attempt, waits for the observed source to go quiet, reloads again if an edit landed mid-attempt, and returns only when that exact attempt is ready.

Its result includes sourceReloadsSinceLastBarrier — how many watch-triggered reloads already ran since the previous barrier — so a jump from generation 4 to 7 is explained rather than surprising.

Identity injection

The CLI sends identity in a header; the supervisor forwards it; the worker resolves it inside the receiver's resolveIdentity hook — the same hook your production middleware fills after decoding a session.

--as globex             →  persona factory in your app entry  →  { identity }
--identity '…'          →  inline identity                    →  { identity }
--client-data '…'       →  merged onto either                 →  { …, clientData }

Three consequences worth internalizing:

• There is no auth to reverse-engineer. PAI middleware derives plain data; the harness supplies that data directly. Nothing downstream can tell the difference, because there is no difference.
• Identity is validated before anything mutates. A dedicated route checks the selected identity against the agent's identity schema, so a malformed persona fails with invalid_identity — carrying issues[] for a schema mismatch, or the failing persona and availablePersonas when a factory throws — instead of a confusing runtime error mid-turn.
• Identity is discoverable. personas resolves every persona to its { identity }, reporting per-persona failures in the same shape threads --all-personas uses, so an agent never reads your app source to learn what a persona means — and never pays for tool schemas to find out.

clientData deliberately does not live in a persona: a persona says who the caller is, clientData says what this request carries, so it can be varied per command without defining a new fixture.

Because storage is partitioned by scope, personas also give you a free cross-tenant test surface: the same thread id is visible to one persona and invisible to another, and the harness can tell you which.

Turn settlement

A "turn" in the harness is not "the HTTP request returned." It is one exact run reaching a durable boundary:

send, respond, and run --wait|--stop all settle this way, which is what removes polling from the caller. Three details make it reliable:

• Results are scoped to one runId. If a queued run is admitted while the target run settles, its output is never attributed to your result.
• Resumed runs keep their id. Answering a pending action resumes the same run, so the harness waits past the stale waiting boundary rather than reporting the state you already had.
• State is refreshed before projecting. A durable run record can turn terminal before the last message update arrives over SSE, so the harness re-reads state after the boundary instead of racing the stream.

The result contract

Every command prints exactly one versioned JSON document on stdout; diagnostics go to stderr. On the wire it is compact — a single line, since indentation is roughly a quarter of a large payload and every byte lands in the calling agent's context. It is shown expanded here for reading:

{
  "schemaVersion": 3,
  "command": "send",
  "ok": true,
  "agent": "support",
  "threadId": "…",
  "runId": "…",
  "outcome": "blocked",
  "run": { "status": "waiting" },
  "answer": null,
  "toolCalls": [
    {
      "toolCallId": "…",
      "tool": "refundOrder",
      "state": "output-error",
      "input": { "orderId": "ord-1001" },
      "error": { "message": "Order ord-1001 was not found in workspace globex." }
    }
  ],
  "pending": [
    {
      "actionId": "…",
      "tool": "refundOrder",
      "suspendName": "approval",
      "payload": { "orderId": "ord-1001", "amountUsd": 129 },
      "resumeSchema": {
        "type": "object",
        "properties": { "approved": { "type": "boolean" } },
        "required": ["approved"]
      }
    }
  ],
  "troubles": [],
  "error": null,
  "usage": { "inputTokens": 1840, "outputTokens": 96, "totalTokens": 1936 },
  "durationMs": 2412,
  "version": { "threadInstanceId": "…", "revision": 12 },
  "daemon": { "instanceId": "…", "generation": 4 }
}

Read it as four independent layers:

• answer — what the agent told the user.
• toolCalls — what actually executed, with per-call input, output, and errors. This is the ground truth, and it is deliberately separate from answer: when a tool reports refunded: true and the prose says the refund was declined, the disagreement is the bug.
• pending — what the run is waiting for, including the suspend payload the approver would see and the JSON Schema of a valid response. Answering an approval is therefore mechanical, not guesswork.
• troubles — why this turn needs attention, in the same vocabulary threads --has-error uses. One check works everywhere, which matters because the exit code describes the run, not the work inside it.

usage and durationMs sit beside each other because they answer the same question about a change you just made: what it cost and how long it took. The per-scope and per-step breakdowns stay in thread --raw.

ok means the command completed and produced an authoritative result for the target run. It stays true when outcome is failed — the command worked, the run did not. Inspect outcome, run.status, and the exit code for the target's fate; ok: false is reserved for command and protocol failures.

Exit codes

A blocked turn exiting 0 is intentional: waiting for an approval is a successful outcome, not an error.

The exit code describes the run, not the work inside it. A tool that failed inside an otherwise successful run still exits 0 with ok: true — correctly, because the agent handled it. A driving loop that wants to know whether anything broke must read toolCalls[].state for output-error, or sweep with threads --has-error; the exit code alone will not tell it.

The error contract

Failures are structured and stably coded, and where a next step exists they name it in CLI vocabulary rather than in prose. details.recovery is present when there is an action to take — not on every failure, because inventing one would be worse than omitting it — and every verb it can contain is published in the catalog's recoveryHints:

{
  "ok": false,
  "command": "send",
  "error": {
    "code": "thread_busy",
    "message": "Thread … is busy (active running run …). Resolve any pending action, or resend with --queue or --steer.",
    "retryable": true,
    "details": {
      "threadId": "…",
      "activeRunId": "…",
      "recovery": ["resolve_pending_action", "send_with_queue", "send_with_steer"]
    }
  }
}

Representative codes and what they tell a driving script to do next:

invalid_identity covers two different failures, and details distinguishes them. An identity that does not satisfy the agent's schemas carries issues[] naming the offending fields. A persona factory that throws carries no issues — the message is the factory's own error — and names the persona plus the availablePersonas to choose from instead. A sweep should stop using that persona; a malformed inline identity should be corrected.

Filters that could never match are rejected rather than returning an empty list — threads --status failed explains that availability returns to idle after a run ends and points at --has-error. A silent empty result is the one answer a debugging tool must never give.

Uncertain mutations

If a mutation is accepted but its outcome cannot be observed, the harness says so instead of guessing. Which code you get depends on how observation was lost:

All four carry retryable: false plus the identifiers to inspect — turn_timeout from a mutation is an uncertain outcome too, not a "try again" signal, and it is the one you will hit most often. Definitive pre-forward rejections (bad arguments, a busy thread) stay retryable. The rule for a driving loop: never blindly repeat a mutation whose outcome is unknown — inspect the thread or the exact run first. A double-sent refund is a real refund.

Two kinds of failure evidence

PAI produces failure information in two different places, and conflating them wastes an agent's time. The harness keeps them separate on purpose:

A tool that threw shows up in thread state as a tool_error. A tool that swallowed its own error and logged instead shows up only in logs — which is exactly the bug class where an agent otherwise has nothing to go on.

How diagnostics are recorded

The supervisor captures the worker's stdout and stderr and writes one JSON record per line, so process output becomes queryable instead of scraped:

{ "harnessLog": 1, "at": "2026-07-27T10:19:55.738Z", "level": "info",
  "source": "app", "stream": "stdout", "generation": 4, "pid": 73255,
  "message": "[refund] order=ord-1003 amountCents=8900 user=sam" }

stderr is recorded as level: "error" and stdout as info, because that is what those channels mean in practice. Every record carries the generation that produced it, so diagnostics can be tied to an exact version of your code the same way results can. Output the harness did not write — a sidecar inheriting a file descriptor, say — is still returned, tagged source: "unknown" rather than silently dropped.

logs filters by --level, --source, --generation, and --grep, and its nextCursor is opaque — it identifies the log session as well as the position, so a replaced log is detected instead of skipped. Passing it back as --cursor returns only what arrived since, so an agent can poll a growing log without re-reading it or the harness keeping state between invocations. One read is bounded to 4 MB and says skippedOlderOutput with windowStartByte when it clamps, so an empty result never has to be taken on faith. It reads the file directly, so a crashed daemon's output is still inspectable — including the supervisor's own record of a worker that exited unexpectedly.

Querying a thread without paying for all of it

thread returns a structured transcript — entries with kind, role, tool, input, output, error, data, messageId, and runId — plus exact state on tool entries — not a rendered string. That matters for an agent twice over: no parsing prose out of an escaped blob, and filters can compose server-side:

harness thread support <id> --items tool --errors --last 5

A tool entry carries the durable data channels it wrote, an attachment or file entry describes the file rather than naming its kind, and every entry carries the messageId that regenerate --message takes — so what an agent can see matches what a person in a UI would see, and can be acted on.

Filters cover item kind (--items), one tool (--tool), one run (--run), failures only (--errors), and a tail (--last). Every result reports totalEntries (the whole transcript) alongside matchedEntries (what the filters kept), so a filter that matched nothing is never mistaken for an empty thread; --last additionally sets truncated: true when it drops matched entries.

Around the transcript sits the rest of what a person would see: the thread title, the pending[] approvals, and queue[] — turns already sent that have not run yet, whose messages use the same entry shape, so a message reads identically before and after it is admitted. The line is deliberate: the default projection is the conversation plus the ids needed to act on it, and framework bookkeeping — cursors, capability flags, thread metadata, per-step usage — lives in --raw, which returns the complete ThreadState for the rare case that needs it.

Failure triage

Thread availability returns to idle once a run ends, so "what went wrong" is a query over durable evidence rather than a status filter:

pnpm harness threads support --has-error --all-personas

Each match carries a troubles array explaining why it needs attention:

Every run in the thread is inspected, in transcript order, and each trouble names its runId — a failed run followed by a successful one is exactly the case a sweep must not miss. --all-personas runs the same listing for every exported persona, tags each thread with the personas that can see it, and reports any persona the agent rejected under skippedPersonas with the reason. Transport failures abort the sweep instead of quietly returning a partial one.

troubles is evidence, not a verdict. A tool that failed on purpose — a guard rejecting a double refund, an approval declined by policy — is recorded here exactly like an unintended failure, because the harness cannot know which you meant. Expect a healthy app to have entries, and read the tool error to decide.

The same honesty applies to thread-visibility answers: thread_unavailable reports checkedPersonas for the personas actually evaluated and skippedPersonas for those the agent could not serve, rather than counting an unusable persona as checked.

Security and isolation

• The supervisor and workers bind only to 127.0.0.1, and the CLI refuses any endpoint that is not loopback HTTP.
• Every route — info, agent traffic, restart, stop — requires the bearer token.
• Discovery lives at .pai/harness.json, written atomically with mode 0600. Prefer local discovery; for an explicit URL prefer PAI_HARNESS_TOKEN over putting --token in shell history.
• A workspace lock prevents a second supervisor from overwriting discovery and orphaning the first.
• The control-plane directory is the unit of isolation. Discovery, the lock, and the default log all live in it, so one directory means one supervisor and every command in a working directory finds the same one. PAI_HARNESS_DIR names a different directory, which is how two sessions in one checkout each get their own daemon instead of taking turns evicting each other. Note that --url alone is not isolation: the token still comes from discovery, so a pinned URL stops working when the slot changes owner.
• On POSIX each worker owns a process group, so reload, stop, and even hard supervisor death clean up app-spawned sidecars.