Server Diagnostics

PAI reports original exceptions at the trusted server boundary while clients and persisted state receive only safe error details.

HTTP Versus Runtime Reporting

The distinction follows the owner of the failing operation:

HTTP request
  |
  +-- receiver cannot complete this request or stream
  |     -> onHttpError
  |
  +-- request successfully admits background work
        |
        +-- work later fails without an awaiting caller,
            or a local tool exception is converted into a safe tool result
              -> onRuntimeError

A Mongo error while the HTTP receiver awaits thread.stop() is an HTTP error, even though storage threw it. A model error after thread.send() admitted a run is a runtime error.

Realtime watch errors are also reported by their consumer. An HTTP SSE watch uses onHttpError; a direct runtime watch returns the rejection to its caller. Only an internal monitor that deliberately consumes a subscription failure reports source: "realtime", operation: "subscribe" through onRuntimeError. This keeps one failure from appearing in both hooks.

Expected validation, not-found, busy, conflict, and authorization responses use their normal public contracts and are not unexpected diagnostic events.

Runtime Reporter

Configure onRuntimeError on createPai() or createAgentRuntime():

const pai = createPai({
  agents: { main: assistantAgent },
  scopeKey,
  onRuntimeError(event) {
    logger.error(
      {
        errorId: "errorId" in event ? event.errorId : undefined,
        source: event.source,
        phase: "phase" in event ? event.phase : undefined,
        operation: event.operation,
        agentName: "agentName" in event ? event.agentName : undefined,
        threadId: "threadId" in event ? event.threadId : undefined,
        runId: "runId" in event ? event.runId : undefined,
        err: event.error,
      },
      "PAI runtime error",
    );
  },
});

Runtime events cover background runs, unexpected local backend tool execution exceptions, best-effort realtime delivery, and capability lifecycle failures. Run events identify phases such as execution, lease heartbeat, and finalization. Tool events identify the local tool name and call whose thrown exception PAI handled through the tool-result lifecycle. When that lifecycle retains the failure, its safe output-error carries the same errorId; a lifecycle may instead recover the exception into a valid output while the trusted diagnostic still records that the exception occurred.

Deliberate tool outcomes do not produce incidents: ctx.fail(), lifecycle rejection, cancellation, missing or disabled tools, pending-action resume outcomes, and provider-emitted tool error chunks. ctx.fail() remains available to mapError under the distinct "fail" operation.

Nor are they redacted. A deliberate failure carries an author-written message, so it reaches the client and the model verbatim and needs no errorId to recover it by — the message is already the account of what happened. Redaction and diagnostic references belong to thrown values.

Omit onRuntimeError for PAI's default server console reporter, pass a function to replace it, or pass false to suppress background reporting. Suppressed failures do not receive a client-visible diagnostic reference.

Warning Reporter

Configure onWarning on createPai() or createAgentRuntime() for conditions the runtime tolerated rather than failed. Today that is one event: stored_record_degraded — a durable record whose stored payload or metadata no longer matches the current schema was read, reported, and returned verbatim instead of failing the read. The stored value was validated when it entered, so a mismatch means a schema moved after the fact; failing the read would make the thread unopenable over ordinary schema evolution.

const pai = createPai({
  agents: { main: assistantAgent },
  scopeKey,
  onWarning(event) {
    logger.warn(
      {
        subject: event.subject,
        reason: event.reason,
        threadId: event.threadId,
        messageId: event.messageId,
        runId: event.runId,
        toolName: event.toolName,
        toolCallId: event.toolCallId,
        err: event.cause,
      },
      "PAI degraded read",
    );
  },
});

subject names what drifted (tool-input, tool-output, tool-data, suspension-input, suspension-resume, data-part, provenance, message-metadata, thread-metadata, run-metadata); messageId, runId, toolName, and toolCallId are present when the value lives on a message, a run, or one specific call. Warnings are deliberately not routed to onRuntimeError: the read succeeded, and a bulk migration would otherwise page whoever watches the error channel once per drifted record.

Like onRuntimeError, the handler is best-effort — a thrown or rejected handler is ignored, so application logging can never make a thread unreadable. Omit it for a console.warn diagnostic, or pass false to silence it.

HTTP Reporter

Configure onHttpError on the HTTP receiver or framework adapter:

const routes = createPaiHonoReceiver({
  pai,
  resolveIdentity,
  onHttpError(event) {
    logger.error(
      {
        errorId: event.errorId,
        phase: event.phase,
        operation: event.operation,
        ...(event.phase === "request" ? { status: event.status } : {}),
        agentName: event.agentName,
        threadId: event.threadId,
        runId: event.runId,
        err: event.error,
      },
      "PAI HTTP error",
    );
  },
});

phase: "request" means the receiver could not complete a request and includes the HTTP status returned to the client. phase: "stream" means an established SSE iterator failed after the response started, so it has no HTTP status.

The option has three modes:

omit it to use the documented default server console reporter;
pass a function to replace the default with your logger or error service; or
pass false to deliberately suppress receiver reporting.

A reporter is best-effort. If it throws or rejects, PAI preserves the original safe response and runtime behaviour.

Run and HTTP events that produce a safe client error share its errorId. A local tool event shares the ID when its final lifecycle outcome remains failed; recovered tool exceptions still have a trusted diagnostic ID, but no failed client record. Log that field as a first-class searchable value. Best-effort realtime and capability events do not produce a client error, so they have no error reference.

If a run fails and recording or cleaning up that failure also fails, PAI emits separate events with the same reference:

PAI-7e57dfc9-...  execution     Model provider failed
PAI-7e57dfc9-...  finalization  Storage write failed

Each record retains its own phase and original exception. PAI does not build a nested error graph.

Logger Integration

Prefer structured fields:

function reportPaiError(event: {
  errorId?: string;
  error: unknown;
  operation: string;
  phase?: string;
  threadId?: string;
  runId?: string;
}) {
  logger.error(
    {
      errorId: event.errorId,
      operation: event.operation,
      phase: event.phase,
      threadId: event.threadId,
      runId: event.runId,
      err: event.error,
    },
    "PAI operation failed",
  );
}

Use the same function for both hooks when your logger accepts both event shapes. PAI does not automatically attach request bodies, headers, prompts, identity, scope objects, client data, or tool payloads.

Original exceptions can still contain sensitive data in their own message, stack, or enumerable properties. See Privacy And Redaction before retaining raw errors.