Context Compaction

createContextCompaction() is a lifecycle helper for summary compaction. It keeps every original message and its visibility unchanged, adds summaries as ordinary transcript-visible lifecycle messages, and tracks the active boundary in server-only thread metadata. While the summarizer runs, public run metadata exposes a typed activity value to clients.

Configure an agent

Declare the utility's state in the agent's private thread schema, then install the lifecycle helper:

import {
  contextCompactionActivitySchema,
  contextCompactionStateSchema,
  createContextCompaction,
  defineAgent,
} from "@pai/core";
import { generateText } from "ai";
import { z } from "zod";

const compaction = createContextCompaction({
  threshold: { maxEstimatedTokens: 100_000 },
  retain: { maxMessages: 8, maxEstimatedTokens: 20_000 },
  async summarize({ messages, signal }) {
    const result = await generateText({
      model: summaryModel,
      system: "Preserve decisions, facts, unresolved work, and corrections.",
      prompt: JSON.stringify(messages),
      abortSignal: signal,
    });
    return result.text;
  },
});

export const agent = defineAgent({
  name: "assistant",
  runMetadata: {
    private: z.object({}),
    public: z.object({
      contextCompaction: contextCompactionActivitySchema,
    }),
  },
  threadMetadata: {
    private: z.object({
      contextCompaction: contextCompactionStateSchema,
    }),
    public: z.object({}),
  },
  model: primaryModel,
  instructions: "Help the user.",
  lifecycle: compaction,
});

Both supplied schemas include their empty values, so the surrounding fields do not need .optional(). Agent registration fails immediately unless the private thread schema preserves cursor state and the public run schema preserves both active compaction metadata and its cleared null value.

Message metadata is different: compaction needs no messageMetadata schema, and the example above declares none. It marks its summary message under the public message bag, and an agent that declares nothing there writes through an unvalidated bag, so the marker survives untouched.

If the agent does declare messageMetadata, that schema owns the bag and must keep the marker — otherwise a plain z.object would silently drop it and compaction would quietly stop finding its own boundary. Registration checks this too:

import { contextCompactionMarkerSchema } from "@pai/core";

messageMetadata: {
  private: z.object({}),
  public: z.object({
    citations: z.array(z.string()).optional(),
    contextCompaction: contextCompactionMarkerSchema.optional(),
  }),
},

.optional() because only the summary message carries the marker.

The threshold object requires at least one explicit maximum, and compaction runs when either supplied maximum is exceeded. Omitting threshold uses a 40-message default; specifying only maxEstimatedTokens enables a token-only policy. Every token maximum is model-specific: the numbers above are illustrative, so size both against the context window of the model you configure. A threshold above that window lets the provider reject the call before compaction ever triggers.

retain.maxMessages caps the recent verbatim suffix instead of requiring an exact count. Retention begins at a user-turn boundary, so the helper may keep fewer messages rather than split a turn. Add retain.maxEstimatedTokens to keep even a small number of unusually large messages from consuming the next context window. The newest user turn remains indivisible when it alone exceeds a retention bound; the helper still summarizes every older eligible turn. Omitting retain defaults to { maxMessages: 6 }.

When either token maximum is configured, provide estimateTokens for model-specific message accounting or use the built-in approximate estimator.

The threshold applies to the projected input for the next model call. When the candidate contains a valid committed model step, the compactor starts from that step's provider-reported inputTokens + outputTokens and asks estimateTokens to count only messages appended after it. Hidden reasoning and provider message framing can make replay differ slightly, so keep a safety margin. Backend or client tool results are added after the provider call but live on the same assistant message, so the suffix may begin with a synthetic assistant delta containing only those unmeasured results. If no valid usage anchor exists, including the first call and the first usable context after an invalidated boundary, the callback receives the full candidate instead.

PAI removes previous user turns' reasoning immediately before the provider call, after compaction has evaluated the context. The estimate does not subtract that reasoning from the previous step's reported usage, and fallback estimation still sees unpruned history. It can therefore trigger compaction earlier than necessary, especially after a reasoning-heavy turn. The next reported model call supplies a fresh usage anchor; no usage totals are changed.

Do not use cumulative run or thread usage for the threshold. Those aggregates sum multiple provider calls and repeatedly count shared prompt prefixes. They remain useful for billing and telemetry.

The callback estimates message tokens only. The usage anchor already includes the previous call's instructions and tool definitions, but those inputs can change between calls. Choose a threshold below the model's full context limit to leave room for those changes, provider framing, and the next output. The same estimator evaluates bounded retained suffixes when retain.maxEstimatedTokens is present. A custom estimator must not report fewer tokens when messages are prepended; this lets retention find the oldest fitting user-turn boundary without testing every overlapping suffix.

Size retention well below the threshold. The usage anchor counts instructions and tool definitions, so the context immediately after a compaction is the retained suffix plus the new summary plus that per-call overhead. When that floor approaches the threshold, the next step exceeds it again with almost nothing older left to compact, so compaction repeats every turn and reclaims little. A large tool set makes the overhead substantial. The two values cannot be validated against each other, because the helper cannot see the overhead.

Immediately before summarization, the helper writes run.metadata.contextCompaction with its start time, projected input, and configured token threshold. The ordinary run.updated stream makes this available on the matching entry in chat.state.runs. The helper clears the field to null after the summary commits or best-effort when compaction fails.

The summarizer runs outside the thread mutation and receives the selected AI SDK ModelMessage[] provider context (role and content), run/thread/step lifecycle inputs, and cancellation signal. It does not receive public PaiMessage or trusted LifecycleMessage values. Persistence ids, timestamps, producers, and message metadata are not copied into its messages. The callback must return non-empty text. The active run lease prevents regeneration and administrative history rewrites while summarization is active. The callback must not rewrite the source messages it is summarizing. Keep external side effects idempotent because a failed run may still be retried by its caller.

A throw, or empty text, fails the run. No summary is appended and the compaction cursor does not advance, so the next run re-enters the same decision. The failure itself is durable, and the activity field written before summarization is cleared only best-effort, so a process that dies mid-compaction can leave it set. A transient failure recovers on retry, while a deterministic one blocks every later turn on the thread. Handle provider refusals and empty completions inside the callback.

The first compaction of a pre-existing long thread must page the context that has no cursor yet. After the first boundary, lookup and reads are bounded by the boundary stack and uncompacted suffix. Usage anchoring also limits token estimation to messages after the latest valid model step in the usual case.

At the start of context resolution, the runtime captures the newest physical message as a fixed upper anchor. Every lazy page remains below it. The thread's active run lease is the serialization boundary: regeneration and administrative history rewrites cannot begin until the run finishes.

Durable shape

The built-in helper owns one flat cursor per thread. It deliberately has no strategy identifier or nested strategy map:

type ContextCompactionPrivateMetadata = {
  contextCompaction: {
    version: 1;
    usageInvalidatedAt?: string;
    boundaries: Array<{
      summaryMessageId: string;
      throughMessageId: string;
    }>;
  };
};

The stack keeps at most 64 boundaries. A stack rather than one pointer lets a partial regeneration discard newer summaries while immediately falling back to the newest older boundary that survived.

Changing thresholds, retention, estimation, or future summarization continues from this cursor. Replacing context management entirely requires the new policy to adopt or migrate the current boundary before its first provider call; simply removing compaction can make an already-long thread exceed its model limit.

Each successful compaction commits two changes atomically while comparing the observed private cursor metadata. Active-run ownership fences the write without rereading every selected message:

• a committed user-role lifecycle message with transcript: true and context: false; and
• a new { summaryMessageId, throughMessageId } boundary in private thread metadata.

The summary appears in ordinary client state with producer: "lifecycle" and relation: "context", so applications can render it as a compaction boundary without a compaction-specific transport. It remains excluded from the default context projection; the compactor injects it explicitly. The compactor validates the referenced message's role, producer, visibility, status, and range endpoint before trusting it. Using user role preserves the original authority of summarized user content; it is not elevated into a system instruction. The active run lease prevents history changes during a long summary. Unrelated root changes such as lease heartbeats remain valid, while changed private cursor state prevents the summary from committing.

When a step creates a summary, downstream resolvers receive that summary plus the retained suffix immediately. Later steps start from the stored summary and page only context-visible messages after throughMessageId. Repeated compaction can summarize the previous summary plus the newer delta.

History rewrites

Regenerating an older user message physically truncates later records. Before that rewrite commits, the compactor's prepareHistoryRewrite hook removes every boundary whose summary or endpoint will not survive. Metadata repair and history truncation therefore succeed or fail together.

thread.retry() uses regeneration semantics and receives the same repair. Trusted or lifecycle code may also change stored message visibility. When at least one visibility.context value changes, the same preparation hook clears the compaction boundary stack and records a new usage-invalidation timestamp in the visibility-update transaction. A transcript-only visibility change does not invalidate context cursors.

After commit, other lifecycle helpers may observe historyRewritten to reconcile external caches or memory. That hook is best-effort; use an outbox or shared transaction for external invariants.

Custom context strategies

Use resolveModelContext directly for moving windows, retrieval, specialized summary layouts, or external memory. Calling next() obtains the composed downstream/default context and is memoized. Calling next(base) supplies a replacement AI SDK ModelMessage[] base while preserving inner middleware. input.history.modelMessages() converts a complete anchored range directly from private native values into provider context. A strategy that needs trusted durability facts may instead call input.history.collect() for LifecycleMessage[], including visibility and private metadata. Raw exclusive input.history.before() pages remain available when a strategy needs to control pagination itself; such callers must handle history.through explicitly. PAI never round-trips the public lifecycle projection back through the SDK.

The built-in compactor decides whether to compact from its durable physical base, commits any new summary and cursor, then calls downstream next(base) exactly once. When it compacts, downstream resolvers receive the new summary plus retained suffix on that same model step. Inner resolvers may add, remove, redact, or reorder the supplied model context without making cursor advancement ambiguous, because the durable compaction decision has already been made.

See the lifecycle reference for the generic API. ADR-0025 records the current compaction decisions and the ADR-0024 constraints it retains; it lives in the repository, under docs/content/docs/internal/architecture-decisions/, and is not part of the published site.