Self-hosting the sandbox runtime

Package: @pai/gke-sandbox-infra

Provide an existing cluster, a verified runtime image in your registry and the caller identities/networks you trust. PAI's Terraform module installs a closed runtime template, namespace permissions and a 20-pod quota for each application. Your infrastructure owns the cluster, cloud IAM, routing, registry and capacity choices. Deploy the application using your existing tools; PAI does not require a separate application-hosting module. The same runtime module can optionally install the sandbox operations dashboard inside that cluster. PAI supplies the dashboard image; you configure user access and resource sizing.

Before installation

The runtime requires configured Kubernetes and Helm providers, gVisor scheduling, an enforcing NetworkPolicy dataplane and a compatible Agent Sandbox controller. This release uses agents.x-k8s.io/v1alpha1 and extensions.agents.x-k8s.io/v1alpha1, with claims targeting SandboxTemplate directly, including cold starts without a warm pool. Verify both the API schema and controller behavior before installation or upgrade. Current GKE beta installations use pool-based claims and require a coordinated compatibility review; enabling Agent Sandbox alone is insufficient. Closed network mode does not require FQDN NetworkPolicy.

Your infrastructure supplies the installer permissions, caller authentication, cluster API access and private routes to the daemon. It also enables cloud APIs and supplies node image-pull permissions. Create this foundation before runtime providers initialize; the module contains no cluster, Google account, IAM or registry resources.

Build the runtime image

From the installed package directory:

docker build --platform linux/amd64 -t my-registry/sandbox-runtime:build -f runtime/Dockerfile .
scripts/verify-runtime-image.sh my-registry/sandbox-runtime:build
# Publish the verified image through your registry workflow, then use its digest.

The Dockerfile pins the daemon source and provides its entrypoint and tools in one container. Its default base includes Node.js; RUNTIME_BASE can select another compatible apt-based toolchain image. The verifier checks the daemon contract and needs the base's Bash and timeout utilities for its TCP probe. Verify your chosen language tools in your build process and own its base-image updates, scanning and publication. There is no PAI-hosted image default.

runtime_image is required and digest-pinned. The SDK rejects a reported daemon version that differs from its contract, but accepts missing metadata for compatibility. This check does not attest image provenance.

Application inside the sandbox cluster

Assume your application already runs in namespace chat, with pod label app=chat and serviceAccountName: server. Its pod uses the projected Kubernetes service-account token. The runtime module does not create this caller account or change the application's deployment.

Configure both providers for that existing cluster:

provider "kubernetes" {
  config_path    = pathexpand(var.kubeconfig_path)
  config_context = var.kube_context
}

provider "helm" {
  kubernetes = {
    config_path    = pathexpand(var.kubeconfig_path)
    config_context = var.kube_context
  }
}

module "sandbox" {
  source        = "./path-to-installed-package/terraform"
  runtime_image = var.runtime_image

  applications = {
    chat = {
      allow_from = [{ namespace = "chat", pod_labels = { app = "chat" } }]
      consumers = {
        server = { kubernetes_service_account = { namespace = "chat", name = "server" } }
      }
    }
  }
}

Review a saved plan and apply. This creates namespace pai-sandbox-chat, template pai-node-runtime, deny-all egress, no warm pool and a quota of 20 pods. The caller allowance permits only TCP 8080/9090. Both selectors form one peer: the pod must have that label and run in that namespace.

Inside the application's pods, construct the provider with the same template policy and the service transport:

import { createGkeSandboxProvider } from "@pai/sandbox-gke";

const provider = createGkeSandboxProvider({
  namespace: "pai-sandbox-chat",
  templates: [{ name: "pai-node-runtime", network: { access: "none" } }],
  transport: "service",
});

The SDK resolves the projected identity and cluster configuration. consumers authorizes Kubernetes operations; allow_from admits daemon traffic. Neither replaces the other. Close the provider when the application host shuts down to release its connections; this does not delete active sandboxes.

The package's terraform/examples/self-hosted supplies the root variables for this example. It uses local state; configure your team's remote backend before a shared deployment. If copying the starter outside the package, replace its relative module source with the installed package's terraform directory or an immutable Git source. Configure private repository access if that source needs it.

Application outside the sandbox cluster

For Cloud Run or an application in another cluster, change both the consumer identity and network peer. Keep the same default runtime and supply the actual existing Google identity plus its verified dedicated source CIDR:

applications = {
  chat = {
    allow_from = [{ cidr = var.caller_cidr }]
    consumers = {
      server = { google_service_account = var.caller_service_account_email }
    }
  }
}

Your infrastructure supplies the caller's Google IAM, credentials, cluster API reachability, private pod routes and firewall rules for TCP 8080/9090. The module binds the supplied email as a Kubernetes User; it creates no Google account or key. runtime/contract.json provides the daemon ports for external firewall configuration before the runtime exists.

Deploy the application with the target cluster's endpoint and base64-encoded CA. Its Google identity alone does not identify which cluster to reach:

import { createGkeSandboxProvider } from "@pai/sandbox-gke";

const provider = createGkeSandboxProvider({
  namespace: "pai-sandbox-chat",
  templates: [{ name: "pai-node-runtime", network: { access: "none" } }],
  cluster: {
    endpoint: process.env.APP_SANDBOX_CLUSTER_ENDPOINT!,
    caCertificate: process.env.APP_SANDBOX_CLUSTER_CA!,
  },
  transport: "pod-ip",
});

The SDK uses ADC for the explicit cluster. A configured kubeConfig or kubeConfigPath/context is another supported Kubernetes authentication path. Daemon connectivity is separate: verify the source Pod/Node/SNAT address the sandbox policy actually sees and determine which other workloads can use it. A CIDR allows all sources in that range. Kubernetes claim RBAC does not authenticate direct daemon requests. The SDK's default port-forward transport cannot reach GKE gVisor sandboxes, so these examples select direct transports.

Defaults and overrides

An omitted or null template allow_from inherits its application's peers. An explicit list replaces them; [] admits no application caller. An enabled dashboard has a separate, explicit administrative network allowance. Quota omission or null uses the default, while any supplied map replaces it. Include pods in that map to retain a pod limit; warm-pool pods count toward the quota.

For a different toolchain and pod sizing, set flat template fields inside the application. This example retains its application's caller allowance:

templates = [{
  name  = "python"
  image = var.python_runtime_image
  resources = {
    requests = { cpu = "500m", memory = "1Gi" }
    limits   = { cpu = "1", memory = "2Gi" }
  }
  extra_env = [{ name = "PYTHONUNBUFFERED", value = "1" }]
}]

Configure the SDK to declare python instead of pai-node-runtime for that example. When declaring multiple templates, give each a distinct name and set SDK defaultTemplate explicitly. The module exports namespace/template names, not a default chosen by sorting. Keep the SDK's declared network and custom root consistent with each installed template.

The old overrides wrapper is rejected. Its supported fields now live directly on the template: resources, extra_env, labels, pod_labels and allow_from. Labels cannot replace runtime policy selectors; environment additions cannot replace HOME or the daemon-version convention.

PAI retains ownership of the container, daemon ports/arguments, gVisor, non-root execution, read-only root filesystem, dropped capabilities and absence of a sandbox service-account token. network.access = "full" selects GKE's managed policy and rejects effective caller peers; it is not arbitrary private-network access. Cluster policy must not broaden these defaults: NetworkPolicies are additive.

Upgrades and acceptance

Outputs are application_namespaces, application_templates, sandbox_operator_role and nullable dashboard. Each application needs a distinct namespace because binding Leases are namespace-scoped. Independent deployments need an explicit isolation decision before sharing one namespace.

For existing module calls, move fields out of overrides before upgrading and review the new quota default. Use quota = {} to deliberately retain an unbounded namespace. For the former cluster-owning module, follow the package's terraform/README.md state-transfer instructions before removing Google resource addresses. Preserve existing cluster settings during that migration.

Chart-only changes appear as Helm updates. A template update does not replace already running workspaces. Destroying this module removes its namespaces and ephemeral sandboxes; destroy runtime resources before deleting the external cluster so its API remains reachable.

Offline validation checks chart isolation, Terraform defaults/overrides and package contents. The local kind fixture lacks GKE gVisor and network enforcement. Test SDK probe/create/command/file operations, a denied caller and denied egress from the actual application identity/network before accepting a GKE deployment. A successful plan or template probe alone cannot prove those paths.