AI Harness

The X4 AI harness is the bounded interface that lets an agent inspect a workspace, infer useful setup, and ask X4 to apply validated changes. It is not a replacement for the registry, sandbox runner, env policy, or deterministic test commands.

Decision

Use a runner-agnostic X4 harness contract and make the official Mastra Harness the implementation dependency for the Mastra runner.

• X4 owns the workspace model, registry schema, env filtering, sandbox startup, preview URLs, status checks, logs, validation, and evidence.
• The harness owns the agent loop: read context, inspect files, run allowed discovery commands, propose registry changes, and verify the result.
• Mastra Harness is the durable orchestration backend for long-running or GitHub-driven workflows.
• The local CLI remains the default backend for Cursor/onboarding workflows, so a developer can use X4 without installing or deploying Mastra.

This keeps local setup simple while still letting X4 reuse Mastra Harness where Mastra already adds value: session state, tool approvals, modes, retries, workflow history, GitHub events, and durable audit trails.

Current State

X4 already has the first pieces:

• x4 ai sandbox-plan writes a redacted Markdown prompt under .x4/ai/.
• x4 ai sandbox-profile asks the configured model for strict JSON and validates the proposed sandbox profile before writing it.
• x4 sandbox run "<goal>" is the human-facing agentic shortcut. It prepares the selected service profile from the goal and then runs normal deterministic sandbox up. If startup fails, it feeds the error, sandbox state, and available logs back into a new profile attempt, up to three attempts by default.
• x4 sandbox up --ai remains the lower-level experimental shortcut for preparing a specific profile before deterministic startup.
• Generated .x4/context.md, .x4/agent-instructions.md, and .x4/commands.md give agents the workspace context.

These commands are useful, but they are not a full harness yet. They provide a bounded prompt/proposal path plus a small retry loop for sandbox run. A full harness adds broader controlled tools, observations, retries, and evidence.

Why Mastra Fits

Mastra should be used when the AI flow needs workflow state rather than just one local command:

• preparing a sandbox profile from a GitHub issue or PR;
• retrying setup after a failed sandbox boot;
• preserving the full prompt/proposal/status history;
• routing failed checks into triage;
• posting evidence back to GitHub through the X4/Mastra bot flow;
• continuing work asynchronously after the developer closes their editor.

Mastra should not be required for the first local onboarding path. A new user opening a repo set in Cursor should still be able to run:

x4 ai sandbox-profile --entry "$PWD" --profile local-preview --write
x4 sandbox up --profile local-preview --serve-preview --wait

Dependency Boundary

Use Mastra's released Harness class for the harness layer instead of rebuilding session management, modes, tool approvals, subagents, and persistent threads inside X4.

The dependency should live behind an adapter:

import { Harness } from "@mastra/core/harness";

That adapter translates between Mastra Harness concepts and X4's stable tool contract:

Mastra Harness concept	X4 mapping
Session and thread state	.x4/ai/ run state, Mastra durable state, and final evidence.
Modes	Plan, inspect, propose profile, start sandbox, verify, summarize.
Tool approvals	X4 approval policy for writes, sandbox startup, cleanup, and PR actions.
Tools	Guarded X4 tools for workspace reads, search, env key listing, registry proposal, sandbox status, logs, and URL probes.
Subagents	Optional focused inspectors for package managers, framework configs, docs, or failed logs.

Keep the adapter boundary explicit because X4 still owns the safety and execution rules. The model and Mastra Harness can decide which approved X4 tool to call next, but they should not bypass registry validation, env filtering, sandbox rendering, preview policy, or evidence collection.

Because the Mastra Harness API is currently new, pin the dependency exactly in the package that implements the Mastra runner and keep the local CLI path working without it. Do not make @mastra/core a required dependency for the basic X4 CLI until the adapter is stable and covered by integration tests.

Codex Subscription Path

The preferred reasoning engine for X4 setup is Codex SDK, authenticated through the user's existing Codex access where possible. That means a developer using Codex through a ChatGPT subscription should not need to paste an Azure or OpenAI API key into X4 just to run local AI setup.

Interactive local setup:

codex login
x4 sandbox up --profile local-preview --ai --ai-engine codex-sdk --wait

Durable automation on a trusted control server or worker should use a Codex access token when the workflow needs ChatGPT workspace governance, Codex entitlements, or subscription-backed usage:

export CODEX_ACCESS_TOKEN=...
x4 sandbox up \
  --profile local-preview \
  --ai \
  --ai-engine codex-sdk \
  --wait

Use a Platform or Azure API key only for the direct-model path:

x4 sandbox up --profile local-preview --ai --ai-engine direct-model --wait

The setup X4 should check before starting a Codex SDK harness run:

• codex is installed on the host that runs the harness.
• The host has an active codex login session, CODEX_ACCESS_TOKEN, or an explicitly configured API-key login.
• The workspace has generated .x4/context.md and registry state.
• The harness runs with a bounded sandbox such as workspace-write for profile preparation.
• Remote worker startup still goes through X4 commands; Codex SDK does not get unrestricted SSH or Docker access.

Harness Tool Contract

The harness should expose X4 tools, not raw unrestricted shell access.

Tool group	Purpose
Workspace	Read .x4/context.md, workspace repo list, entry repo, branch, registry path, and sandbox profile.
Search	Run bounded file discovery such as rg, package metadata reads, docs discovery, and config discovery.
Registry	Read the registry, propose a sandbox profile patch, validate it, and write only after validation.
Env	List env file candidates and env key names. Env values stay out of prompts and logs.
Command	Run allowed discovery commands from repo roots, such as package-manager script listing and non-mutating diagnostics.
Sandbox	Run x4 sandbox up/status/logs/down/clear through X4, not through model-authored shell.
HTTP	Probe health URLs and preview routes, storing status and snippets as evidence.

The model can reason over tool results, but X4 remains the policy boundary.

Safety Rules

• The model cannot change the repo set. The workspace already defines it.
• The model cannot receive or print secret env values. It can see key names and selected env file paths only.
• The model cannot run arbitrary shell. Commands go through an allowlist and are scoped to workspace repos or .x4/.
• The model cannot write registry YAML directly. It returns structured JSON, then X4 validates repo ids, service ids, commands, ports, routes, dependency edges, images, Dockerfiles, and inline env values.
• Service startup always goes through deterministic x4 sandbox up.
• Evidence comes from X4 status, Docker/worker state, logs, and HTTP checks, not from the model claiming success.

Sandbox Preparation Flow

The baked-in flow should look like this:

Local Runner

The local runner is the default for Cursor and command-line use. It should:

• run inside the current X4 workspace;
• use the configured provider from the X4 .env;
• write prompts, proposals, and observations under .x4/ai/;
• return a normal command result to the user;
• require no Mastra service.

This is the right runner for onboarding because it keeps the first successful demo path short.

Mastra Runner

The Mastra runner should use Mastra Harness to implement the same X4 harness contract, but add:

• durable workflow state;
• resumable runs;
• retries and backoff;
• GitHub issue/PR event context;
• bot comments/checks generated from X4 evidence;
• async execution on X4 workers/testboxes.

The runner should call X4 tools and CLI commands. It should not duplicate registry parsing, env policy, sandbox rendering, preview routing, or evidence rules.

Implemented Command Shape

Current commands can remain as the simple path:

x4 ai sandbox-plan --entry "$PWD" --run
x4 ai sandbox-profile --entry "$PWD" --profile local-preview --ai-engine codex-sdk --write
x4 sandbox up --profile local-preview --wait

The harness-backed path is:

x4 ai sandbox-profile \
  --entry "$PWD" \
  --profile local-preview \
  --ai-engine codex-sdk \
  --write

x4 sandbox run is the preferred natural-language wrapper around the same preparation step plus deterministic startup:

x4 sandbox run "frontend, backend, and admin dashboard" --ai-engine codex-sdk

x4 sandbox up --ai remains available when a script already knows the exact profile:

x4 sandbox up --profile local-preview --ai --ai-engine codex-sdk --wait

The backend is Mastra. The engine defaults to a configured direct-model provider when one is complete, otherwise codex-sdk.

Every AI-backed sandbox profile run writes:

.x4/ai/runs/<run-id>/goal.md
.x4/ai/runs/<run-id>/prompt.md
.x4/ai/runs/<run-id>/response.md
.x4/ai/runs/<run-id>/proposal.json
.x4/ai/runs/<run-id>/state.json

Only sandbox profile preparation currently gets AI assistance. Workspace creation, registry validation, env filtering, sandbox startup, cleanup, and PR operations remain deterministic.

Implementation Plan

1. Expand the Codex SDK path from single-turn proposal generation into a bounded inspect/propose/verify loop.
2. Persist Mastra Harness state in the deployed Mastra app instead of using only local run artifacts.
3. Add guarded X4 tools for discovery commands, sandbox status, logs, and HTTP probes.
4. Wire Cursor to expose a Prepare Sandbox With AI action that shows the proposed profile, validates it, and then lets the user start the sandbox.
5. Promote x4 sandbox up --ai from experimental only after local and Mastra harness runs produce the same validated registry profile shape.

Non-Goals

• Do not make Mastra a mandatory dependency for local X4 onboarding.
• Do not let AI execute arbitrary Markdown as shell.
• Do not let AI select hidden repos outside the current workspace.
• Do not let AI infer or synthesize secrets.
• Do not replace registry profiles with transient model output.