Migrating to the bounded engine

The default runtime is now bounded, and the CLI's default driver is cdp. Treat this as an execution compatibility change before releasing another package version. The package name and current published version are unchanged in this review branch.

Adopt the new path

1. Choose Goal/Success for agent-directed execution, or retain Steps/Expected for scripted execution. See Markdown goals. A goal needs observable success conditions and a configured agent profile.
2. Replace Assert outcome with a concrete observable assertion. For example, assert the confirmation text and destination URL required by the acceptance criteria. Semantic verdicts are not release gates in the new runner.
3. Select CDP and configure Jev or a local selector profile. Exact, unique labels work without a model. Keep selectors and credentials out of Markdown.
4. Seed a test tenant and declare approved input values or environment references. Use isolated, resettable data for actions that submit or change application state.
5. Run a representative smoke set. Compare failure detection, false passes, ambiguity rate, model-call count, cost completeness, and p95 runtime against the previous runner.
6. Update consumers to accept inconclusive, flaky, schema-v2 reports, nullable costs, and attempt-specific artifacts. Gate on observed results, not a confidence score.

Keep a suite on the previous engine

{
  "driver": { "name": "playwright" },
  "execution": { "engine": "legacy" }
}

Legacy CDP is also available by choosing driver.name: "cdp". The old runner moved to src/runtime/legacyRunTest.js; the maintained native driver moved from examples into src/runtime/drivers/nativeCdp.js. The example import remains a compatibility export. Existing regression tests explicitly exercise the legacy engine; new tests exercise the default path separately.

Bounded goals can now choose observed frames and pages explicitly. Legacy scripted frame commands are not automatically translated; migrate their intent into goals and validate against your application. See context and authentication boundaries. Current screenshot settings work; browser traces and journey recording remain legacy features. After hooks run within the original deadline, so external fixture cleanup should also be recoverable by the factory after worker termination.

Configuration and schema changes

The new selector section chooses element selection independently of the general authoring model. Profile selection uses provider: "profile" plus the profile name; otherwise a default merged provider: "jev" will continue selecting Jev.

Supported bounds are maxSteps, maxToolCalls, maxModelCalls, and maxCandidates. The last value must be at most 254, reserving one Jev option for NONE. Cleanup is allowed one extra lifecycle event beyond the tool limit. Requests are limited to 96 KB. A deadline aborts model transport and closes the browser context; execution does not resume when a late response arrives.

Report schema v2 is additive for existing result fields. cost remains model-only; runCost includes configured elapsed-time infrastructure estimates. Preserve null totals in downstream systems. Do not convert them to zero. Tool counts refer to framework operations, including observation, assertion polls, model requests, and context lifecycle, not individual CDP protocol messages.

The finite-choice local-model response must contain a choice, confidence, and a probability entry for every offered reference and NONE. Missing, extra, out-of-range, inconsistent, or tied outputs stop execution. This differs from the old planner's generated action list.

Quality evidence and visual checkpoints

Bounded HTML and agent JSON reports expose separate accessibility, Web Vitals, and visual results. Each check records its policy, status, reason, measurements, and evidence. Disabled and unavailable checks never appear as passes. A failed journey skips end-of-test accessibility and vitals checks.

Configure checks explicitly:

{
  "execution": { "quality": {
    "a11y": { "mode": "fail", "maxSerious": 0 },
    "vitals": { "mode": "warn", "lcpMaxMs": 2500, "clsMax": 0.1 }
  } },
  "visual": { "mode": "fail", "baselineDir": "visual-baselines", "updateBaselines": false }
}

Modes are off, warn, and fail. Threshold failures in fail mode fail the test; unavailable required evidence makes it inconclusive. Warning mode preserves the functional verdict and records findings. Accessibility uses axe; vitals measure lab LCP and CLS, not INP or field Core Web Vitals compliance.

Add Visual: checkout under a goal's Success section, or Capture visual checkpoint "checkout" in a scripted test. The navigator cannot replace this assertion. Action screenshots alone do not perform visual regression.

Visual checkpoints compare exact image bytes, with separate baseline and actual images and a hash difference record. They do not offer pixel tolerances or heatmaps. Use stable data, fonts, viewport, and browser versions. Baselines are scoped by suite, test, target, browser, and checkpoint name.

Missing baselines are unavailable and are never created automatically. Set visual.updateBaselines: true explicitly to capture candidates. This returns review-required and an inconclusive verdict. Review the images, commit approved baselines, then rerun with updates disabled. Keep baseline updates outside unattended release gates.

The legacy HTML reporter remains active for legacy execution and retains its compatibility coverage. Bounded execution uses the new reporter; both retain the full JSON artifact.