Limits, multi-page journeys, and login reuse

Configurable budgets

These defaults apply to the normal bounded runner. Demo profiles can override them.

A starting configuration for a longer journey:

{
  "execution": {
    "bounds": {
      "maxSteps": 150,
      "maxToolCalls": 1000,
      "maxModelCalls": 100,
      "maxCandidates": 254
    },
    "timeouts": {
      "testTimeout": 600000,
      "navigationTimeout": 30000,
      "actionTimeout": 10000,
      "expectTimeout": 10000
    }
  }
}

The packaged examples/config/long-journey.config.json contains these larger budgets with scoped discovery. Add your existing navigator/selector profiles and explicit origins for your application.

This is a budget example, not a guarantee that a model will complete 150 steps. Keep the model request timeout shorter than the overall test deadline. Larger budgets can increase latency and inference cost. The two stale-target recovery attempts and unchanged-state loop detection are fixed safeguards, not configurable retries for a whole journey.

Multi-page support and current boundaries

There is no separate page-count limit. A test keeps its isolated browser context while navigating between pages, so its cookies and storage stay available. The step, call, and time budgets span the entire journey rather than resetting on each page.

A candidate is an eligible interactive element in the active document or selected scope. It is not a page count or a total across the journey. The default strict discovery mode preserves existing behavior: a page exceeding the candidate budget returns candidate_overflow.

For large pages, enable scoped discovery:

{
  "execution": {
    "discovery": { "mode": "scoped" },
    "allowedOrigins": ["https://app.example", "https://accounts.example"],
    "bounds": { "maxCandidates": 80 }
  }
}

The observation represents all eligible controls through named groups. Oversized groups are subdivided into ranges. The navigator chooses an observed scope and receives its controls or smaller groups. Every choice stays within the candidate budget, including Jev's maximum of 254 elements plus NONE. Scoped discovery needs a candidate budget of at least two. Selecting a scope consumes a navigator action and its associated calls. It does not silently discard the rest of the page. An agent can return to the document scope to inspect other controls; detached scope nodes reset discovery. This is DOM-based grouping, not visual segmentation or semantic search. Exact-step specifications retain their explicit vocabulary and should use a smaller starting page when they exceed the strict limit.

Goal observations include frame and page references. The navigator can explicitly switch to an observed nested frame, a cross-origin frame on an allowed origin, or a tab/popup, then return to the main frame or original page. Popup creation does not silently switch a bounded test's active page. Frame switching waits for a committed document within the navigation timeout and overall test deadline. Persistent blank or srcdoc documents are outside the current frame contract. Assertions inspect the selected document. Quality scans and screenshots cover the active page; they are not a combined audit of every frame visited.

Goals default to the suite origin. To span domains, list every permitted HTTP(S) origin in execution.allowedOrigins, including the suite origin and required identity-provider origins. Absolute navigation targets and document redirects are checked against that list; subresource fetching is not a network firewall. Relative goal navigation resolves against the observed document URL. A denied document request produces origin_policy. New popup targets are policy-checked after attachment, so this is not a guarantee that their first request can never leave the browser.

Nested frames and cross-site frame interaction have real-browser regression coverage. Rotated or scaled frames stop with unsupported_frame_transform; covered frames stop rather than clicking through an overlay. Closed shadow roots, canvas-only controls, CAPTCHA solving, and universal identity-provider/MFA support are not provided. These boundaries cannot be removed by increasing budgets.

Other fixed limits include a 96 KB model request, an 8000-character goal, and at most 20 success conditions. These limits are separate from the number of pages visited.

Login capture and reuse

The recorder and saved authentication profiles are retained. Their compatibility boundary matters:

Record a login from a local terminal:

npx webtest-ai record --url https://your-app.example/login --save-auth --auth-profile demo-user --output specs/login-recording.md

Complete the login in the recorder's browser, then stop recording as prompted. This writes a profile under playwright/.auth/demo-user.json. Do not commit saved sessions or package them with a demo. A Playwright storage-state capture normally saves cookies and localStorage; do not assume every app's sessionStorage or IndexedDB authentication is portable. The native CDP storage format can also retain sessionStorage when captured through that driver, but the Playwright recorder does not populate it by default.

Reference the saved profile in suite frontmatter:

suite: authenticated journey
baseUrl: https://your-app.example/dashboard
auth:
  mode: reuse
  profile: demo-user

To detect an expired session and recreate it before a test, extend the auth frontmatter:

auth:
  mode: reuse
  profile: demo-user
  validationPath: /api/session
  validationText: Authenticated
  loginPath: /login
  usernameEnv: DEMO_USERNAME
  passwordEnv: DEMO_PASSWORD
  usernameLabel: Username
  passwordLabel: Password
  submitName: Login
  successPath: /dashboard
  successText: Welcome,
  timeoutMs: 10000

Validation requires a successful HTTP response at the configured URL, without redirecting to a login page. validationText adds a browser-visible check. Without an explicit validation endpoint, existing saved-state reuse is preserved and the report warns that validation was not configured. validate: false explicitly skips validation. refresh: false prevents automatic login if state is missing or invalid, returning auth_required. The CLI refresh-auth option can request a fresh login; within a suite it refreshes once per profile and validates subsequent uses.

UI refresh uses exact, unique accessible control names. It supports a configured IdP start link and optional mfaLabel, mfaSubmitName, and environment/TOTP credentials through the existing credential resolver. It does not attempt arbitrary authentication discovery or solve interactive challenges. Native state capture retains cookies and storage for the final active origin; multi-origin storage and IndexedDB portability are not promised. A recorded profile can contain multiple origins for reuse.

Authentication work counts toward tool-call and elapsed-time budgets. Concurrent tests in one process serialize preparation of the same profile, then use isolated contexts. Separate processes should use different profiles or externally coordinate their state files. Refresh happens before execution; the runtime does not replay a partially completed journey if login expires mid-test. Fresh profiles are saved atomically with owner-only file permissions. Reports show reuse/refresh source, validation status, and actionable failure guidance without session contents.

Regression coverage includes manual-state reuse across three pages and a second isolated test, plus two concurrent 15-page goal journeys starting with expired state. The latter verifies one login refresh, session validation, redacted model input, and a clear failure when refresh is disabled. These use deterministic model responses to validate runtime behavior, not to measure a live model's planning accuracy.