Try the beta in an isolated project

This guide targets 0.2.0-beta.1. The npm beta tag was still 0.1.0-beta.3 when checked on September 27, 2026 (UTC). The source repository is private. Before publication, use a maintainer-provided candidate tarball; after publication, pin the exact version. Neither path needs repository access.

Prerequisites

Use Node.js 18 or newer, npm, and Chrome or Chromium. Node 24 is covered by CI. A hosted model needs credentials and network access; local inference needs a running model server. Start with the installation check below before adding model configuration.

Install the candidate tarball

Create a fresh folder and put asserthive-webtest-ai-0.2.0-beta.1.tgz in it:

mkdir webtest-trial
cd webtest-trial
npm init -y
npm install --save-dev ./asserthive-webtest-ai-0.2.0-beta.1.tgz
npx playwright install chromium
npx webtest-ai --version
npx webtest-ai --help
npx webtest-ai demo

Alternatively, once that exact version is published:

npm install --save-dev @asserthive/[email protected]

The demo is a deterministic rehearsal. It validates browser execution and reporting; it does not prove live model quality. A native Chrome installation can also be used through WEBTEST_AI_CDP_CHROME. Installing Chromium once avoids browser discovery problems on a new machine.

Run against public websites

Copy the packaged read-only examples into your own project:

node -e "const fs=require('node:fs'); fs.cpSync('node_modules/@asserthive/webtest-ai/examples/public-sites','public-sites',{recursive:true})"
npx webtest-ai run --suite public-sites --config public-sites/webtest-ai.config.json

The examples check Wikipedia article rendering, Hacker News newest submissions, and Example Domain. They use exact steps and real public pages, with no model fixtures. Accessibility and LCP/CLS collection run in warning mode. Public content, access policies, bot protection, TLS, or connectivity may change and can make a run fail or inconclusive. The examples explicitly allow up to 254 candidates. Larger pages can still return candidate_overflow; use a smaller page for exact-step checks, or enable scoped discovery for goal-based journeys. These are opt-in smoke checks, not a default CI dependency or proof of compatibility with every website.

To watch the browser, add --headed to the run command. Run one site first if troubleshooting:

npx webtest-ai run --suite public-sites/wikipedia.md --config public-sites/webtest-ai.config.json --headed

Switch to a live navigator

Use Goal/Success Markdown from writing specs. Copy a packaged examples/config/goal-agent*.json template into your project, set its model and endpoint, and name it in the run command's --config argument. Keep API credentials in environment variables. Exact-step public checks do not establish live navigator accuracy.

For the packaged demo with an external Codex session:

npx webtest-ai demo session headed

This process waits for a separate agent to read fresh requests and reply. It is not an automatic use of your subscription. In an installed project the helper is:

node node_modules/@asserthive/webtest-ai/demo-site/sessionControl.js next

Set WEBTEST_AI_SESSION_CONTROL to the path printed by the running demo. Submit the selected action as JSON on stdin to the same helper with reply. See session instructions. The demo mocks Jev; missing subscription usage remains unavailable rather than zero.

Check the evidence

Open the printed HTML report. Verify the site URL, verdict, screenshots, model identity when applicable, tool calls, and Quality checks. Agent JSON stores quality.a11y, quality.vitals, and quality.visual. LCP and CLS are lab measurements; INP is not collected. A disabled or unavailable measurement is not a quality pass.

A result may pass its functional checks while showing quality warnings. Inconclusive means the runtime could not establish the result, not that the website passed. Failed, inconclusive, or blocked results produce a nonzero CLI exit. See quality policies for gates and visual baselines.

Maintainer release checks

Build the tarball with npm pack, install it into a clean project, verify the exact version, run the bundled demo, and run an opt-in public smoke set. Test a live navigator separately. Publish and update the npm tag only after review, then repeat the consumer install from the registry. A local tarball pass does not prove the published registry package is the same build.

Validation recorded for this candidate

On September 27, 2026 (UTC), an isolated consumer project installed the candidate tarball and passed three read-only exact-step checks: Example Domain content, Hacker News navigation to newest submissions, and Wikipedia article rendering. Reports contained real axe findings and lab LCP/CLS measurements. This was not a live-model benchmark.

A separate Wikipedia section-navigation check reached the candidate bound and was inconclusive. The first Hacker News run also exceeded the default 80-candidate bound; its repeated run passed with the explicitly configured 254 limit. Network failures during setup led to a native CDP navigation-error fix. Public checks do not cover authenticated applications, cross-origin frames, popups, or every website.

See limits, multi-page journeys, and login reuse before configuring longer authenticated flows.