Getting Started

Install

npm install --save-dev @asserthive/webtest-ai@beta
npx playwright install chromium

After installation, npx webtest-ai resolves the local CLI. The package includes the bounded browser runtime, the Markdown goal parser, reporting, and the demo site.

Write a goal

Create specs/homepage.md with the outcome you want to verify:

---
suite: homepage
baseUrl: https://example.com
---

# Homepage loads

## Goal
Open the public homepage and verify that the welcome content is visible.

## Success
- Text: Example Domain

A goal describes intent and success. It does not need CSS selectors, XPath expressions, or a prescribed click path.

Run it

npx webtest-ai run --suite specs/homepage.md

Reports are written under artifacts/reports/. The HTML report is for people, the agent JSON report is for automation, and the calls ledger is for detailed execution analysis.

Try the included demo

npx webtest-ai demo

The Everyday Goods demo runs twelve bounded Markdown goals covering product search, cart state, quantity, checkout, delivery, validation, reload persistence, and separate shipping and billing forms.

Mode	Use	Model behavior
`demo`	Repeatable protocol and reporting showcase	Local deterministic agent and selector fixtures
`demo local`	Open-source model on your machine	Configured Ollama or compatible navigator
`demo live`	Configured provider and Jev	Live navigator plus real Jev selection
`demo session`	External agent session bridge	External navigator, with selector mode chosen by configuration

Configure Jev

Jev is a specialized element-choice service. It receives a fresh candidate set and the navigator’s element intent. It is used only when local exact matching cannot identify one unique candidate.

{
  "agent": { "profile": "navigator" },
  "selector": { "provider": "jev", "model": "jev-1.13.0" },
  "models": {
    "profiles": {
      "navigator": {
        "provider": "openai-compatible",
        "model": "your-navigator-model",
        "endpoint": "http://127.0.0.1:8000/v1"
      }
    }
  }
}

TYPESAFE_API_KEY=your-key
WEBTEST_AI_MODEL_CONFIG=webtest-ai.config.json
npx webtest-ai demo live

What is verified

The runtime executes only supported bounded browser actions.
Approved data values are the only values a model may enter.
Fresh candidate references are revalidated before interaction.
Success conditions are fixed by the Markdown contract.
Uncertain model or selector decisions become INCONCLUSIVE instead of guessed clicks.

Legacy step-based specs remain supported. Use the bounded goal format for new agent-factory suites, then add explicit assertions for every outcome that matters.