Commands

Run

tapstep test <flow.yaml | dir> [--driver web|android|ios|app[:port]] [-v]
tapstep test app.flow.yaml --headed        # real browser window
tapstep test app.flow.yaml -c              # watch mode: re-run this flow on save
tapstep test flows/ --tags smoke --shards 4   # directory run: tag filter, 4 flows at once
tapstep live <flow.yaml> [--port 7878]     # interactive live view in the browser

--driver web|android|ios|app[:port] picks the target (app = a desktop app with the test bridge, default port 9223 — see Desktop apps). Without it tapstep auto-detects: a URL appId means web, otherwise the connected Android device or booted iOS simulator. A flow with a vscode: header runs on the VS Code driver, and a flow with device: none runs without any device.

--browser chrome|chromium|edge|brave|opera|vivaldi|yandex|firefox|managed or a path to a browser executable picks the web browser (managed = pinned Chromium in ~/.tapstep/browsers). --viewport WxH, --locale en-US tune the browser; --video-scale 1–4 records --video at N× the viewport (sharp demo footage). -e KEY=VAL overrides flow variables; --env-file FILE loads extra dotenv files over the auto-discovered .env; --env-set NAME runs against a named environment from the config’s environments: (else its defaultEnvironment), loading .env.NAME with it — see config.yaml.

Other test flags:

--tags a,b / --exclude-tags a,b — filter a directory run by flow tags:.
--shards N — run up to N flows of a directory concurrently.
--config <config.yaml> — workspace defaults: flows, tags, ai, report; flags win — see config.yaml.
-c / --continuous, --video, --capture-network — single flow only: re-run on save, record a video next to the flow, capture network traffic.
--analyze — AI triage of a failed flow (needs an ai: block in the config or TAPSTEP_AI_ENDPOINT/MODEL/KEY).
--strict-selectors — fail the run when a step passed only because one of its or: alternatives matched. Without it such a step is green and the run says ⚠ N step(s) passed on an alternative selector.
--report json|junit|html|allure, --output <file|dir> — see Reports.
Tuning, in ms: --wait-timeout, --settle-timeout, --settle-stable-for, --poll-interval; retries: --action-retries, --scroll-attempts.

live also takes --driver, --browser, --headed, --viewport, --locale and the tuning flags.

Inspect & tooling

tapstep inspect [target] [--measure] [--driver …] [--browser …]   # compact screen snapshot
tapstep list-devices                   # browsers, Android devices, iOS simulators
tapstep start-device android|ios [--name <avd|sim>]   # boot an emulator / simulator
tapstep doctor                         # check each driver's external tools
tapstep privacy install                # on-device PII masking model (~945 MB download)
tapstep privacy status                 # privacy mode: enabled? model installed?
tapstep validate <flow.yaml | dir>     # parse-check flows and load their includes, without a device
tapstep fix-paths [path] [--write] [--all]  # respell ../ runFlow paths from the project root
tapstep init [path] [--samples] [--agents]  # starter flow / samples / agent setup
tapstep skill status                   # per agent: installed? same version as this binary?
tapstep skill install                  # agent skill for Claude Code / Codex / Gemini CLI / Cursor / OpenCode
tapstep skill refresh                  # rewrite installed copies with this binary's version
tapstep mcp [--driver app:9223] [--browser …]   # stdio MCP server for an AI client
tapstep appmeta <apk|ipa|app>          # parsed build metadata as JSON
tapstep bugreport [path]               # zip local diagnostics for an issue
tapstep update [--check]               # self-update to the latest release

tapstep mcp defaults to the connected device, else web. tapstep update runs skill refresh afterwards, so installed agent skills follow the binary; skill refresh never installs anew — that is skill install.

tapstep validate reads a flow (or a whole directory, .helper.yaml files included) and reports what test would refuse before a device is booted. It also warns about a ${VAR} an environment leaves open and names which environments miss it — the check that catches a test recorded against one environment before another one runs it. Helpers are not checked that way: their variables come from the caller.

Include paths

Since 0.13.12. An older CLI or desktop app reads a runFlow: path from the including file’s folder only, so a root-spelled include in a nested flow does not load there — upgrade both before running fix-paths --write.

A runFlow: path is looked up in the including file’s folder first, then from the project root (the folder of the nearest config.yaml). Written from the root — helpers/login.helper.yaml — the same line works in a flow at any depth, so a group of steps copied from one test into another keeps running. The rule is the same for every file a step names: runScript:, a request: schema, uploadFile:, installApp:, and the helpers of before:/after: hooks.

validate loads each include from where the file lives and reports per line a helper that is not there — naming the file of that name it found in the project — required params nobody passes, and cycles, and exits non-zero; a helper outside the project is a warning.

tapstep fix-paths rewrites includes written with ../ (or ./, or \ separators) into that root form. It only prints the change by default:

$ tapstep fix-paths flows
flows/checkout/cart/pay.flow.yaml
     4 - ../../../helpers/login.helper.yaml
     4 + helpers/login.helper.yaml
1 path(s) in 1 file(s) would change — run again with --write to apply

--write saves the files; --all also respells includes that name a helper beside the flow (login.helper.yaml) from the root. An include that would then find a different file, or one outside the project, is left as it is.

Import & export

tapstep import playwright <spec.ts|dir> [-o flows]        # codegen output -> flows
tapstep export playwright <flow.yaml|dir> [-o playwright] # a flow -> *.spec.ts
tapstep export extester <flow.yaml|dir> [-o extester]     # a vscode: flow -> *.test.ts

Where an export lands is the project’s setting, not a per-command habit: without -o both write into the folder config.yaml’s exports: block names, and only without that into playwright/ and extester/ at the project root. The desktop app and the MCP tools read the same setting; an exports: block that does not parse stops the export rather than quietly falling back to the default folder.

Every exported file opens with a // tapstep: line naming the flow it came from and the hashes of both sides, so a later export can tell whether the file is still what the flow says, whether the flow has moved on since, and whether someone edited the file by hand.

Import reads what playwright codegen writes: one flow per test(...), locators mapped to selectors (getByTestId → testId, getByRole → role + name, .nth() → index, .filter({ hasText }) → containsDescendants). Whatever it cannot carry over — loops, helper functions, xpath locators, arguments computed at run time — is reported with its line and written into the flow as a # TODO comment where it happened. Nothing is dropped in silence, so the report is the list of what still needs a person.

export playwright writes the browser half of a flow. Steps with no Playwright equivalent (the mobile catalog, AI assertions) become // TODO comments and are listed in the summary. ${VAR} becomes process.env.VAR, so the exported spec reads its values the same way the flow did; an env value that is itself an expression (TITLE: "Order ${Date.now()}") becomes one const at the top that every step reads, as the run worked it out once. Each element is found through a small ui() helper that looks in the page and in every frame inside it, because a flow never says which document holds an element — an <iframe> preview needs no frameLocator written by hand. A tapOn followed by typing becomes one fill(), and an assertion that narrows on text keeps both halves (expect(locator).toHaveText(…)). The flow’s before: / after: — the whole resolved chain, project and testspace hooks included — become the spec’s test.beforeEach / test.afterEach; a flow with no hooks gets no blocks. A flow with a vscode: header exports as a standalone Electron spec instead: it installs the flow’s vsix into a throwaway profile and launches the installed VS Code through Playwright’s Electron launcher, the way VS Code’s own smoke tests do. A vscodeCommand becomes the palette title it answers to, and an id with no known title is left as a // TODO rather than exported as a command doing something else. The palette matches a command id as well as a title, and the three clean commands it hides once their state already holds are typed as the id for that reason — a miss there is read as nothing to do, and on a workbench in another language an English title misses every time. The rest are titles, so an export runs against a VS Code in English; set "locale": "en" in argv.json for one that is not.

export extester is the other way to leave with a VS Code flow: a vscode-extension-tester test — Mocha and chai over Selenium, run by extest. It refuses a flow without a vscode: header rather than writing something that cannot run, and a directory export walks past the flows that have none, saying how many. The first export into a folder also writes a README.md with the extest commands for it (get-vscode, get-chromedriver, install-vsix, run-tests, or setup-and-run in one go), the --code_version this machine’s VS Code reports, and the workspace the runner must open with -r. In the tests themselves: one TIMEOUT constant is the budget for every wait in a file (the flow’s own per-step timeouts are deliberately not carried over), and elements are found through a withWebview(...) helper that tries the workbench document and then each iframe.webview → #active-frame in turn, because a flow records that a step has an element, not which document holds it.

Reports

--report json|junit|html|allure prints a json/junit/html report to stdout, or writes it to --output <file>; allure writes an allure-results/ directory in the current dir (or --output <dir>). A directory run with --report produces one combined report.

With --config config.yaml carrying a report: block (format, output — defaults html → reports/, relative to the config’s folder) every run writes <flow>.<ext> there without any flag; a directory run writes one combined file. --report/--output still win.

A block — a runFlow into a .helper.yaml, a repeat or a retry — is one step named after what it ran, with its own steps nested under it, so a failing helper shows which of its steps failed. The top-level list still counts one step per command, which is what CI reports index by.

A step that had to act on a page that never settled says so in every format: the JSON step carries "settled": false and an "unsettledReason" (Loading, Network(2), Mutating, Navigating or Timeout), JUnit adds a <property name="unsettled"> to that test case, Allure an unsettled parameter, and the HTML an acted while loading chip on the row. A step on a settled screen carries neither field — see how a run knows the page settled.

A step that passed only because one of its or: alternatives matched says so the same way: the JSON step carries a drifted object (used, primary, alternative), JUnit a <property name="drifted">, the HTML a drifted chip on the row, and the run summary counts them. Such a run still exits 0 — --strict-selectors is what makes it fail.

Exit codes & failure artifacts

0 — every flow passed. 1 — flows ran and some failed. 2 — a config or infra error (bad flag, no device, unparseable config), so CI can tell a red suite from a broken runner.

On a failed run tapstep prints the screen at failure and writes <flow-stem>-failure.png — plus <flow-stem>-failure.network.json when network traffic was recorded — under reports/, printing the saved path:

inside a project (the folder of the nearest config.yaml, or --config) — the project’s reports dir: report.output when set, else reports/. The name is prefixed with the flow’s folders under the project root, so flows/auth/login.flow.yaml fails into reports/flows-auth-login.flow-failure.png;
outside a project — reports/ next to the flow file.

Never the current directory. The network log can carry request data, so keep reports/ in your .gitignore — tapstep prints a one-time hint when a project’s .gitignore does not ignore it (it never edits the file).

tapstep login [--token tapstep_pat_…]   # browser SSO; --token saves a personal token
tapstep logout
tapstep token create --name <name> [--expires-days N]   # a personal token for CI, shown once
tapstep token list                      # your tokens — id, name, prefix, last use; never the secret
tapstep token revoke <id|name>

tapstep login signs in via browser SSO by default; --token saves a token instead, and TAPSTEP_TOKEN in the environment counts as signed in. The token comes from tapstep token create, which opens the browser once and prints only the token on stdout (… | gh secret set TAPSTEP_TOKEN works); list and revoke sign in through the browser the same way. A tapstep_pat_… token is verified against the API once a day and cached in ~/.tapstep/token-check.json: revoked or unknown stops the command, an unreachable API never stops the run (a warning only if the token was never verified on this machine). Any other TAPSTEP_TOKEN value still works with a warning until 0.15 — see CI / GitHub Actions.

Every command except login, logout, token, doctor, bugreport, privacy, update, schema, validate, fix-paths, skill status and skill refresh requires a prior sign-in; once signed in, everything works offline.