AI agents

tapstep exposes its whole toolbox — see the screen, tap, run a test, heal a selector — to AI agents. There are three ways in, from zero-setup to bring-your-own-agent.

Built-in AI chat (desktop)

The ✨ icon in the desktop app opens a chat panel next to your tests. The agent sees the workspace and the device you pick: it records steps via snapshot/act, writes .flow.yaml files, runs them and heals failing selectors. Every file edit shows up as a diff in the transcript, and a version checkpoint is saved before the first change.

Five providers, in the chat’s ⋯ menu → AI settings:

• Anthropic API key — stored in the OS keychain. Endpoint and model are configurable (ai.endpoint, ai.model in config.yaml).

• OpenAI-compatible — any chat/completions endpoint: a local Ollama or LM Studio, a self-hosted vLLM, or a hosted API. Point it at e.g. http://localhost:11434/v1, name a tool-capable model, and nothing leaves your machine. The API key is optional — local servers need none.

• Claude Code — no API key: the chat drives a locally installed Claude Code signed in to your Claude subscription. Install it once and you’re done:

  npm install -g @anthropic-ai/claude-code
  claude   # sign in once

• Codex — same idea for ChatGPT plans: npm install -g @openai/codex, then codex login.

• Gemini CLI — npm install -g @google/gemini-cli. Google retired the free personal sign-in, so create an API key at aistudio.google.com/apikey (it has its own free quota) and paste it into the app — it lives in the OS keychain and is handed to the CLI per run.

Connect an external agent (MCP)

Settings → App → Agent endpoint starts a local MCP server bound to a device — localhost only, token-protected. The same row copies a ready command:

claude mcp add --transport http tapstep http://127.0.0.1:PORT/mcp \
  --header "Authorization: Bearer TOKEN"

Any MCP client works the same way (Cursor, Windsurf, …): point it at the URL with that header. The agent gets snapshot, act, run_flow, heal and friends against the selected device.

CLI workspaces

In a repo, scaffold agent config once:

tapstep init --agents

This writes .mcp.json (so Claude Code in that folder picks up the tapstep tools automatically) and an AGENTS.md cheat sheet with the flow-file conventions.

There is also an agent skill — a deeper playbook (CLI + flow reference, working conventions) the agent loads on demand:

tapstep skill install

It installs for every detected agent — Claude Code (~/.claude/skills), Codex (~/.codex/skills), Gemini CLI (~/.gemini/skills). Idempotent: re-run it after a CLI update to refresh.

The desktop app also ships an integrated terminal (the Terminal button in the status bar, or Ctrl+</kbd>) — handy for running claudeortapstep` right in the workspace.